Machines as Agency: Artistic Perspectives [1. Aufl.] 9783839406465

Table of contents :
Content
Editorial
Agency
Toward Embodied Musical Machines
Mechano-Poïia
Embodied Cognitive Science as a Paradigm for Music Research
Gil Weinberg, Scott Driscoll. The Robotic Percussionist – Bringing Interactive Computer Music into the Physical World
…as…
Impossible People. Some Thoughts on the Cultural History of the Android
Man and Machines. Psychological notes on a difference and its fading
Questions of Style. Subjects, Things and Shared Agency in Popular Articulations
“You See the Videos But I Know How the Machine Functions”. Human-Robot Interaction and Its Media Representations
Machines
Krimhild Becker. Reality is a pretence for fantasy
Gary Cass, Alan Mullett. Bioalloy. Designing a Cyborg’s Evolutionary Future
LivingRooms
Bill Vorn. Adaptive Machines. For Interactive Robotic Art Installations
Biographies

Citation preview

Machines as Agency. Artistic Perspectives — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Christoph Lischka, Andrea Sick (Eds.) — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Machines as Agency. —

Artistic —

Perspectives —

University of the Arts Bremen Press Series 04 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Imprint — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — University of the Arts Bremen Press Series 04 — — — — — — — — — — — — — — — — — Machines as Agency. Artistic Perspectives

« — — — — — — — — — — — — — — — — — « — — — — — — — — — — — — — — — — Editors

Distributed in North America by:

Christoph Lischka, Andrea Sick Proof-Reading

Sabine Melchert, Stefanie Möller Graphic Design

David Lindemann Corporate Design

Transaction Publishers Rutgers University 35 Berrue Circle Piscataway, NJ 08854

Stefan Bargstedt, David Lindemann, Matthias Wörle Printed by

Majuskel Medienproduktion GmbH, Wetzlar Bibliographic information published by Die Deutsche Bibliothek

Die Deutsche Bibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data are available on the Internet at http://dnb.ddb.de

©2007

transcript Verlag, Bielefeld

All rights reserved. No part of this book may be reprinted or reproduced or utilized in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publisher. www.transcript-verlag.de ISBN 978-3-89942-646-5

Tel: (732) 445-2280 Fax: (732) 445-3138 toll free 888-999-6778

Funding — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

« — — — — — — — — — — — — « — — — — — — — — — — — — — « — — — — — — —

Senator für Kultur der Freien Hansestadt Bremen

Bremen Marketing GmbH

Hochschule für Künste University of the Arts Bremen (FuE)

Atlas Elektronik Bremen

EFRE Europäischer Fonds für regionale Entwicklung

Content — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — Christoph Lischka, Andrea Sick. Editorial

10

— — — Agency — Jin Hyun Kim. Toward Embodied Musical Machines —

18

Christoph Lischka. Mechano-Poïia —

36

Lüder Schmidt. Embodied Cognitive Science as a Paradigm for Music Research —

48

Gil Weinberg, Scott Driscoll. The Robotic Percussionist – Bringing Interactive Computer Music into the Physical World

66

— — — …as… — Peter Gendolla. Impossible People. Some Thoughts on the Cultural History of the Android.

88

Friedrich Wolfram Heubach. Man and Machines. Psychological notes on a difference and its fading. —

104

Andrea Sick. Questions of Style. Subjects, Things and Shared Agency in Popular Articulations —

118

Jutta Weber. “You See the Videos But I Know How the Machine Functions”. Human-Robot Interaction and Its Media Representations —

136

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

  — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

— — — Machines — Krimhild Becker. Reality is a pretence for fantasy —

156

Gary Cass, Alan Mullett. Bioalloy. Designing a Cyborg’s Evolutionary Future —

164

f18 Institute. LivingRooms —

176

Bill Vorn. Adaptive Machines. For Interactive Robotic Art Installations

182

— — — Biographies

194

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

8

y K

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Christoph Lischka, Andrea Sick >

v

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Ö Editorial f v

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

8

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Christoph Lischka, Andrea Sick. Editorial — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — —

Editorial —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Today, both art and science are getting more and more involved with advanced technologies: digital media, bioengineering, smart materials, (distributed) actor-sensor-systems, robotics – to mention just a few of them. Artists often focus their research upon the “limits” of these technologies, in working on realizations of esthetic concepts as well as simultaneously reflecting critically their achievements, and thus pushing technological development towards new and unexpected horizons. Especially in the academic fields of research in art and science, the activities in both of these areas remain institutionally quite unrelated. It seems important then to dramatize, support and deepen the existing interfaces between art, science, and technology – transgressing traditional principles and styles of research, and selectively overcoming the side-by-side coexistence in favor of an integrated “laboratory of the future”. In 2006, the University of Bremen hosted the 10th RoboCup, an internationally reputable event, where advanced robotics technologies are brought into competition in a soccer-based scenario. We took this as a welcome opportunity to bring together artists, scientists, technologists, and theorists in a Symposium (SMARt 2006), primarily to identify the current state of the art in robotic-based technologies, discuss recent developments in corresponding interdisciplinary projects, and to envisage prospects for future development. Complementary, several artistic robotics projects were presented at different places in the City of Bremen. The topics included Autonomous Systems, Artificial Life, Biomorphs, Cyborgs, Philosophy, Media Theory, and Embodied Cognition. As SMARt 2006 was conceptualized as an informal event – intentionally in the spirit of ancient symposia in Greek academia – we decided not to publish proceedings in the usual manner. Instead, we present here a book with loosely coupled contributions of most of the conference‘s participants: Machines as Agency. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

10 11 — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — —

Due to the interdisciplinary and cross-boundary character of this symposium, it is almost impossible to organize these contributions in a linear, hierarchical vein – we would create rather artificial taxonomies and dependencies. Thus, we setup three main balance points, which should be seen as kind of “supernodes” in a widespread, complex network of articulated thoughts, actions, and esthetic experiences: Agency ... as ... Machines. Agency

This part collects four papers, mainly about topics of embodiment and agency in general. Jin Hyun Kim delineates the dis- and re-embodiment of the sound generation process over the past 50 years, focussing on the conflicting relationship between phenomenological and mechanistic narratives of the body, primarily in the context of robotics. She expects intensive discussions in this field in the near future. Christoph Lischka analyzes the dominance of computational metaphors within contemporary technoscience, and its prospects for the emerging field of convergent technologies. Particularly in the life-sciences, he complains about a lack of awareness of recent conceptual developments in physics, biology, and philosophy – arguing for a move towards a symmetric, post-computational praxis of mechano-poïia. Lüder Schmidt outlines a framework for music cognition based on dynamical systems theory. Summarizing the current state of research in Cognitive Science of Music, he describes a dynamics-based concept of embodiment which is extended to include music-related cognition, resulting in a model of “man as a musical machine”. First steps towards a robotics-based implementation are presented. Gil Weinberg and Scott Driscoll describe Haile, an improvisational robotic percussionist. This robot implements both perceptual modeling techniques as well as algorithmic music capabilities, and combines them with the visual and gestural richness of acoustic music playing. Haile is intended as a test-bed for novel forms of musical human-machine interaction. ... as ...

The second part accumulates contributions broaching the issue of “being as ...” – oscillating between the humans and machines, without (hopefully) getting trapped into mainstream categories. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Christoph Lischka, Andrea Sick. Editorial — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — —

Peter Gendolla opens this section with an analysis of the cultural history of the Android. Automata, as they can be found already with the ancient Greek and Chinese, were conceived both as slaves and “useless playthings”. Inspired by La Mettrie, Gendolla then analyzes the various pathways of the idea of machines as extensions or substitutes of the human. But things are changing: due to progress in neurotechnology, the very border between man and machine vanishes. Friedrich Wilhelm Heubach argues as psychoanalyst. The seemingly straight attribution of robots as machines is challenged by events like 9/11. People described as “machines of evil” blur the difference of man and machines, providing a rich repository for unconventional thinking. Heubach gives several illuminating examples, covering Turing, Kismet, cyborgs, sex, mom and paradise, with a short digression concerning “Die Verbesserung von Mitteleuropa”. Andrea Sick investigates the idea of “shared agency” between man and machine, and its popular articulations. She suspects that events like the RoboCup competition elude the dichotomies of subject and object, natural and artificial, man and machine, instead of overcoming them. One of her main questions is: How does the space open up between different styles that find their different echo in the different tones of different disciplines? Jutta Weber analyzes the role which media representations play in research strategies, particularly in robotics. She first introduces the field of personal service robotics, and presents some preliminary thoughts on the growing impact of media representation; she then extends these to Human-Robot Interaction and technosciences in general. Machines

The third and last part compiles accompanying texts for some of the exhibits. Krimhild Becker comments on her artwork as a painter, and how she uses photography to explore the boundaries between man and machine. E.T.A. Hoffmann‘s narrative “The Sandman” is a beginning for Becker to open the sight for seduction, perfection, and the ambiguity of the artificial and “natural”, machine-man and man-machine. Gary Cass and Alan Mullett present their project “Bioalloy”. In their research, Cass and Mullett explore possible futuristic evolutionary pathways of the living and the non-living. “Bioalloy” engages with the possibilities of interfacing a living system to a machine, eventually creating an artificial life entity – a Cyborg, thereby predicting an evolutionary leap in the 21st century where cybernetic entities are becoming more viable. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

12 13 — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — —

f18 Institute, an international artist group located in Hamburg, introduce their performance-installation “LivingRooms”: an ambient and a situation which applies to ideas and dreams to improve our daily life and environment. Starting with electrified kitchen via electronic bureau and virtual space towards “Augmented Environment” or “Ambient Intelligence”, a mixture of virtual and real space creates a picture which we can relate to our past and future. Bill Vorn gives an overview of his research on adaptive machines for interactive robotic art. The aim of robotic art projects is to induce empathy from the viewers towards characters which are nothing more than simple articulated metal structures; their strength relies therefore on subverting the perception of these creatures by simulating organic and metabolic functions, and by creating dynamic virtual architectures.. Bremen, July 2007 Christoph Lischka, Andrea Sick — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

rr Q Q id i ul_ g

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

01

v § +9ln

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

J

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

>

vvvv

K JM d H 3 O — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

UH Agency

ss

77

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

rH ILp c g h

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

k

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Jin Hyun Kim :

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

s

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Toward Embodied Musical Machines

r

c

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Jin Hyun Kim. — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Toward Embodied —

Musical Machines —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — Disembodiment of the sound generation process

The development of digital signal processing and computer technology has given rise to digital realizations of sound generation processes, in other words: sound generation by numbers. These numbers, acting as numeric representations of sound events, can be taken into account as samples of the sound pressure wave (cf. Mathews Technology; Roads et al.). The techniques of digital sound synthesis are based on the modeling of musical sounds by means of time functions. These functions allow computers to realize sound events in terms of algorithms which – expressed as programs by means of programming languages – serve as instructions for the computer. The computer has been considered as a musical instrument since the end of 1950s due to its capacity to generate sound algorithmically (Mathews, Digital Computer). The term “instrument” in this context refers to “an algorithm that realizes (performs) a musical event” (Dodge and Jerse 72). This idea of the computer as a musical instrument, as illustrated in figure 1, was implemented in the series of Music N, which had mainly been developed at Bell Laboratories from 1957 to 1968. The instrument consists of units of the computer program whose numerical input and output values are connected. Each unit receives numbers as input values that are then processed by an algorithm. Algorithms of an instrument calculate the samples of audio signals. The unit generators, which are specified in figure 1 as oscillators generating periodic waveforms, are used in order to generate audio signals that may be re-used for further processing. The idea of the computer as a musical instrument is therefore characterized by the algorithmization of sound generation processes, which implies their informatization. « — — — — — — — — — — — — — — — — « — — — — — — — — — — — — — — — — — 1 This decoupling of information and materials is a

general characteristic of algorithmic sound generation even in the case of physical modeling developed since the 1990s, in which material properties of a

physical-acoustic instrument responsible for the sound generation process are simulated. Some physical attributes of the modeled instrument – e.g., a violin – are given with a list of parameters such as the volume of

— — — — — — — — — — — — — — — — — — — — — — — — —

18 19 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 01 — — The computer as a musical instrument — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

This means that information is considered as crucial to a model and constitutive for digital sound synthesis. In the case of the signal models described above, information is contained in the numerical representation of the sound pressure wave in terms of frequency, amplitude, and duration. Hence a musical instrument that is intended to generate sounds does not necessarily need to possess a material property, but instead it is sufficient for it to operate algorithmically. In this way, information is separated and differentiated from materials, as in the case of traditional music instruments which generate sounds.1 Accordingly, algorithmization of the sound generation process implies disembodiment in the sense of separation of information from materials, in short, de-materialization. This idea of disembodiment seems to be closely related to the Cartesian mind/body dualism, in which a material phenomenon is considered as a secondary substance that is subsumed by a primary substance (mind). Katherine N. Hayles, an American literary theorist, characterizes the privilege of information patterns over material instantiation in algorithmic informatization as a typical posthuman point of view, from which “embodiment in a biological substrate is seen as an accident of history rather than an inevitability of life” (Hayles 2) and the body is viewed as the original prosthesis that can be manipulated, extended and replaced (cf. Hayles). Accordingly there are “no essential differences of absolute demarcations between bodily existence and computer simulation, cybernetic mechanism and biological organism, robot teleology and human goals” (Hayles 3), since all material objects are considered to be “interpenetrated by flows of information, from DNA code to the global reach of the World Wide Web” (Hayles 14). « — — — — — — — — — — — — — — — — « — — — — — — — — — — — — — — — — — the body, the bowing point, and the placement of the bridge. Though the instrument’s body is taken into account for a sound generation process, this body consisting of information patterns is considered as

decoupled from its materiality that has a resisting power in relation to the musician on the one hand and provides a context-sensitive condition according to different environments on the other.

Jin Hyun Kim. Toward Embodied Musical Machines — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — —

In short: information separated from the material medium that instantiates it becomes dominant (Hayles 13). Let us get back to the de-materialization of a musical instrument through the algorithmization of sound generation. A starting question for discussion in this article is whether it is enough to characterize a musical instrument solely by its capacity to generate sounds, as in the case of the computer as a musical instrument. In the latter case, a bodily action guided by physical energy which actuates a physical musical instrument becomes unnecessary, since algorithmic computation is sufficient for sound generation. Therefore, algorithmization of sound generation leads to a shift away from a skill-based access to sound generation processes, accompanied by the double feedback of tactile and aural senses, to a knowledge-based access. A rising number of approaches to integrating the physical body into algorithmic process of sound generation in recent decades may be interpreted as the re-instigation of bodily mediated sound processing. However, the re-instigation of the physical body itself does not seem to necessarily act as a solution for the disembodied process of algorithmic sound generation, for digital technologies can design the materiality so as to be controlled thoroughly by an information pattern from a posthuman point of view, as briefly described above. Hence it appears necessary to discuss the concept of embodiment in detail as opposed to that of the disembodiment induced by digital technologies, especially due to the fact that the physical body, whether biological or artificial, has been more and more integrated into algorithmic sound generation in interactive music and new media performance. Embodiment? Paradoxes of corporeality in the bodily-based algorithmic process of sound generation

The diverse strategies of integration of the physical body into algorithmic sound generation, which up to now have been observed in the context of interaction between bodily movement and sound, however, show paradoxes concerning the concept of corporeality. One concept that most often underlies technical strategies of enhancing human expressivity is the body as a material object, instantiating the mental substance, which is fitted with tools in order to externalize internal expressivity. Technology in this context is used to extend or replace the physical body which itself is considered an extension of the mental substance following the Cartesian mind/body dualism (cf. Kim and Seifert 140-141). — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — —

20 21 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 02 — — Seeds, from Ephémère (1998) — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

This idea of enhancing the physical body underlies many strategies of musical interface design, which aim at an extension of the human senses, using human-computer interfaces. An example is the so-called hyperinstrument project at the Media Lab of the Massachusetts Institute of Technology (MIT). This project, in which intelligent musical instruments have been developed since 1986, is based on the assumption that mechanical acoustic musical instruments limit possibilities of controlling sound synthesis by means of human gestures. Hence, they aim at enhanced human expressivity that must “transcend the traditional limits of amplifying human gestuality” by augmenting control dimensions of a musical instrument by means of interactive computer systems (Machover 3). Here, technology serves as an extension of the physical body. The other concept of corporeality that is usually observed in virtual reality and in cyberspace is the body as a construction. This concerns the aspect of manipulable control of the body, which implies that corporeality is not associated with a pre-given materiality, but is rather seen as an effect of discursive techniques. In other words, the body appears as a construction which can be modified and manipulated into any desired form. The body becomes an unresisting structure and passive construction. The physical materiality herein is not taken into account, for it is not conceived as a substantial aspect of the body. This concept of corporeality seems in this way to reproduce the Cartesian dualistic ontology instead of dissolving it. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Jin Hyun Kim. Toward Embodied Musical Machines — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Char Davies’ artistic projects, however, are noteworthy due to the attempt to reaffirm the participant’s corporeality by use of immersive technologies. For instance, in Ephémère (1998) an interaction with an immersive audiovisual virtual environment allows the participant to experience herself or himself as a “mortal fleshy body” embedded in “a living, flowing world”, namely “the ephemerality of being” (Char Davies). In this way Davies directs one’s attention toward being embedded in the environment by means of the body using virtual reality in contrast with a widespread idea of digital manipulable constructions of the body. Embodiment: a philosophical root (Merleau-Ponty)

Such an alternative concept of corporeality that seems to be of importance to embodiment, which has recently been coming into the focus of current discourses on the body, was proposed by the French phenomenologist Maurice Merleau-Ponty in the 1940s. In his theory of perception, which had been developed in his major works La Structure du comportement and Phénoménologie de la perception, he focuses on the bodily mediated aspect of perception. His main research perspective is directed toward “the relations between consciousness and nature, between interiority and exteriority” (Merleau-Ponty, Phenomenology 489). The behaving actor is for Merleau-Ponty not a fully-fledged subject founded on disembodied consciousness. He rejects any notion of a disembodied I that may be ‘embodied’ in the body acting in space and time. As John Wild writes rightly in his foreword in the English translation of La Structure du comportement that “human behavior is [for Merleau-Ponty] neither a series of blind reactions to external ‘stimuli’, nor the projection of acts which are motivated by the pure ideas of a disembodied, wordless mind” (Wild XIV), the relation between the behaving actor and the world, which may not be conceived as ‘linear’ causation, becomes a main topic of Merleau-Ponty’s philosophy. At the beginning of the “sense experience” chapter of Phenomenology of Perception, Merleau-Ponty supports the thesis that “perception does not present itself in the first place as an event in the world to which the category of causality, for example, can be applied, but as a re-creation or re-construction of the world at every moment” (Merleau-Ponty, Phenomenology 207). Sensations are for him not reducible to “a certain indescribable state or quale” (Merleau-Ponty, Phenomenology 209), but rather a mode of access to the world, a being-to the world, in which form and content meet: “The sentient and the sensible do not stand in relation to each other as two mutually external terms, and sensation is not an invasion of the sentient by the sensible. It is my gaze which subtends colour, and the movement of my hand which subtends the — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

22 23 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

object’s form, […] and in this transaction between the subject of sensation and the sensible it cannot be held that one acts while the other suffers the action, or that one confers significance on the other” (Merleau-Ponty, Phenomenology 214). A theory of perception is for Merleau-Ponty at the same time a theory of the body. The subject of perception is, according to Merleau-Ponty, not a pure subject without content, but the body (Merleau-Ponty, Phenomenology 206). He argues against classical psychology, which treats the body as an object among other objects, and notes that the body is not itself a fully external object, which can be observed, nor is purely internal to consciousness. Rather, the body is comprehended as the body-as-lived (or the lived body), which Merleau-Ponty also calls the phenomenological body or the body-proper (corps propre), “by performing it myself and in the degree to which I am a body which raises itself toward the world” (Merleau-Ponty, Phenomenology 90). Therefore, the relation between the body-as-lived and physical objects cannot be described as a causal relation. Objects are not located “out there” (Merleau-Ponty, Phenomenology 72) – i.e. separately from the bodily action – but in relation to the specific field of action. Our bodily mediated perception is closely coupled with action. Perception is a performative act: “the ‘perceptual side’ and the ‘motor side’ of behaviour are in communication with each other” (Merleau-Ponty, Phenomenology 209-210); “every perceptual habit is still a motor habit” (Merleau-Ponty, Phenomenology 153). According to Merleau-Ponty, habits, which refer to repeated actions through skillful copying, provide the stability of perception. Merleau-Ponty’s theory of perception challenges an age-old dualism between subject and object, mind and matter, perception and action, “inner” and “outer”. The relation to each other is characterized by “circular causality” (Merleau-Ponty, Structure 15). A theory of embodied perception, which can be traced back to Merleau-Ponty, is hence characterized by action-perception loops and embeddedness in the world mediated by the body. The idea of embodiment may be conceived as a basis for mediated presence, being-to the world. In other words, we have the experience of being-to the world with and by means of the body. Embodiment: mediated mentality

Following the idea underlying embodiment developed by Merleau-Ponty, the body may not be considered as a transparent mean which – as a kind of carrier – transfers ‘intended’ inner processes into the external world. What becomes of importance is rather the mediality of the bodily-based acts taking place in an interactive relation to the world. From the information technological point of view, a question to be concerned — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Jin Hyun Kim. Toward Embodied Musical Machines — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — —

here is how an agent is related to its environment. Some aspects have been discussed in current discourses on presence that in the context of information technology means the experience of being in a technologically mediated environment. However, most research on presence takes it for granted that technical mediation aims at immediacy, in other words: “the perceptual illusion of non-mediation, i.e. the extent to which a person fails to perceive or acknowledge the existence of a medium during a technologically mediated experience” (Lombard and Ditton). This seems to be based on the assumption that a technologically non-mediated form of communication, e.g., face-toface communication – in other words: body-to-body copresence, is an unmediated and ‘natural’ form of communication. Therefore it appears necessary to point to problems concerning this myth of a media-free communication. With respect to speech communication, speech has often been considered as a means of expressing amodal ‘language of thoughts’ as if they might precede an act of speech communication. Since this ‘ex-pression’ is considered to be manifested itself not only in utterances, but also in speech-guiding non-verbal gestures such as mimic and hand gestures, a face-to-face communication is viewed as a natural format for understanding each other. An alternative theory that pays attention to a speech-mediated recursive relation of a subject to itself was developed by Wilhelm von Humboldt in the 18th and 19th century. According to him, mental processes are indispensable to be made objective in the vocal announcement: “The intellectual activity” remains – according to Humboldt – “to a certain extent traceless passing”, unless it doesn’t become “external through utterances in speech and perceptible for our senses” (Humboldt Bd. VI 152; Bd. VII 53; Bd. V 174 – as cited in Linz 52). The performative act of utterance is a medial stage in which a merely ‘seemingly’ mental system renders capable of being constituted as a mental system just through its ‘external’ medial trace (cf. Jäger, Zeichen/Spuren 20). Ludwig Jäger, a German media theorist, takes up the theory of Humboldt in developing his “trace theory of mind”. He claims that distinguishable language concepts are not pre-existent in mental ‘thoughts’, but rather constituted through a medial transcription that affords ‘re-reading’. These medial traces in the course of their iterated acts lead the ‘mind’ to self-reference in a form of designation of concepts. Seemingly pre-existent inner thoughts to be designated are constituted or completed in a subsequent act of designation (cf. Jäger, Transkription). When we conceive our speech – according to the trace theory of mind – not as a means of externalizing pre-existent and media-free thoughts, but rather as a medium that completes seemingly pre-existent inner thoughts — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

24 25 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

or intentions, it is not possible to think that there is a non-mediated form of (speech) communication. Even a face-to-face communication is a highly mediated communication form, in which not only the mediality of means by which we communicate, but also the mediality of mentality comes to the foreground. The trace theory of mind is hence considered as a mediation theory of mentality, which is directed against the metaphysics of presence, i.e. the idea of immediate self-presence of cogito. Embodiment in cognitive science

This idea of mediated mentality is similar to that underlying a newer approach in cognitive science after cognitivism and connectionism, which is called “embodied cognition” (Clark, Being there; Sporns), “embodied cognitive science” (Pfeifer and Scheier), “embodied artificial intelligence” (Iida, Pfeifer, Steels, and Kuniyoshi) etc. Cognitivism, which views the mind as a symbolic computation, is based on the assumption that inner symbols’ strings – defined syntactically within the framework of Allen Newell’s and Herbert A. Simon’s “physical symbol systems” – include discrete, independently manipulable elements, which the philosopher and cognitive scientist Andy Clark refers to “symbolic atoms” (Clark, Embodiment). These elements are regarded in accordance with semantic elements, which can be identified in a sequential and logical description of relevant mental states. In connectionism, which extends research on cognition by using artificial neural nets as an operational computational model based on the processing of neural nets in the brain, the idea of symbolic representation has been challenged and instead the idea of distributed representation has been settled. Inner vehicles of semantic contents are viewed not as static inner symbols, but rather as complex structure of inner processes. But both cognitivism and connectionism are based on the assumption that there is a pre-given external world which needs ‘internal’ representation and that a cognitive system relates to the external world by means of internal representation. Here, cognition is concerned with inner models of the external world. Clark refers to these approaches as “isolationism” for it marginalizes the physical body and the environments, concentrating on inner complexity (Clark, Embodiment). Isolationistic cognition separates internal and external processes as well as perception and action from cognition, which may be expressed by the idea of a “sense-think-act cycle” (Clark, Embodiment) or the “the sense-model-plan-act framework” (Brooks). A new research approach in cognitive science, often called embodied cognition, which has been intensively developed since the 1990s, questions both traditional approaches in cognitive science, cognitivism and connectionism. A core idea of this — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Jin Hyun Kim. Toward Embodied Musical Machines — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — *

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 03

— — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — The premotor cortex in monkeys — — — — — — — — — — — —

new paradigm is that cognition is conceived neither as “the recovery of a pregiven outer world” nor as “the projection of a pregiven inner world” (Varela, Thompson, and Rosch 172), but in reliance on “being in a world that is inseparable from our bodies, our language, and our social history – in short, from our embodiment” (Varela, Thompson, and Rosch 149). Thus, the coupling of motor and sensory processes, action and perception as well as inner and outer processes comes into focus. Francisco J. Varela, who understands cognition as embodied action, places an emphasis on cognitive processes which depend on “the kinds of experience that come from having a body with various sensorimotor capabilities” that are embedded in an environmental context, whether biological or cultural (Varela, Thompson, and Rosch 173). In other words – speaking with Clark, cognitive processes are considered as emerging from the “continuous reciprocal causation” between mind/brain, body, and environment (Clark, Being There 163-166). Thus both Varela and Clark direct their research focus toward the coupling of action and perception as well as motor and sensory processes “in which internal and external processes are interwoven” (Sporns 395). Evidence for the coupling of perception and action, sensory and motor processes has been provided by recent neuroscientific findings on a neural “execution/observation matching system” (Rizzolatti and Luppino 889), which is known as a mirror system “which allows the brain to recognize actions as well as execute them” (Arbib et al. 975). The mirror neurons discovered by Rizzolatti et al. in area F5 (see figure 3), the inferior part of the premotor cortex in monkey, are involved in both perceptual and motor behavior. This implies that “cortical premotor areas are endowed with sensory properties” (Gallese, Embodied Simulation 24). Further neurophysiological experiments provided evidence for neural mirrormatching mechanisms in humans similar to mirror neurons (e.g., Cochin et al.; Hari et al.). The so-called neural execution/observation matching system is characterized — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

26 27 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

as a sensorimotor integration process supported by the “parietal-premotor cortical networks” (Gallese, Inner Sense, Embodied Simulation). The posterior parietal areas not only serve to process purely sensory information, but also play an important role in motor control. The premotor and parietal areas are not considered to have separate and independent functions, but neurally integrated ones, which results in there being coupled relations between sensory and motor processes, action control and action representation (Gallese, Embodied Simulation). The neuroscientist Vittorio Gallese suggests interpreting the motor activity of mirror neurons in terms of an “efference copy” of the motor program signal that would act as a kind of “simulator” of the programmed action, which serves to predict its consequences within the framework of a “forward model” (Gallese, Shared Manifold Hypothesis 40). Imitation, mirroring oneself in the behaviors of other persons, is therefore based on simulation mechanisms, through which one may ascribe the observable behaviors of other persons to their mental states and processes (mind-reading) and feel oneself into the objects of imitation (empathy). Accordingly, imitation, the capacity to translate observed objects or actions into mimicked ones, is not seen as a copy of something objectively given, but as an interactive relation to reality. The neural execution/observation matching system is therefore viewed as the basis for founding the nature of cognitive processes on embodied (social) interaction. A definition of embodiment given by the roboticist Kerstin Dautenhahn seems to summarize the aspects of embodiment in the context of cognitive science briefly described above: “Embodiment means the structural and dynamic coupling of an agent with its environment, comprising external dynamics (the physical body embedded in the world) as well as the phenomenological dimension, internal dynamics of experiencing and re-experiencing of self and, via empathy, of others. Both kinds of dynamics are two aspects emerging from the same state of being-in-the-world” (Dautenhahn 135). A recent approach to embodiment in cognitive science that is necessarily closely connected with robotics, in particular, so-called real-world robotics, emphasizes not only the physical body itself, but rather bodily mediated interaction of an agent with its environment. Hence, embodiment that implies situatedness of an agent in the world seems to serve as a condition for human-machine interaction, which is not characterized by linear causality, that is reaction of machines on human inputs, but by circular human-machine interplay related to a coupling of inner and outer processes, perception and action, mind/brain, body and environment, and information and materials. In this way, human-machine interaction that is embodied – in other words: mediated — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Jin Hyun Kim. Toward Embodied Musical Machines — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 04 — — — — — — The vBow — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

by the physical body embedded in the world – serves to dissolve subject/object and mind/body dualisms. Embodied interaction in algorithmic sound generation

As has been shown so far, embodiment does not simply imply material instantiation, but a bodily being embedded in the world by interacting with it. Embodiment therefore cannot be comprehended as the binary relation such as representation, designation, and denotation. Rather, embodiment is related to performance that succeeds in showing itself, in other words, in being present and making a ‘meaning’ emerge as an effect. Returning to the starting point of this article, a disembodied process of sound generation induced by algorithmization may not be simply embodied solely through the integration of the physical body into algorithmic sound generation. What is of importance is rather the way in which the body is related to its (sound) environment. In case the human body is re-instigated, approaches of tangible and social computing may be taken into account for interface design (cf. Dourish), through which the body gets marked so that the medial aspect of the body, which gives rise to a coupling of internal and external processes, action-perception loops, becomes the focus of attention. Tangible musical interfaces, particularly haptic musical interfaces that provide the capability of providing haptic feedback, may be taken as an example. Designing haptic musical interfaces aims to combine haptic feedback supported by the materiality of interfaces with auditory feedback induced by algorithmically generated sound. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — —

28 29 — — — — — — — — » — — — — * 05 — — ‘Excuse Me ?’ — — — — — — — — — — — — — — — — —

Embodied interaction formed by integrating the artificial body into the algorithmic process of sound generation is closely related to the emergence of the alive occurring during the sound generation process. The artificial body, which appears to be a source of generating the sounds, becomes of great importance to embodied interaction in which an observer acts as the participant interacting with an embodied agent. In Sympathetic Sentience (I: 1995-1996; II: 1996-1997) by Simon Penny and Jamieson Schulte, an interactive sound installation, twelve identical electronic units mounted on the ceiling and walls of a darkened room, were used to generate “emergent complexity” of rhythmic and melodic patterns (cf. Penny and Schulte). In Sympathetic Sentience I, each unit was capable of the most extremely simple on/off chirping rhythm. It passed its rhythm to the next via infrared signal, then combined its own rhythm with the data stream it received, and passed the resulting new rhythm along. In this way, the visitor could experience recognizable, but unpredictable, complex patterns of rhythmic sound and interrupt the chain of communication among the units by moving through the space as well as let this chain build-up a new rhythm cycle (cf. Penny). ‘Excuse Me ?’ (2006) by Tom Davis is an emergent sound installation consisting of agents (violin bodies) listening to their environment and recreating their aural stimuli through a matching process of incoming audio to their own internal database of sounds (cf. Tom Davis). Sonic structure evolves both during interaction between agents and (the participant in) their environment and during interaction between agents, which is used to follow the first act of interaction. Two noteworthy musician-robot interactions in the context of live electronic music were performed at SMARt 2006. In par_cho|r : fugue by Christoph Lischka, a ball robot ‘listens to’ and ‘analyzes’ music played on a contrabasson and in this way acquires some kind of ‘hearing’ knowledge. This ball robot moves within a certain defined space and plays a contra part, modulating gestures of contrabasson sounds. This leads to improvisation of two ‘musicians’, which becomes evident in evolving musical structure and in music playing gestures of both as well. This metaphor of the robot as a musician becomes more obvious in Haile (2004-2006) by Gil Weinberg and Scott Driscoll — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Jin Hyun Kim. Toward Embodied Musical Machines — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — —

(Weinberg and Driscoll 2007). The anthropomorphic robot as a percussionist makes a perceptual analysis – detection of note onset, amplitude and pitch – while listening to the plays of a human percussionist and accordingly plays the drum by imitating and transforming playing results of other percussionists. To this end, Haile provides the opportunity of improvising with other human percussionists. Hence algorithms underlying embodied interaction, when applying to musical performance, may be characterized by improvisation, which mostly leads to emergent musical structure evolving during performance even in the tradition of human musicianship. Indeed there have not yet been many approaches to embodied interaction in the context of algorithmic sound generation. However, we may expect intensive discussion in the near future as to how embodiment of an algorithmic process of sound generation may (re-)direct one’s focus toward bodily mediated processing of sound generation and perception and at the same time give rise to an unprecedented esthetic enjoyment. This will be concerned with a re-reading of the underlying ideas of esthetics based on bodily intelligence. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

30 31 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — References Arbib, Michael A. et al. “Synthetic Brain Imaging, Grasping, Mirror Neuron, and Imitation.” Neural Networks 13.8-9 (2000): 975-997.

Gallese, Vittorio. “The ‚Shared Manifold‘ Hypothesis: From Mirror Neurons to Empathy.” Journal of Consciousness Studies. 8.5-7 (2001): 33-50.

Brooks, Rodney. “Intelligence without Reason”. Proceedings of the 1991 International Joint Conference on Artificial Intelligence. 1991. 569-595.

Gallese, Vittorio. “Embodied Simulation: From Neurons to Phenomenal Experience.” Phenomenology and the Cognitive Sciences. 4 (2005): 23-48.

Cochin, S. et al. “Perception of Motion and qEEG Activity in Human Adults.” Electroencephalography and Clinical Neurophysiology. 107 (1998): 287-296.

Hari, R. et al. “Activation of Human Primary Motor Cortex During Action Observation: A Neuromagnetic Study.” Proceedings of the National Academy of Science USA 95. 1998. 15061-15065.

Clark, Andy. Being There. Putting Brain, Body, and World Toghether Again. Cambridge, MA: The MIT Press, 1997 Clark, Andy. “Embodiment and the Philosophy of Mind.” Ed. Anthony O’Hear. Current Issues in Philosophy of Mind (Royal Institute of Philosophy Supplement 43). Cambridge: Cambridge University Press, 1998. 35-52. Dautenhahn, Kerstin. “Embodiment and Interaction in Socially Intelligent Life-Like Agents.” Ed. Chrystopher L. Nehaniv. Computation for Metaphors, Analogy, and Agents. Berlin: Springer, 1999. 102-142. Davies, Char. “Ephémère: Landscape, Earth, Body, and Time in Immersive Virtual Space.” Ed. Roy Ascott. Reframing Consciousness. Exeter: Intellect Books, 1999. 196-201. Davis, Tom. ‚Excuse Me ?‘. 2006. Last access 5. January 2007 . Dodge, Charles and Thomas A. Jerse. Computer Music. Synthesis, Composition, and Performance. 2nd ed. New York: Schirmer, 1997. Dourish, Paul. Where the Action Is. The Foundations of Embodied Interaction. Cambridge, MA: The MIT Press, 2001.

Hayles, N. Katherine. How We Became Posthuman. Virtual Bodies in Cybernetics, Literature, and Informatics. Chicago, IL/London: The University of Chicago Press, 1999. Iida, Fumiya, Rolf Pfeifer, Luc Steels, Yasuo Kuniyoshi. Embodied Artificial Intelligence: International Seminar. Dagstuhl Castle, Germany, July 7-11, 2003, Revised Selected Papers. Berlin: Springer, 2004. Jäger, Ludwig. “Zeichen/Spuren. Skizzen zum Problem der Zeichenmedialität.” Eds. Georg Stanitzek and Wilhelm Voßkamp. Schnittstelle: Medien und kulturelle Kommunikation. Cologne: DuMont, 2001. 17-31. Jäger, Ludwig. “Transkription. Überlegungen zu einem neuen interdisziplinären Forschungsparadigma.” Eds. Walter Bruno Berg, Rolf Kailuweit and Stefan Pfänder. Migrations et transcriptions: Europe et Amérique latine de voies en voix. [in print], Freiburg, 2007. Kim, Jin Hyun and Uwe Seifert. ”Embodiment: The Body in Algorithmic Sound Generation.” Contemporary Music Review. 25.1-2 (2006): 139-149. Linz, Erika. “Die Reflexität der Stimme.” Eds. Cornelia Epping-Jäger and Erika Linz. Medien/Stimmen. Cologne: DuMont, 2003. 50-64. Lischka, Christoph (2007): http://www.zeitmedien.de

Gallese, Vittorio. “The Inner Sense of Action: Agency and Motor Representations.” Journal of Consciousness Studies. 7.10 (2000): 23-40.

Lombard, Matthew and Theresa Ditton. “At the Heart of It All: The Concept of Presence.” Journal of Computer Mediated-Communication. 3. 1997. Last access 12 January 2007 < http://jcmc.indiana.edu/ vol3/issue2/ lombard.html>.

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — »

Jin Hyun Kim. Toward Embodied Musical Machines — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Machover, Tod. Hyperinstruments: a Progress Report 1987-1991. MIT Media Laboratory White Paper, 1992. Mathews, Max V. “The Digital Computer as a Musical Instrument.” Science. 142/3592 (1963): 553-557. Mathews, Max V. The Technology of Computer Music. Cambridge, MA: The MIT Press, 1969. Mathews, Max V.and John R. Pierce. “Der Computer als Musikinstrument.” Ed. Klaus Winkler. Die Physik der Musikinstrumente. Heidelberg: Spektrum Akad., 1992. 170-177. Merleau-Ponty, Maurice. Phenomenology of Perception. 1945. Transl. Collin Smith. New Jersey: The Humanities Press, 1962.

Sporns, Olaf. “Embodied Cognition.” Ed. Michael A. Arbib. Handbook of Brain Theory and Neural Networks. 2nd ed. Cambridge, MA: The MIT Press, 2003. 395-398. Varela, Francisco. J., Evan Thompson and Eleanor Rosch. The Embodied Mind. Cognitive Science and Human Experience. Cambridge, MA: The MIT Press, 1991. Weinberg, Gil and Scott Driscoll (2007): this volume: 66 – 82. Weinberg, Gil and Scott Driscoll. “Toward Robotic Musicianship”. Computer Music Journal. 30.4 (2006): 28-45. Wild, John. “Foreword”. Maurice Merleau-Ponty. The Structure of Behavior. 1942. Transl. Alden L. Fisher. Pittsburgh: Duquesne University Press, 1963. xii-xvii.

Merleau-Ponty, Maurice. The Structure of Behavior. 1942. Transl. Alden L. Fisher. Pittsburgh: Duquesne University Press, 1963. Nichols, Charles. “The vBow: A Virtual Violin Bow Controller for Mapping Gesture to Synthesis with Haptic Feedback.” Organised Sound. 7.2 (2002): 215-220. Penny, Simon. Sympathetic Sentient. 1996. Last access 5 May 2007 < http://adaweb.walkerart.org/context/ events/moma/bbs5/penney.html>. Penny, Simon and Jamieson Schulte. Sympathetic Sentient. Last access 5 May 2007 . Pfeifer, Rolf and Christian Scheier. Understanding intelligence. Cambridge, MA: The MIT Press. 1999. Rizzolatti, Giacomo et al. “Premotor Cortex and the Recognition of Motor Action.” Cognitive Brain Research. 3.2 (1996): 131-141. Rizzolatti, Giacomo and Giuseppe Luppino. “The Cortical Motor System”. Cognition. 31.6 (2001): 889-901. Roads, Curtis et al. The Computer Music Tutorial. 4th ed. Cambridge, MA: The MIT Press, 1996.

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — »

32 33 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — Images * 01 The computer as a musical instrument (following Mathews and Pierce 173) * 02 Seeds, from Ephémère (1998).

Last access 12 February 2007. * 03 The premotor cortex in monkeys. http://www.cognition.ens.fr/~alphapsy/blog/images/ Rizzolatti-monkeyF5s.jpg. Last access 7 January 2007. * 04 The vBow (Nichols 219) * 05 ‘Excuse Me ?’ (Installation in International Conference on Auditory Display (ICAD) 2006). http://www.sarc.qub.ac.uk/~tdavis/Performances.html. Last access 12 February 2007.

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Christoph Lischka >

t

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

SN

o

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

`

a

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Christoph Lischka. — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — —

—

—

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — I

Art and science maintain an (often astounding) syzygy around technology already for millennia – beginning at the latest with the Aristotelian distinction of poiesis/techne and praxis (Aristotle, passim). Both poiesis and techne are concerned with the making of a product; but whereas the focus of the former is a skillful, intuitive creation by an artisan or even artist, the latter exhibits a rather mediated, reflective construction of the desired result. Praxis, in contrast, means the auto-oriented, self-reflective development of an attitude, a kind of “being-in-the-world” which has to be maintained indefinitely. In the subsequent history, there was an ambiguous attitude towards poiesis/techne, particularly in the arts. On the one hand, “materials” were rarely considered as being purely “passive” – being just the object of exclusive control by an “agency”; instead, most practitioners were aware of the complex feedback they will gain from the very stuff they‘re trying to shape, and of the self-will (obstinacy) and resistivity which will eventually superimpose their own activity yielding surprising “results”. On the other hand, there was a strong tendency to conceptualize the complex interaction between material and “making” as a dichotomy of passive data and active control; along the lines of metaphors of (God-like) creation, generation, production, and agency a passive, dead substrate was put into opposition to constructive processes, controlled by an autonomous agent: homo faber (Latour 2002, 344-359). It is well-known that science – especially during modernity – got intimately interwoven with techne, and eventually developed into what is now called techno-science. Techne was molded by scientific concepts, and – vice versa – science embodied — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

36 37 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

techne. The limited space doesn‘t allow for a more detailed discussion; also, this would be rather out of focus in the current context (q.v. Lischka 2005). But – just in a few words: seeded by the ancient metaphysical tradition of the Greeks, contemporary techno-science is deeply anchored in a set of (cartesian) dualisms of, e.g., subject-object, life-dead, mind-matter, control-data, et cetera. Within this modern scaffolding, it exposes itself as a reductionist materialism, which, in its most radical elaborations, implies a deterministic, discrete ontology – unfolding an algorithmic “Theory of Everything”: the universe as a computer (Zuse 1969, Wikipedia: Edward Fredkin, Digital Physics, Lloyd 2006). Poiesis disappears. What remains, is an elaborated “High-Church-Computationalism” (Dennett), based on Newtonian Physics under the control of Universal Turing Machines. II

As, e.g., Popper (Popper 2007) and Wilson (Wilson 2003) have revealed, the background for most of the contemporary research in art and sciences as well is given by the rapid development and overwhelming success of computational technologies, particularly “Virtual Reality”, “Networks”, and “Robotics”. This development in turn resulted in a strong convergence of the respective institutional embodiments of “interdisciplinary” or “transdisciplinary” activities, just as it is abundantly documented by numerous well-known “Art-‘n‘-Technology”-Centers. Although artists successfully adopted the various digital practices, they, even more important, naively adjusted to the underlying conceptual framework also. In fact, the computational paradigm was almost nowhere put into question: techno-science enslaved art, and, as a consequence, the built-in control metaphor of digital media infected artistic thinking and practices. This can be best seen by an analysis of recent discourses on so-called “Generative Art” (Galanter 2003): by letting machines assist the artist during his/her “creation”, the implicit idea of autonomously acting “algorithmic generators” – pro-ducing kind of “result” – is seamlessly and naively incorporated into the underlying esthetic attitude. Poiesis again disappears. There is no “material” anymore. It disintegrates into a network of myriads of digital switches, constrained by complicated algorithmic dependencies. The former obstinacy of the material is – at best – substituted by the break-downs of the hard- and software, reflecting the faulty implementation of the (often anonymous) programmer. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — —

Thus – in conclusion – at the beginning of the 21st century there seems to be a broad consensus among scientists, (media) artists, and engineers about their computational metaphysics; it even seems that people don‘t see an alternative to this ideology anymore – the either approve of it, resign, or even revert to romantic dystopia. III

Most of the advanced technologies of the last century are … “James-Bond-Technologies”: sophisticated electronic and mechanical devices which constrain processes on the micro-scale level. The corresponding scientific disciplines (mathematics, computer science, material sciences, physics, chemistry) are – more or less – well separated, and “boundary experiences” are governed by well-defined transdisciplinary protocols. This situation is changing dramatically. Recent advances in nanotechnology, molecular biology, information technologies, and brain research open the possibility of a new, thoroughly integrated field of nano-scale research where these disciplinary boundaries are melting away, and a fascinating area of unprecedented, but at the same time challenging opportunities emerges (Bainbridge et al. 2006, Roco et al. 2002). One of the most ambitious projects within this new field of convergent technologies is “Synthetic Biology”: the idea of the fabrication of life (ETC group 2007). “Synthesis” here means much more than just “genetic engineering”; synthesis stands for the assembly and organization of basic organic materials in such a way that they exhibit life-like behavior. The ultimate aim is the complete control over the prebiotic processes which eventually will result in a living cellular and/or multicellular organism: controlled bio-materials. “Naturally”, then, life itself is reduced to a programmable “machine”, under the control of … itself? As we have already seen, the technoscientific framework was adopted by the new media arts seamlessly, without any attempt to take a critical stance (q.v. Popper 2007, Wilson 2003 for a comprehensive overview). Most of the new “bio-art” turns out to be a rather superficial examination of the respective sciences, just placing common technologies of genetic engineering in a public esthetic context, forcing a playful, unhindered attitude towards bio-engineering, without questioning the underlying conceptual and technological machinery.1 Moreover, when contemporary bio-artists are arguing for an intimate relation between “life and art” by introducing Art as a Living System (Sommerer et al. 1999), and simultaneously exploiting naively the computational framework, they reveal an extreme technical subjugation of life, a reductionist enframing of biology – wiping away the very distinction between praxis and techne. « — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 1 q.v. BioBricks Foundation,

in particular their iGEM competition.

38 39 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — IV

A techno-science of the nano-world, as described above, inherits from the “classical” framework (at least) two serious problems: (1) it does not account for novelty, and (2) it fails to catch crucial aspects of living organisms. As was discussed by various authors (Fontana et al. 1994, Kauffman 2003 – q.v. Lischka 2005), classical dynamical systems (the underlying formal framework of classical techno-science) suffer from a pre-configured state-space which excludes from the very beginning any change in its structural layout. The dynamics is timereversible, and a trajectory within these spaces is nothing but a multiple-choice procedure between predefined possibilities. Novelty just collapses into an epistemological epiphenomenon. But as novelty plays a central role both in chemistry and biology, in order to put them on a sound theoretical basis, Dynamical Systems Theory has to be extended and changed. Organisms are not machines (Rosen 2005, Cornish-Bowden et al. 2007). In particular, they cannot be described as (universal) Turing Machines. They are even a challenge within the framework of naive set theory (Nomura 2003). Thus, as far as art and life are considered prominent topoi for novelty, we have to admit that contemporary mainstream theory-building and technical attitude miss the whole point. We need an expanded physics, beyond classical reductionist thinking – a physics, which would meet the challenges of biology without reducing life to a mere mechanistic, crystalline equilibrium. We also would need a different interactional pattern describing our embedding into the network of “things” – beyond a mere technical, heteronomic enslaving. V

It is astounding then, how audaciously the technoscientific ideology still dominates most of our current discourse2 – despite the fact, that this profound criticism arose from several sources already during the last century. To name just a few (we completely bypass, for instance, the whole bunch of philosophical arguments stemming from continental philosophical traditions, e.g. Heidegger, Merleau-Ponty, Deleuze): — Actor-Network-Theory (Wikipedia: Actor Network Theory), basically

developed as a conceptual framework for “Science Studies” by Michel Callon and Bruno Latour, advanced (especially through Bruno Latour) to an ambitious, thorough criticism of modernity, putting into question these very « — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 2 As can be experienced quite pertinently

in the context of the so-called „Sokal-Affair“ (Sokal/Bricmont 1999)

— — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — —

dualisms, and arguing for a less static, more dynamic oriented account of “Nature” as a complex agency network. The principal distinctions between culture and nature, or, subject and object (among others), are substituted by a “parliament of things”, where man, hybrids, and natural “entities” are communicating and interacting with equal status and power (Latour 2004). — Theoretical Biology. Besides mainstream biology, several attempts were made

during the last century to unslave theoretical biology from the reductionist con­ straints of the underlying conceptual framework of the physically-oriented lifesciences. As a matter of course, this does not imply any reversion to vitalism or similar modern and pre-modern thinking. Instead, the focus is put on the analysis of distinguishing types of organization, which separate living from non-living systems, aiming at a kind of “logic of life”. Probably well-known are Bertalanffy‘s General Systems Theory (Bertalanffy 1951) and so-called 2nd-order cybernetics, particularly Maturana/Varela and von Foerster (Maturana, Varela 1951; von Foerster 1951). Usually, less well-known is the approach by Robert Rosen (Rosen 1999, 2005). Carrying forward a relational ansatz to biology of his teacher Rashevsky (Rashevsky 1961), Rosen emphasized in his analyses the phenomenon of “causal closure”, which principally separates organisms from machines. An impressive body of research papers still needs a more detailed reconstruction of his basic argument, which challenges the current “central dogma” of computational biology – organism as machine – fundamentally.3 — New Physics. It is almost fun over the years to realize, how physics itself, biol-

ogy, and cognitive sciences etc. are trying to play down the role of quantum physics and related areas – just in order to avoid to accept the strange implications for their mainstream epistemology and ontology. And it is even more fun to see how they desperately try to conceal the very fact, that they still don‘t have any idea of what quantum physics actually is about. There are only a handful of scientists who are at the least discussing the foundational problems (Zeilinger 2003). But, as nano-scale technologies heavily rely upon quantumphysical concepts and formalisms, it should be an urgent need to explore the possibilities of new, advanced explanatory patterns in biology, brain research, and computability – not in order to reinvent the (n+1)th version of an efficient Turing Machine (Deutsch 1998), but rather to elicit uncommon aspects and behaviors which might eventually lead to a new, enhanced physics able to entail biology in a non-reductionist manner.4 « — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 3 There are some recent publications, reconstructing

and continuing Rosen‘s research on (M,R)-Systems; particularly interesting are attempts to relate Rosen‘s, Maturana‘s, and Varela‘s concepts in a unified framework. See, for instance: (Cornish-Bowden et al. 2007)

40 41 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — VI

So – are there any prospects? Indeed, their is a huge network of networks of promising discourses where we just had a glimpse at. Furthermore, their is an urgent need to counterpoint the mainstream approach of the emerging convergent technologies. We can start at different, already quite mature discussions in order to knot and unknot, tie and untie the existing networks, moving the mainstream subversively toward a post-digital, post-technical praxis, which could persist as an appropriate embedding of a trans-human culture. In particular: — Cosmopolitics. In his research, the Belgian chemist Ilya Prigogine developed

an ontology of open systems with serious implications for our self-conception with respect to nature. From his point of view, the statistical character of micro-scale experiences is not reducible to a pure epistemological epipheno­ menon, but indicates a principal “ontological” dimension of nature. This in turn introduces irreversibility, and a fortiori an arrow of time. In contrast to a classical world view, this implies both an open future – liberating systems from pure determinism –, and kind of autonomy for non-human beings also (Prigogine 2003). Following up these theoretical results, the philosopher Isabelle Stengers developed an interesting conceptual framework, where humans and non-humans are mutually engaged in a cosmopolitical network (Stengers 1997, Stengers 2000). Interestingly though, Stenger‘s ideas ressemble to a great extent Bruno Latour‘s concept of a Parliament of Things (Latour 2004). — Mathematics. As was mentioned above (Nomura 2003), contemporary math-

ematics suffers from its strong binding to a set-theoretical foundation – at least from the point of view of theoretical biology. Fortunately, there are several developments within mathematics pointing to alternative settings, as, e.g., Category Theory (Bell) – which was already extensively used by Robert Rosen – , Non-Commutative Geometry (Connes 1994), Synthetic Differential Geometry (Kock 2006), Pointless Topology (Johnstone 1977), and Toposes (Barr et al. 1985). All these approaches are related to Category Theory in one way or another, as this theory turns out to be a general alternative framework to set theory as foundation for mathematics; but what is at least as interesting are new concepts of the continuum which might shed new light on the limitations of a purely discrete ontology. « — — — — — — — — — — — — — — — — — « — — — — — — — — — — — — — — — — 4 Recent examples of how quantum phenomena can

be successfully applied within biology can be found, for instance, with (Brukner et al. 2005, Engel et al. 2007, Summhammer 2006).

These examples, exploiting quantum entanglement, are unconceivable within the traditional, classical framework.

— — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — Physics and Biology. Beyond the developments in physics and biology –

mentioned already in the previous section – there are other interesting topics, especially those in cosmology. The discourses around Quantum Gravity offer a lot of inspiring concepts and prospects; in particular, background-independent theories like Loop Quantum Gravity (Rovelli 2004, Smolin 2002, 2006) undermine our intuitive understanding of space and time profoundly. Artificial Chemistry, an emerging field within Artificial Life Research, also provides new insights through its criticism of standard dynamics as non-constructive (Dittrich et al. 2001). Whether these digitally simulated models will lead to a thorough understanding of the emergence of life, however, remains open. In a similiar vein argues Stuart Kauffman (Kauffman 2003), emphasizing a principal limitation of computational models with respect to novelty. As a consequence, it could be necessary to migrate to soft machines anyway, be it chemotons (Ganti 2003, 2004), synthetic (proto-)cells (PACE), or just hybrids (Bakkum 2004). VII

Within this overwhelmingly complex network of heterogeneous, contrasting, controversial, and conflicting positions, a pure, simple computational ontology seems rather inappropriate – even simple-minded. The era of convergent technologies, especially as life is concerned, needs an alternative to a purely technical, control-oriented stance over “nature”. The arts – as they are traditionally concerned with an obstreperous attitude – might be the right place to articulate an alternate path: overcoming techne again by poiesis, agitating for a mechano-poïia of life … eventually coming down with a paroxysm of the trans-human as the upcoming praxis. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

42 43 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — References Aristotle. Nicomachean Ethics, VI, 4, 1140 a 2ff; 5, 1142 b 3ff; Magna Moralia, I, 34, 1197 a 3ff; II, 12, 1211 b 27ff; Politics, I, 2, 1254 a 6. Bainbridge, William Sims, and Mihail C. Roco. Managing Nano-Bio-Info-Cogno Innovations: Converging Technologies in Society, Springer, 2006. Bakkum, D. J., Shkolnik, A. C., Ben-Ary, G., Gamblen, P., DeMarse, T. B. and Potter, S. M. (2004). Removing some ‚A‘ from AI: Embodied Cultured Networks. Embodied Artificial Intelligence. Iida, F., Pfeifer, R., Steels, L. and Kuniyoshi, Y. New York, Springer. 3139: 130-145. Barr, Michael, and Charles Wells. Toposes, Triples and Theories (Grundlehren Der Mathematischen Wissenschaften), Springer-Verlag, 1985. Bell, John L.. Observations on Category Theory. (http://publish.uwo.ca/~jbell/Observations.pdf) Bertalanffy, Ludwig von. General System Theory: A New Approach to Unity of Science, John Hopkins Press, 1951. BioBricks Foundation. (http://www.biobricks.org/)

Engel, Gregory S., Tessa R. Calhoun, et al. „Evidence for Wavelike Energy Transfer Through Quantum Coherence in Photosynthetic Systems.“ Nature. 446.7137 (2007): 782–86. ETC group, 2007. Extreme Genetic Engineering: An Introduction to Synthetic Biology. (http://www.etcgroup.org/upload/publication/602/01/ synbioreportweb.pdf) Foerster, Heinz von. Cybernetics Circular Causal and Feedback Mechanisms in Biological and Social Systems, Josiah Macy, Jr Foundation, 1951. Fontana, W.; G. Wagner, and L. W. Buss Beyond digital naturalism. Arficial Life, 1/2:211–227, 1994. Galanter, Philip. What is Generative Art? Complexity Theory as a Context for Art Theory (http://www.philipgalanter.com/downloads/ga2003_paper.pdf) Ganti, Tibor. Chemoton Theory: Theory of Living Systems (Mathematical and Computational Chemistry), Kluwer Academic / Plenum Publishers, 2004. Ganti, Tibor, James Griesemer, and Eors Szathmary. The Principles of Life (Oxford Biology), Oxford University Press, USA, 2003.

Brukner, Caslav; Nikola Paunkovic, Terry Rudolph, Vlatko Vedral. Entanglement-assisted Orientation in Space. (arxiv.org/abs/quant-ph/0509123v1), 2005

Johnstone, P.T. Topos Theory (London Mathematical Society Monographs), Academic Press Inc.,U.S, 1977.

Connes, Alain. Noncommutative Geometry, Academic Press, 1994.

Kauffman, Stuart A. Investigations, Oxford University Press Inc, USA, 2003.

Cornish-Bowden, Athel; M. L. Cárdenas, J.-C. Letelier & J. Soto-Andrade (2007). Beyond reductionism: metabolic circularity as a guiding vision for a real biology of systems. Proteomics, 7, 839–845

Kock, Anders. Synthetic Differential Geometry (London Mathematical Society Lecture Note Series), Cambridge University Press, 2006.

Deutsch, David. The Fabric of Reality: The Science of Parallel Universes and Its Implications, Penguin (Non-Classics), 1998. Digital Physics. (http://digitalphysics.org/) Dittrich, Peter; Jens Ziegler, and Wolfgang Banzhaf, 2001. Artificial Chemistries - A Review (http://ls11-www.cs.uni-dortmund.de)

Latour, Bruno. Die Hoffnung der Pandora. Untersuchungen zur Wirklichkeit der Wissenschaft, Suhrkamp, 2002. Latour, Bruno. Politics of Nature: How to Bring the Sciences Into Democracy, Harvard University Press, 2004. Latour, Bruno. Science in Action: How to Follow Scientists and Engineers Through Society, Harvard University Press, 1988.

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — »

— — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — —

Lischka, Christoph. Ζῷον als Υλη: Die Vermachtung des Lebendigen, in: Industrialisierung und Technologi­ sierung von Kunst und Wissenschaft, hrsg. E. Bippus, A. Sick, Bielefeld 2005 Lloyd, Seth. Programming the Universe: A Quantum Computer Scientist Takes on the Cosmos, Jonathan Cape, 2006. Maturana, H.R., and F.J. Varela. Autopoiesis and Cognition: The Realization of the Living (Boston Studies in the Philosophy of Science), Springer, 1991. Nomura, T. 2003. Formal description of autopoiesis for analytic models of life and social systems. In Proceedings of the Eighth international Conference on Artificial Life (December 09 - 13, 2002). R. K. Standish, M. A. Bedau, and H. A. Abbass, Eds. MIT Press, Cambridge, MA, 15-18. PACE - Programmable Artificial Cell Evolution. (http:// bruckner.biomip.rub.de/bmcmyp/Data/PACE/WWW/ PACE/index.html) Popper, Frank. From Technological to Virtual Art (Leonardo Books), The MIT Press, 2007. Prigogine, Ilya. Is Future Given?, World Scientific Pub Co Inc, 2003. Rashevsky, Nicolas. Mathematical Principles in Biology and Their Applications, Charles C Thomas, 1961. Roco, Mihail C., and William Sims Bainbridge. Converging Technologies for Improving Human Performance: Nanotechnology, Biotechnology, Information Technology and Cognitive Science, Springer, 2002.

Rovelli, Carlo. Quantum Gravity (Cambridge Monographs on Mathematical Physics), Cambridge University Press, 2004. Smolin, Lee. Three Roads to Quantum Gravity, Perseus Books Group, 2002. Smolin, Lee. The Trouble With Physics: The Rise of String Theory, the Fall of a Science, and What Comes Next, Houghton Mifflin, 2006. Sokal, Alan, and Jean Bricmont. Intellectual Impostures, Profile Books Ltd, 1999. Sommerer, Christa, and Laurent Mignonneau. Art as a Living System: Interactive Computer Artworks. Leonardo, Vol. 32, No. 3 (1999), pp. 165-173 Stengers, Isabelle. Power and Invention: Situating Science (Theory Out of Bounds Series, Vo 10), University of Minnesota Press, 1997. Stengers, Isabelle. The Invention of Modern Science (Theory Out of Bounds), University of Minnesota Press, 2000. Summhammer, Johann. Quantum Cooperation of Two Insects. (arXiv:quant-ph/0503136v2), 2006 Varela, Francisco J. Principles of Biological Autonomy, Prentice Hall PTR, 1979. Wikipedia: Actor Network Theory. (http://en.wikipedia. org/wiki/Actor_network_theory) Wikipedia: Digital Physics. (http://en.wikipedia.org/ wiki/Digital_physics)

Rosen, Robert. Essays on Life Itself (Complexity in Ecological Systems), Columbia University Press, 1999.

Wikipedia: Generative Art. (http://en.wikipedia.org/ wiki/Generative_art)

Rosen, Robert. Life Itself: A Comprehensive Inquiry Into the Nature, Origin, and Fabrication of Life (Complexity in Ecological Systems), Columbia University Press, 2005.

Wikipedia: Edward Fredkin. (http://en.wikipedia.org/ wiki/Edward_Fredkin) Wilson, Stephen. Information Arts: Intersections of Art, Science, and Technology (Leonardo Books), The MIT Press, 2003.

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — »

44 45 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

Zeilinger, Anton. Einsteins Schleier. Die Neue Welt der Quantenphysik, C.H.Beck, 2003. Zuse, Konrad. Rechnender Raum, Vieweg, 1969.

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Lueder Schmidt :

b

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

p

Embodied Cognitive Science as Paradigm for Music Research

e w

F

§

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Lüder Schmidt. — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — —

Embodied Cognitive —

Science as a —

Paradigm for Music —

Research —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 1. Cognitive Science of Music (CSM)

The scope of cognitive science (CS) is commonly circumscribed by giving examples of research considered relevant or by listing scientific disciplines contributing to the examination of cognitive phenomena. These lists may vary somewhat, but a core membership of psychology, artificial intelligence / computer science, linguistics, neuroscience, and philosophy seems to be generally accepted (e.g. Gardner; Miller; Wilson; Boden; and Strube). A definition, however, is hardly given, and even in The MIT Encyclopedia of the Cognitive Sciences (Wilson) an entry for cognitive science is conspicuously lacking. Two exceptions to this trend should be noticed, however: Barbara von Eckardt reconstructs CS as a coherent scientific discipline, albeit immature. This is achieved by identifying different sets of assumptions and related questions that characterize the domain of CS, the fundamental theoretical approach to the domain (“substantive assumptions”) and resulting methodological commitments. The domain is broadly described as human cognitive capacities; according to the two substantive assumptions, — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

48 49 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

these are to be understood as computational (substantive assumption 1) and representational (substantial assumption 2) devices ([sic], von Eckardt, e.g. 50). In her monumental history of the field Margaret Boden, too, stresses the need to integrate the views from the various “cognitive sciences” (Boden 12). She defines CS as “the study of mind as machine, covering “all aspects of mind and behaviour” and drawing “on many disciplines” (Boden 9). In two ways the latter definition appears to be broader than the former: the “computational device” is explained by von Eckardt with reference to a computer as characterized in the standard literature on computer science (Boden 105), whereas the term “machine” used by Boden seems to apply to a wider range of artifacts. The human cognitive capacities are traditionally taken to refer to mental activities such as thinking, conceiving, and reasoning (e.g. Reber and Reber, entry “cognition”), the scope of “all aspects of mind and behaviour” again including a wider range of phenomena. The talk about embodied cognitive science as indicated below is related to the different approaches apparent in these definitions, claiming to transcend the limits set by (an orthodox interpretation of) the first definition in a substantial way (e.g. Pfeifer and Scheier). A common aspect of the two approaches is the reference to technical artifacts, which on the one hand provides theoretical concepts for investigations, on the other hand creates the need to be explicit in the formulation of theories, which then can be implemented and tested by the design of model systems, e.g. computer simulations. According to our view, the challenge posed by becoming explicit in theorizing about and by modeling all aspects of mind and behavior relevant to “music” should be taken as seriously in cognitive science of music as the possible theoretical stimulations by ideas adopted from “cognitive sciences”. 2. State of CSM

According to the ambitions implied by this view, CSM should come up with (designs for) artifacts that incorporate abilities also exhibited by humans in contexts deemed to be musical. A rough summary of research relevant to CSM – e.g. on music cognition –, however, can only point to modeling attempts that look at restricted areas of musical structure, take as input highly simplified representations of music and produce output that needs to be interpreted in terms of data gained in empirical investigations or in traditional analyses of musical structure. Converging interpretations of simulation output and empirical data are taken as evidence supporting the theoretical assumptions implemented in the model. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Lüder Schmidt. Embodied Cognitive Science — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — —

Typical examples of research in music cognition include investigations of attributes of local musical events – tones – such as pitch and timbre: data from psychological tests, such as (dis-) similarity judgements, are transformed into geometrical constructions, which in turn are interpreted as (models of?) internal representations of the attributes in question (e.g. on pitch (Shepard), on timbre (Grey; Donnadieu). The integration of local events can be construed after similar lines: geometrical configurations which are regarded as visualizing mental representations of mutual relations of musical features are derived from judgements about elements fitting into a context, extensively described for the case of tonal organization by Krumhansl. Other approaches rely on rule systems inspired by linguistic and / or gestalt psychological considerations to derive descriptions of sound scenes in general (Bregman) or more specifically musical structure (most prominently Jackendoff and Lerdahl) which are again interpreted as a listener’s internal representations. Corresponding modeling approaches are occasionally classified according to the well-known (well-founded?) opposition of connectionism and symbolic AI (e.g. Toiviainen). Experimental work based on similarity judgement is more easily associated with neural network models as exemplified by Leman: There, output of a so-called auditory model is used as input to a Kohonen self-organizing map. After a sufficient amount of training, tonal centers could be demonstrated to arise, i.e. areas within the network that responded most strongly to stimuli within a specific musical key. The topology of the tonal centers could be interpreted in terms of relationships familiar from traditional music theory (circle of fifths) and was compatible with the results of Krumhansl. In a more recent example, Krumhansl and Toiviainen used data derived from judgements of key distances to train a Kohonen map. The network was incorporated into a key-finding model operating on a highly reduced musical input (pitch numbers ranging from 1 to 12 and onset / offset times), whose output then was compared with judgements of musically trained listeners. A similarly reduced input (“piano roll representation”) is used by Temperley in the implementation of a system based on preference rules (inspired by Lerdahl and Jackendoff) to automatically provide analyses of musical pieces. Scheirer integrates rules from auditory scene analysis and more general psychoacoustic data to automatically extract musical features from the output of an auditory model, calling his procedure musical scene analysis (Scheirer, chapter 5). In summary, the systems described may be characterized as follows: A rather restricted set of musical features is addressed, mostly conforming to the “narrow” — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

50 51 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

interpretation of cognition cited above and possibly leaving out (more?) important aspects of musical experience. There is a tendency to study and model these features in isolation, aiming at self-contained descriptions or explanations. This form of particularization, however, may entail neglecting the issues of coherence and closure of the musical domain. Despite the argument of converging evidence, system performance remains difficult to evaluate as a model of human cognitive processes (see Wang and Brown for a detailed discussion), because input as well as output is quite remote from realistic situations including musical stimulation and human music-related behavior. For these reasons it appears desirable to integrate modeling attempts into systems, in the following referred to as agents that can exhibit more or less appropriate behavior within musical contexts. 3 Role of the Body – Embodiment

These last remarks are complemented by a growing amount of evidence that the investigation of cognitive phenomena can benefit from taking into account conditions and processes previously not regarded as pertinent to cognition proper: Since the 1980s, the role of corporeal interaction and embodiment of an agent (human, animal, or artifact) has increasingly come into focus from a wide range of perspectives. With respect to spatial hearing, early observations were made by Wallach who pointed out the role of active head movements of a listener to disambiguate otherwise ambiguous directional cues; recently, possibly underlying neural mechanisms in the auditory system of the cat have been described (Young and Davis). Within cognitive linguistics, the importance of interacting in a physical environment with a body possessing specific properties for the formation of concepts underlying abstract reasoning (Lakoff and Johnson) and mathematics (Lakoff and Núñez) has been described. The human body as a medium of communication and interaction in musical contexts has gained ever-increasing interest: Body movements are investigated as a means to convey expressive or emotional aspects. More fundamentally, within the Musical Gestures project music-related bodily movements are considered to be intimately connected to the formation of musical concepts and the organization of musical behavior. A related but quite different role is assigned to bodily movements in the context of new ways of sound production: New interfaces and instruments are developed that take different kinds of body related signals as variables in the generation process providing — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Lüder Schmidt. Embodied Cognitive Science — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — *

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 01

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — Notions of embodiment, redrawn after Ziemke [41]. — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

new and enhanced possibilities for interaction between humans and technical artifacts. The broad range of possible devices is illustrated e.g. by the series of conferences on New Interfaces for Musical Expression (NIME, www.nime.org) – understanding musical behavior, however, doesn’t appear to be a primary goal of this work. A cognitive science approach to music that aspires to integrate these aspects, which may be called embodied cognitive science of music (Clark, Embodied Cognitive Science; Pfeifer and Scheier), should adopt the use of complete autonomous agents as a basis for modeling. 4 Notions of Embodiment

The term embodiment is intuitively appealing, but difficult to capture precisely. Ziemke presents a summary of previous attempts of definition proposed with regard to embodied cognition and autonomous agent research and offers a quasi-hierarchical set of notions of embodiment (* 01) to capture the discussion so far.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

52 53 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

The notions are, in the order of increasing restrictiveness: — structural coupling — historical embodiment — physical embodiment — organismoid embodiment — organismic embodiment.

His presentation suffers slightly from presupposing a view that restricts cognition to living organisms, which leads to blurring the concepts cognition, life, and embodiment in the explanations given. The most permissive notion, structural coupling, is adopted from Maturana and Varela. It is provided to allow for agents without a body in any obvious sense (software agents, alife creatures) to be included in the discussion. Taking this notion to be more inclusive than physical embodiment, however, seems to be misleading: Agents as programs, i.e. as structures in a purely logical sense, do not interact with their environment and therefore do not exhibit any form of coupling. For structural coupling to arise the agent program must be implemented and running, i.e. it must be realized by some physical processes. These processes, however, need not be tied to any specific and readily identifiable material substrate. This seems to me to be the main aspect of the characterization that there is no body “in the usual sense”. The distinction drawn here between logical structure and realizing physical processes may reflect Maturana and Varela’s differentiation of organization and structure: “By organization we mean those relations that must be given among the constituent parts of something so that it will be recognized as a member of a certain class. By the structure of something we mean the constituent parts and the relations that in a concrete manner constitute a certain unit and realize its organization.” (Maturana and Varela 54, translated from the German edition).

By the notion of historical embodiment the observation is captured that not only the present structure is of importance for the way agent and environment interact, but that also the course of previous mutual influences may have contributed to the present form of coupling. This means that among others phenomena of learning, development, adaptation (on the side of the agent) as well as structuring of the environment can be taken into account. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Lüder Schmidt. Embodied Cognitive Science — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 02 — — — Coupling of agent and environment, — — — redrawn after Beer [2], — — — — see also Arbib [1], 58 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

As indicated above, the differentiation of physical embodiment from the notion of structural coupling seems unclear. As a possible way for a system to qualify as physically embodied, Ziemke quotes Brooks’ requirement of the presence of sensors and actuators integrated into a material structure; a first attempt to delimit the body of a living organism might be to take the “biological skin bag” (Clark, Natural Born Cyborgs 5) as a boundary. The remaining two notions, organismoid and organismic embodiment, again direct attention to the specific ways agent-environment coupling is implemented, i.e. to the presence and location / morphology of sensors and actuators as well as the integration of their respective activity into coherent and well adapted behaviors. Whereas organismoid (organism-like) embodiment is intended to be applied to both artificial and living systems, organismic embodiment is reserved for the latter – which leaves the problem of distinguishing the living from the non-living / artificial. The discussion presented by Ziemke (section 4.5) leaves room for further debate. In this discussion, there is a strong focus on the individual agent and its coupling to a more or less unspecified environment; the role of the environment is discussed with regard to shaping the specific coupling. However, some more consideration might be given to the role of the environment as part of processes mediating agent-environment interaction, as well as to the presence of other (similar) agents within the environment. According to the interpretation offered here, embodiment in any of the senses discussed above is tightly connected to some sort of physical processes going on, problems to be solved including: — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

54 55 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

What kinds of processes are involved? What are the ways of (re-)configuring these processes? What are the mutual interactions between processes? How can the coherence of processes be defined in a way that reflects the identity of an agent? 5 Dynamical Systems Theory as a Framework

The theory of dynamical systems has repeatedly been proposed as a theoretical framework to describe the interactions between an agent and its environment (e.g. Arbib; Beer; Steels; van Gelder; Pfeifer and Scheier; Pfeifer and Bongard); related ideas can be traced back at least to the work of Ashby in the 1950s (van Gelder section 6.11; Boden chapter 4.viii.c; Beer 181). We will take up and slightly extend the semi-formal formulation of Beer, trying to relate the notions of embodiment discussed above to this framework. Following Ashby, Beer distinguishes between agent and environment as two mutually coupled dynamical systems (* 02). The agent is referred to with the symbol A, the environment with E. Agent and environment are essentially treated symmetrically, the main focus, however, rests in accordance with the above discussion on the agent. The present state of the agent is assumed to be described by a set of time dependent variables, in vectorial notation given as ➞xA; in the same manner, the environment is described by a set of variables ➞ xE. In addition, Beer introduces a second set of more stable quantities to characterize both agent and environment, respectively, called parameters and denoted by u➞A for the agent, u➞E for the environment. These may refer to material properties of (parts of) an agent such as density, elasticity, number of neurons / synapses, and are intended to describe persistent structural properties of agent and environment. As argued below, however, a clear-cut distinction between state variables and parameters will be difficult to draw. The coupling of agent and environment is introduced into Beer’s scheme by two functions describing the mutual influences: the “motor” function M( x➞A ) takes as arguments the agent’s state variables to specify the influence exerted on the environment by the agent, the “sensory” function S( x➞E ) captures the influences on the agent produced by environmental conditions.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Lüder Schmidt. Embodied Cognitive Science — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — —

The time course of agent and environment states (i.e. agent and environment behavior) is formally described by two coupled sets of coupled differential equations, the “equations of motion”: x➞A = A ( x➞A ; S( x➞E ); u➞A ) x➞E = A ( x➞E; S( x➞A ); u➞E ) As presented, the scheme is well adapted to the example application of developing an insect-like walking agent: the parameters are partly fixed by the design of the material body, partly determined as free parameters in an artificial neural network used to control leg movements. Adjustment of the network parameters is achieved by genetic algorithm search in a process external to the dynamics of agent-environment interaction as described by the equations of motion. Thus, the agent’s description doesn’t seem to allow for any persistent changes due to interaction or for a history in terms of the notions of embodiment. To incorporate an agent’s history, i.e. to open up the scheme for the description of adaptive properties, at least some of the parameters need to be subject to temporal change instead of being considered as constants. This change is mediated by the state variables x➞A and partly by the values of the other parameters, i.e. u➞A= u➞A ( x➞A ,u➞A ). The time change of the parameters should be captured by the dynamical description of the agent, too, so that the agent’s equation of motion is modified to ( x➞A ,u➞A ) = A ( x➞A ; S ( x➞E ) ; u➞A ) Thus, from a formal point of view, there is no distinction between state variables and parameters, and it turns out to be a matter of perspective and convenience which characterizing quantities are regarded as state variables or parameters. One criterion at hand may be the time scale of change. In the context of neuroscience, Arbib (66) proposes a separation according to the time scales of short term vs. long term memory, taking instantaneous nervous activity as representing state, more slowly changing synaptic strengths as an example of system parameters. A quite different range of time scales is discussed by Pfeifer and Bongard (chapter 3.5): “hereand-now” refers to an agent’s short-term behavioral mechanisms, the ontogenetic time scale to development and learning within an individual agent’s lifetime, and even the evolutionary (phylogenetic) perspective is taken into consideration. These remarks were intended to illustrate that the notions of embodiment found in the discussion of embodied cognition / embodied cognitive science can be accommodated within the framework of dynamical systems theory or the discussions around it. This can be taken as an indication that the approach of embodied cognitive science — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

56 57 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

does not raise any new foundational issues for cognitive science beyond those hiding behind the contrasting definitions cited in section 1, which concern among others the interpretation of the term computation (see e.g. Goldin and Wegner) and the applicability of the theory of dynamical systems wearing a deterministic hat (Prigogine). Important contributions, however, are made in the following ways: — Attention is drawn to aspects of cognitive processes that were previously ne-

glected. This leads to new ways of understanding observed phenomena within the “old” framework, i.e. the theoretical foundation is explored more thoroughly and more fully. — This is supported and facilitated by new theoretical and technological means,

in turn giving rise to new developments. — Of special interest to the realm of music (and other fields which are similarly

considered to be cultural phenomena), interaction is taken up not just as an interesting artistic feat, but as a fundamental condition. — In the design of interactive artifacts, theoretical work can be integrated and put

to a test under increasingly realistic conditions. New problems, however, arise in evaluating the performance of these artifacts. 6 First Steps

Research along these lines requires a high degree of technical and technological competence, which is not abundant within a rather small musicological institution. Nevertheless, some steps in the desired direction might be achieved, partly due to the cooperation with the DFG research project C10: Artistic Interactivity in Hybrid Networks of the SFB 427: Media and Cultural Communication. As described elsewhere (Schmidt; Schmidt and Seifert), it is intended to integrate two small mobile autonomous robots (Khepera III) into musical interaction. For this purpose, the robots are equipped with onboard cameras and sound processing modules as well as WLAN connections; an interface that allows accessing the robot control via the open sound control protocol is currently in preparation. The expressive capabilities of the robots are extremely restricted – part of the project is to investigate what forms of interaction between robots and humans and among the robots will be possible. This work has been put off, however, by delays in the technical development of the robotic platform, especially the camera module, and by administrative / organizational problems. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Lüder Schmidt. Embodied Cognitive — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 03 — LEGO robot used and modified in class. — — — — — — — — — — — — — — — — — —

Science — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

To integrate these ideas into the curriculum, a seminar on musical robotics was conducted in the summer term 2006. Topics included the presentation and discussion of artistic, sound/music related uses of robots, aspects of developmental robotics and some of the ideas indicated above. In a practical part, students were given the opportunity to experiment with a Lego Mindstorms RCX robot (* 03) and to explore in small-scale problems the discrepancies between neat behavioral descriptions in a graphical programming language and the observed behavior of the robot in a hard-tocontrol physical environment. Inspired by this seminar, a master’s thesis project was conducted – mainly at the Department of Speech, Language, and Hearing of the Royal Institute of Technology, Stockholm – that attempted to investigate the connection of robotic gestures and musical expression (Burger). More specifically, parameters derived to describe the movements of instrumentalists that were asked to express certain basic emotions (Dahl and Friberg) were incorporated into the control of a Lego Mindstorms NXT robot. The robot was programmed to exhibit the “emotions” happy, angry, and sad. The means of expression were the overall motion pattern of the robot and up- or downmovements of a pair of robot arms. In addition, a melody was prepared to express the — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

58 59 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

same emotions according to an established rule system. In two test sessions observers were asked to rate the motion patterns of the robot according to the intended emotions in two test conditions: movement alone or accompanied by the melody prepared to express the same emotion. In this kind of set-up, evidence from different fields of research is combined in the design of an agent exhibiting observable behavior. Still lacking, however, is the ability of the agent to react to environmental conditions and to enter into interaction. An early idea to integrate a module to analyze musical acoustical input, e.g., had to be abandoned because of technical limitations (and time available). Conclusion

As will be evident from this presentation, presently there is no coherent framework of an embodied cognitive science of music. A fair amount of work, both theoretical and practical, will be required to put together these ideas loosely drawn from music research and cognitive science in general. Following these lines, we believe, will be highly stimulating and beneficial for the understanding of man as a musical machine. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Lüder Schmidt. Embodied Cognitive Science — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — References Arbib, Michael. The Metaphorical Brain. An Introduction to Cybernetics as Artificial Intelligence and Brain Theory. New York: Wiley, 1972.

Grey, J. M. “Multidimensional scaling of musical timbres.” Journal of the Acoustical Society of America. 61 (1977): 1270 – 1277.

Beer, Randall D. “A dynamical systems perspective on agent-environment interaction.” Eds. Philip E. Agre and Stanley J. Rosenschein. Computational Theories of Interaction and Agency. Cambridge, MA: MIT Press, 1996. 173 – 215.

Krumhansl, Carol L. Cognitive Foundations of Musical Pitch. New York: Oxford University Press, 1990.

Boden, Margaret A. Mind as Machine. A History of Cognitive Science. Oxford University Press, 2006. Bregman, Albert S. Auditory Scene Analysis. The Perceptual Organization of Sound. Cambridge, MA: MIT Press, 1990. Burger, Birgitta. Communication of Musical Expression from Mobile Robots to Humans. Recognition of Music Emotions by Means of Robot Gestures. Master’s thesis (forthcoming), KTH Stockholm. Clark, Andy. “An Embodied Cognitive Science.” Trends in Cognitive Science. 3 (1999): 345 – 351. Clark, Andy. Natural-Born Cyborgs. Minds, Technologies, and the Future of Human Intelligence. Oxford University Press, 2003. Dahl, Sofia and Anders Friberg “Visual perception of expressiveness in musicians’ body movements”. Computer Music Journal (submitted). Donnadieu, Sophie. “Mental Representations of the Timbre of Complex Sounds.” Ed. James W. Beauchamps. Analysis, Synthesis, and Perception of Musical Sounds. The Sound of Music. New York: Springer, 2007. 272 – 319. Gardner, Howard. The Mind’s New Science. A History of the Cognitive Revolution. New York: Basic Books. 1985 / 1987. Goldin, Dina and Peter Wegner. Refuting the Strong Church-Turing Thesis: The Interactive Nature of Computing. Preprint, September 2005.

Krumhansl, Carol L. and Petri Toiviainen. “Tonal Cognition”. Eds. Peretz Isabelle and Robert J. Zatorre. The Cognitive Neuroscience of Music. New York: Oxford University Press, 2003. 95 –108. Lakoff, George and Mark Johnson. Philosophy in the Flesh. The Embodied Mind and Its Challenge to Western Philosophy. New York: Basic Books, 1999. Lakoff, George and Rafael E Núñez. Where Mathematics Comes From. How the Embodied Mind Brings Mathematics into Being. New York: Basic Books, 2000. Leman, Marc. Music and Schema Theory. Cognitive Foundations of Systematic Musicology. Berlin: Springer, 1995. Lerdahl, Fred and Ray Jackendoff. A Generative Theory of Tonal Music. Cambridge, MA: MIT Press, 1983. Maturana, Humberto and Francisco Varela. Der Baum der Erkenntnis. Die biologischen Wurzeln des menschlichen Erkennens. Scherz Verlag, 1984/1987. Miller, George A. “The cognitive revolution: a historical perspective.” Trends in Cognitive Science, 7.3 (2003): 141 – 144. Miranda, Eduardo Reck and Etienne Drouet. “Evolution of musical lexicons by singing robots.” Proceedings of TAROS 2006 Conference - Towards Autonomous Robotic Systems. Gilford (UK), Surrey University, 2006. Musical Gestures Project. Last access May 2006 . Pfeifer, Rolf and Josh Bongard. How the Body Shapes the Way We Think. A New View of Intelligence. Cambridge, MA: MIT Press, 2006. Pfeifer, Rolf and Christian Scheier. Understanding Intelligence. Cambridge, MA: MIT Press, 1999.

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — »

60 61 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

Prigogine, Ilya. The End of Certainty. Time, Chaos, and the New Laws of Nature. New York: Free Press, 1997. Reber, Arthur S. and Emily S. Reber. The Penguin Dictionary of Psychology. London: Penguin Books, 2001.

Wang, DeLiang and Guy J. Brown. “Fundamentals of Computational Auditory Scene Analysis.” Eds. DeLiang Wang and Guy J. Brown. Computational Auditory Scene Analysis. Principles, Algorithms, and Applications, Hoboken, NJ: Wiley, 2006. 1 – 44.

Scheirer, Eric D. Music Listening Systems. PhD thesis, Massachusetts Institute of Technology, Program in Media Arts and Sciences, Cambridge, MA, 2000.

Wilson, Robert A. and Frank C. Keil, Eds. The MIT Encyclopedia of the Cognitive Sciences. Cambridge, MA: MIT Press, 1999.

Schmidt, Lüder. “Towards an ‘embodied cognitive science of music’: Incorporating mobile autonomous robots into musical interaction.” APSCOM05. Seoul, 2005.

Young, Eric D. and Kevin A. Davis. “Circuitry and Function of the Dorsal Cochlear Nucleus.” Eds. Donata Oertel, Arthur N. Popper and Richard R. Fay. Integrative Functions in the Mammalian Auditory Pathway New York: Springer, 2002. 160 – 206.

Schmidt, Lüder and Uwe Seifert. “Körperlichkeit und Musik.” Neue Zeitschrift für Musik. 167.4 (2006), 44 – 45. Shepard, Roger N. “Structural Representations of Musical Pitch.” Ed. Diana Deutsch. The Psychology of Music. New York: Academic Press, 1982. 343 – 390.

Ziemke, Tom. “Are robots embodied?” Eds. Christian Balkenius, Jordan Zlatev, Hideki Kozima, Kerstin Dautenhahn and Cynthia L. Breazeal. Proceedings of the First International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems. Lund: Lund University Cognitive Studies, 85, 2001.

Steels, Luc. “Homo cybersapiens oder Robo hominidus intelligens: Maschinen erwachen zu künstlichem Leben.” Eds. Christa Maar, Ernst Pöppel and Thomas Christaller. Die Technik auf dem Weg zur Seele. Forschungen an der Schnittstelle Gehirn / Computer. Reinbek: Rowohlt, 1996. 327 – 344 Strube, Gert Ed. Wörterbuch der Kognitionswissenschaft. Stuttgart: Klett-Cotta, 2001. Temperley, David. The Cognition of Basic Musical Structures. Cambridge, MA: MIT Press, 2001. Toiviainen, Petri. “Symbolic AI versus Connectionism.” Ed. Eduardo Reck Miranda. Readings in Music and Artificial Intelligence. Harwood Academic Publishers, 2000. 47 – 67. van Gelder, Timothy. The dynamical hypothesis in cognitive science. Behavioral and Brain Sciences (draft version September 1997). von Eckardt, Barbara. What Is Cognitive Science? Cambridge, MA: MIT Press, 1993. Wallach, Hans. “On sound localization”. The Journal of the Acoustical Society of America 10 (1939): 270 – 274.

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — »

Lüder Schmidt. Embodied Cognitive Science — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — Images * 01 Notions of embodiment, redrawn after Ziemke [41]. * 02 Coupling of agent and environment, redrawn after Beer [2], see also Arbib [1], 58 * 03 LEGO robot used and modified in class.

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — »

62 63 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

WGil p Weinberg, VScott Driscoll

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

> — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Robotic ? ? The Percussionist – WBringing Interactive Computer Music the k oginto Physical World

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

V

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Gil Weinberg, Scott Driscoll. — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — —

The Robotic —

Percussionist – —

Bringing Interactive —

Computer Music into —

the Physical World —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

The paper presents a perceptual and improvisational robotic percussionist that listens to live players, analyzes perceptual musical aspects in real-time, and utilizes the product of the analysis to play along in a collaborative manner. The robot, named Haile, is designed to combine the benefits of computational power, perceptual modeling, and algorithmic music with the richness, visual interactivity, and expression of acoustic playing. We believe that when interacting with live players Haile can facilitate a musical experience that is not possible by any other means, inspiring humans to collaborate with it in novel expressive manners, leading to novel musical results. Haile can, therefore, serve as a test-bed for novel forms of musical human-machine interaction, bringing aspects of computer music into the physical world both visually and acoustically. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

66 67 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — Background and motivation

Computers have been programmed to compose music for more than half a century, although the expressive and creative content of algorithmic music has always been questioned (Papadopoulos and Wiggins). Music generated by people, on the other hand, while benefiting from human expression, emotion, and creativity, is limited by the computational capability and dexterity of the human mind and body. Researchers in the field of interactive music systems have been attempting to facilitate collaborations between humans and computers in an effort to inject algorithmic composition with human expression and to infuse novel computational ideas into the human musical experience. Such interactive music systems use computational models to analyze human input as well as perform and compose music based on theoretical foundations in fields such as music theory, signal processing, cognition, artificial intelligence and human-computer-interaction (Rowe). Computer speaker-based interactive music systems, however, are hampered by their inanimate nature, which does not provide players and audiences with physical and visual cues that are essential for creating expressive musical interactions. For example, motion size often corresponds to loudness and gesture location often relates to pitch. These cues provide visual feedback, help players anticipate and coordinate their playing, and create an engaging experience for the audience by providing a visual connection to the sound. Computer speakerbased interactive music systems are also limited by the electronic reproduction and amplification of sound through speakers, which cannot fully capture the richness of physically generated acoustic sound. The main motivation behind this work, therefore, is to utilize robotics to create an interactive musical system that provides the visual cues and acoustic richness that are required for expressive and creative interaction with humans. Current research directions in musical robotics focus on sound production and rarely address interactive and perceptual aspects of musicianship such as listening, analysis and improvisation. Such automated devices include both Robotic Musical Instruments – mechanical constructions that can be played by live musicians or triggered by pre-recorded sequences (Jordà; Singer, Larke et al.; Dannenberg, Brown et al.), and Anthropomorphic Musical Robots – hominoid robots that attempt to imitate the action of human musicians (Takanishi and Maeda; Sony; Toyota). Haile was designed to address these limitations based on the concept of “Robotic Musicianship” (Weinberg and Driscoll). Through perceptual and improvisatory modeling Haile is designed to bring the computer into the physical world both acoustically and visually. Unlike computer speaker-based interactive music systems, a physical anthropomorphic robot can cre— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Gil Weinberg, Scott Driscoll. The Robotic Percussionist — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — —

ate familiar, acoustically rich, and visual interactions with humans. In order to create such intuitive and inspiring social collaboration with humans, Haile is designed to analyze music based on computational models of human perception and to generate algorithmic responses that are humanly impossible (“listen like a human, improvise like a robot”). The generated sound is acoustically rich due to the complexities of real life systems whereas in computer-generated audio, acoustic nuances require intricate design and are ultimately limited by the fidelity and orientation of speakers. Moreover, unlike speaker-based systems, the visual connection between sound and motion allows humans to anticipate and synchronize their gestures with the robot, leading to more familiar, intuitive, and expressive musical experiences. Related work

A number of research fields relate to our goal of achieving “Robotic Musicianship”, including Musical Robotics, Human-Robot Interaction, Machine Musicianship, and Music Perception Modeling. In recent years, the field of musical robotics has received commercial, artistic, and academic interest, addressing a variety of musical instruments, including chordophones, aerophones, membranophones and idiophones. Several approaches have been explored for robotic stringed instruments utilizing solenoids, servomotors, and electro valves to slide over and pick guitar strings in a variety of manners (Jordà; Singer, Larke et al.). More sophisticated control has been developed for the anthropomorphic robot flutist (Takanishi and Maeda), which uses a complex mechanical imitation of human organs (including mechanical lungs, lips, fingers, and tongue) in an effort to accurately reproduce human subtleties. Other examples for aerophone robotic instruments are Toyota’s Robotic Trumpeter (Toyota) and the Rae’s Autosax (Rae). Efforts have also been made to develop robotic percussionists for idiophone and membranophone instruments. The ModBots, for example, are miniature modular instruments that utilize a rotary motor or a linear solenoid and can be affixed to virtually any structure (Singer, Feddersen et al.). A more elaborated mechanism is used in the Thelxiepeia project where an effort has been made to capture a wider timbral variety by using several actuators to produce a richer set of sounds (Baginsky). It is important to note that current research directions in musical robotics focus on sound production and rarely address interactive and perceptual aspects of musicianship. Current work on musical robots is also rarely informed by studies in human-computer interaction, which attempt to define a framework and taxonomy for human-robot interaction (Yanco and Drury; Thrun), and in particular issues such as gesture coordina— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

68 69 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

tion (Waldherr, Romero et al., Prassler, Bank et al.) and collaboration (Fong, Thorpe et al.; Severinson-Eklundh, Green et al.). A related area of work in this regard is the field of Machine Musicianship, which addresses computational systems that analyze, perform, and compose music with computers based on theoretical foundations in fields such as human-computer interaction, music theory, computer music, cognition, and artificial intelligence (Rowe). Scholars from a variety of fields have explored different approaches for the design of such machine musicianship systems. One of the earliest research directions in this area was the Score Follower, in which the computer tracks a live soloist and synchronizes MIDI (Dannenberg, Vercoe), and recently audio (Orio, Lemouton et al.) accompaniment to musical input. Related efforts have been made to design computerized beat tracking applications to analyze and match tempi and meter in real-time (Scheirer; Davies and Plumbley). A more improvisatory approach is taken by systems such as “Cypher” (Rowe), “Voyager” (Lewis) and the “Continuator” (Pachet). Here, the software analyzes musical input and responds by controlling and manipulating a variety of parameters such as melody, harmony, rhythm, timbre, and orchestration. In the field of music perception modeling, which can be seen as a subset of Machine Musicianship, researchers explore how humans perceive music and attempt to define mathematical approaches that model musical percepts. Research in the area addresses computational approaches for modeling low- and high-level rhythmic, melodic, and harmonic percepts. Lower level cognitive rhythmic modeling addresses percepts such as note onset, amplitude and pitch detection using audio sources (Puckette) as well as MIDI (Winkler). Higher-level rhythmic percepts include concepts such as stability (Dibben; Desain and Honing), tension (Lerdahl and Jackendoff; Narmour), and similarity (Arom; Tanguiane; Schmuckler). These perceptual models, however, have not been implemented in live interactive systems in an effort to enrich humanmachine collaboration. Goals and challenges

As part of our effort to construct a robotic percussionist that can demonstrate musicianship we identified a number of challenges in design, mechanics, perception, and interaction. Our main challenge in designing Haile’s physical body was to create a mechanical device that encourages humans to collaborate with a machine in an expressive and intuitive manner. The robot’s shape, construction materials, and the manner in which technology is embedded in the physical form had to support intuitive and engaging musical interaction. In the area of mechanics, our main challenge was — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Gil Weinberg, Scott Driscoll. — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 01 — Haile’s anthropomorphic design. — — — — — — — — — — — — — — — —

The Robotic Percussionist — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

to develop a dexterous robotic apparatus that would be able to accurately translate perceptually based performance algorithms into a rich acoustic and visual performance. In perception, our challenge was to implement models for low-level percepts such as note onset and pitch as well as high-level musical percepts such as rhythmic stability and similarity, allowing the robot to obtain a meaningful representation of the music it listens to. Based on this perceptual analysis, an additional challenge was to develop responsive improvisation algorithms, generating musical responses that intuitively relate to the humans’ input. In interaction design our goal was to develop performance algorithms that would enable the robot to collaborate with human players in a meaningful and inspiring manner, using transformative and generative methods both sequentially and synchronously. Physical Design

In order to encourage familiar and expressive interactions with human players, Haile’s design is anthropomorphic, utilizing two percussive arms that can move to different — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

70 71 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

locations along the drum’s radius and strike with varying velocities. The robot was designed to play a Native American Pow Wow drum – a unique multi player instrument that supports the collaborative nature of the project. In order to match the natural aesthetics of the Native American Pow Wow ritual, we chose to construct the robot from wood. A design made by Clint Cope was used as a basis for Haile’s appearance. The wooden parts were made on a CnC wood cutting machine and constructed from several layers of plywood that were glued together. Metal joints were designed to allow shoulder and elbow movement as well as leg adjustability for different drum heights. While attempting to create a warm and organic look for the robot, it was also important that the technology would not be completely hidden, so that players and learners would be able to see and understand the robot’s operation. We therefore left the mechanical apparatuses uncovered and embedded a number of LEDs on Haile’s body, which provide an additional representation of the mechanical actions (* 01). Mechanics

Haile controls two robotic arms; the right arm is designed to play fast notes, while the left arm is designed to produce larger and more visible motions that produce louder sounds in comparison to the right arm. Both arms can adjust the sound of strikes in two manners: different pitches are achieved by striking the drumhead in different locations along its radius, and volume is adjusted by hitting with varying velocities. To move to different positions over the drumhead, each arm employs a linear slide, a belt, a pulley system, and a potentiometer that provides feedback (* 02) . Unlike robotic drumming systems that allow hits at only a few discrete locations (Jordà; Raes), Haile’s arms can strike anywhere on a line between the center and the rim of the drum, moving between the two ends in about 250 ms. The right arm’s striking mechanism is loosely based on a piano hammer action and consists of a solenoid driven device and a return spring (* 03) . The arm strikes at a maximum speed of 15 Hz (faster than the left arm’s maximum speed of 11 Hz), but it cannot generate a wide dynamic range or provide easily noticeable visual cues. This limits Haile’s expression and interaction potential. The left arm was designed to address these shortcomings, using larger visual movements, and a more powerful and sophisticated hitting mechanism. Whereas the striking component of the right arm is about the size of a finger and can only move 2.5 inches vertically, the entire left forearm takes part in the striking motion and can move up and down eight inches. A linear motor and an encoder located at the elbow are used to provide sufficient force and control for the larger mass and motions (* 05). — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Gil Weinberg, Scott Driscoll. — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 02 — The right arm slider mechanisms — — — — — — — — — — — — — — — —

The Robotic Percussionist — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — * 03 — — The right arm striking mechanism — — — — — — — — — — — — — — — — — — — — —

In an effort to provide an easy-to-program environment for Haile, we decided to use Max/MSP (Cycling74 2005), an intuitive graphical programming environment that can make the project accessible to composers, performers, and students. Our first one-armed prototype incorporated the USB based Teleo System (MakingThings 2005) as the main interface between Max/MSP and Haile’s sensors and motors. Low-level control of the solenoid-based right arm’s position was computed within Max/MSP, which required a continuous feed of position updates to the computer. This consumed much of the communication bandwidth as well as processor time on the main computer. The current two-arm mechanism utilizes multiple onboard microprocessors for local low-level control as well as Ethernet communication with the main computer. The new system, therefore, facilitates much faster and more sophisticated control (2ms control loop) and requires only low bandwidth communications with the operating computer. Perception

As a test bed for musical human-robot interaction, we developed a number of in­ dependent perceptual modules for Haile, which can be embedded in a variety of combinations in compositions and other interaction schemes. Each module addresses one perceptual aspect, from hit onset, amplitude, and pitch detection, through beat and density analysis, to rhythmic stability and similarity perception. We base our low-level modules for hit onset and amplitude detection on the Max/MSP bonk~ ob— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — —

72 73 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 04 * 05 — — Haile’s right arm design The linear motor and encoder provide closed-loop — control over the position and velocity of the left — arm while the gear motor and potentiometer control — — distance from the center of the drum. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

ject (Puckette), and adjust its output to the unique character of the Pow Wow drum. Bonk~ provides effective onset attack detection but its frequency band output is insufficient for accurate pitch detection of the Pow Wow’s low and long reverberating sounds. Since bonk~ is hard-coded with a 256 point analysis window, the lowest frequency it can analyze is 172Hz – too high for the Pow Wow drum, which has a natural frequency of about 60 Hz. Moreover, pitch detection is complicated when high frequency hits are masked by the long decay of the previous low-pitched strikes. To address these issues, we wrote a Max/MSP external object that uses 2048 point FFT windows to determine the magnitude and derivative of lower frequency bins. By taking into account the spectral changes in addition to magnitudes, we can better determine whether energy in a particular frequency band came from the current hit or from previous ones (* 06). Other relatively low-level perceptual modeling modules are beat detection, utilizing Tristan Jehan’s beat~ Max/MSP object based on (Scheirer), and density detection, where we look at the number of note onsets per time unit to represent the density of the rhythmic structure. We also implement a number of higher-level rhythmic analysis modules for percepts such as rhythmic stability, based on (Desain and Honing), and similarity based on (Tanguiane). The stability model is based on the relationship between pairs of adjacent note durations that are rated according to their perceptual expectancy. This depends on three main criteria: perfect integer relationships are favored, ratios have inherent expectancies (i.e., 1:2 is favored to 1:3 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Gil Weinberg, Scott Driscoll. The Robotic Percussionist — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 06 — Magnitude plots from a 60Hz, 300Hz, and 5kHz frequency band over several low and — high-pitched hits showing the relatively slow decay of the low-pitched hits. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

and 3:1 is favored to 1:3), and durations of 0.6 seconds are preferred. The expectancy function may be computed as: r

Eb(A,B) = ∫ (round(r)-r)x|2(r-floor(r)-0.5)| p 0

x round(r)d dr Where r = max(A/B,/B/A) represents the (near) integer relationship between note durations, p controls the shape of the peaks, and d is negative and affects the decay rate as the ratios increases. This function is symmetric around r=1 when the total duration is fixed (* 07a). Generally, the expectancy function favors small near-integer ratios and becomes asymmetric when the total duration varies, exhibiting the bias toward the 600 ms. interval (* 07b). Our similarity rating is derived from Tanguiane’ binary representation, where two rhythms are first quantized, and then given a score based on the number of note onset overlaps and near-overlaps. In order to support real-time interaction with human players, we developed two Max/MSP externals that analyze and generate rhythms — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — —

74 75 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 07a 07b — — Basic expectancy of intervals A and 1-A (a) and 0.3 and B (b), — reproduced from Desain (Desain and Honing). — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

based on these stability and similarity models. These externals have recently been embedded in a live interaction module that reads measure-length rhythmic phrases and modifies them based on desired stability and similarity parameters. Both parameters vary between 0 and 1 and are used together to select an appropriate rhythm from a database of pre-analyzed rhythms. A stability rating of 1 indicates the most stable rhythm in the database, 0.5 equates to the stability of the input rhythm, and 0 to the least stable rhythm. The similarity parameter determines the relative contribution of similarity and stability. Interaction Design

The main challenge in designing the interaction with Haile was to implement our perceptual modules in a manner that would lead to an inspiring human-machine collaboration. The approach we took to address this challenge is based on our theory of interdependent group interaction in interconnected musical networks (Weinberg). At the core of this theory is a categorization of collaborative musical interactions in networks of artificial and live players based on sequential and synchronous operations with centralized and decentralized control schemes. For example, in sequential decentralized interactions, players create their musical materials with no influence from a central system or other players and can then interact with the algorithmic response in a sequential manner (* 08). In a synchronous centralized network topology, on the other hand, players modify and manipulate their peers’ music in real-time, interacting through a computerized hub that performs analysis and generative functions (* 09). — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Gil Weinberg, Scott Driscoll. The Robotic Percussionist — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 08 — A model of sequential decentralized interaction. Musical actions are taken in — succession without synchronous input from other participants, and with no central — — system to coordinate the interaction — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — —

More sophisticated schemes of interaction can be designed by combining centralized, decentralized, synchronous, and sequential interactions in different directions, and by embedding weighted gates of influence between participants (* 10). Based on these ideas, we developed six different interaction modes for Haile: Imitation, Stochastic Transformation, Perceptual Transformation, Beat Detection, Simple Accompaniment, and Perceptual Accompaniment. These interaction modes utilize different perceptual modules and can be embedded in different combinations in interactive compositions and educational activities. In the first mode, Imitation, Haile merely repeats what it hears based on its low-level onset, pitch, and amplitude perception modules. Players can play a rhythm and after a couple of seconds of inactivity Haile imitates it in a sequential call-and-response manner. Haile uses one of the arms to play lower pitches close to the drumhead center and the other arm to play higher pitches close to the rim. In the second mode, Stochastic Transformation, Haile improvises in a call-and-response manner based on players’ input. Here, the robot stochastically divides, multiplies, or skips certain beats in the input rhythm, creating variations of users’ rhythmic motifs while keeping their original feel. Different transformation coefficients can be adjusted manually or automated to control the level of similarity between their motifs and Haile’s responses. In the Perceptual Transformation mode, Haile analyzes the stability level of users’ rhythms, and responds by choosing and playing other rhythms that have similar levels of stability to the original input. In this mode Haile automatically responds after a specified phrase length. Imitation, Stochastic Transformation, and Perceptual Transformation are all sequential interac— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — —

76 77 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 09 * 10 — — A model of synchronous centralized interaction. A combination of centralized, decentralized, — Human and machine players are taking musical synchronous and sequential musical actions in — actions simultaneously, and interact through a an asymmetric topology with weighted gates — — computerized hub that interpret and analyze the of influence. Based on Weinberg — input data — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

tion modes that form decentralized call-and-response routines between human players and the robot. Beat Detection and Simple Accompaniment modes, on the other hand, allow synchronous interaction where humans play simultaneously with Haile. In Beat Detection mode, Haile utilizes the Max/MSP object beat~ to track the tempo and beat of the input rhythm. Although beat~ can be effective for pre-recorded audio, in a live setting human players naturally adjust to the robot’s tempo, which leads to an unsteady input tempo that is difficult for beat~ to follow. Haile therefore uses beat~ to listen for a short period (5-10 seconds) and then locks the tempo before joining in. A simpler, yet effective, synchronous interaction mode is Simple Accompaniment, where Haile plays pre-recorded MIDI files so that players can interact with it by entering their own rhythms or by modifying elements such as drumhead pressure to modulate and transform Haile’s timbres in real-time. This synchronous centralized mode allows composers to feature their structured compositions in a manner that is not susceptible to algorithmic transformation or significant user input. The Simple Accompaniment mode is also useful for sections of synchronized unisons where human players and Haile play together. Perhaps the most advanced mode of interaction is the Perceptual Accompaniment mode, which combines synchronous, sequential, centralized and decentralized operations. Here, Haile plays simultaneously with human players while listening to and analyzing their input. It then creates local call-and-response interactions with different players, based on its perceptual analysis. In this mode we utilize the amplitude and density perceptual modules that are described above. While Haile plays short looped sequences (captured during the Imitation and Stochastic Trans— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Gil Weinberg, Scott Driscoll. The — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 11 — The composition “Pow” for the first one-arm — prototype of Haile as performed in concert at the — — Eyedrum, Atlanta, GA. — — — — — — — — — — — — — — — — — — — — —

Robotic Percussionist — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 12 — — The composition Jam’aa for two darbuka players — and robotic percussionist as performed in concert — at Hamaabada Art Center, Jerusalem, Israel. — — — — — — — — — — — — — — — — — — — — — — —

formation modes) it also listens to and analyzes the amplitude and density curves of human playing. It then modifies its looped sequence, based on the amplitude and density coefficients of the human players. When the rhythmic input from the human players is dense, Haile plays sparsely, providing only the strong beats and allowing humans to perform denser solos. When humans play sparsely, on the other hand, Haile then improvises using dense rhythms that are based on stochastic and perceptual transformations. Haile also responds in direct relationship to the amplitude of the human players so that the louder humans play, the stronger Haile plays to accommodate the human dynamics, and vice versa. Compositions

Two compositions were written for the system, each utilizing a different set of perceptual and interaction modules. The first composition, titled Pow, premiered at the Eyedrum Gallery in Atlanta as part of the Listening Machines concert in January 2005. The second piece, titled Jam’aa, was commissioned by Hamaabada Performing Art Center, and premiered in Jerusalem, Israel in March 2006. It was later also performed in cities such as Paris, Bremen, Odense, Boston, and Atlanta. (For video clips from concerts see - http://www-static.cc.gatech.edu/~gilwein/Haile.htm) Pow

The composition Pow, written for one or two human players and a one-armed robotic percussionist playing a Pow Wow drum, served as test case for some of Haile’s early — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

78 79 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

mechanical, perceptual, and interaction modules. The piece begins with call-and-response routines, featuring Haile’s onset, amplitude, and pitch detection in Imitation mode. It develops into an improvisatory section where Haile utilizes the Stochastic Improvisation mode. Here, analyzed pitch, amplitude, and rhythmic data are used to generate simple stochastic manipulations such as hit division, and pitch averaging. A structured section then follows, which utilizes the Simple Accompaniment mode, where human players interact with Haile in a synchronous manner, taking turns as soloists based on a prerecorded 7/4 beat MIDI file. The piece ends with a short showcase of Haile’s mechanical abilities, featuring a fast trill that slides back and forth along the drum’s radius. Jam’aa

The composition Jam’aa (“gathering” in Arabic) builds on the unique communal nature of the Middle Eastern percussion ensemble, attempting to enrich its improvisational nature, call-and-response routines, and virtuosic solos with algorithmic transformation and human-robotic interactions. In Jam’aa, we added to the sonic variety of the piece by incorporating two robotic arms and by including other percussive instruments such as darbukas (goblet shaped middle-eastern hand drum), djumbes, and tambourines. Here, Haile listens to audio input via directional microphones installed inside two darbuka drums played by humans. It responds by playing a Pow Wow drum using two arms, while other human players join the drum circle, supporting the beat without interacting directly with the robot. In some sections of the piece the left arm merely provides the beat while in other sections it participates in the algorithmic interaction. Jam’aa utilizes interaction modes that were not included in Pow, such as Perceptual Transformation, where Haile responds based on its cognition of rhythmic stability and similarity, and perceptual accompaniment, in which Haile modifies its drumming in real-time based on the perceived density and loudness of human playing. We also developed a new response algorithm for Jam’aa titled Morphing, where Haile combines elements from two or more of the motifs played by humans, based on a number of integration functions. Future Work

A user study that was conducted to evaluate Haile’s rhythmic human-robot interaction effectiveness (Weinberg G. and Driscoll) led to the identification of a number of directions for further contributions. Based on the user responses, we plan to extend — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Gil Weinberg, Scott Driscoll. The Robotic Percussionist — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — —

our previous perceptual, mechanical, and interaction research to melodic and harmonic music. The positive results from the perceptual user study on rhythmic stability and similarity will drive the implementation of melodic and harmonic high-level percepts such as attraction, similarity, and tension, allowing the proposed robot to infer musical meaning from pitched-based input. In order to provide melodic and harmonic responses to such analysis, the robot will be designed to play a mallet instrument – the marimba. Findings from the user studies regarding the mechanics of the robotic percussionist emphasized the importance of anthropomorphic design for the creation of familiar and inspiring interactions with humans. The study also emphasized the need for larger, more visual movements and richer acoustic variety. Our proposed robot, therefore, will have a multi jointed anthropomorphic form, and will be designed to generate a wider sonic variety using various hitting techniques. One of the important aspects evaluated in the user study addressed the prospect of creating novel humanrobot musical experiences that cannot be generated by humans. Here, users’ responses suggested the need for better and more sophisticated algorithmic response and new playing techniques that will inspire humans to play and think about music in novel manners. In order to address this requirement, we plan to research and implement new approaches for interaction schemes such as beat detection and style matching, as well as new interactive algorithmic composition techniques using cellular automata, fractals and genetic algorithms. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

80 81 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — References Arom, S. „De L‘ecoute a l‘analyse des musiques centrafricaines.“ Analyse Musicale. 1 (1985): 35-59. Baginsky, N. A. The Three Sirens: A Self Learning Robotic Rock Band. 2004.

Narmour, E. The Analysis and Cognition of Basic Melodic Structures. Chicago: University of Chicago Press, 1990.

Cycling74. Max/MSP, 2005. www.cycling74.com; www.makingthings.com

Orio, N., S. Lemouton, et al. Score Following: State of the Art and New Developments. International Conference on New Interfaces for Musical Expression, Montreal, Canada, 2003.

Dannenberg, R. An On-line Algorithm for Real-Time Accompaniment. International Computer Music Conference Paris, France, 1984.

Pachet, F. The continuator: Musical interaction with style. International Computer Music Conference, Göteborg, Sweden, 2002.

Dannenberg, R. B., B. Brown, et al. McBlare: A Robotic Bagpipe Player. International Conference on New Interface for Musical Expression, Vancouver, Canada, 2005.

Papadopoulos, G. and G. Wiggins. AI Methods for Algorithmic Composition: A Survey, a Critical View and Future Prospects. Symposium on Musical Creativity, Edinburgh, Scotland, 1999.

Davies, M. and M. Plumbley. Beat Tracking With A Two State Model. IEEE International Conference on Acoustics, Speech and Signal Processing, Philadelphia, USA, 2005. Desain, P. and H. J. Honing. Rhythmic stability as explanation of category size. Proceedings of the International Conference on Music Perception and Cognition Sydney, 2002. Dibben, N. „The Perception of Structural Stability in Atonal Music: The Influence of Salience, Stability, Horizontal Motion, Pitch Commonality, and Dissonance.“ Music Perception 16.3 (1999): 265-295. Fong, T., C. Thorpe, et al. Collaboration, Dialogue, and Human-Robot Interaction. Robotics Research: The Tenth International Symposium. 6 (2003): 255-270. Jordà, S. Afasia: The Ultimate Homeric One-man multimedia- band. The International Conference on New Interfaces for Musical Expression, 2002. Lerdahl, F. and R. Jackendoff. A Generative Theory of Tonal Music. Cambridge, Massachusetts: MIT Press, 1983. Lewis, G. „Too Many Notes: Computers, Complexity and Culture in Voyager.“ Leonardo Music Journal 10 (2000): 33-39.

Prassler, E., E. Bank, et al. Coordinating the Motion of a Human and a Mobile Robot in a Populated, Public Environment. IEEE Int. Conf. On Field and Service Robotics FSR ‘01, Helsinki, Finland, 2001. Puckette, M. (1998). Real-time Audio Analysis Tools for Pd and MSP. International Computer Music Conference, 1998. Raes, G. W. Robotic Instruments. 2005. http://logosfoundation.org/instrum_gwr/automatons Rowe, R. Interactive Music Systems: Machine Listening and Composing. Cambridge, MA: MIT Press, 1992. Scheirer, E. „Tempo and beat analysis of acoustic musical signals.“ Journal of the Acoustical Society of America 103.1 (1998): 588-601. Schmuckler, M. A. „Testing Models of Melodic Contour Similarity.“ Music Perception 16.3 (1999): 295-326. Severinson-Eklundh, K., A. Green, et al. „Social and collaborative aspects of interaction with a service robot.“ Robotics and Autonomous Systems 42.3-4 (2003): 223-234. Singer, E., J. Feddersen, et al. LEMUR’s Musical Robots. International Conference on New Interfaces for Musical Expression, Hamamatsu, Japan, 2004.

MakingThings. Teleo, 2005. www.making.things.com

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — »

Gil Weinberg, Scott Driscoll. The Robotic Percussionist — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — —

Singer, E., K. Larke, et al. LEMUR GuitarBot: MIDI Robotic String Instrument. International Conference on New Interfaces for Musical Expression, Montreal, Canada, 2003. Sony. QRIO conductor Robot. 2003. . Takanishi, A. and M. Maeda. Development of Anthropomorphic Flutist Robot WF-3RIV. Proceedings of International Computer Music Conference, 1998. Tanguiane, A. Artificial Perception and Music Recognition. New York: Springer-Verlag, 1993. Thrun, S. „Toward a Framework for Human-Robot Interaction.“ Human-Computer Interaction 19.1-2 (2004): 9-24. Toyota. „Trumpet Robot.“ 2004. . Vercoe, B. „The Synthetic Performer in the Context of Live Performance. International Computer Music Conference, Paris, France, 1984. Waldherr, S., R. Romero, et al. „A Gesture Based Interface for Human-Robot Interaction.“ Autonomous Robots 9.2 (2000): 151-173. Weinberg, G. „Interconnected Musical Networks - Toward a Theoretical Framework.“ Computer Music Journal 29.2 (2005): 23-39. Weinberg G. and S. Driscoll. „Towards Robotic Musicianship.“ Computer Music Journal 30.4 (2006): 28-45. Winkler, T. Composing Interactive Music: Techniques and Ideas Using Max. Cambridge, MA: MIT Press, 2001. Yanco, H. and J. Drury. A Taxonomy for Human-Robot Interaction. AAAI Fall Symposium on Human-Robot Interaction, Falmouth, MA, 2002.

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — —

82 83 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

W f I bx Iv e f

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

02

3

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

B d f è v

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

t

V

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

?

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

>

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

K MI ss t uB

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

... as ...C

8? H v

Q$

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Peter Gendolla >

nD ß

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

9Impossible 0 People Thougts nSome on the WCultural History

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

n

of the Android — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Peter Gendolla. — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — —

Impossible People —

Some Thoughts on —

the Cultural History —

of the Android —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — It’s not pleasant to discover you were invented. (Jack Slater alias Arnold Schwarzenegger alias Arnie Schwarzenegger in LAST ACTION HERO)

Technical inventions do not simply make work easier, they also allow their constructor to gain an understanding of himself, of how he functions. This human impulse to explore internal connections by projecting external mechanisms onto them is nowhere more evident than in the automaton (Greek: “that which acts of itself”). Be it by the ancient Chinese, by the contemporary Japanese in their factories, by the chip-smiths in Silicon Valley, automatons have always been constructed to act as useful helpers, slaves, no less – the word robot originally signified laborer – but also as “useless playthings.” Automatons today are in the process of undoing the Babylonian linguistic confusion as computer translation programs. Moreover, they have been disguised as nightingales or ducks, have entertained royalty at the Baroque courts in complete automaton theaters, and these days constitute one of the industrial branches with the — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

88 89 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 01 — — — — Le joueur de flûte traversière de Vaucanson — — — — Le joueur de galoubet et de tambourin de Vaucanson — — — — — — — — Le canard de Vaucanson — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

largest turnover, as Nintendo, Sony or Sega systems. In this very tradition a new selfimage of the individual and his social body has always been playfully documented, as if in passing. In his book “L’Homme machine” (Man a Machine) of 1747, the notorious 18th century philosopher Julien Offray de La Mettrie made perhaps the most disputed proposal for a new image of man disconnected from traditional metaphysical foundations, an image which, in its basic elements, finds its continuation today in proposals by biologists to improve the human genome. Some years before La Mettrie, Jacques de Vaucanson is someone who vigorously promoted the “isolation” of the spiritual powers and the expansion of the circles of knowledge, plus the subsequent “segregation of professional occupations,” that is to say, someone who warmly welcomed and ingeniously furthered the technologies of the division of labor. Like no others before them, his automata, the “Flute Player” (1738), the “Tambourine Player” and the “Duck” (* 01) (both 1739), stimulated the mechanical fantasies of the 18th century and beyond – from La Mettrie to Reimarus and Goethe (Matthes 121). What is less well known, though, is just how closely connected work and play were for Vaucanson, the interplay of skillfulness and ingenuity, which were concentrated in his main obsession: to isolate or separate the controls or steering force from the movement of the body/machine parts, to transfer these to their own steering/control sector or organs, and to divide them into the process sections controls/transmission/operation, enclosed in flexible hollow bodies – a technique we today would call module or black box systems. Vaucanson was appointed director of silk manufacture in Lyon in 1740. Here, long before Jacquard, he developed automatic controls for looms.1 To do this, he transferred the cam controls used in mechanical toys and musical boxes to the work machines, reversing the principle and thus developing perforated disc controls. « — — — — — — — — — — — — — — — — — « — — — — — — — — — — — — — — — — 1 See Hans H. Hiebel Kleine Medienchronik. Von den ersten Schriftzeichen zum Mikrochip, who also mentions the French mechanician P. Falcon, who used sets of small perforated wooden slates to guide looms;

he refers to Vaucanson only as a builder of automatons. The imaginative interaction between work and play is closer in inventors, engineers, and artists.

Peter Gendolla. Impossible People — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — —

One could regard this expansion of the circles of knowledge and the isolation or segregation – later McLuhan would call this extension – of the effectiveness of spiritual powers as also being responsible for the technical inventio. Social differentiations go hand in hand with greater demands on the individual spiritual powers, resulting in an overtaxing of those powers. On the one hand, this very pressure on the individual capacities leads to the invention of technical aids, from Vaucanson to the current programs of Artificial Intelligence, irrespective of whether they control washing machines, prostheses, or whole factories (on prosthesis see Berr). On the other hand, and in interaction with this, aesthetic perception, the perception of perception, is worn down: what happens to sensations, how are senses expanded, filtered, intensified, shaped or tinted,2 when the body that thus perceives itself is extended or restructured by technical artifacts? The fantasies about machine-men entertained by writers from E.T.A. Hoffmann to Asimov or Gibson can be seen as a direct reaction to this. Indirectly the various 18th century literatures, cults or cultures of Empfindsamkeit (Sensitivity) can be regarded as in a way articulating the internal reactions to the external aids. It is no coincidence that as early as 1777 Goethe, who had contributed considerably to this culture with his “Leiden des jungen Werther” (Sorrows of Young Werther) of 1774, distanced himself in his “Triumph der Empfindsamkeit” (Triumph of Sensitivity) from such inwardness: from that “theatrical whim” (Goethe) about love, i.e., “the electricity of tender hearts,” for an artificial woman, surely the direct model for Hoffmann’s famous doll Olimpia in the story “Der Sandmann” (The Sandman) of 1815. Another direct reaction to this can be found in the so-called literature of horror, the terrifying visions of the Gothic novel, an aestheticization of the experience of the individual and social division of labor, from Wackenroder’s “Märchen von einem nackten Heiligen” (Tale of a Naked Saint) to Mary Wollstonecraft Shelley’s “Frankenstein, or The Modern Prometheus” – still one of the most frequently filmed stories in cinema history: What else do these literary works articulate but the sudden terror in the face of amputations, the destruction of physical wholeness, of the possibility of undivided sensations? La Mettrie’s “L’Homme machine” (1747) declares man to be a “very enlightened machine”; man is the inspired, the enlightened machine, as opposed to all other machines, the animal-plant-heaven-machines. This recourse taken by the man of the Enlightenment to his own body, this application of rationalism’s main mechanical metaphors to the organic, was encouraged by an 18th century preoccupation that is « — — — — — — — — — — — — — — — — — « — — — — — — — — — — — — — — — — 2 As with the corresponding metaphor in Heinrich von Kleist’s famous Kant crisis, which refers to nothing more than a technical “support” instrument, i.e. spectacles. 3 Which has not been solved yet. Heinz von Foerster made an interesting, if equally inadequate suggestion at a congress in Berlin early in 1997. With reference

to Kant, for whom consciousness is that alertness, attentiveness or brightness ought to accompany all our thoughts and emotions, he defines consciousness as a certain break in the inner neuronal routines. Consciousness, he argues, emerges the moment they are interrupted, when they do not simply run on, but run wrongly, when functions oppose one another, that is to say, have to be reorganized. If one wants to under-

90 91 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

still vexing today, the emphatic fascination with this watch, i.e., the human body, “constructed with such skill and ingenuity” that, astonishingly, it winds itself up, or stops other clocks, apparently with just as much passion. What is it that draws our attention to our own material composition, what makes the doctor or philosopher claim that the light of reason is both the principle differentiating us from the non-human machines as well as the principle of every living cohesion in any being whatsoever – thereby embroiling him in the real problem of the consciousness debate: Does consciousness exist to greater or lesser degrees?3 Inspired by La Mettrie and Vaucanson, one could formulate a thesis, which though impossible to prove here, might, if padded out with some material, be a useful basis for some further thoughts on already existing or possibly imminent substitutes for man: mechanizations of the body, from the restructuring of the individual body with the help of prostheses to its integration into large – for example military – machine systems, could go – sometimes – hand in hand with the amputation of the respective organs and their functions. These in turn produce phantom pains,4 sensations in no longer existing organs. Art, literature and their diverse media, among other things, come to terms in a particular way, i.e. aesthetically, with this manifold process of substitution or transference. One consequence of this claim would be that civilizatory, disciplinary, and industrial processes are always reflected in the history of the arts. Not directly or simply, however, but rather in pain, that is to say, in the highly varied forms in which phantom pains are processed, among other things, in a historically very diverse aesthetic production and in ever new phantomatic objects articulated in ever new media. Such a thesis inspired by La Mettrie could be pursued in three different discourses: Bellmer’s surrealistic discourse, Freud’s psychoanalytical discourse, and McLuhan’s media-theoretical discourse. In the thirties, Hans Bellmer conceived a very peculiar counterpart in the history of artificial man by varying and transforming what he called the “doll.” He accompanied his construction of the doll with extensive notes, commentaries, and interpretations. He understood “the various expressive categories: physical pose, movement (...), tone, word, graphics, design of objects (...) as born of one and the same mechanism,” in keeping with the model of the “reflexes provoked by a toothache,” a technique of pain mastery, the re-routing and thus control of an original sensation of pain, a kind of cramped hand. The “cramped hand is an artificial excitation center, a « — — — — — — — — — — — — — — — — — « — — — — — — — — — — — — — — — — stand consciousness as light, as the Enlightenment did, then here in this context more as a will-o’-the-wisp in search of new exits. The difference between animals, men, and machines would then in fact be only a gradual one, measured according to the number and diversity of strategies available for mastering disruptions.

4 “Phantom-limb pain, sensation of pain in an amputated limb. The phantom-limb pain is caused by the fact that the nerve fibers responsible for the sensations in the amputated limb are still present in the main nerve. When the nerve stump is stimulated at the point of the amputation (where the amputated nerve strands are not insulated), sensations are triggered which the brain ‘projects’ onto the missing part of the extremities.” (Translated from: Der große Brockhaus 141)

Peter Gendolla. Impossible People — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — *

— — — — — — — — — — — — — — — — — — — — — — — — — — — — 02

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — Circuitry — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

‘virtual tooth’ that diverts the stream of blood and nerves from the real center of the pain, directing it to itself in order to cancel it out.” In Bellmer’s view, all aesthetic production comes about as a reaction to a too intense impression, a pain, a disruption of perception and of inner homeostasis. The fantasies that then ensue create their own “excitation centers,” self-made phantom pains. He understands their various material manifestations – for example, his “doll” (* 02) – as “a consequence of liberating transferals that lead from the suffering to its image. The expression, and what it contains in the way of pleasure, is a pain that has been displaced, a liberation.” (Bellmer 73). What he was attempting was nothing less than to continue to reformulate the pleasure principle, or the functioning of the psychic apparatus, which Freud undertook in his study “Beyond the Pleasure Principle.” The persistent dreams and fantasies of traumatic events related in analytical practice by war invalids or train accident victims – the loss of an arm or leg – cannot really be understood with the help of the wish fulfillment model put forward in the “Interpretation of Dreams.” Freud structures the apparatus in a more complex way. According to him, the dream/trauma repeats the experience that has come over the subject suddenly or violently from outside – “the main thrust of the cause (seems) to lie in the moment of surprise, the shock” (Freud 222)5 – and destroys the subject’s psycho-physiological integrity, but the dream repeats it as self-inflicted injury, though staged by the dreamer himself. In his way, physical wholeness is reconstructed phantasmagorically. The third continuation of the thesis comes from the father of current media theories, Marshall McLuhan. If his concept of all technology - from the wheel to the processor - as “extensions of men” were to adhere to a simple logic of extension and intensifica« — — — — — — — — — — — — — — — — — « — — — — — — — — — — — — — — — — 5 For a discussion of this model see Jacques Derrida Freud and the Scene of Writing 196-231; Wolfgang Schivelbusch Geschichte der Eisenbahnreise.

6 McLuhan too refers to the toothache, illustrating his theory with an apparatus that technically implements Bellmer’s and Freud’s theses: “Battle shock created by violent noise has been adapted for dental use in the device known as audiac. The patient puts on headphones and turns a dial raising the noise level to the point that he feels no pain from the drill.” (54).

92 93 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

tion, we could safely leave it to the optimists or pessimists of the digital age. In his “Understanding Media” McLuhan first formulates the effects of such extensions on the psyche and on society, the main focus of his investigation. “Any extension, whether of skin, hand, or foot, affects the whole psychic and social complex.” (Mc Luhan, 19) He then expands on this concept. With reference to the kind of medical research that looks upon every extension of the person as “auto-amputation,” as a means of maintaining inner balance, he construes technical systems as results of disruptions. “In the physical stress of superstimulation of various kinds, the central nervous system acts to protect itself by a strategy of amputation or isolation of the offending organ, sense, or function. (...) Physiologically, the central nervous system, that electric network that coordinates the various media of our senses, plays the chief role. Whatever threatens its function must be contained, localized, or cut off, even to the total removal of the offending organ.” (Mc Luhan 52).6 So art (Bellmer’s) “takes pain in hand,” the dream (Freud’s) enables the Ego to reconstruct itself, the external media (McLuhan’s) rescue the internal media from super-stimulation and contain the pain. If one understands pain as nothing more than the signal for the severance of representations in the neuronal system from their matter, from cells to organs, and as the destruction of the close cohesion between signified and signifier in the individual body, then there are in fact two possible consecutive effects: The first is that the signal-sign complex – the phantom pain in the sensitive virtual organ – becomes independent. The second is its extension, the placing of the organ outside the body, liberation from the phantom pain by a kind of manifestation of the phantom. The construction of aesthetic objects or technical systems – the two cannot be separated here – goes hand in hand with the anesthetization of the corresponding individual organs. By being transferred to the outside, the functions of the organs order, extend, and intensify the capacities of the individual, and the species, both to perceive and to act. All three proposals attempt to explain very different modes of expression – art, dream/trauma, (media) technology – by means of a virtual, not immediately visible, but very effective object, as a processing of phantom pains concentrated in this object, as a thus regulated interplay between inside and outside.7 These attempts are to be continued here, and technical constructions and aesthetic perceptions are to be seen in certain respects as parallel phenomena. « — — — — — — — — — — — — — — — — — « — — — — — — — — — — — — — — — — 7 This is to be understood literally. Freud illustrates his theory with the so-called “Fort-Da” (gone-back) game played by his grandchild, who used a wooden spool on a string to transform the disappearance/appearance of his mother into a game and thus make it more bearable. One can therefore read every “Once upon a time” in literature as an aestheticization or anesthetization of

an injury or pain and its transformation into a game or story. Freud could have found one of the most amusing transformations in Kafka’s story about Blumfeld, the “elderly bachelor” who fails to attach the little hopping balls to a string, unlike Freud’s grandchild. Literature thus adopts psychoanalysis, while at the same time distancing itself from it.

Peter Gendolla. Impossible People — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 03

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — Les Automates — — — — — — — — — — — — — — — — — — — — — — — — — — — —

In 1760 Friedrich von Knaus presented his “miraculous writing machine” to the public. “The desired text is transferred onto a horizontally positioned cylinder by means of tiny pins. These pins strike keys which move the curved disks of the desired letters by means of a lever.” (Beyer 57). This technology was developed to temporary perfection in the androids produced by the Jaquet-Droz family and their mechanic Jean-Frédéric Leschot: the “Draftsman,” the “Musician,” and the “Author.”8 (* 03) The latter was able to combine up to forty characters so as to produce any desired text. The free programming of the automatons thus achieved reveals the cultural orientation and social models of the respective historical periods all the more clearly. If the “Author” writes “Long live the city of Albrecht Dürer” during a presentation in Nuremberg in 18oo, and if the “Draftsman” portrays mainly such high-ranking personalities as Louis XV, it is evident that this representation of feudal power has today given way to a catchy salute of welcome in the advertising world. While the clockwork automatons in the early modern era were modeled on the myths of the gods or the history of Christian salvation, and in the 18th century served to represent the power of the sovereign and at the same time the universal craftsmanship of the middle-class citizen, this technical skill becomes the real object of the man-machine constructions in the 19th and finally the 20th century: to reproduce the whole complex apparatus called man, albeit free of pain and defects, in a state of equilibrium. Engineers and artists, scientists and writers – significantly enough, mainly men – now work uninterruptedly on this project of the species, as if they somehow wanted to catch up with women’s natural productivity and finally overtake it some day, with beings that are naturally « — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 8 They can still be viewed today at the historical museum in Neuchâtel in Switzerland, and every Sunday morning they are in action.

94 95 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

more faultless than those to date, thereby eliminating the difference between the sexes.9 However, these constructions initially differed from the “full-bodied automatons,” which now tend to drift into the toy production sector: series productions of talking dolls or quick little fighting robots for putting under the Christmas tree. In the wake of the inventions of Reis, Bell, Edison, Marconi, and many others toward the end of the 19th century, the individual senses, their perceptions and forms of articulation, are reproduced, specialized or intensified in technical systems; speech and hearing, projection and vision are transformed, and the system of the technical media is further developed. Society is not just trying to maintain itself at a new level or to expand its productivity through these projections and extensions. It is moreover attempting to redefine itself, from the tiniest element to the largest system, from the individual to the state itself. By thus effectively transforming the individual senses into technical media, sensorsystems for example, however, the two disintegrate, both individual and state. The technical reproduction, storage, and transport of voice and ear, eye, nose, and skin, divides the individual up into a field of sensory exchange processes. When national institutions are networked and coupled with global information processes, vital economic, political, and social decisions are transferred from the traditional legislative and executive “bodies” to dynamic, re-coupled media processes which are no longer localizable in space or time. In order to be somehow able to grasp this disintegration and reformation, which run counter to a traditional understanding of decision-making processes, and get an impression of the new bodies and their new communities, there are probably only these two figures which in equal measure awaken both technical euphoria and culture-criticism in the early 21st century: the android, robot, cyborg, or whatever the new individual is called – “I, robot”, the film based on Asimov’s stories, was catching our attention the last year - and his state supervisor, Big Brother, the Securitate, the intelligence services, the CIA… or the Internet, the greatest conspiracy medium of all time. In the early 19th century Joseph Faber built his famous talking machine “Euphonia,” (* 04) which imitated human speech better than all prior attempts. It could well be the model, utilized by cultural critics, for the many artificial women that populate literature. Faber’s talking woman did not bring him anything like the luck, fame, and money that, for example, Kempelen’s “Chess Player” (* 05) had brought its owner, Maelzel, who even had it challenge Napoleon to a game of chess.10 Faber destroyed « — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 9 Even the authors of the extensive literature on this project are mainly male, cf. Bernhard J. Dotzler, Peter Gendolla, Jörgen Schafer, Eds. MaschinenMenschen. Eine Bibliographie. Only when feminist cultural criticism became established was women’s attention drawn to this attempt to eliminate them, recently, for example, by Donna J. Haraway Simians, Cyborgs, and Women: The Reinvention of Nature.

Peter Gendolla. Impossible People — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 04 — — Euphonia 1835 — — * 05 — — Chess Player — — — — — — — — — — — — — —

his machine and committed suicide, a fate prefigured in Hoffmann’s story about the “Sandman” and repeated in Villiers de L’Isle Adam’s “L’Eve future” (1886), and in 20th century science fiction literature, Lawrence Durrell’s “Nunquam” from 1970, for example or Bryan Forbes‘ “The Stepford Wives” from 1975. Here literature is just continuing the work on that vexing “phantom” which holds societies and cultures together at their core: that material-immaterial system of signs, language’s possibilities for combination. These stories, which have long since moved from the medium of the book to the theatre, cinema, and the latest media, there to be further processed, really have only one positively fixed idea: that it should be possible to talk to a selfsteered machine, that the human Ego and the technical Id should be able to enter into communication with one another. Long before Turing’s test – which tries to prove that when a technical medium is interposed, machine communication is indistinguishable from human communication on the basis of the data sent and received alone – writers, artists, and philosophers were working on this idea, doubtless in the interests of their own peculiar productivity. The idea that a wonderful machine – the machine – might one day be able to answer these lonely artists as they shape their texts, pictures, and sculptures, and might in fact be able to enter into an open dialogue with their creators, this displacement of the self, is surely their most irresistible temptation. The creators are not interested in passive partners, but want an active, interactive as we say today, counterpart that is unexpectedly independent, something whose movements, replies, actions are completely unpredictable.11 « — — — — — — — — — — — — — — — — — « — — — — — — — — — — — — — — — — 10 The “Chess Player” was so famous around the

world that E.A. Poe felt called upon to prove, merely on the basis of the newspaper reports on its presentation that an intelligent dwarf must have been hidden

in the man-machine. Cf. Edgar Allan Poe Maelzel’s Chess Player.

96 97 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

Consequently, the idea thus far pursued with the help of Bellmer, Freud, and McLuhan that technical artifacts are widening and replacing or even amputating the human apparatus might have to be extended, possibly even canceled out or better rethought from a new level. This new level is provided by developments in information technology and neurological research that have begun to reconnect those elements which they at one time had divided, expanded or replaced – namely artifact and nature. By directly coupling them they are now questioning the opposition artificial vs. natural or human in general. A long tradition of theories is questioned here, theories that regard technology (especially in self-directed, automated systems) only as a mechanism of replacement or support, as liberation from heavy work, or as management of overly complex processes as mentioned above. It is questioned, however, in a rather playful manner, the exchanging of thoughts with chat boxes, dialogue-systems and software-agents. It is not the machine-like, standardized or calculable aspect of human beings that is displaced here, thereby functionalizing or economizing it. It is rather coincidence, surprise, emergence, risk or illusion, in a word: it is the unexpected that becomes conceivable in the ‚dialogues‘ mentioned above; one’s own and the others‘ ideas begin a new type of playful conversation with each other. Until now only a well-formulated share, a limited mode of those processes was transferred from the mind to the computer that create sentences, ideas or images. Everything else was happening within the body, not only in the mind. Consequently, the exchange between inside and outside on whichever interface only aligned two automated processes or internal routines with calculated or calculating algorithms. Even the offer on a chatterbot for lovers‘ conversations between boyfriend or girlfriend worked in just this way: Cyberito, http://www.liebste.de/ (* 06) Probably the most advanced partner seems to be Kaily. (* 07) “Kaily live: she can talk and understand.

Download a free Kaily

spot. You can ask Kaily everything she searches the web . and finds an answer. Every morning Kaily gives a weather forecast and reminds of birthdays and appointments. Kaily is your personal assistant: she reads you the latest news.” (http://www.nice-technologies.de/ last access 11 July 2006)

« — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 11 On other aspects of this unpredictability see Peter Gendolla, Thomas Kamphusmann, Eds. Die Künste des Zufalls.

Peter Gendolla. Impossible People — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 06 — * 07 — Cyberito erstellt für Sie kostenlos einen Digitalen Liebesbrief — Kaily — mit Bild und Musik. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

All these processes of exchange between man and machine however retain a boundary – a boundary of the skin, of the retina, or of the eardrum. Even though the computer increasingly approaches this boundary through more precise sensors it does not yet cross it. Nevertheless, the evolutionary process is continuing; the small but decisive step towards a revolution is probably done at the point at which the direct connection between computer and brain is passing into both directions, where the one-directional connections change into bi-directional ones. The December 2004 issue of a Berlin magazine (* 08) reports of a computer that works without a mouse since it can receive commands directly from the brain. This time a woman, Verena Araghi, starts the conversation with the machine: “You are waiting for my thoughts. You‘ll be able to see them on the monitor immediately. My scalp is cooled down by the gel on 128 electrodes; every one of them is a little guardian scanning a part of the cerebral cortex under my cranium.” (Araghi 30)

A paraplectic has been enabled to raise himself (quite laboriously) through the collaborative efforts of a German-French team of neuromedical specialists who implanted a chip. At least wheelchairs will soon be controlled like this and intense research is going into the direction of prosthetics. Machines can process thoughts much faster than the body, since the brain initiates movements almost half a second before the arms or the legs react. Neurophysi— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

99 — — — — — — — — — — — — — — — — — — — — — — — — — — — Berlin — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — *

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 08

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

98 » — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

cists want to take advantage of this. For example cars that are equipped with the brain-computer-interface BCI could be able to tighten the safety belts just before an accident. The scientists will also develop an electrode-cap without cables with which one can move around freely. Whether we are talking of implanted chips or, as above, of sensors attached to the skin: the boundary is becoming permeable. The possible people – robots, androids, men-machines as projections of ourselves, and this self are starting an exchange. In other words: The game of imagination, the internal process of projections that only needed stimulation in order to be set in motion by itself, as well as communicative devices like literature, images, and films are suddenly skipped. To rephrase: Whereas the inner and the outer world could previously communicate only via complex detours, via whole series or layers of ever-new codes in a contradictory and paradoxical way – a self, that is always locking on projections of itself – now this communication is replaced by a bi-directional exchange, interior routines and computer programs are in a direct exchange. Even though it is still a very weak bi-directionality since the computer actually cannot say more than R, L or F (right/left for the hands and F for feet) nevertheless a new conversational game has been initiated which is a game of ideas of the other kind. “The machine knows me now. It reacts best to my thoughts for the right hand and the right foot. We are ready for the . My pong-racket is a flat black bar. With that I have to fend off — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Peter Gendolla. Impossible People — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 09 — — — Voder — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — a small green ball from the lower edge of the monitor thrown by the computer from the upper edge. My thoughts of a movement of my right hand pull the racket to the right, those of the right foot pull it left. If I am catching the ball, I‘m getting the point, if I miss it counts for the computer. I want to win this game at all cost. But the racket all of a sudden is dancing from one side to the other and does not reach the ball any more. The machine is rebelling. Or is ambition blocking my brain? I am fighting tensely but without success. On the upper left the monitor is showing that I‘m already behind 4:11. I‘ll loose heavily. But I don‘t care any more.” (Araghi 30).

In view of the defeat the test person is becoming indifferent, thereby relaxing and finally winning 20:18. Certainly this is not the last word spoken. For now, I‘m going to leave it to VODER, (* 09) a speech synthesis device that was developed by Homer Dudley and presented to the public at the World Exhibition in New York in 1939. “Good afternoon, radio audience”12

« — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 12 Listen to: http://www.ling.su.se/staff/hartmut/ kempln.htm. Last access 24 December 2005.

100 101 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — References

Images

Araghi, Verena. „Man-Machine.“ Berlin. The Magazine From The Capital. 1 December 2004.

* 01 Vaucanson. Heckmann, Herbert: Die andere Schöpfung. Frankfurt/M. 1982

Berr, Marie-Anne: Technik und Körper. Berlin: Reimer, 1990.

* 02 Bellmer, Hans: Die Puppe. Die Puppe / Spiele der Puppe / Die Anatomie des Bildes. http://www.amazon.de/Puppe-Puppe-Spiele-Anatomie-Bildes/dp/3548032818/ref=sr_1_2/303-8285855-1 627430?ie=UTF8&s=books&qid=1183372174& amp;sr=1-2.Berlin 1982. Download: 2 July 2007

Bellmer, Hans. Die Puppe. Frankfurt am Main, Berlin, Wien: Ullstein, 1976. Beyer, Annette: Die faszinierende Welt der Automaten. Uhren, Puppen, Spielereien. Munich: Callwey, 1983. Der große Brockhaus. 14, 1992. Derrida, Jacques. “Freud and the Scene of Writing.” Writing and Difference. London: Routledge, 1978. 196-231. Dotzler, Bernhard J., Peter Gendolla, Jörgen Schafer, Eds. MaschinenMenschen. Eine Bibliographie. Frankfurt am Main: Lang, 1992. Freud, Sigmund. „Jenseits des Lustprinzips“. Studienausgabe. Ed. Alexander Mitscherlich, Angela Richards, James Strachey. Psychologie des Unbewussten. Frankfurt am Main: Fischer. 3, 1975. Gendolla, Peter, Thomas Kamphusmann, Eds. Die Künste des Zufalls. Frankfurt am Main: Suhrkamp, 1999. Haraway, Donna J. Simians. Cyborgs, and Women: The Reinvention of Nature. New York: Routledge, 1991. Hiebel, Hans H., Ed. Kleine Medienchronik. Von den ersten Schriftzeichen zum Mikrochip. Munich: Beck, 1997.

* 03 Jaques Droz, Les Automates Heckmann, Herbert: Die andere Schöpfung. Frankfurt/M. 1982 * 04 Fabers „Euphonia“ http://www.mambo.ucsc.edu/psl/smus/faber.gif Download: 11 November 2006 * 05 Kempelen, Schachtürke Heckmann, Herbert: Die andere Schöpfung. Frankfurt/M. 1982 * 06 Cyberito http://www.liebste.de/ Download 2 July 2007 * 07 Kaily http://www.nice-technologies.de/ Download 2 July 2007 * 08 Berlin. The Magazine From The Capital December 1, 2004 * 09 Voder http://ptolemy.eecs.berkeley.edu/~eal/audio/voder.gif Download 2 July 2007

McLuhan, Marshall. Understanding Media: The Extensions of Man. New York: Signet Books, 1964. Matthes, Dieter. „Goethes Reise nach Helmstedt und seine Begegnung mit Gottfried Christoph Beireis. Braunschweigisches Jahrbuch. Wolfenbüttel. 49 (1968): 121. Poe, Edgar Allan. “Maelzel’s Chess Player”. Southern Literary Journal. April 1836. Schivelbusch, Wolfgang. Geschichte der Eisenbahnreise. Munich, Vienna: Ullstein, 1977.

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — »

g m q

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Friedrich Wolfram Heubach >

P

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

k

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

q

«S

Man and Machines Psychological notes on a difference and its fading

P

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Friedrich Wolfram Heubach. — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — —

Man and Machines —

Psychological notes —

on a difference and —

its fading. —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

To approach the subject via an anecdote of a bizarre coincidence: I was just working on an essay for the catalogue compiled for the Cologne exhibition “Ex machina – a history of robots from 1950 to the present days”, when the events of September eleven shocked the world and the ensuing media-reports provided me unexpectedly with material for my reflections on man and machines: All these reports were marked by the irritation in face of the indifference demonstrated by the assassins towards their own life and the lives of the many people they killed. But these reports proved to be especially alarmed by what they described as the engine-like precision shown in the planning of the attacks, in calculating their precise point in time, in coordinating the moves of the different groups and the clockwork-like accuracy with which these attacks were eventually executed devoid of all human impulses, just as if they were – as one could read there “unfeeling robots” or “machines of evil”. And in fact do these events impressively demonstrate how thoroughly humans can negate their own and other people’s “human nature” – negate everything this nature said to be. What is more, these events are revealing in rare clarity what enables — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

104 105 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

people to functionalize themselves as machines and other humans as objects: a deep, often enough religious conviction, high ideals or a belief in a mission. In other words, something which is not only characteristic for the Taliban-terrorists – something which even their enemies claim to be and are very proud of it, too. And last but not least these events raise a question referring to a central topic of my speech, namely the apprehensions many people have when they think of future “humanlike machines”: what is it that people will have to fear of future increasingly human-like machines they never had to fear in other humans since ever? I leave the answer up to you and turn to the difference between man and machine and to the question of its possible annihilation. This question can be asked in very different forms. The most popular is: Will there be machines, which are like humans somewhere in the future? In this version, the question itself is, due to its ontological claim, boring and of no interest. To speculate about the existence of future machines and to ask whether their being might one day resemble that of human beings is as for free and nothing – that means: leading to too many answers – as that question asked ever since “What is the being of man” has proved to be. But if one can’t help to take this difference between man and machine as an ontological one and if therefore one wonders anxiously, whether there will be machines one day, which are no longer distinguishable from man, whether a machine can really be as a human is, – then this silly question would at least gain some weight if asked the other way around: Will there one day still be humans different from machines, can human beings become machines, become like machines? Put this way, the ontologically asked question about the difference between man and machine would not be more reasonable but at least a lot more insinuating – given the events of our days, not only of September eleven. For somebody who isn’t philosophically minded enough to like questions only, if they come up as last ones, the question of the possibility of a humanlike machine is only a reasonable one, if reduced to an analysis of the conditions under which human beings regard something as man or machine. Thus, the ontological question has become an epistemological one and the difference between man and machine lies no longer in their being but in the patterns controlling the process of perceiving and recognising an object as man or machine. This was – contrary to popular interpretation – precisely the problem which Turing was dealing with when asking the famous question whether it would be possible to construct a machine which would not be recognized as a machine but believed to be a human — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Friedrich Wolfram Heubach. Man and Machines — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — —

being by the people communicating with it when asked in writing and answering accordingly. The fact that this question can be answered in the affirmative doesn’t affirm at all the possibility of humanoid machines – contrary to what some people think. It only illustrates an old fact: if you make skilful use of the mechanics of human imagination, if you serve the patterns of recognition with sufficient subtlety, you trigger the experience of “human”, you can give a human touch to everything. This machination of the human, this make-belief of “human” is a possibility professionally exploited ever since by agents as different as legacy-hunters or politicians, priests or simulators, artists and actors. And the Turing-Test only proves that this machination/mimicry of the human can also be mechanically organized. But there is another question, which ought to be asked in connection with humanlike machines. Why is it that this topic, with its long history in myth and literature, has captivated such a general interest in our days and become such a popular issue? It would mean to mistake the trigger for the cause, if we made what has been recently presented to the public as the newest achievements in the field of developing humanoid machines responsible for it. And it would also mean to ignore the fact that these achievements are still far from having the significance, which is attributed to them by their authors who are eager to attract public attention. These achievements, which are presented as an important step towards the development of humanoid machines are fairly modest compared with what has already been achieved with regard to the mechanization of man (I will come back to this point later). There is a reason why people are currently fascinated by the topic of humanoids and why they were intrigued by this topic long before they started manufacturing machines which may have aroused the suspicion of being more than just machines: It is the old and great pain humans have always taken to define the difference between thing and man, between subject and object, it is this old pain the discussion about “artificial life”, “bionic robots”, “androids”, “humanomata” etc. is dealing with. It does it in a very modern, technological, avant-garde guise but in its arguments it does not shy away from any metaphysical volte quite like the old discussion about matter and spirit. If the topic “man-machine” is regarded as an anecdotal variant on the old problem of the difference between man and object, a development can be discerned which covers a much wider area than its popular equation with the specific engineering project of creating a humanoid machine suggests. If we consider phenomena such as robots in manufacturing and, more recently, service and medical care robots, diagnosis and therapy computers and other so-called — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

106 107 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

“intelligent” machines including “smart weapons” (target-seeking, feedback-controlled missiles), we might argue that man has made considerable progress in developing humanoid objects. On the other hand, the extent to which objects have been incorporated into the human body is quite striking: hearing aids, contact lenses, pacemakers, coils, synthetic arteries and joints, chemical anti-depressants etc. So we might also argue that man has made some progress in creating an objectified object-like human being. If we identify the same logic at work in this two developments – the logic of an increasing objectual self-substitution of man – in that sense that human beings are replaced not only in the physical and psychological functions which define man, but also in their social function by objects, by machines. – And if we make a step further in this logic, then the following will become conceivable: humans may experience such a degree of prosthetisation and armouring (mechanical reinforcement) with regard to both their organic and interactive competences – thanks to functionally equivalent apparatuses – that eventually the difference between man and matter, man and machine will become blurred. The relationship between man and machine will resemble that of a brain-dead person kept alive artificially by means of machines. There will be more and more events, which can neither be clearly defined as belonging to the world of objects nor to the world of humans. They will belong to both worlds. In the example of the brain-dead person, turning a lever means switching off a machine and killing a human being at the same time. The fading of the difference between object and man which has just been described as an event taking place in the physical, material structure of them, is at the same time, is also – and perhaps even more – an event on the level of cognition. The concrete physical changes taking place in what we used to take as different entities need not necessarily be so great as to make it difficult for us to keep up that difference. It is sufficient that these changes soften somewhat the logic/concept of this difference, – relativising some of its former self-evident criteria. Let me give you one example: “giving an answer” – “answering” has always been regarded as a and perhaps the characteristic of the living world, whereas the world of objects/all matter has always been regarded as mute. This much-quoted “muteness of the matter” (“Stummheit der Materie)”, which is said to have always terrified human beings to such an extent that they eventually resorted to bizarre things such as art and the law of causality in order to animate these objects to make them answer – to get some answer, some meaning out of this frightening load of agonizingly meaningless matter and muck called “ the world”, – this muteness of the matter has disappeared — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Friedrich Wolfram Heubach. Man and Machines — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — —

with the advent of machines. This is at least the impression we get when confronted with all those affable automats and machines which have become part of our everyday lives and which speak to us, ask us questions, and give instructions and information. This is all the more true of “Kismet”, the famous humanoid robot-head from the MIT laboratory, which apparently learns in an interactive way to express its feelings, to answer with mimic reactions comparable to that of a toddler – when addressed by somebody. And in fact, all people who have come into contact with Kismet report how deeply touched they were by the touching way Kismet displays its being touched by the attention given to him in his mechanical face. But is it really the matter, the material, that – in the body of that machine by the name of “Kismet” – finally feels so touched by man that it finally, finally answers him?? No, it isn’t – for all those years the machine had a female German theologian as an attachment figure. What is standing there in front of the deeply affected humans, called Kismet, is not a machine giving answers but a machine reproducing the human patterns/the schemata of “being answered”. It produces a signal, which humans have learned to interpret as an answer to the same degree as in their world of experience a way of behaviour has proved worthwhile which takes that signal for an answer. The reason why anybody who has come into contact with Kismet has developed a human, a human-like relationship with the machine is not because it answered them but because they answered themselves with the help of the machine by responding to their own patterns of “being answered” provided by the machine. It is not necessary for the machine to be able to give answers in order to be regarded as giving answers, just as it is not necessary for a weeping willow to be sad in order to be identified as a weeping willow. All that is required is a formal pattern, a schema of recognition. So is it all a load of rubbish?! No, it is just the widespread woolly thinking of people who think the fact that their conclusions about the world proved themselves worthwhile in the sense that they led to expected effects – that on their level it was possible to affect the world in a favoured way – that this would also prove their truth. In short: the function of the “being answered”-schema, just like the function of all other cognitive schemata, is not so much comprehension of the real, of what is given outside there, let alone comprehension of its truth, but just to activate a reaction which has so far been successful and useful in coping with that reality. The conclusions, which are drawn by means of this schema need not necessarily be correct as long as this schema activates a behaviour which leads to results the acting person experiences — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

108 109 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

as pleasant or useful. And people are perfectly capable of behaving towards a machine or an object as if it was a human being and vice versa, if it seems conducive to their emotional needs or if it fits into their ideas of salvation – that’s not new at all. It does not require a machine like Kismet or people like the Taliban to realise that. From what I have said so far, and from what is mainly said and written about the topic of humanlike machines, one might come to the conclusion that the fading of the difference between man and thing – between subject and object – is something inevitably terrifying human beings. It can and does indeed terrify them, but basically with regard to the way they see themselves or the way they are supposed to see themselves in the theories of philosophers, theologians and other dubious experts on humanity. Outside of theory, in reality, however, this fading has always been a source for both fear and desire. It does not require psychological astuteness to conclude that this fading causes fear on the level of consciousness to the extent that it provides an opportunity to fulfil unconscious desire. In other words: as disconcerting as it can be for a human being, as offending as it may be to his rational mind, when he can’t identify himself anymore in that either-or between being a subject and being an object, his logical thinking is based upon, as willingly he abandons himself to the fading of that difference in his concrete life and does he enjoy his experience of being an object or of being both, object and subject at the same time: Sexuality is the best, but not the only, example for that. Someone riding a bicycle enjoys himself being the blind, reliably working muscle-machine and as the far-sighted pilot who is sovereignly steering that machine. On the dance floor, we enjoy ourselves as this spinning and turning mass and at the same time as the bold master of all that kinetic uproar. In front of a mirror, the subject tries to please itself as an object and we all try to overcome that lifelong boring must of getting up by routines and rituals, by turning us into an as conscious-free as well as an adapted automat. Being a machine has never been only a horror but always a dream of precision and a lustful challenge, too – long before James Brown sang his song “Like a Sex Machine”. And being an object has never been just a nightmare either but has always been a basic principle behind a lot of games (not only children’s games) and a common pattern of many redemption fantasies and practises. And exactly because the subject gets so much relief and lust out of its self-mechanization and its self-objectification (not to mention the benefits, society draws out of it) these dimensions of human behaviour are constantly disregarded and discriminated by all those dumb minds claiming that — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Friedrich Wolfram Heubach. Man and Machines — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — —

the human being is a higher one (and who like to dumb it down to) a spiritual one and therefore insist that man and machines are worlds apart. They never tire of pointing at the unique and immeasurable complexity of the human as the barrier, which condemns every attempt to build a human-like machine to fail. Two remarks on that: First, the complexity they are celebrating like a privilege of humans is something that arouses rather mixed feelings in every more thoughtful person. And everybody with higher demands would very much prefer to see this complexity somewhat reduced, – not only in himself, but in the Taliban and American presidents as well. Second, the barrier to the attempt to construct a human-like machine might be less one of this principally inherent nature but more an economic one. What is the advantage of constructing such a humanoid machine – apart from proving that it is possible – even if it were comparable in its faculties to a fourteen-year-old Indian or Malaysian as long as the prime-costs of this boy or girl are so much less than the costs of producing the robot? My conclusions concerning the general and publicly voiced apprehension about the possibility of human-like machines are the following: There may be sensible reasons behind it all and they might express realistic worries about the future, but basically this apprehension serves to maintain a narrow-minded pretentious view of man. A view which completely denies the lust and comfort humans can find in their self-mechanization and without which there would be neither dances nor armies, neither sexual reproduction nor games, neither grace nor skills. That’s one aspect of the man-machine debate, but there is yet another one: this debate obviously offers a very welcome opportunity to the public: palavering about the future of machines and about the mischief they will inflict upon us humans; people will eventually find words for what they are doing with and to themselves already – making them feel sometimes somewhat uneasy but are far from questioning or changing. So far my remarks on the irrational aspects, to what is psychologically questionable in the current ideas about the evil future, the fading of the difference between man and machine will lead to. These irrational motives behind the pessimistic scenarios of what future humanlike machines might bring, are regularly criticised by those thinkers and scientists, who like to think of themselves as sober representatives of rationality, open and free of prejudice towards technological matters. For them the possibility of humanlike machines mainly offers marvellous chances to free mankind of long- enough endured — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

110 111 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

limits and burdens. Which may be true, but are the sources of that inspiration and euphoria they manifest in their scenarios of the future really more pure, less obscure than those of the apprehensions and the anxiety this future induces to others as described before? Is it really nothing but the interest into the technically possible /feasible, is it only the intellectual challenge in developing increasingly human-like machines, what makes them so enthusiastic about it? It is – I frankly admit – somewhat audacious to question that here, at this meeting of robotic-fans and other hardcore technology-enthusiasts. But as unlikely as it is that anybody here will consider my arguments seriously, as free I am feeling to develop them in quite an unserious way, that is to say babbling about occurrences and conditions, lying long, long, very long behind us. In the beginning, in paradise, in the Garden of Eden, when man was still inno­ cent he was a pure machine, an artful gadget of God or – so to speak – a machination of God. An apparatus out of God’s hands, driven by His breath and running strictly following God’s commandments. Then came the well-known Fall of Man and as a punishment for disobeying his operating-manual God condemned this machine “man” to operate, to steer himself by and on its own: He condemned him to be an automat from now on, a machine which steers itself. Seen that way all the fantasies of regaining Paradise became readable as the dream of automats with steering-problems – as the dream of eventually becoming a machine again, to be once more that apparatus, functioning firmly determined by God’s will and being free, being redeemed from the tedious tasks of self-regulation and auto-determination. Could it be, and this suspicion could come up, that in these supposedly science-based technological scenarios about the fusion of man and machine only very old, just weakly secularised fantasies of redemption are at work? But there exists another variant of the dream humans have about their happiness in or as a machine: the womb or uterus fantasies. In these dreams, the automat “man”, weary of the routines of self-regulation, overstrained by the task to run himself, imagines himself back into the uterine symbiosis where he was just an ego- and task-free submachine of the great mom-machine, complaisantly determined in its closed loop system. Could it be, and this suspicion is not that far-fetched, that in these supposedly science-based technological scenarios about the fusion of man and machine very primitive, however highly technologified symbiotic fantasies are at work? I don’t want to deepen or prove that suspicion, but please read the chapter “der bio-adapter” in Oswald Wiener’s book: “Die Verbesserung von Mitteleuropa”, where — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Friedrich Wolfram Heubach. Man and Machines — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — —

Wiener not only anticipated many of the man-machine topics popular today, but with the stringency and malice of its argumentation this text outruns most of today’s visions by large and in its ingenious, eye-opening perversion equals the idea Samuel Butler developed in the two chapters “Book of the Machines” of his book “Erewhon”. – And just like it, it is far from receiving the attention it deserves. In his text Wiener describes a technical device, a machine-like installation, the “bio-adapter” which adapts itself to an individual, gradually taking over more and more of its functions, at the same time improving them, substituting more and more what this individual experiences as the world, and in doing so shaping it closer to its expectations, needs and wishes, so that this machine “forms a link between the inadequate cosmos and the unsatisfied human being” and eventually permanently bridges the gap by a complete fusion of that adapter with everything it was adapting to. The description Wiener gives of this final state is nothing else than a description of man’s past in mom’s womb: the technological realization of a uterine ideal of life. (Wiener himself makes a short remark in that sense.) The only irony is that this state achieved towards the end of a long technological development by means of the “bio-adapter”, blurring the borderline between man and world and bringing the “perfect solution of all the problems in the world” – that this symbiotic state marks the very beginning in the development of a human being and it creates heavy problems to the extent that it is not surmounted. One might not share the suspicions I brought up, that these supposedly rational and strictly technological scenarios of the final dissolution (the happy end) of the border between man and machine might follow oldest human fantasies of redemption and symbiosis. But at least the recourse on early infantile conditions offers an interesting, heuristically fruitful perspective to understand the extremely contradictory feelings and attitudes towards machines, towards all machinery. If we consider that a new-born baby, yet far from being a “person”, can only perceive the mother as a responding, feeding, nappy-changing, dandling and warming programme which, long before this maternal event-complex can be grasped by the child as a personal unit, must be for quite a long period nothing more than an physical unit of repeated actions, meaning a machinery in the very sense of the word. If we further bring to mind, what not only the psychoanalyst Eric Ericson has said about the mother’s feeding and caring behaviour, about the importance its regularity has for the development of what he calls “primary trust” or “primary distrust”, – then the early experiences made with that reliable or frustrating maternal machinery might explain why the image of machines and all machinery is of this deeply ambivalent — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

112 113 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

emotional quality. But this ambivalence has still other aspects. When one sees that most artificial man-made creatures featured in literature and myths are female, one could be tempted to see in the ambivalent attitude towards machines something of this ominous ambivalence towards woman often dealt with in literature. But let me leave the field of speculations and come to what might be drawn as a conclusion. Like many of the so called “great” or “fundamental” differences, as for example the difference between man and woman or that between evil terrorism and holy missions, the difference between object (machine) and man is not as deeply grounded as it is still generally believed today. From a moral and educational point of view it might have been sensible to pass the difference as fundamental and at least that was done with good intentions. From an aesthetic point of view the difference has always been negligible, and intellectually the difference was only as great as it offered heuristic benefits. But if one accepts for once the profundity of the difference and also the apprehension its abolition arouses in many people, then one has a good advice yet to give them: Instead of torturing themselves with the question where the border between man and machine is exactly running and instead of coming up with ever more metaphysical subtleties, they should devote themselves to a less complicated but much more catchy question: From which side will the border be crossed one day? Won’t humans be like objects or machines rather than objects or machines be like humans sooner or later? How ever it will be, the leave the inability to distinguish any longer between man and machine would force us to take, wouldn’t be the leave from that difference – we still could achieve it by our behaviour and in our mind – but it would just be the leave from the idea that we could find this difference in the world, from the idea of it as an one given ontologically. But why continuing making it, when it doesn’t “exist”? This question leads us to the only solid but very human reason for this difference and its making, - and to the most intelligent argument against its fading: it entertains. Whatever has been said about it and whatever value has been attached to it, it is all very, very far from the empirical evidence all this fuzz about the difference gives so amply to its entertainment-value. What I want to say about this difference between man and machine and its very human raison d’être (which by the way applies to the other great differences mentioned here, too) leads me to my final assumption: A machine will prove to be humanlike only when this machine begins wondering when confronted with a man or a machine: — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Friedrich Wolfram Heubach. Man and Machines — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — —

Is that a machine or is that something like me, a human? – And when in the end this machine appreciates the entertainment gained from that question more than any answer eventually given to it. – Shouldn’t we have something to learn from that machine?! — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

114 115 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

WAndrea p Sick §>

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

:!!

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

!

rrrQuestions of Style Subjects, Things and Shared wAgency in Popular m Articulations

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Andrea Sick. — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Questions of Style —

Subjects, Things —

and Shared Agency —

in Popular —

Articulations. —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

If we consider the relations between subjects and “things” and, in a context of technical mediation, seek to represent this relationship as one of shared agency, describing its manifestations is also a question of style. Because style, I should like to contend, is a sharp instrument that may promote a vulgarisation of science. Style brings implementation and representation to such articulations as the popular RoboCup competition with its even more popular Sony Four-Legged League, Honda’s Asimo humanoid, and the cuddly pillow robot known as “The HUG”. The question is to what extent such articulations elude or even ensnare the dichotomies of subject and object, natural and artificial, man and machine, rather than overcoming them. How does the space open up between different styles that find their echo in the different tones of different disciplines? To what extent do these echoes create a polyphony that ultimately results in — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

118 119 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

a vulgarisation of science? In what follows, I should like to address these questions of style from two angles: The first part of my text will discuss the term “style” with regard to its etymology. The aim is to clarify “style” as a set of instruments and to identify the challenges it poses for interdisciplinary approaches of the kind practised in all projects and research questions presented here. In that sense it is not immediately surprising that style, traditionally, is particularly significant in artistic processes. The second part of my text spells out questions of style. For our present purpose I will concentrate on two fields or aspects of representation. Based on these questions, I will discuss technical mediation – a recurrent concept in Latour’s network of shared agency – and its stylistic effects in an ambivalent context: The anthropomorphic style (in robotics) and the style of vulgarised naturalisation (of science in the field of robotics). Style

As mentioned earlier, stylistic effects are mostly attributed to the arts. Style seduces, eludes and ensnares. In this view, it is the instrumental opposite of the objectivity that generates technical and scientific facts. So if we consider style as an instrument that is indispensable for producing the object or rather, things, the etymology of the word “style” points to this semantic field. In German, Stil is documented since the fifteenth century and derives from Latin stilus (stylus) both referring to a pointed object, stake, stem, stylus. It can also be a spur, quill, dagger or stiletto. One point of comparison between concrete and figurative meaning is, for example, that it was evident from a piece of writing how (i.e. with which stylus) it had been written. French philosopher Jacques Derrida, who has explored the etymology further, says: “The épéron, which is translated sporo in Frankish or High German, spor in Gaelic, is pronounced spur in English. ... the English spur, the épéron, is the “same word” as the German Spur: or, in other words, trace, wake, indication, mark.” (Derrida, Spurs 39-41)

If style refers to a purpose-bound way and means of written and verbal expression or a characteristic form of expression in art and culture, it characterises the way something is done. The term “style” stands for a characteristically distinct way of performing human activities or the manifestations thereof. It emerges through the – not always conscious, but always coherent – choice, assessment and application of a distinctive manner or technique by which something is done, created or performed (Merriam-Webster). The modus. It is subject to continual change. There are styles in — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Andrea Sick. Questions of Style — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — —

art, writing, painting, music, interior design. There are styles of thinking, programming styles, fashion styles and lifestyles. Derrida’s analysis and critique of ontology led him to tackle “Nietzsche’s styles” with undermining reflections on appearance. He points out that styles can protect and veil, abstracting the object from the Here and Now. Shielding it. Style describes the How of what – following Heidegger’s essay “The Question Concerning Technology” – could be called entbergen (revealing, disclosing, bringing out of concealment). In that sense technology itself, which Heidegger distinguishes from “anything technological” and refers to as Ge-stell (enframing), could be called style. “Enframing means the gathering together of that setting-upon that sets upon man, i.e. challenges him forth, to reveal the real, in the mode of ordering, as standing-reserve. Enframing means that way of revealing that holds sway in the essence of modern technology and that is itself nothing technological.” 1 (Heidegger 302)

Following Derrida we can conclude: Style is the challenge to approach a line of thinking, not as metaphysics does, according to a logic of opposition, but according to an undermining, supplementary logic of appearance. Which stylistic effects emerge in an interdisciplinary network as described by Latour? The theory developed by the French philosopher of science Bruno Latour also reaches beyond concepts of opposition and in particular beyond the dichotomy of people and objects. Instead, Latour proposes the concept of shared agency, which cannot be brought in line with the style of any particular discipline. Crossing borders, he explores disciplines ranging from art to the philosophy of science, media studies, philosophy and ethnography. He also translates style as tone when he speaks of the analysis of different objects and calls for an appropriate style for each. Discussing a comic strip, a steel bowl, an article by Louis Pasteur or a religious painting, he says, requires different terms and an appropriate tone. Adjusting the style offers the possibility of ensuring that each object is treated with respect, and prevents the same problematics from being carried over from one object to the next (Latour, Technische Vermittlung 29-64). Yet despite his call for a change of tone depending on the object, Latour repeatedly emphasises that his concept is based on the assumption that we live in laboratories. He says there is no difference between the world inside the laboratories and outside of them, and I think that is a very important point. The sharp distinction between research laboratories inside whose walls theories and phenomena are tested, « — — — — — — — — — — — — — — — — — « — — — — — — — — — — — — — — — — 1 Heidegger adds: „The word stellen (to set upon) in the name Ge-stell not only means challenging. At the same time it should preserve the suggestion of another stellen from which it stems, namely that producing and presenting (Her- und Darstellen) which,

in the sense of poiesis, lets what presences come forth into unconcealment.” (Heidegger 302)

120 121 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

and an outside political situation in which non-experts have to get by with judgements, opinions and passions, says Latour, gets blurred if we take the concept of shared agency seriously (Latour, Tatsachen 20). Latour’s own style is often described as popular, experimental and witty: “All our cherished boundaries, us and them, modern and pre-modern, the West and the rest, are nothing but self-delusion, meant to dig a trench where no trenches are. Our distinctions all follow straight lines manifested in the number and stability of so-called actants in networks. What we take for modernity is nothing but a string of safety-pins, tape-measures and particle accelerators. However, since all these things have a part in the way the world is going, they should have a say and a right to participate in the ‘parliament of things’.” (www.readme.cc)

Latour makes a stand for hybrid forums in which styles mingle and boundaries are crossed. He assumes that within these forums there is a technical mediation that divides action up among the different actants. Actants can be human or non-human. Mediation refers to an event and is always related to other events. The shift of tone, the hybrid forums, would entail a shift in the action of language which, if we follow Nietzsche and Derrida, is nothing but illusion, its performative force being seduction. Nietzsche writes in “The Gay Science”: “This has given me the greatest trouble and still does: to realize that what things are called is incomparably more important than what they are. [...] Originally almost always wrong and arbitrary, thrown over things like a dress and altogether foreign to their nature and even to their skin [...] We can destroy only as creators. – But let us not forget this either: it is enough to create new names and estimations and probabilities in order to create in the long run new ‘things’.” (Nietzsche 84)

What does it mean to hit the tone appropriate to the object? If it already exists or if it were just created? What does it mean to respect the object if it is itself made manifest as a stylistic effect? If it does not make its appearance or gain representation unless there is style? Is it not, for example, style that mimics the apparent respectability and objectivity of science? Is it not style that veils the thing which in itself eludes percep— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Andrea Sick. Questions of Style — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — —

tion? The stylistic effect is the distance necessary for a network of “things” to unfold, making visible the things as such. What object could there be, were it not for mediation, terms, style? My aim here however is not so much to explain as to suggest possible questions. Questions about styles will not enable us to join the choir promising interor transdisciplinary salvation. They do however enable us to sharpen those spurs and perhaps perforate the veil cast by style – even if no wide vistas open beyond, only a glimpse of new effects. Derrida has shown that the question of style is a question of writing, a question of a spurring or spurning operation that is stronger than any content, any hypothesis or any meaning. The sharpened spur tears the veil not only to perceive or generate the thing itself, it also dissolves the very opposition that Bruno Latour is also seeking to circumvent. Doesn’t that entail the destruction of a fetish, Derrida would ask. Latour therefore proposes factishes, types of action which do not fall into the choice between fact and belief but achieve a combination of facts and fetishes. Both fact and fetish have a common element of fabrication, fabrication also being an element of the spurring operation. Style is how the objects of scientific study are constructed or fabricated. We could say that the objects of scientific study have no natural existence, they have to be constructed, so that nature is also a stylistic effect. The objects studied in the humanities and in the natural sciences thus have more and less extensive histories of their own. They are implemented at different levels and for different periods of time in the various fields of knowledge, where they unfold strong or less strong effects (Rheinberger 270-294). Bruno Latour takes this thought to its logical conclusion: Because not a single scientific fact can be considered as given, because each fact is constructed in a concerted effort by the (scientific) community, we would do better to cultivate a certain readiness to work with hybrid explanations. That also involves mixed styles which strike different tones and whose veils pile up on each other like the skins of an onion. Let me illustrate this with two questions of style. They concern both representational issues and the way the objects of science are constructed. They concern anthropomorphism and the vulgarisation of science. Anthropomorphism: A Question of Style

What is the function of anthropomorphism? This question is inherent in the many fantastic tales about automatons and, since the last century, robots, as well as doppelgaengers and dream images. It is the point of departure for the description of manmachine relations, even though it implies the kind of dichotomous modern concept, which Latour advises us to avoid. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

122 123 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

Anthropomorphism is predicated on two principal concepts: a. The machine is an apparatus that saves human effort by generating and

transmitting power. b. The machine functions as a model for explaining human capabilities.

These two concepts are based on three possible explanations of the relation between man and machine: I. The first explanation assumes an ontological difference between people

and technology, which is based on categorical distinctions like artificial and natural. The difference between natural and artificial is less one of form than of genesis. What is generated by art has its origin in a generating agent. In the case of technical artefacts this explanation would therefore seek to establish their essence previous to existence. II. A second explanation assumes and seeks to establish functional equiva-

lence: man and machine have the same function. However, it is also assumed that technology serves to extend human capabilities. While the first concept seeks to level difference by emphasising similarity of form, the second concept emphasises similarity of function. III. Now, a third possibility is to use technical artefacts as models for explain-

ing human capabilities. The technical emerges as a medium in the process of modelling and discovery. Following this approach opens the door to fears that technology may one day overwhelm us humans. This is a position Latour identifies in Heidegger. All three explanations assume a functional correspondence between man and machine. This requires the machine or technology to be deceptive (Krämer 208-221). If, for example, a huntsman’s trap is to be effective, it has to fool its prey (Krämer 211). The Turing test also defines deception as a sign of success: if the machine is functioning properly, it can convince an interlocutor that it is in fact a human being (Krämer 212). Another example is Vaucanson’s legendary mechanical duck (see Gendolla, this book) which was even able to demonstrate its digestive functions. These deceptions and wiles, which can be referred to as a modus of style, are a prerequisite for the machine to be put in motion. Philosopher Sybille Krämer has shown that cultural techniques (for example symbolic machines), the machine we have inside, can undermine the ontological difference or perhaps indifference of man and machine. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Andrea Sick. Questions of — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 01 02 03 04 — — — — — The HUG Pillow Proxy — — — — — — — — — — — — — — — — — — — — — —

Style — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

That would also mean interpreting human activities as incorporations of technical processes. What I will do here is to present different styles that seek to achieve anthropomorphism. More specifically, I will juxtapose replacement or prosthetic concepts, which aim at similarity in form or function, helpers designed to replace a specific human activity, with adaptive, autonomous ecosystems. Links to artistic or aesthetic issues, commercial (merchandising) and scientific concerns may also emerge. The Hug Pillow Proxy2

The HUG was developed by a team of designers from various disciplines at Carnegie Mellon University. The project, which was publicised in numerous popular magazines. This is how the researchers and developers describe it: “The Hug presented here is a challenge to the familiar telecom products. A visionary design born out of research with elders, the Hug addresses a very human need for physical closeness in remote communications. It uses the same network infrastructure as many appliances today, but places a new face on human product interaction.”

So the project is aimed at replacing intimacy and care, at fulfilling human needs. This is achieved by functional similarities – the pillow gives you a hug – conveyed through the visual and haptic resemblance to “human arms and torsos” and the familiar pillow shape. The HUG is soft and organic. Anthropomorphic qualities suggest that the product should be held in the arms. It seems that the form fits the human body comfortably, facing forward towards the upper torso or backwards sitting in the lap. This is what the developers write: “We believe as designers, that we can shape future robotic products to be more appropriate and sensitive to the human « — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 2 The Hug. 10 April 2007 .

124 125 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — experience. […] The Hug facilitates remote voice and tactile communication between distant family members and friends. […]”

To illustrate how The HUG accomplishes this goal, they offer the following scenario: “Mary lives on her own in a one-bedroom apartment. Her daughter Jean, who she is close to, lives many hours away. Mary feels frustrated that she is not able to share in her grandchildren’s day-to-day experiences of growing up. Her daughter Jean worries about Mary being alone. The Hug helps Mary feel more connected to Jean and her children. Mary and Jean each have a hug and can use it to communicate and send ‘physical’ greetings to one another.“ […] Communication is initiated by squeezing the left paw, and speaking the receiver’s name into the microphone. Once a connection is established, senders can squeeze, stroke, hug or pet the Hug, activating pressure sensors and accelerometers inside. Communication is terminated by squeezing the right paw. The base stores caller information that is used to create a Hug network. A Hug network is a closed set of people. Hug info for this group is programmed into the base using a memory card or a standard telephone interface.”

The HUG sets up a network of technical mediations while at the same time resulting from a style that assumes the functional and representational similarity of man and machine, which is used to further the project’s commercial success. This assumption is the style, the instrument that raises and penetrates the veil. The Tortoises

The tortoises (originally Machina speculatrix) constructed in 1948/49 by neurophysiologist William Grey Walter were electromechanical robots that reacted to their environment. In a paper on cybernetics and philosopher of science Andrew Pickering describes the tortoises as a non-modern experiment (Pickering). Though not originally conceived of as art, the tortoises were in the 1960s identified by art historian Jack Burnham as the starting point of a new style in art that saw sculpture as a system (Conrads). The tortoises “showed how really rather complex forms of structured behaviour could emerge from some very simple circuitry” (Pickering 8). They soon became objects of scientific inquiry in their own right. The idea was to approximate and explore the structure of the human brain by constructing an electromagnetic model. The — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Andrea Sick. Questions of Style — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 05 06 — Mirror Dance — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — * 07 — — Mating Dance — — — — — — — — — — — — — —

tortoise “materially exemplified a theory of the brain and how it works” (Pickering 6). It is a performative and embodied brain shaped as a turtle. Pickering describes the tortoise’s brain as one that functioned as a switchyard between the motors and the sensors, and not, importantly, as a hierarchical controller running the show from above. In the absence of the sensors or the motors, the tortoise’s brain was just a handful of inert components (Pickering 6). [Examples for its performative aspect are (* 05 06)] Walter described how a tortoise passing a mirror would be attracted to the reflection of its own light, which would then be extinguished as the tortoise locked onto its image. The light would reappear as the scanning rotation of the front wheel set back in, attracting the tortoise’s attention again, and so on. The tortoise would thus execute a kind of mirror-dance, “flickering, twittering and jigging like a clumsy Narcissus.” (Walter quoted in Pickering 7). The connections between purpose and environmental feedback are obvious. The tortoises also reacted to one another, executing a kind of mating dance in which they would repetitively lock onto and then lose interest in one another. (* 07) In contrast to earlier approaches in the artificial intelligence movement, the brain was here understood as a performative, embodied organ. Pickering rather briefly says that “while AI was antidualist in one sense – it sought, after all, to simulate human intelligence on a machine, the computer – in another sense AI beautifully reproduced a common-sense mind-body dualism, by casting cognitive processes as essentially representational, as other than bodily processes, and as hierarchically controlling the latter.” (Pickering 7). Pickering points out that Walter himself recognised the implications of the mirror dance, namely, that “the tortoises could be seen as performative technological artefacts as well as models of the brain […], an adaptive system in the sense that it reacted in real time to the circumstances in which it found itself. The complexities of a tortoise’s behaviour emerged from its engagement with a complex — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

126 127 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 08 — — — Lego Tortoises — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

environment and varied with the specific contours of the environments in which it found itself; they were not built in by its designer; they were not programmed or hardwired.” (Pickering 9). The tortoise machines had a lively, purposeful character, “as if the thermostat wants to keep the temperature constant come what may”. And, Pickering adds, “we should also think about temporality here” (Pickering 10). The tortoise creates its ecosystem and explores the existing one. The tortoise itself becomes a style of brain science. Why is it, however, that the tortoise was as popular then as the RoboDog is today? If so many people have heard of the tortoises, it is not because of their impact on the sciences of the mind and the brain or because of their unusual style of engineering. It is because they were fun. Walter put them through their paces and people liked them. He demonstrated the first two tortoises, Elmer and Elsie, in public in 1949 even though they were still quite unreliable. Three of the tortoises built by Bunny Warren were exhibited at the Festival of Britain in 1951, others were shown regularly throughout the 1950s (Pickering 11) (* 08). Despite their scientific purpose, the robot tortoises were objects of popular entertainment much like their canine counterparts are today. Their popularity may be ascribed to the anthropomorphism attributed to them: people knew they were machines, but they behaved like living beings, pursuing goals, dodging obstacles, pursuing… Pickering describes their effect as one of wonder: the tortoises transgressed the boundaries between “the animate and the inanimate, the organic and the inorganic, the living and the dead” (Pickering 12). Or perhaps one might say that they evoke this sense of wonder because they remind us of something we have always known and suppressed, in particular because they react to their environment. In that case, we could also speak of the uncanny in the sense coined in Sigmund Freud’s discussion of E. T. A. Hoffmann’s tale “The Sandman” (Der Sandmann). — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Andrea Sick. Questions of Style — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Walter was a neurophysiologist, a pioneer in the field of electroencephalography and a published novelist. The tortoises made him an engineer, a brain scientist, writer and TV-pundit – predestined, as it were, for the kind of popularity designed into projects and prototypes like The Hug. Following Bruno Latour, we might describe this popularity as a style of the vulgarisation of science. Vulgarisation and Science: A Question of Style

“Vulgar” derives from Latin vulgaris, mob, common people. “Vulgar” can describe something generally used, applied, or accepted. From the nineteenth century onwards, however, it came to mean simplified, not scientific, superficial, and that is the semantic field I would here like to look at. Latour somewhat cryptically distinguishes between the “Gay Science” we know from Nietzsche, the Science in Action, and its vulgar representation in the press. He described their relation in an interview conducted by Gustav Roßler in 1997. Roßler pointed out that in his collection of essays “Le Clef de Berlin” Latour rose up in arms against the “vulgar” representation of science in the press and even called for a constitutional right to ignore the sciences. Latour explained that he used “vulgarisation” to object to the representation of science as ready-made facts, as scientific findings, or as he calls it, matters-of-fact. Rather than making science public, he said, the dilettanti of science move in on science from the public domain. Science as ready-made facts implies transcendence, with no relation to the practice of scientists, and for Latour this vulgarisation is a way of saying “Let the debate cease!” (Latour, Roßler 43). A vulgar view of science emerges where science is presented in terms of immutable facts and a firm belief in them. In a rival concept Latour proposes a system of humans and non-humans whose actions position them in a network where propositions are temporary and science emerges as a risky, passionate undertaking. In that sense one could also define “vulgar” as referring to a style that generates immutable facts in the first place and bolsters a transcendence in science. In an essay entitled “From Realpolitik to Dingpolitik – or How to Make Things Public”, Latour writes: “Transparency and immediacy are bad for science as well as for politics; they would make both suffocate. What we need is to be able to bring inside the assemblies divisive issues with their long retinue of complicated proof-giving equipment. No unmediated access to agreement; no unmediated access to the facts of the matter. […] Why should we suddenly imagine an eloquence so — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

128 129 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — devoid of means, tools, tropes, tricks and knacks that it would bring the facts in the arenas through some uniquely magical transparent idiom?” (Latour, Realpolitik 12)

Now, if vulgarisation describes a technique that simplifies things in order to make them more accessible, vulgarisation also lends itself to putting disciplines into what might then look like dialogue – not in order to bring up new structures and questions, but to confirm existing ones. In the interview with Roßler, Latour says: I don’t really believe in disciplines. Nevertheless, there is no confusion of relations for me. I use them just as I use the different styles which we talked about earlier, as a function of the different questions. As you know, laboratories always have different instruments and methods, used depending on the type of experiment that is being conducted (Latour, Roßler 45). Following Latour’s theory we may assume that the choice of technique results from the networks themselves because, Latour says, there is no outside of the collective, outside the circuits of give and take. In the interview with Roßler he says, “in regimes of proposition it is important to differentiate in terms of characteristic modes of association […] I for one have no qualms about differentiating, you can’t measure everything by the same yardstick, but this differentiated view is predicated on networks and nervatures rather than fields, spheres or surfaces.” (Latour, Roßler 48).

By mapping the heterogeneous kinds of association empirically, it may be possible to open up a complexity that outwits the vulgarisation of science as demonstrated for example by The Hug. That circumvents it and unfolds stylistic effects in the network of instruments, techniques and texts, which themselves unfold their effects in the network and resist the pull towards “established findings and immutable facts”, towards a readymade science. Such a complexity could not be opened up by establishing a new system of experts functioning as mediators for the rest of society. In heterogeneous networks, Latour suggests, the expert position is continually undermined, with eloquence being one of the undermining factors (Latour, Tatsachen 33). The question of style relates to a network in which nothing is beyond question and in which the different areas are involved in processes of mediation whose outcome has always needed to be explored. Facts are revealed to depend on style, on the question of how to make things public, or, in Latour’s words: “Matters-of-fact now appear to our eyes as depending on a delicate aesthetic of painting, drawing, lighting, gazing, convening, something that has been elaborated over four centuries and that might be — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Andrea Sick. Questions of Style — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — —

changing now before our very eyes.” (Latour, Realpolitik 13). Latour emphasises that facts can only be registered in the minutes. Hence it is not surprising that in his view, there is no going back, no trusting in the transcendence of nature or rationality to save (Latour, Realpolitik 32). That which we could call nature would also emerge as style, the sharp object determining the modus that it is in itself. “No innovation without representation” (Latour, Tatsachen 35) could be the slogan that would also shatter the dream of naturalising all social relations. The vulgarisation of science, which is predicated on immutable facts, will show that none of these processes are independent of the increasing commercialisation of scientific processes, which affects both the generation and the administration of what we call knowledge. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

130 131 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — References Conrads, Martin. “Menschen, Maschinen, Schildkröten – Kybernetische Kunst im Zeitraffer.” de:Bug. 25 August 2005. Last access 1 Juli 2006 . Derrida, Jacques. Spurs. Nietzsche’s Styles / Éperons: les styles de Nietzsche (English/French edition) Trans. Barbara Harlow. Chicago, 1979. Heidegger, Martin. “The Question Concerning Technology“. Martin Heidegger – Basic Writings. Ed. David Farrell Krell, London, 1978. Krämer, Sybille, Maschinenwesen, “Ein Versuch über den Anthropomorphismus in der Technikdeutung hinauszukommen“. Autonome Maschinen, Eds. Thomas Christaller, Josef Wehner. Wiesbaden: Westdeutscher Verlag, 2003. 208-221. Latour, Bruno and Peter Weibel. Introduction to Making Things Public – Atmospheres of Democracy, Exhibition. Catalogue, 2005. English citations are to , 10 April 2007.

Andrew Pickering, “The Tortoise against Modernity: Cybernetics as Science and Technology, Art and Entertainment.” 30 May 2006. English citations are to . Friedrich Nietzsche. The Gay Science. 30 May 2006. Translation from . Read me, Book tip: Wir sind die modern gewesen. Versuch einer symmetrischen Anthropologie, Bruno Latour, 10 April 2007, Hans-Jörg Rheinberger. “Cytoplasmic Particles: The Trajectory of a Scientific Object“. Biographies of Scientific Objects. Ed. Lorraine Daston. Chicago and London, 2000. 270–94. The Hug. 10 April 2007 .

Latour, Bruno. “Über technische Vermittlung, Philosophie, Soziologie, Genealogie“. Technik und Sozial­theorie. Ed. Werner Rammert. Frankfurt am Main. New York, 1998. 29-82. Latour, Bruno im Gespräch mit Gustav Roßler. “Ein neuer Empirismus, ein neuer Realismus“, Mittelweg 36.1 (1997) : 40-52. Latour, Bruno. Von der Realpolitik zur Dingpolitik. Berlin: Merve, 2005. Latour, Bruno. “Von Tatsachen zu Sachverhalten. Wie sollen die neuen kollektiven Experimente protokolliert werden?“, Kultur im Experiment. Eds. Henning Schmidgen, Peter Geimer, Sven Dierig. Berlin: Kadmos, 2004. 17-37. Andrew Pickering, “Mit der Schildkröte gegen die Moderne: Gehirn, Technologie und Unterhaltung bei Grey Walter“. Kultur im Experiment. Eds. Henning Schmidgen, Peter Geimer, Sven Dierig. Berlin: Kadmos, 2004. 102-123.

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — »

Andrea Sick. Questions of Style — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — Images * 01 02 03 04 Carnegie University, The hug http://www.cmu.edu/research/centers.shtml#Robotic. Download: 2 June 2006 * 05 Grey Walter‘s Tortoises „Machina Speculatrix“ http://www.cerebromente.org.br/n09/historia/ documentos_i.htm Download: 15 May 2007

* 06 Grey Walter‘s Tortoises „Machina Speculatrix“ http://socialfiction.org/?tag=cybernetics Download: 2 June 2006

* 07 Grey Walter‘s Tortoises: „Mating Dance“ http://jwgibbs.cchem.berkeley.edu/science1951/ speculatrix.html Download: 15 May2007

* 08 Lego Mindstrom Version of Grey walter‘s Turtles http://www.bcp.psych.ualberta.ca/~mike/Book2/ Robots/Walter/index.html Download: 15 May 2007

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — —

132 133 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Jutta Weber >

F

W — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

W

»You See the Videos But I Know How the Machine Functions«. Human-Robot Inter­action and Its Media Representations

g h pp

p

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Jutta Weber. — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

‘You See the Videos —

But I Know How the —

Machine Functions’. —

Human-Robot Inter­ —

action and Its Media —

Representations — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — “the image, the imagined, the imaginary -- these are all terms that direct us to something critical and new in global cultural processes: the imagination as social practice. […] The imagination is now central to all forms of agency, is itself a social fact, and is the key component of the new global order” Arjun Appadurai

Look at all the fancy websites in personal service robotics. We see beautiful (female) roboticists repairing their sophisticated robot systems or colourful robot cubs exploring the world. There are sweet baby seal robots being petted by happy and engaged seniors in a home for elderly people or smart assistant systems are busy to bring — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — —

136 137 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 01 * 02 * 03 — — DFG-Group Learning Humanoid Robot Paro with elderly people; Robotcub by — Robots, Univ. Freiburg Shibata et al. Eu-Project Cogniron — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

orange juice to TV watchers in the living room. This imaginative kind of staging of new personal robots is not (only) part of commercial robot companies but also part of the business for scientific research and academic working groups. Surfing many websites of personal service robotics during my research I was quite astonished about the ubiquitous media representation of fancy artefacts and engaged scientists. At informal occasions at robotic conferences or workshops I could listen to many roboticists complaining about the growing workload through the pressure to present their work in professional and appealing ways. Science communication would hinder them from working properly, having all these journalists around asking for interviews, giving tours all day through their lab, sitting at photo sessions presenting their latest prototypes. In personal communication they complained that media representation is going to consume a growing amount of their time and energy which they would prefer to spend on their ‘actual’ research. At the same time the investments in popular and successful representation of robotic labs are growing rapidly. You find more and more robotics labs where photographers, artists and web-designer are hired to support the presentation of latest research and to make their performance more professional. Shining websites, press releases and flyers are produced in which cute-looking artefacts, innovative researchers and their latest breakthrough strategies are performed. Science Festivals like the annual robot soccer competition RoboCup, Science Fairs, Science Museums and artists in residence are other popular strategies and events to attract public attention and to heighten the performativity of one’s research and latest artefacts. Robotics – like Artificial Intelligence before – is quite effective in commodifying its latest achievements and visions. The growing interest in media representation – at least in robotics – seems to be predominant in the area of personal service technologies. Personal robotics focuses on robots for leisure, entertainment, therapy-assistance, the household or care. In this — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Jutta Weber. ‘You See the Videos But I Know How the Machine Functions’ — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

field the every-day user (or end developer) comes into play for the first time in the history of robotics. And this is also the reason why Human-Robot Interaction works a lot on strategies and mechanism to immerse the user in the interaction with (personal) robots and to find ways to build up social and emotional relations between its artefacts and its users. In contrary to personal service robotics, industrial robotics as well as professional service robotics have been fields, where traditionally ‘hard’ technology was developed for industrial and military applications. These robotic systems were relevant for experts only (engineers, computer scientists, funding agencies, military, industry, etc.). But beside the new interest in every-day users and the media-friendly shaping of artefacts (humanoids, zoomorphic artefacts) to immerse these users, I think there are other general dynamics in the culture of technoscience that support the increased interest in media representation in technosciences such as personal service robotics. Recent (techno)science policy, the perception of technosciences as part of culture as well as the growing impact of the techno-imaginary in the entertainment industry and everyday life are also main factors which lead to a greater attention towards media representation in new and emerging technosciences. In the following I will introduce the field of personal service robotics or HRI, sketch some of its visions as well as problems and present some preliminary thoughts on the growing impact of media representation in Human-Robot Interaction and technosciences in general – a development which got some attention in Technoscience Studies (Haraway; Franklin) but rarely in media studies up to now. Human-Robot Interaction: Computing the Human as Socio-Emotional

Over the past years we can observe a profound paradigm shift from rational-cognitve concepts, algorithms and top-down modelled automats towards socio-emotional interaction as well as a loss of distance towards tight couplings of system and environment, machine and humans, in the field of artificial intelligence, computer science and especially robotics (see Hayles; Crutzen; Wegner). For example, early Artificial Intelligence (AI) concentrated on symbol processing, while behaviour-based robotics from the late 80s of the 20th century on orients itself more towards biologically-inspired approaches which “played down the personification of machines” (Suchman 2). Today, we experience a shift towards socially-inspired AI and a new interest in the interaction between humans and machines. In the field of Human-Computer In— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

138 139 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

teraction, the invention of the desktop, mouse and icons are part of this development, which shows a clear tendency to hide the complexity of the machine towards the user in favour of supposedly attractive interfaces with ready-made functions. This trend is perpetuated in Human-Robot Interaction. Industrial and professional robotics had been a field for experts only where industrial robots and professional devices were developed. These were programmable machines that didn’t offer any challenges in terms of human-computer interaction. Everyday users didn’t come into play. But since the mid 90s Human-Robot-Interaction (HRI) has become an important and rapidly growing field in AI. HRI is an interdisciplinary field in-between robotics, AI, cognitive science, (developmental) psychology, user testing, biology (esp. ethology), and partly sociology. HRI develops robots not for professional or industrial applications but for the personal service economy (entertainment, education, therapy-assistance, household, care, etc.). Therefore today’s personal service robotics faces the same challenge like Human-Computer Interaction: They have to take the user and private consumer into consideration when conceptualizing, designing and developing their systems. As the economic importance of personal service robotics is rising, HRI is slowly and increasingly regarded as an important field and is more and more accepted by the robotics community (see Kiesler and Hinds; Rogers and Murphy). For example, some people even regard the field of personal service robotics as the new candidate for the next digital revolution. Bill Gates amongst others is speculating whether the personal robot will be – similar to the PC in the 70s – the next multi-media and ubiquitous device that will revolutionize our every day life: “[…] when I talk to people involved in robotics – from university researchers to entrepreneurs, hobbyists and high school students – the level of excitement and expectation reminds me so much of that time when Paul Allen and I looked at the convergence of new technologies and dreamed of the day when a computer would be on every desk and in every home. And as I look at the trends that are now starting to converge, I can envision a future in which robotic devices will become a nearly ubiquitous part of our day-to-day lives.” (Gates 1). Robot Infants

The term ‘social robot’ or ‘sociable robot’ was coined by Aude Billard and Kerstin Dautenhahn (Billard and Dautenhahn) and Cynthia Breazeal (Breazeal, Designing, — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Jutta Weber. ‘You See the Videos But I Know How the Machine Functions’ — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Breazeal, Emotion) – all three are leading researchers in the field of social robotics respective Human-Robot Interaction. The vision and ambitious goals of the strong approach in Human-Robot interaction can be exemplified by the research goals of the research project COGNIRON – The Cognitive Robot Companion. This research project is funded with six million Euros by the European Community and half a million by the Swiss State, running from 2006-2009. On their official website the goals of the project are described in the following way: “The project will develop methods and technologies for the construction of such cognitive robots able to evolve and grow their capacities in close interaction with humans in an openended fashion. The robot is not only considered as a ready-made device but as an artificial creature, which improves its capabilities in a continuous process of acquiring new knowledge and skills. […] The design of cognitive functions of this artificial creature and the study and development of the continuous learning, training and education process in the course of which it will mature to a true companion, are the central research themes of the project.” (http://www.cogniron.org/InShort.php; my emphasis, JW).

The credo of social robotics is to develop machines which adapt in a ‘natural’ and ‘intuitive’ manner to humans – while in any kind of human-computer interaction it is the other way round up to now. Developmental psychology is supposed to help to achieve this aim. Many researchers hope to implement ‘curiosity’ into their machine to support the system-environment interaction, to make the robot ‘experiment’ and ‘learn’ like children do in their early years. This is the background of the model of caregiver-infant in Human-Robot Interaction. The human (expert or user) is modelled as the caregiver, while the robot is given the role of the infant. This idea builds on the amazing (and quite reductionist) claim that the caregiver-infant (most of the time imagined as a mother-child relation) is the most simple of human-human relationships and therefore the best model for social interactions in Human-Robot Interaction. The background of the strong approach of Human-Robot Interaction is the creation of autonomous and self-learning robots that can be educated and which might develop their own categories, decisions and even purposes. The concept of the caregiver-infantrelationship and of social learning via the interaction with other humans can be found in a variety of research approaches (see Breazeal; Fong, Nourbakhsh and Dautenhahn; — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

140 141 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

Kaplan) in Human-Robot Interaction. Traditionally, the relation between engineer / user and machine was modelled as a ‘master-slave’ relation in the sense of the engineer programming the robot, giving it orders to fulfil, etc., Human-Robot Interaction uses models and theories not only from the fields of (developmental) psychology, but also from cognitive science, ethology and sometimes sociology (see Burghart and Haeussling), thereby aiming at the implementation of social and emotional competencies. Knowing the state of art in robotics, this approach looks a bit overambitious (see Weber, Helpless Machines, Human-Robot Interaction). In contrast, the weak approach in Human-Robot Interaction claims only to imitate social behaviour and interactions. These researchers regard the copy of human social behaviour to be a more promising goal for real world systems in the near future. In both approaches, social and especially emotional intelligence is a central concept. Mostly, the concept of social or emotional intelligence refers to an understanding of social interaction that is interpreted in terms of pregiven social mechanisms, like for example a scheme – developed originally in psychology – of six pre-given basic emotions or five personality types (Ekman; Breazeal, Designing) or that so-called social norms (Petta and Staller; Wilhelm, Böhme and Gross). As the modelling of emotions, social behaviour and competencies is a quite complex task, researchers are using simple, reductionist models to translate their visions and concepts into action. The behaviourist implications of their models are striking and I doubt that a caregiver-infant model, which reproduces a quite stereotypical mother-infant relationship in a bourgeois nuclear family, is very attractive to the enduser – or may be it’s my hope. The question is whether it looks attractive for people to train their personal robot in their leisure time and to occupy themselves with the education of an one-and-only robot-child. Sometimes I’ve got the feeling that this might be a very specific idea, dreamt by professional as well as hobby roboticists who are – at least partly – fond of dreams and techno-imaginations about artificial partners, techno-playmates and never-tiring servants. AIBOS for the (White) Middle-Class No-Kids Techno-Educated Users

May be more attractive for the future (white) middle-class no-kids techno-educated users are social robots which are modelled as a pet-toy-gadget. These robots are made for entertainment, edutainment, and leisure. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Jutta Weber. ‘You See the Videos But I Know How the Machine Functions’ — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Think of the hundreds of thousands of people who bought their thousand-dollar AIBO just for fun – not only in Japan, but also in Germany, in the UK, France and other countries. In AIBO-online discussion they show off their personal robo-dogs, they compete in the production of new behaviour patterns of their artefacts. Their owners want to demonstrate their own capabilities: their technological know-how, experience or creativity. I guess that these people are not interested in educating their expensive personal robots to educate a future partner. In a way the engineers have cleverly outsourced some of their tiring, long-during work of product testing – which is now partly done by the users. But nevertheless, I think, that training a robo-dog is more about sport, competition and tricky, inventive solutions, just like playing chess or Go – and not about developing your own artificial creature for home work, for future partnerships and social relationships in a human-like way. What might be interesting about this new kind of outsourcing is the fact, that it obscures the roboticists´ authorship in the Human-Machine relationship (Suchman 7f). The roboticists who developed the software and its basic functions become invisible. At the same time, the intense and on-going testing of AIBO through its users contains some participatory aspects with regard to technology design – and it helped enormously to improve the features of AIBO. On the other side I agree with Natalie Jeremijenko that AIBO is not about learning technology in a profound way but about consuming the latest upgrades etc. Another strand of social robots modelled in the pet-owner relation is that of care robots. Think for example of Paro (Shibata et al.) – a kind of artificial baby seal with the function of a teddy bear for elderly people in Japan. It was developed to substitute traditional relationships between animals and humans – not at least because animals are often forbidden in homes for elderly people and – I guess – because they are much cheaper in terms of costs and care in the long run. In this context of ‘therapy-assistance’, nurturing responses via zoomorphism and baby schemes etc. (Breazeal, Designing) are triggered to immerse the user in the interaction. These simple-minded owner-pet relationships with care robots obviously work with the reinforcement of stereotypical social relations. (Weber, Helpless Machines) Personal Robots and Media Representation

As I mentioned in the beginning, professional representation of technoscientific research and development in the field of personal service robots is very common and wide-spread. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

142 143 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

Professional photographers, web designers, video artists etc. are hired for the presentation of latest developments – even in the context of so-called basic research. Brilliant photos, entertaining videos and impressing media reports can be downloaded on the websites of scientific institutes and research laboratories. Researchers are more and more involved in press work and science communication. Every other day they produce short videos for demonstration purposes which show the achievements of their latest developments. We see video clips of well-functioning robots that talk, climb stairs, dance, play tennis, or serve tea. The sweet and supposedly intriguing artefacts are often presented by happy researchers and preferable one of the few female roboticists around in robotic labs. They are portrayed as engaged and somehow caring engineers improving their artefacts. Other media-related activities and actors in the field are sightseeing tours for camera teams through the lab, the professional organized annual RoboCup, Lego Mindstorm Robot Courses for Kids (as part of so-called ‘Educational Robotics’), multiple kinds of Science Fairs and Festivals with robots at the front, as well as the staging of major achievements with press conferences and releases – especially media-effective with humanoids and zoomorphic robots. But the question here is: why is media representation so much more important than ten or twenty years ago? How does the compatibility of research findings to the media influence scientific and engineering work? And how does the work of representing itself influence one’s work? Funding and Media Representation

What becomes more and more obvious, is that not only managers but researchers invest a growing amount of their time in promoting and communicating their scientific findings and new artefacts – in the hope to influence the public opinion on technoscience in general and especially their own research. When asked, researchers mostly comment this promotion work as part of the funding acquisition. They are convinced that without a proper amount of public attention for their research the chances for funding radically decline. National or international funding – at least in Germany – has never been as central for academic and scientific research as today. For example, in the 70s and 80s German universities as well as single departments had much more independent funds and grants for their research. Today, renowned and international relevant research is heavily dependant on money from national and international funding agencies. In such — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Jutta Weber. ‘You See the Videos But I Know How the Machine Functions’ — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

a situation the proper staging of one’s department, faculty or institute as an innovative, pioneering and promising research group doing frontier research is pure gold. Strengthening the Fake. Or: Consumer Technologies and Media Representation

But this is only one of many reasons. Another one, especially with regard to personal service robotics, is the need to now ‘care’ about the user and to immerse the future consumers in their (future) project. So, broad media awareness is very attractive to researchers. Another important factor here is that media such as video or simulation opens up new opportunities for implementing the fake of sociality into the human-robot interaction and thereby to compensate the malfunctioning of machines and to veil the simple‘mindedness’ of still many artefacts. Video production in these technosciences often takes weeks of preparation and it often shows a single moment of well-function. For example, the navigation of the robot, its open dialogue with a user or the reaction to several users at the same time might have worked only a single time in weeks – but the important thing is that it is now on the video. To use photographs and videos is part of the immersion of the user. It is part of the roboticists’ job to convince the user of the credibility, sociality and intelligence of the artefacts. And video and photographs help to ‘embody’ anthropomorph behaviour, gender stereotypes, or simple schemes of emotions in the robot. Science Policy & the Crisis of Legitimacy in Technoscience

Another point in media representation is the problem of the legitimacy crisis in technoscience. For a long time science had the function of a truth discourse to fill the ethical vacuum of modernity. The idea of an objective and value-free science was central to the self-understanding of the modern, fragmented and rationalized world. In post modernity, master stories about linear technological progress as well as the truth function of science is at stake. Technoscience – this hybrid of science and technology devoted to applied research – needs new (narrative) strategies to secure its primacy and legitimacy in the production of truth. Science Communication – and with it media representation – is one way to reinterpret today’s fusion of science and technology with other realms, with our daily lives. Science communication and media politics is one way of actively shaping the reconfiguration of technoscience as ‘cultural practice — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

144 145 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

and practical culture’ (Haraway 66). In this process, science as culture also puts itself a friendly face of social practice on – a practice that is performed as open to the public, attractive, user-friendly, and the result of innovative and genius breakthrough work. And it is mediated as a complex process in which many different agents (machines, humans, animals, etc.) produce meanings and contribute to the endeavour of our ‘knowledge society’. I think this picture fits very well to today’s performativity of technoscience with its Science Museums, Science Fairs, Children’s university, shining ads and newsletters and its tight involvement with industries, entrepreneurs, investors, and start-up companies. A main message for selling technoscience and stabilizing its legitimacy is the following: Technoscience is fun and it has a high entertainment value. “[…] another level of this syntax can be defined as an imaginary. Not in the technical sense of a psychoanalytic pre-symbolic realm of undifferentiated totipotency, but in the more quotidian sense of a realm of imagining the future, and reimagining the borders of the real; life itself is dense with the possibility of both salvation and catastrophe.” Sarah Franklin (my emphasis; JW) Technoscience as Soap Opera: The Techno-Imaginary in Robotics & AI

Donna Haraway has pointed out the soap opera qualities of technoscience itself. She writes that apocalyptic stories as well as those of salvation “are bedfellows in the soap opera of technoscience” (Haraway 8). Artificial Intelligence as well as robotics are wonderful and fruitful playground for the techno-imaginary supporting their own ends. We find a quite sophisticated state of the art of enlivening visions and technoscientific images of bright and comfortable futures that need to be translated into action or picturing threats that must be avoided with the help of these cybersciences. Personal service robotics mobilizes futures of digital wellness and comfortable security which will free us from stress, data and work overload. For example, robots will take care of our old parents or young children, undertake internet researches for us finding a cheap flight or a hotel for the next conference, they clean our home while we are away or travel with us to serve as social companion – let it be a friend or a sex mate (Brooks) – so that we do not feel alone. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Jutta Weber. ‘You See the Videos But I Know How the Machine Functions’ — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

And there are apocalyptic stories of robots out of control that forget about the decent three rules for robots (Asimov) and are only concerned with their own rights and needs. One part of these kind of stories are artificial intelligent artefacts that become smarter than their constructors and might decide to destroy humanity. (Apocalyptic stories about armies of unemployed workers by advanced automatisation in the personal service economy seem to be less media-effective – I guess they are too realistic.) Anyhow, the deterministic message is: new technoscientific opportunities and disasters are waiting for us we might desire or fear but we have to live with the consequences of technoscience. And there is not much room left for the idea of intervention and participation in the genesis and production of technology. These tales live by the fusion of fact and fiction. A feeling of unpredictability of the scientific development, the complexity and disorder of sociotechnical processes even reinforces the impact of the techno-imaginary and stirs irrational fears and hopes. The difficulties in differentiating between fiction and fact are rising in contemporary technoscientific research (Nordmann) and support the power of the techno-imagi­nary. The permanent growth of robotics in the last 10 or 20 years, the differentiation of approaches and subfields as well as multidisciplinarity makes it even for experts more difficult to assess the contemporary state of the art in robotics, to differentiate between future or even science fiction scenarios, applications and artefacts soon to come, and real-world research projects. The narratives of salvation as well as apocalyptic visions are a quite productive way to intensify the symbolic capital of robotics. Robotics is staged as a highly powerful playground, with ubiquitous influence and far-reaching cultural and societal consequences: “Indeed, the promise of technoscience is, arguably, its principal social weight. Dazzling promise has always been the underside of the deceptively sober pose of scientific rationality and modern progress within the culture of no culture. Whether unlimited clean energy through the peaceful atom, artificial intelligence surpassing the merely human, an impenetrable shield from the enemy within or without, or the prevention of aging ever materializes is vastly less important than always living in the time zone of amazing promises. In relation of such dreams, the impossibility of ordinary materialization is intrinsic to the potency of the promise. Disaster feeds radiant hope and bottomless despair, […]” (Haraway 41; emphasis given). — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

146 147 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

I think, that it is not by chance that science fiction is a common reference point of roboticists, and that future scenarios of the human and the machine, the human and the robot, are central topics of discussion in the media. It is amazing that at the same time while Human-Computer Interaction is occupied to redefine the image of the robot as helpful, friendly and social, many Hollywood science fiction movies – like for example, The Bicentennial Man, A.I. or I, Robot – present robots as sensible, emotional and caring creatures of the future. It is not the image of the robot terminator but of the robot as helpful servant, of the intelligent and cooperative creature which is predominant at the moment. In a time of growing interest in commercial applications of personal service robotics, the relationship between humans and machines is pictured as mostly friendly. Remaining ambiguities work with pity for the highly intelligent and emotional robot creatures that are treated carelessly by humans. Patterns of suspension build on the tragedy of sensitive robots that are regarded only as a mean but not as an aim in themselves (A.I.), on ontogentic narratives of becoming intelligent and sensible (The Bicentennial Man) or on narratives that play with the traditional fears of robots as heartless and mean machines – where we learn in the end that these embodied creatures are not dangerous but supportive for humans (I, Robot). Our relationship seems to become quite harmonious with robots at the beginning of the 21st century. Roboticists and / in Media Representation

But what do roboticists think about the role of media representation and its impact on their work? How do they receive their own techno-imaginary tales and those of their colleagues? Having done quite a lot theoretical as well as empirical research in this field I was asking myself these questions. Mentioning regard apocalyptic stories about wild-going robots eliminating humanity, many roboticists comment that these are extreme stories which are told only by a few, kind of weird roboticists who should not to be taken seriously. Being a bit insistent I hinted at many EU research programs as well as EU-funded projects that use a quite dubious rhetorics of innovation and breaking news like – for example – the mentioned EU research project COGNIRON that promises the development of an artificial creature as companion. In informal discussions many technoscientists agree that the application-oriented science policy of the EU support the rhetorics of science fiction, of rhetorics of break through and innovation, about marvellous achievements and the favoured salvation stories – to be successful in getting funding. A German roboticist mentioned once that this is not only the case with the EU but also with the BMBF (German Ministry of Education and Research) while the proposals — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Jutta Weber. ‘You See the Videos But I Know How the Machine Functions’ — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

for the DFG (German Research Society) are those with a more scientific character. It seems that two different kinds of funding at least partially lead to different kind of rhetorics: Funding policy of application-oriented research which is oriented towards consumer technologies evokes rhetorics of promise, radical innovation and salvation, while funding policy of basic research might build more on rhetorics of incremental knowledge acquisition and scientific progress. Inclined to use the rhetorics of science fiction and marvellous innovation I wondered whether this also changes research itself. (How) Do – for example – the narratives on caring, credible and social machines which researchers have to put so much effort in change their research in the long run? One expert comments on that in the following way: “You see the videos but I know how the machine functions. And you get the impression perhaps that the machine is actually looking at a person but I actually know what’s going on ...., you know, it’s like being behind a camera, you don’t believe in the movie, because you’re creating the movie, so you know what it is. So, no I don’t think so. […] I see you (the recipient / user) as an observer who interprets and starts putting a lot of intention that are not there in the first place.” (expert interview).

The expert kind of claims a clear-cut differentiation between the work of faking sociality in the robot and telling glorious narratives about self-learning creatures from the everyday work in the lab. But there are other moments where it becomes obvious that for example videos – originally produced in the context of media representation – develop their own life and epistemological consequences, like we know it from computer simulation: Listen for example to the following working description from the construction of a humanoid: “we knew that these were analogous servo-motors and we tested them, whether they are sufficient, and we didn’t realize, that at end position the power is not at full range, and that is the problem, why the robot might not walk properly. We have a video in which it walks, but we are sure, that it will not improve, that is why we now build a new version” (expert interview; my emphasis).1 « — — — — — — — — — — — — — — — — — « — — — — — — — — — — — — — — — — 1 “…wir wussten, dass es analoge Servos sind und wir haben getestet, ob die Kraft ausreicht, aber uns ist nicht so direkt aufgefallen, dass er die Kraft nicht wirklich an der Zielposition erreicht, sondern nur weit entfernt von der Zielposition und das ist so ein

bisschen das Manko, warum er wahrscheinlich nicht richtig gehen wird. Also wir haben ein Video, wo er so ein bisschen dahinschlurft, aber wir sind uns auch sicher, dass es nicht wesentlich besser wird, deswegen bauen wir auch gerade wieder eine neuere Version, …”

148 149 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

On the one hand the researchers are sure that there are main problems which will hinder the robot from walking properly. But nevertheless the researcher mentions a video in which the robot managed to walk. Having in mind all the efforts of producing the promotion videos which is only loosely linked to the ‘actual’ research, it is astonishing that he mentions it at all. The question would be whether video originally made for promotion purposes does not also fulfil the function of proof in scientific contexts though it is also used for implementing the fake of sociality into the humanrobot interaction. May be creating the movie and being behind the camera does not always help not getting immersed in the stories of promise. The production of personal service robotics is built on the evocation of social and emotional feelings of the users towards robots and on performing the fake of sociality in the human-robot relation. This produces its own effects in the long run. The heterogeneity, multidisciplinarity and complexity of sociotechnical processes make the differentiation between fiction and fact even more difficult and support the impact of the techno-imaginary – which has always been an important part of the story-telling of technoscience. Its narratives of promise are today systematically reinforced by application-oriented funding policy that demands these rhetorics of apocalypse and salvation. Technoscience as cultural practice and practical culture today needs not only to be attractive, user-friendly and produce marvellous breakthrough work but also to guarantee a high entertainment value. In this framework, being behind the camera might not be so different from sitting in the cinema hall on a Saturday evening, eating popcorn, drinking beer and enjoying the movie. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Jutta Weber. ‘You See the Videos But I Know How the Machine Functions’ — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — References Billard, Aude and Kerstin Dautenhahn. “Grounding Communication in Situated, Social Robots.” Proceedings of the Towards Intelligent Mobile Robots Conference. Technical Report Series, Department of Computer Science, Manchester University, Manchester, UK, 1997. Last access April 2004 . Breazeal, Cynthia. Designing Sociable Robots. Cambridge, MA, USA: The MIT Press, 2002. Breazeal, Cynthia. „Emotion and sociable humanoid robots.“ Ed. E. Hudlika. International Journal of HumanComputer Studies. 59 (2003): 119-155. Brooks, Rodney. Flesh and Machines. New York: Pantheon Books, 2002. Burghart, Catherina and Roger Haeussling. „Evaluation Criteria for Human Robot Interaction.“ Proceedings of the Symposium on Robot Companions: Hard Problems and Open Challenges in Robot-Human Interaction. AISB‘05 Convention. Social Intelligence and Interaction in Animals, Robots and Agents. 12-15 April 2005. University of Hertfordshire, Hatfield, UK, 2005. 23-31. Christaller, Thomas, Michael Decker, Joachim-Michael Gilsbach, Gerd Hirzinger, Karl Lauterbach, Erich Schweighofer, Gerhard Schweitzer and Dieter Sturma. Robotik. Perspektiven für menschliches Handeln in der zukünftigen Gesellschaft. Berlin et al.: Springer, 2001. Crutzen, Cecile. “ICT-Representations as Transformative Critical Rooms”. Agents of Change. Eds. Kreutzner, Gabriele and Heidi Schelhowe. Opladen: Leske + Budrich, 2003. 87-106. Ekman, Paul. “Are there Basic Emotions?” Psychological Review. 99.3 (1992): 550-553. Fong, Terrence, Illah Nourbakhsh and Kerstin Dautenhahn, Eds. “A Survey of Socially Interactive Robots”. Robotics and Autonomous Systems. 42 (2003): 143-166 (24). Franklin, Sarah. “Life Itself: global nature and the genetic imaginary”. Department of Sociology, Lancaster University. 2001. Last access 15 May 2001 .

Gates, Bill. “A Robot in Every Home.” Scientific American. 16 Dez 2006. Last access 28 December 2006 . Haraway, Donna J. Modest_Witness@Second_Millenium. FemaleMan©_Meets_Onco- Mouse™. Feminism and Technoscience. New York, London: Routledge, 1997. Hayles, N. Katherine. “Computing the Human.” Eds. Jutta Weber & Corinna Bath. Turbulente Körper, soziale Maschinen. Feministische Studien zur Technowissenschaftskultur. Opladen: Leske & Budrich, 2003. Kaplan, Frederick. Developmental Robotics. 2006. Last access September 2006 . Kiesler, Sara and Pamela Hinds, Eds. Special Issue of Human-Computer Interaction. 19.1-2, 2004. Nordmann, Alfred. “Ignorance at the Heart of Science. Incredible Narratives on Brain-Machine Interfaces”. Ed. Johannes Ach, Münster. 2007 . Last access January 2007 . Petta, Paolo and Alexander Staller. “Introducing Emotions into the Computational Study of Social Norms: A First Evaluation.” Journal of Artificial Societies and Social Simulation. 4. 1, 2001. Rogers, Erika and Robin Murphy. “Human-Robot Interaction”. Final Report for DARPA/NSF Workshop on Development and Learning. 2001. Last access 1 April 2006 . Shibata, Takanori, Kazuyoshi Wada, Tomoko Saito and Tanie Kazuo. “Human Interactive Robot for Psychological Enrichment and Therapy.” Proceedings of the Symposium on Robot Companions: Hard Problems and Open Challenges in Human-Robot Interaction. AISB 2005 Convention Social Intelligence and Interaction in Animals, Robots and Agents, University of Hertfordshire, 2005.

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — »

150 151 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

Suchman, Lucy. “Human/Machine Reconsidered.” Centre of Science Studies, Lancaster University, Lancaster LA1 4YN, UK, 2003. Last access February 2004 . Weber, Jutta. “Helpless Machines and True Loving Caregivers. A Feminist Critique of Recent Trends in Human-Robot Interaction.” Journal of Information, Communication and Ethics in Society. 3.4, Paper 6 (2005): 209-218. Weber, Jutta. “Human-Robot Interaction”. Ed. Sigrid Kelsey. Handbook of Research on Human-Computer Interaction. Idea Group Publisher (under review), 2008. Wegner, Peter. “Why interaction is more powerful than algorithms.” Communications of the ACM, 1997. 80-91. Wilhelm, Torsten, Hans-Joachim Böhme and Horst-Michael Gross. “Classification of Face Images for Gender, Age, Facial Expression, and Identity.” Proceedings of the International Conference on Artificial Neural Networks ICANN ’05. Springer Verlag, 1 (2005): 569-574.

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — —

U sx bEMx I r V N

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

03

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

y8 NJebc ie t

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

ßßß sr

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

>

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

c — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

9+ :na d ` g H Ki f m s g

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Machines vy

K cL 9Ut

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Krimhild Becker >

g n n

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

3Z

Reality is a pretence for fantasy

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Krimhild Becker. — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — —

Reality is a —

pretence for fantasy —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

As an introduction I would like to quote Dr. Ulli Seegers who wrote the following text for my exhibition “Olympia the prototype” in Leipzig in 2003: “[…] In her works Kriemhild Becker has been discussing representations of the body for several years: her photo works show expressionless art-bodies in a strange perspective: human-like and artificial, familiar and yet completely unfamiliar at the same time: the image of the body appears as real and virtual — as bearer of the living, yet still as its lifeless representative — an in-between state: the peculiar androgynous figures without visible individual facial features remain captivated strangely expressionless in a state between creation and disappearance. Partly floating, partly blurred, loosing themselves in a diffused manner in the dark image, the sometimes glassy, sometimes metallic or shocking pink anthropomorphic beings seem to move following a mysterious logic: an accumulation of bodies, torsos, legs, arms and hands. — The androids, robots and plastic humans are generated from a different, distant world and in a state of transition. Transitions from a state of nothing, or better a state of not-yet, from nothing into something or from something into nothing. Becker’s photo works breathe a strong physicalness without showing one single human body at all. Even the format chosen for the photo works (height two metres) of which two overlapping lengths form one unit, represents a subtle reference — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

156 157 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — to the body size: the body-like as substitute and projection screen. The art-bodies begin to move in alternation with the incidence of light […].”

I will try to introduce you to the world I move and work in: I ask myself the following three decisive questions over and over again: What is in the space between man and machine? What is in the space between life and death? What happens in the interspaces unknown to us? The first human robot of the word – WABOT 1 – was constructed in Tokyo in 1973. Progress was and is being made in frantic speed. That one can call the human body a machine means no news for you, of course: all inner organs are totally dependent from one another; they interlock like gears and keep us alive. If a gear stops, or an organ fails, it can come to a standstill. So, considering the circumstances, does the aforementioned interspace exist at all? In my work I am looking for what might be the absent. I would like visualise both interspaces in my image language just like a mirror does. In order to achieve that I use robots, androids, art-bodies. They are the projection screen for my needs. With these art-bodies I want to ask for the being. In my images is no reference to space, no recognisable ground one can stand on, top and bottom are nullified. My art bodies have no ground beneath their feet. A new space – an image space – extends and opens up the depth of space. It is important to stress that I only work with daylight: it shines through my models, illuminates them, — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Krimhild Becker. Reality is a pretence for fantasy — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — —

provides them with energy and power, just like robots and androids are in reality fed with energy to make them fulfil their functions. Light puts order into chaos. Light gives life. Both works exhibited here bear a strong reference to E.T.A. Hoffmann who wrote “The Sandman” in 1817. I should probably mention that Hoffmann is one of the great writers of German romanticism. In the Sandman, the character “Olympia” represents seduction with many faces. She is seduction, she irritates and she has the power to deceive. Nathaniel – one of the main characters next to Olympia – falls in love with her and is infatuated with her beauty. In this state of confusion he cannot face life anymore and commits suicide. In his awareness Olympia, the robot, has become a human being and his friends have turned into robots. For this mistake he voluntarily paid with his life. This narrative was decisive for my discussion with Olympia’s facial expression resulting in a series of heads to which both images presented here belong. The yearning for the artificial human has always existed, and this yearning is continuously supplied with new energy. In this context I would also like to mention Fritz Lang who produced his wonderful film “Metropolis” in 1926/1927. Today, it seems as if we have nearly reached the goal of our dreams. The human body is no longer implicitly unique; it can be reproduced – transformed in many ways, spare parts can be integrated into it. In the past it was attempted to adapt artificial humans or dolls to the human archetype and the reality of our life as closely as possible, today this process of adaptation is left to us, the human. An individual body expression can be changed in accordance with the respective fashion trend – as can be distinctively observed in models, male or female. This, just to touch in short the fields of fashion and contemporary taste. If these assumptions are taken into consideration, do the geneses of the world religions still hold validity for us humans? In everyday life we talk about communication, but that does not mean we are actually talking with each other – we are talking to or are entertained by various machines: radio, television, computers etc. One can fall in love aided by technology, the dating channel makes that possible. Feelings have undergone changes, it is no longer implicitly necessary to become acquainted to each other in person. Is there a chronic shortage in time? In the electronic age, patience and concentration needed for a longer ability to listen are increasingly lost. One has to get precise quickly; otherwise no one will listen to you anymore. And what does that mean for our future? Will we have — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

158 159 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

longer conversations with artificial humans? Will we all have an artificial partner at home? Will they be programmed to twist around our thoughts (see Nathaniel in E.T.A. Hoffmann’s Sandman)? If so, who will program them? Do we disappear into an illusory world to escape reality? Or is that our reality? Does that promise us a welcome enrichment to our existence? Will we no longer have the time to live our spiritual and physical needs? Robots and androids are supposed to be a facilitation, a gain of time for us. According to my opinion, these machines consume a lot of our time and can make us dependent and addicted. All that leaves its traces and in my work I show these traces – I make them visible. For me, the relation between materialness and transcendence is of central importance. I am following the traces of the non-material with my art-bodies, I try to visualise the interspace. The perfect human images are already at home in virtual reality. Machine-man and man-machine. The idea of an artificially created human always goes along with a changed view of the human body which no longer appears as being unique and unchangeable, but has often replaced human awareness. We can find the type of human being we would prefer to multiply in commercials and advertisements. The artistic way of expression will change of course, too, simply by the changing materials to be used in future. In the past years many things I used to work with have disappeared from the field of photography. I had to reorganise myself– but that actually meant an enrichment for me: I found the approach to colour. For me this change has become a positive one. Now I ask for the space unknown to us in colours. The problem remained the same – the material has changed. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Krimhild Becker. Reality is a pretence for fantasy — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — —

We find ourselves in a new modern age that is linked to social changes. For artist and art exciting times do lie ahead. “New times – new problems” Please allow me to quote from the catalogue “DOLLS BODIES AUTOMATA”, (K.S. and P. Müller-Tamm, Eds. Puppen·Körper·Automaten. Phantasmen der Moderne. Ausstellungskatalog der Kunstsammlung Nordrhein-Westfalen, Köln 1999.) “Soon the machines will develop away from us. Super-intelligent machines will inherit the universe, and the biological humans, their ancestors, will become a historic reminiscence[…]”

The other prediction: Who will be the most important personality of the year 2050? The answer to this question is: The one virtual avatar in the net that will be indistinguishable from a human. I do not want to believe that this is about to come, it most certainly depends on the human being’s readiness to be manipulated and his/her ability to make him-/herself aware of procedures. On the other hand, utopias are of utmost importance for our life, our being. Utopias are the humus for reality. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

160 161 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

?

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Gary Cass Alan Mullett >

st

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

v

( Bioalloy Designing Cyborg's x g aEvolutionary WFuture

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Gary Cass, Alan Mullett. — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — —

Bioalloy —

Designing a Cyborg’s —

Evolutionary Future —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Bioalloy is a designed Cyborg. The project researches a possible futuristic evolutionary pathway of the living and the non-living. The project engages with the possibilities of interfacing a living system to a machine, to create an artificial life entity or a symbiotic hybrid. In the twenty first century where cybernetic entities are becoming more viable, Bioalloy predicts an evolutionary leap, exploring the possibilities of a living system interacting with a machine to create a Cyborg that will grow and nurture its own “skin” (* 01). Presently, very few machines in‘corp’orate any organic material.

In the future, will evolution only link the living? Emerging from the metallic shell, and out of the primordial ooze, three white ossified strands, coil, mutually embraced. The living begins to flow, as the bruising bleeds to red. A slimy, soft, skin forms as the machine courses to life. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

165 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — After several hours of a bacterial culture circulat— ing through the ossified strands of Bioalloy, the — bruising and bleeding has led to the formation of a — — biological skin. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — *

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 01

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — —

164 — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Presently, the two main theories of evolution prescribe that the evolutionary development of living organisms are governed by chance variations caused by random acts of biological processes. These theories are: — Darwin’s 1859 Origin of the Species by Means of Natural Selection. Darwinian’s evo-

lutionary random biological acts have created a competition resulting in the survival of the fittest mutant. If the chance variation was advantageous in an ever changing world, the mutant would survive to fight another day. Many other variations would have been fatal. — Symbiogenesis was originally introduced by Mereschkowsky in 1905 and further

championed by Margulis in 1981. Symbiogenic’s evolutionary random biological acts were the merging of two or more organisms giving rise to a chimera and survival of the new alliance. Many of these alliances would have been fatal. Are these two theories of evolution mutually opposed to each another? Or will they again, collaborate as they have in the past in the continuing evolution of living systems? And, in the future, as the viability and availability of Cyborgian systems further increases, can these evolutionary partners again combine to link both the living and the non-living? — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Gary Cass, Alan Mullett. Bioalloy — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — —

The cooperation of Symbiogenesis that leads to the formation of chimeras will compete successfully against natural selection and endure Darwin’s survival of the fittest. The newly formed mutant will travel further on the evolutionary pathway of life, and depending on strengths or weaknesses accrued through this evolution, may or may not spawn further survival. We would like to pose this question; (If we can be so bold as to use the human evolutionary pathway as an example to illustrate our narrative)! At an evolutionary fork in the road, early primates had to choose life’s path. The human species went one way, with the remaining primates following the other route. Which of the two paths was evolutionarily most demanding? After posing this question to many people over many years, some thought that humans had chosen the more demanding road as it was paved with guns, wars and death. Others believed that, as humans were required to work for a living rather than sit lazing in the sun eating the proverbial banana, humans must have had it harder... Most forgot to look at the bigger picture, the evolution of a species rather than individual sufferance. Diversity is the key. As there is much more diverse primate speciation compared to the human species, the primates had a far more demanding evolutionary pathway! Primates had to adapt to their environment rather than adapting the environment to themselves! The minimal variation found in the human pathway implies that an easier evolutionary path was navigated. The remaining primates’ path was paved with hardship; hence variations which occurred in the development of these species were essential for survival. All of these random repetitive variations and adaptations have played a major part in the evolution of biological living systems. Like Erwin Schrödinger‘s aperiodic crystal repeating its structure as it grows, biological life resembles this repetitive nature. Repeated millions of times over millions of years, the processes of life have led to the wonderful three-dimensional patterns seen in organisms, hives, cities and planetary life as a whole (Margulis and Sagan). Will the unpredictable random repetitive nature of biological evolutionary processes play a major role in the Cyborg’s evolutionary future? Where we have been is the best predictor of where we are going (Gray). The term Cyborg was first coined by Manfred Clynes and Nathan Kline (1960) by combining the words cybernetic organism. A Cyborg is a self-regulating organism that combines the natural and the artificial together in one system. Cyborgs do not have to be part human, for any organism/system that mixes the evolved and the — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

166 167 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

made, the living and the inanimate, is technically a Cyborg (Gray). Gray goes on to say ‘this would include biocomputers based on organic processes, along with roaches with implants and bioengineered microbes.’ This early predictor of the use of microbes in Cyborgian systems perhaps unintentionally points to yet unforeseen evolutionary possibilities. Paul Davies in ‘The Fifth Miracle’ searches for the origin of life, not just; what is life! The giant leap that was made from the non-living to the living, the animation of an inert object into an autopoietic1 being; the spark of life! Once we discover the spark, then we may be able to apply it to other systems such as Cyborgian systems to create a truly autonomous, self-maintaining and self-replicating life form. The word Cyborg in recent times seems to have been hijacked by Hollywood, which has caused the public’s perception of the Cyborg to be very anthropomorphic by using such stars of science fiction movies such as the Terminator trilogy (19842003), Star Trek series (1966 – ) and Bicentennial Man (1999). All of these examples are very conceptual with the Cyborg image used as a metaphor for many sociological, political and post-human paradigms. More recent novels such as Michael Crichton’s Prey has moved this ontological opinion away from the human-like form to illustrate a Cyborg system that integrates bacteria and machine to produce a nanorobotic swarm camera. Presently many realised Cyborgian systems are attempting to change this speciesist ideology by designing provocative machines that do not have any human biological substance or form. Projects such as Ken Rinaldo’s Augmented Fish Reality, SymbioticA Research Group’s Fish and Chips and Tsuda, et al. Robot Control: From Silicon Circuitry to Cells have all steered away from human substances and forms. To illustrate the complexity we use forms of simplicity. The simplistic or even primitive appearance of Bioalloy is quite deliberate, not only because of its process driven form, but it illustrates one important factor concerning evolution; start simple and progressively develop complexity. To begin with complexity may jeopardize the initiation. The commencement of biological evolution started with organisms far more primitive even than modern day microbes. With this in mind, when designing and assembling a Cyborgian System, one must initially think simplistically. Do not attempt to construct a Cyberdyne system T800, model 101 (Cameron et al.) straight off the factory floor, but keep it simple initially and gradually evolve successful iterations along the lines of the living! « — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 1 Autopoiesis coined by Chilean Biologists Humberto Maturana and Francisco Varela meaning auto (self) and poiein as in poetry (making).

Gary Cass, Alan Mullett. Bioalloy — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 02 — — — Bioalloy’s main non-living part. — — — The inverted fume-hood to remove — — — — — — Bioalloy’s stink and the Barnacle — — — shaped receptacle to dam the — — — biological fluid. The ossified — — — — — — strands with their primitive shape — — — bruise, bleed and form skin. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Bioalloy’s main non-living part can be broken into three sections; inverted fume-hood, ossified strands and barnacle, each having a vital function (* 02). As the fumes emitted by Bioalloy’s biologicals can become rather pungent and offensive to some, an inverted fume-hood was constructed. Initially serving as the plinth supporting Bioalloy, its very important odour removal role became an intricate part of the system. As Cyborgian systems become more viable, they may have their own distinctive smell. The waste products that are excreted, whether gas, liquid or solid from the Cyborg will have to be dealt with. This initial Bioalloy system catered for the gas phase but liquid and solid excrement had to be removed manually. The barnacle which houses the machinery and dams the biological soup has been shaped to honour the one creature that fascinated Charles Darwin.2 Inside the contoured metallic shell and under the biological fluid are housed the pumps and motors that circulate this fluid around the machine. As the fluid is recycled, the barnacle acts as a receiving vessel returning the fluid from the ossified strands to the pumps. Made entirely from aluminium, the barnacle may play an important role as an electrode (and with the acidic nature of the biological fluid) in the production of electricity. It is this production of electricity that may evolve autonomy in Bioalloy. This evolutionary path may lead Bioalloy to achieve autopoiesis and be recognized as living. One of the major hurdles of creating a Cyborgian system is that (unlike a robotic system) the living part or biological life form must maintain a certain level of hydration. Without water there is no life as we know it! Another of the barnacle’s functions is to separate the fluid from the electronics. This then dictates shapes and structural design to avoid the inevitable disaster if the fluid and electronics meet. Therefore some creative and improvised construction must be sought. « — — — — — — — — — — — — — — — — — « — — — — — — — — — — — — — — — — 2 Bioalloy is in the form of a barnacle to emphasis the magnitude that one Chilean barnacle had on postponing the publication of the most significant book in evolutionary science. There is a story (Stott, 2003) that Charles Darwin carried a Chilean barnacle on a journey

around the world, from the South American beach back to London, preserved in a jar of wine spirits. When he has finished finding homes for all the 1,529 species he has collected on the Beagle, he will return to the puzzle that the creature‘s strange anatomy

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

168 169 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — Skeletal System Ossification Bruising & Bleeding Skin Formation — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 03 — — Descriptors given to the ossified strands. The metallic endoskeleton is — ossified with cellulose. Once the machine springs to life, the biologicals bruise, — bleed and finally form an outer skin over the strands. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Bioalloy’s ossified strands have following descriptors (* 03) Skeletal System: Alloy (the combination of two or more metals) forms the

major component of the non-living skeletal system of Bioalloy. Ossification: The ossification process coats the skeletal system in white

‘bone like’ cellulose. Bruising: Bruising begins when the red biological fluid flows through the al-

loy endoskeleton and impregnates the ossified coating. Bleeding: During the bruising process or a short time after, through small

fractures in the ossified coating, the bleeding will commence. Irregular vein like patterns will form down the length of the ossified strands. Skin Formation: Once the biological fluid comes in contact with air a slimy,

soft, skin forms. Ageing: As the Bioalloy matures the skin dries, darkens and wrinkles. Large

fluid filled blisters give the once aesthetically pleasing (for most) white strands a more grotesque look. With age comes smell. Bioalloy is no different, with a very distinctive pungent odour that will cling and remain with by-passers. Dietary Requirements (When Cyborg’s start drinking your wine):

The diet of the biological symbiont is wine. Red wine will create a red Cyborg and white wine a white one. « — — — — — — — — — — — — — — — — — « — — — — — — — — — — — — — — — — presents; and then he will write this Chilean barnacle‘s evolutionary biography--a puzzle that will take him eight years to think through. Eight years, from 1846 until 1854, devoted entirely to barnacles? By 1842 Darwin had already sketched out his theory of evolution by

natural selection. But he pushed it all aside, squirreling it away to work on the barnacle riddle. What was so compelling about these invertebrates that Darwin chose to postpone the completion of his major work – Origin of Species – for their sake?

Gary Cass, Alan Mullett. Bioalloy — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — —

Skin formation of Bioalloy occurs on the surface of the ossified strands (* 03). The ossified strands have a primitive and simple form due to the complexity of the methodology. As the Bioalloy group has little references to other methods of ossifying the metallic endoskeleton, the shapes have be influenced and process driven. Other systems that are similar to Bioalloy’s ossification are that of water purifying filter production. However due to manufacturing patents and company secrecy, the Bioalloy group was not allowed to use privileged information. ‘Cellulose fibres and glue!’ was the only clue we had. Therefore much research has been done on the making of the strands and future works will continue to increase the complexity of the skeletal system and constant improvement of the ossification process. It is within the strands that interaction between the organic and inorganic takes place. The skeletal system with its inorganic metallic origins is interfaced with the organic origins of the white ossified cellulose. This cellulose has a plant origin which is interfaced with bacterial cellulose as the skin formation occurs. The skin is comprised mainly of bacterial cellulose but with a red pigmentation, gives its menacing meaty/animal organ tissue appearance. This biofilm shield (Deacon) will protect both the alloy and bacterial colony from the outside world. The Age of the Bacteria (Gould) is now, has been in the past and probably will be in the future. These tiny primitive organisms with their immense diversification seemed to have driven evolution and controlled life. If all bacteria were to cease to exist, life as we know it would end. Symbiogenic evolution of the biological world has been dominated by bacteria. The bacteria, we believe will play a major role, not only in biological evolution but in the evolution of Cyborgian systems. Will this change the narcissistic opinion that humans have on this Gaia world and the belief that we are the sentient beings? As bacteria are the living part of Bioalloy, we will research the coming together of the living and the non-living and examine what role the bacteria will play in the Bioalloy system. The cellulose skin that cloaks Bioalloy’s ossified strands is synthesized by bacteria called Acetobacter.3 Acetobacter are rod shaped bacteria, 1-4µm in size. They are aerobic (requiring oxygen) and can be found to be motile or non-motile. Acetobacter (commonly called ‘Mother’) are distinguished by their ability to convert ethanol (wine) to acetic acid (vinegar). Another feature of Acetobacter is the synthesis of large quantities of micro fibrils of pure cellulose. One explanation for the synthesis of this cellulose might be to keep the bacteria close to the oxygenated surface, thus forming a cellulitic raft. From an ecological view point, the Acetobacter has evolved to deal « — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 3 http://www.botany.utexas.edu/facstaff/facpages/ mbrown. Last access May 2006.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — —

171 — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 04 — — The imperfections of Cyborgian — Systems. This wound on the — damaged ossified strand will — — eventually self heal. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — —

— — — — — — — — — — — — —

— — — — — — — — — — — — —

— — — — — — — — — — — — —

— — — — — — — — — — — — —

— — — — — — — — — — — — —

— — — — — — — — — — — — —

170 — — » — — — — — — — — — — — — — — — — — — — — — — — —

with and reduce the high alcohol and acidic environments of decaying plant material left behind by yeasts and other micro flora (VanDemark and Batzing), which can be detrimental to the remainder of us mere mortals. As for the limitations and imperfections of the human species, it would appear that Cyborgian systems will have their own imperfections (* 04). If construction of the ossified strands is flawed, the biological fluid will rip the strands apart, pressure drops and the skin formation falters. However, due to the Cyborgian System containing a living entity, the wound is self repairable; self healing. The Acetobacter begin to deposit their cellulose waste product, blocking the gash and preventing the flow of the biological fluid. Once the bleeding is reduced to its usual trickle, skin formation continues. As Cyborgs are part living, are they mortals? Bioalloy is a designed machine that in‘corp’orates biological matter, researching a possible futuristic evolutionary pathway of the living and the non-living. The unpredictable random repetitive nature of biological evolutionary processes will play a major role in the Cyborg’s evolutionary future. Symbiogenic theories believe that bacteria have controlled biological evolutionary processes and as Bioalloy’s living part is bacterial, then will Symbiogenic theories drive the Cyborg’s evolution? In the future, evolution will link the living and the non-living! Bioalloy is an ongoing research endeavour into artistic Cyborgian systems developed in the F.N.A.S. laboratories; scientific collaborators to SymbioticA, The University of Western Australia. Other art/science projects that the Bioalloy group is involved with can be found at www.bioalloy.org — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Gary Cass, Alan Mullett. Bioalloy — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — References Asimov, I., R. Silverberg and N. Kazan. The Bicentennial Man. (novel The Positronic Man) (short story The Bicentennial Man). Buena Vista Pictures, 1999. Cameron, J., G. A. Hurd and H. Ellison, 1984. The Terminator. Orion Pictures Corporation (1984-1997) Metro-Goldwyn-Mayer (1998-present).

Tsuda, S., K-P. Zauner and Y-P. Gunji. Robot Control: From Silicon Circuitry to Cells. Ed. A. J. Ijspeert et al. BioADIT 2006, LNCS 3853. Berlin Heidelberg: SpringerVerlag, 2006. 20-32. Van Demark, P. J. and B. L. Batzing. The Microbes. The Benjamin/Cummings Publishing Company Inc., 1987.

Clynes, M. E. and N. S. Kline. “Cyborgs and Space”, Astronautics. September 1960. 26-27 and 74-75. Crichton, M. Prey. Harper Collins, 2002. Darwin, C. Origin of the Species by Means of Natural Selection. Ed. John Murray, London, 1859 Davies, P. The Fifth Miracle. The Search for the Origin of Life. Allen Lane. The Penguin Press. Australia, 1998. Deacon, J. New Scientist. 31 August 1996. 32. Gould, S. G. Life’s Grandeur. London: Jonathon Cape, 1996. Gray, C. H. Cyborg citizen. New York: Routledge, 2002. Margulis, L. Symbiosis in Cell Evolution: Life and its Environment on the Early Earth. W.H. Freeman. New York, 1981. Margulis, L. and D. Sagan. What is Life. Uni of California Press Simon and Schuster, 1995. Mereschkowsky, C. Über Natur und Ursprung der Chromatophoren im Pflanzenreiche. Eur. J. Phycol., 34 (1905): 287–295. Rinaldo, K. Augmented Fish Reality. ARS Electronica. Linz, Austria, 2004. Roddenberry, G. Star Trek; the Next Generation. Paramount Pictures, 1987. Stott, S. Darwin and the Barnacle. Faber & Faber Ltd U.K, 2003. SymbioticA Research Group. Fish and Chips. ARS Electronica. Linz, Austria, 2001.

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — »

172 173 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

f18 institute >

f

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

ssnLivingRoomsGqVVVV ed

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

ß

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

f18 Institute. — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — —

LivingRooms —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Our world, especially in terms of communication, production and transport as well as “at Home”, consists largely out of embedded, capsulated machines. We take and use these “Blackboxes” in a self-granted and self-evident way as most simple tools like pencil or hammer. The complexity of technology, development, mechanic and electronic of our everyday devices is not translated into our reception. The system behind them – creation, historical and material conditions under which they are produced and user-optimisation is in no way transparent and not enough reflected. This applies also for our non-motorised or non-electrified surrounding shaped through architecture and urbanism. All kinds of systems rule our everyday lives; they determine developments and inventions which should serve us and which should make our life and daily living systematically “more efficient, easier and better”. The LivingRooms performance-installation approaches this phenomena through creating an ambient and a situation which we can apply to several ideas and dreams to improve our daily life and environment. Starting with electrified kitchen via electronic bureau and virtual space towards “Augmented Environment” or “Ambient Intelligence”, a mixture of virtual and real space creates a picture which we can relate to our past and future. The LivingRooms installation consists of kitchen, sleeping-room, bath, living-room and hobereau-area. Rooms form a standard flat, ca. 70m² floor space, with no walls, but doors in between. Each area is equipped with ubiquitous devices and tools – furniture, kitchen devices, accessories and tools. Devices function to a certain degree, however, their function is not created by use, they are a subject of their “own” dynamic. The flat and the objects take on a life of their own; it seems the flat generates a possible inhabitant in virtual state. A course of action with furniture and accessories arises/starts which then increasingly runs into a somehow escalating independence. Single elements leave the inner space of their areas, stray through the other units and even leave the living-area out into the outer world of the gallery. Other elements start — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — * 01 02 — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — —

176 177 — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

their life on the spot – some kind of orchestra made out of kitchen-devices, tools and trunks, chairs and tables jumps in rocking rhythmic and absurd dance. Part by part the objects calm slowly down, the mobile furniture move back to their stations and the virtual inhabitant moves back to bed, the light closes down. The LivingRooms installation offers the visitor a possibility to explore his/her own everyday world which is very often determined with ideas of automation and duty. A moment of sensual and aesthetic reception of an almost normal situation is created with the intention to offer the opportunity for the visitor to find his/her own position to related topics, even more so, through confusion and questioning. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

f18 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * — — — — — — — — — — — — — — — — — — — * — — — — — — — — — — — — — — — — — — — — — * — —

Institute. LivingRooms — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 03 — — — — — — — — — — — — — — — — — — — 04 05 06 — — — — — — — — — — — — — — — — — — — — — 07 * 08 09 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

178 179 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — Images * 01 02 03 07 Photo: Nellie Baartz

* 04 05 06 08 09 Photo: Katrin Dopener

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

oBill Vorn > p

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

v

o W f

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Adaptive Machines For Interactive Robotic Art Installations

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Bill Vorn. — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Adaptive Machines —

For Interactive —

Robotic Art —

Installations —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — Abstract

Since the last fifteen years, we have been creating public shows using robotics, sound and light, and processes inspired by Artificial Life. The aim of our robotic art projects is to induce empathy from the viewers towards characters which are nothing more than simple articulated metal structures. The strength of the simulacra relies on subverting the perception of these creatures. By implementing sets of basic reactive behaviors in the robots and by creating an immersive audiovisual context that establishes an environment for the works, an inevitable reflex of anthropomorphism and projection arises, as it does towards any dynamic phenomenon that challenges the senses. We present robotic machines not as specialized and virtuoso automatons but rather as expressive animated artworks. We also explore the reformulation of sound and light applications by simulating organic and metabolic functions and by creating dynamic virtual architectures.« — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

182 183 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — Robotics as Art

Robotic Art is an emerging artistic domain where research and creation are intimately related. If we think of the few artists around the world actively involved in this field, we discover that each one of them has in some way or another developed new technologies, techniques and methodologies of production that enable the creation of innovative works of art integrating robots, machines and automatons. Moreover, these works are raising fundamental philosophical and sociological questions about the relationships between human beings and machines, between the real and the artificial, and between the living and the non-living. From Karel Capek’s famous play R.U.R. (1920) to Nam June Paik’s Robot K-456 (1964) to contemporary electronic arts and new media, robots have invaded the realm of artistic expression. Artists have been exploring robotics for a few decades now, sometimes on their own, but often in collaboration with engineers and scientists. In Foundation and Development of Robotic Art, Eduardo Kac states “As artists continue to push the very limits of art, traditionally defined by discrete and inert handmade objects, they introduce robotics as a new medium at the same time as they challenge our understanding of robots” (Kac 60). In the last 20 years, artists like Mark Pauline,1 Christian Ristow,2 Eric Paulos,3 Chico Macmurtrie,4 Ken Rinaldo,5 Simon Penny,6 Stelarc,7 Symbiotica,8 Robotlab,9 Erwin Driessen and Maria Verstappen,10 France Cadet 11 and Jim Whiting,12 just to name a few, distinguished themselves by their particular artistic application of robotic principles. Well-known Canadian artists like Max Dean,13 Norman White,14 Reva Stone15 and Istvan Kantor16 also used robotics in many of their works. In the 1960’s, Norman White established himself as a pioneering figure in the field with his work on electronics, motors and gears. Robotic art is not a single homogeneous discipline, rather it is a mixture of multiple technological areas involving mechanics, electronics, programming, as well as multimedia. In the same manner, our research-creation agenda does not focus on one single problem or one field of study; it encompasses a wide variety of research projects that all have one thing in common: producing a work of art as a final outcome. This is why we do simultaneously address research and development projects in parallel robotics, artificial muscle actuators, reinforcement learning (RL) and simulation of adaptive behaviors. « — — — — — — — — — — — — — — — — — « — — — — — — — — — — — — — — — — 1 http://www.srl.org 2 http://christianristow.com 3 http://berkeley.intel-research.net/paulos/ 4 http://www.amorphicrobotworks.org 5 http://accad.osu.edu/~rinaldo 6 http://www.ace.uci.edu/penny 7 http://www.stelarc.va.com.au 8 http://www.symbiotica.uwa.edu.au 9 http://www.robotlab.de

10 http://www.xs4all.nl/~notnot 11 http://www.cyber-doll.com 12 http://www.leipzig-online.de/bimbotown/

whiting-e.htm 13 http://www.susanhobbs.com/artists/Dean/ MDimages.html 14 http://www.normill.ca 15 http://www.revastone.ca 16 http://www.istvankantor.com

Bill Vorn. Adaptive Machines — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 01 — — — One of the Hysterical — — — Machines (2006) — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Since its early stages, our artistic work has been strongly influenced by scientific advances in Artificial Life. We are particularly interested in studying and developing cellular automatons, artificial social systems, genetic algorithms, adaptive behaviors and reinforcement learning processes in order to produce innovative works of art. Our conceptual framework is not only based upon Artificial Life, but also immersive environments, connectionism, reactivity and artificial behaviors (implanted and emergent), with the underlying intention of producing an aesthetic medium out of machines. Our approach is based on simulating living organisms by stimulating a reflex of empathy from the viewers towards responsive machines. Our work is defined by an aesthetics of empathy and anthropomorphism from human reactions engendered by animating abstract mechanical structures. Suspension of disbelief is an inevitable reflex experienced by most viewers that allows the artist to bring life into inert matter. Underlying Problematics

What makes a machine turn into a living creature? Behavior is a keyword in anthropomorphic automaton design and actualization (Arkin). A certain level of realism may be achieved by the illusions induced by actions and reactions of the animats and the machines: the success of this dynamic form of computer-mediated communication may be measured by the effectiveness of the simulacrum. An effective simulation of the living is the result of different parameters acting to trigger impressions and empathy (visual appearance, sound emission or physical movement, for example), but behavior may be seen as the most convincing one as it gives a strong impression of autonomy and self-consciousness. Uncertainty also plays an important role in the behavioral relation with the viewer. Animated metal parts in a robot or dots on a computer screen can be seen as being — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

184 185 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

alive if they move and react in a non-repetitive and unforeseeable way, giving a strong impression of self-decision and autonomy. Do Artificial Life creatures have to be figurative representations (anthropomorphic or zoomorphic) to be convincing? The premise of our hypothesis is that, as long as they manifest autonomous behaviors in the interaction process, agents could bear any abstract visual form. Recent research in Artificial Life and robotic technology encourages a new kind of art form that combines artificial morphogenesis, immersive environments, interactivity and reactivity with multiple theatrical cognitive machines (robotics, automation and animatronics) to achieve aesthetic results. We often use the expression “theatrical machines” to describe physical and autonomous robotic agents integrating some kind of multimedia objects in their ontology (sound, light, video, etc). The examples of application using this new form of expression include emulation of realistic creatures and lifelike systems, conceptual exploration in the aesthetics of artificial perceptions, behaviors and interactions, embodiment of machine mechanisms, etc. The research we pursue is principally based on perception: the viewer‘s perception of the robot and the robot‘s perception of the environment as well as itself. Perception guides the effect created on the viewer. This is why the work is steered by this fundamental question: is it possible to create an impression of life simply through human-machine reactive behaviors of abstract robotic structures? Our work is based on the merging of aesthetic, philosophic and scientific questions related to machine ontology, its awareness, perception and potential “sentience”. An ontology describes how the world in which the agent lives is constructed, how this world is perceived by the agent and how the agent may act upon its world. Robotic ontology is generally defined by a formal description of the component parts and their individual role and behavior. Our research also investigates the notion of the artificial construction of “self” as the leading theme of our creation projects. Construction of subjectivity through perception leads to aesthetics, thus engaging into the very subject matter of art. Research-Creation

Since 1992, we have been creating more than fifteen different works involving robotics, audiovisuals, theatrical set-ups and interactivity with the public. Of our most recent projects, Red Light (2005) and Hysterical Machines (2006) are certainly good examples of the type of work that has been produced over the years. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Bill Vorn. Adaptive Machines — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 02 — — — The Red Light interac- — — — tive robotic installation — — — — — — (2005) — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

The Hysterical Machines robotic installation is very much inspired of a previous work based on the “misery of the machines” (Bill Vorn, LP Demers, La Cour des Miracles 1997). It is conceived on the principle of deconstruction, suggesting dysfunctional, absurd and deviant behaviors through a functional machine. It operates on a dual-level process expressing the paradoxal nature of Artificial Life. The first prototype of the hysterical machine (renamed Prehysterical Machine) has been presented in many international events since 2002. We have built nine more machines inspired by the prehysterical prototype that are part of a larger scale environment. Each hysterical machine has a spherical body and eight arms made of aluminum tubing. It has a sensing system, a motor system and a control system that functions as an autonomous nervous system (entirely reactive). Some machines are suspended from the ceiling and their arms are actuated by pneumatic valves and cylinders. Pyroelectric sensors allow the robots to detect the presence of viewers in the nearby environment. They react to the viewers according to the amount of stimuli they receive. The perceived emergent behaviors of these machines engender a multiplicity of interpretations based on single dynamic pattern of events. Red Light is an interactive robotic environment which aims to question, reformulate and subvert the notions of behavior, projection and empathy that usually characterize the relationship between human beings and machines. This project evokes a certain “deviance of the machines” as it would exist in the hottest areas of a fictive world populated exclusively by these cybernetic creatures. This installation project also explores techniques and technologies related to parallel robotics and to pneumatics with the construction of home-made pneumatic muscles. Eight machines react to the presence of viewers by generating sound and light and by moving their body in a very organic but unusual way. Each robot is an assembly of four segments joined — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

186 187 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

by twelve air muscle actuators. Six machines are hanging from the ceiling and two machines are convulsing on the floor. The artificial characters in the Red Light environment are complex machines that could generate a wide number of possible behaviors. These behaviors are also adaptive in response to what the machines perceive and the way viewers decide to interact with them: by touching the robots, by moving around them, by simply standing in front of these untamed tentacles. The title of this installation project evokes a particular situation/context where the actors are expressing themselves through unpredictable behaviors that may seem completely wild or crazy, where the interpretation of these behaviors may even lead to believe in their own true existence. Red Light is a place where the human qualities of the machine and the machinic nature of man are intermixed and become blurred. Along our artistic work, we pursue research and development projects in parallel mechanisms, artificial muscle actuators, reinforcement learning and simulation of adaptive behaviors. A parallel mechanism is a mechanical system that is connected to its base by two or more independent kinematic chains (assemblage of links and joints). A pneumatic muscle (also called McKibben actuator) is a flexible air piston made of inflatable material that contracts when activated. In recent years, we have been working with different types of parallel mechanisms (for example, the two robots mounted on Stewart platforms in Le Procès) and pneumatic muscles (like the suspended robot arms in Red Light) and we believe that they can provide unusual types of physical motion that can produce a more organic feel to our machines. We explore various designs and build several experimental prototypes of machines that make use of these technologies to create lifelike artificial creatures. Such a perspective requires a sophisticated use and development of behavioral agents in a complex and highly structured environment. It also implies the simultaneous development of both hardware and software as platform for real-life applications. Our projects distinguish themselves through the development of interfaces and specialized algorithms that further explore the proprioceptive aspect of the machine (that is “perception of the self”). In other words, how does the machine learn to adopt proper behavior by “experimenting” the limit of its joints, movements, etc, in response to the modifications of its inner states and environment changes. Such changes include but are not restricted to: learning to travel in a restrictive space, vision of surrounding environment, recognition of multiple sound and speech sources as well as deciphering them to adopt proper responsive behavior. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Bill Vorn. Adaptive Machines — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — * 03 — — — The Convulsive Machine — — — from La Cour des — — — — — — Miracles (1997) — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — Current and future work

Three major artistic projects are actually at the conceptual stage of development and are envisioned for the next three-year period of production: The Grace State Machines project addresses the subjective attribution of living being characteristics to “Finite State Machines”. It also points to the paradoxal nature of Artificial Life: how could it be life if it is artificial? This project is conceived as a performance/installation inspired by classical ballet and modern dance. It focuses on the movement of physical bodies in space and how machines can express the notions of lightness, balance, gracefulness, and so on. Five different machines will be built, each one representing the individual character of a fictive dance/theater play. Even if they will be quite different visually, each robot will be able to produce a wide range of physical movements by using extensions of its body. The robots will be following a general choreography but will also be able to get out of this loop and react as an individual to the proximity of the visitors. Untouched (Partie de chasse) is an installation project that aims to turn an industrial robot arm into a reactive organism. We will use the Fanuc M16iB industrial robot installed in the Alab studio at the Hexagram Institute. For security reasons, the installation will require to divide the space in two different areas: the interaction space and the robot space. In order to detect the presence of viewers in the interaction space, we plan to use a system that can localize and track sound sources, a machine vision system, as well as an elaborate set of sensors. Only the industrial robot will be occupying the robot space, viewers will be kept at distance. Live video images will also be transmitted from cameras filming the robot and cameras mounted on it. The particularity of this project resides in bypassing the normal programming paradigm of the robot in order to have it execute real-time commands instead of a predefined — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

188 189 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

sequence of actions. Many artists have used industrial robots in the past (Ken Goldberg, Stelarc, Robotlab) but they have always used them as simple automatons, more or less the same way they are normally used in car factories. None has ever tried to turn them into autonomous reactive creatures. With this project, we want to build a sensitive and responsive machine conceptually based on adaptive and evolutive behaviors. The Protozoic Machine project will integrate a sound source localization and tracking system and artificial muscle actuator technologies to produce an interactive robotic artwork. With this project, we want to build a single complex responsive machine able to interact with the viewers and express a wide range of lifelike behaviors (from purely reactive to wild and deviant). Contrarily to popular entertainment robots (like the Aibo robot dog, for example), the Protozoic Machine is deliberately built as a pure machine and does not visually represent any particular living being. Even if some of its parts are conceptually defined as “eyes”, “arms” or “legs”, it is made as an assembly of functional parts and somewhat of an abstract shape. Only its behaviors will turn the robot into a believable organism. Some of these behaviors will also be controlled by a reinforcement learning algorithm that will allow the machine to evolve over time, in different interaction contexts. The sound localization system will be driving two pan-tilt surveillance cameras that will act as the eyes of the creature, constantly observing the viewer and creating the impression of being observed. The robot will also have relative mobility, which will allow it or some of its parts to move toward or away from the viewers, as it senses the point of origin of the voices or the sounds in the environment. As the robot‘s physical extensions will be based on air muscle actuators, the viewers will also be able to touch it in different ways and interact in a tangible manner with responsive feedback. Acknowledgements

We would like to thank the Hexagram Institute for Research-Creation in Media Art and Technology (Montréal, Canada) for their support; the Canada Council for the Arts; the Conseil des arts et des lettres du Québec; Martin Peach, who has been a dedicated multitasking technician since many years; as well as the numerous graduate students who have been working as research assistants on many of these projects. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Bill Vorn. Adaptive Machines — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — References Arkin, R. C. Behavior-Based Robotics. Cambridge, Massachusetts: The MIT Press, 1998. Cadet, F. . Dean, M. . Driessen, E., M. Verstappen . Goldberg, K. The Robot in the Garden: Telerobotics and Telepistemology in the Age of the Internet. Cambridge, Massachusetts: The MIT Press, 2000. Kac, E. “Foundation and Development of Robotic Art”. Art Journal. 56.3 (1997): 60-67. Kantor, I. < http://www.istvankantor.com>. Macmurtrie, C. . Pauline, M. . Paulos, E. . Penny, S. . Rinaldo, K.. Ristow, C. . Robotlab. . Stelarc. . Stone, R. . Symbiotica. . White, N. . Whiting, J. .

— — — — — — — — — — — — — — — — » — — — — — — — — — — — — — — — — — —

190 191 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

U

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

I r

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

mBx Öwh g

>

t W lD§ nf u

c — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

K Lc

uu8GH imw l h JBW r t E Öfgzc F 8 tß m f r

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Biographies

Biographies — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — — Krimhild Becker

Krimhild Becker studied Painting at the “Kölner Werkschulen” (Cologne, Germany) 1960-1966. 1970 she began making photographs, substituting the paint-brush by a camera. Krimhild Becker is working as an artist, editor, and curator. National and international exhibitions. Art residence Greece and USA (New York). Kunstfonds Bonn e.V. (1991-1992). Krimhild Becker lives in Cologne (Germany). Gary Cass

Gary Cass is a key scientific collaborator with SymbioticA: the art and science colla­ bo­ra­tive research laboratory based in the School of Anatomy and Human Biology at the University of Western Australia. As a scientific technician with the Faculty of Natural and Agricultural Sciences; teaching laboratories, his broad range of skills have been utilised over the last ten years. He is capable of tutoring and demonstrating many aspects of agricultural and related scientific practices. He has worked with many art projects engaged with biological specimens in these labs. His innate ability to teach science led to him co-ordinating and running the SymbioticA Biotech Art Workshop across Australia at Universities and Art Festivals, Kings College London and the University of California at Irvine. Scott Driscoll

Scott Driscoll came to the Music Technology Group at Georgia Tech (Atlanta USA), with a M.S. in Mechanical Engineering focusing on controls, and also a fair amount of coursework in computer science and electrical engineering. Before losing all of his time to engineering school, Scott played classical and jazz piano, clarinet, and composed a bit here and there. f18 institute

The institute for art, information and technology was founded in Hamburg (Germany) in 1996 by the artists Gwendolin Taube, Stefan Doepner and the engineer Lars Vaupel and has today become a pool of various artists and technicians who work internationally as well as on-site in Hamburg. The institute creates technified environments and robotic installations for public areas, which leads to better understanding of the relationship between man and technology. They work with modern technology and try to reflect the common ideas of future inventions from the present and the past. Mostly they connect this technology with everyday applications and surroundings to offer a — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

194 195 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — —

way of finding one‘s own approach to understanding social and technical developments. Realization Bremen: Jan Cummerow, Stefan Doepner in collaboration with Dennis Tan und Marc Pira.; www.institutf18.org Peter Gendolla

Peter Gendolla has studied art history, philosophy and literature in Hanover and Marburg/Lahn, received his Ph.D. in 1979 and completed his habilitation in 1987. He has been professor for literature, art, new media and technologies at the University of Siegen (Germany) since the end of the year 1996. Peter Gendolla is director of the University’s publishing house UniVerSi. 2 and spokesperson of the research institute “Medienumbrüche” where he is head of the project B6, “Literatur im Netz / Netzliteratur” Friedrich Wolfram Heubach

Friedrich Wolfram Heubach (born 1944) studied Psychology, Sociology, and Art History, and received the diploma and PhD degrees. 1968 co-founder and editor of the art journal “interfunktionen” (– 1974). 1984 State doctorate (“Habilitation”) in Psychology. 1985-1989 Professor of Psychology at the University of Cologne, and later at the University of the Arts Hamburg. Since 1992 Chair of Psychology and Pedagogy at the Academy of the Arts Düsseldorf. Jin Hyun Kim

Jin Hyun Kim studied musicology and philosophy at the Seoul National University and the University of Hamburg. Since 2002, she is a member of the interdisciplinary research group “Media and Cultural Communication“ (SFB/FK 427) funded by the German National Research Foundation (DFG) at the University of Cologne (Germany). From 2002 to 2004, she worked at the subproject on Interactive Music and gesture controlled musical interfaces. Since 2005, she has been engaged in the subproject “Artistic Interactivity in Hybrid Networks” dealing with Interactive Media Art including interactive audio programming (live coding), robotic arts and Artificial Life Art in cooperation with the Academy of Media Arts (KHM Cologne). Christoph Lischka

Christoph Lischka studied Composition, Piano, Musicology, Philosophy, and Mathematics at Cologne University of Music, University of Cologne, and University of Bonn. — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Biographies — — — — — » — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Software engineer, artist, research scientist, and university lecturer (e.g. Fraunhofer Institute, Cologne University of Music, Academy of the Arts Düsseldorf) in the fields of Artificial Intelligence, Music Theory, Cognitive Science, Philosophy, and Robotics. In his current research the focus is put on the interplay of art and convergent technologies (NBIC), particularly nanobiotechnology. Since 2007, Christoph Lischka is professor of Poietic Machines (Autoaktive Systeme) at the University of the Arts Bremen. Alan Mullett

Alan Mullett has worked in the information technology disciple for over ten years and has a keen interest in the amalgam of art and science. Possessing a unique mix of mechanical, electronic and scientific knowledge, he brings a creative and innovative approach to Bioalloy. Alan is extremely enthusiastic and has unique ideas about the direction of Bioalloy, in both its design and creation. Alan holds the position of Information Technology Manager in the Faculty of Natural and Agricultural Sciences at the University of Western Australia. This is a position which frequently requires innovative solutions to problems and one which exposes him to a wide range of groups and activities. He is also a member of the Technical Advisory Group to the University. Lüder Schmidt

Lüder Schmidt studied physics in Darmstadt and Kiel, Systematic Musicology in Hamburg. Since 2000 he is scientific assistant at the Institute of Musicology, Cologne University (Germany); since spring 2004 co-operation with sub-project C10: Artistic Interaction in Hybrid Networks of SFB/FK 427: Media and Cultural Communication. Andrea Sick

Andrea Sick studied German Language and Literature, Political Science, Cultural Studies, and Art History in Heidelberg, Bremen and Hamburg. Andrea Sick is professor at the University of Arts Bremen (Germany) for Media Theory and Cultural Studies and responsible for the creation and concept of a network between Art and Science in the New Media Technologies. Focus of work and research: relations between technological media and cultural production, research on transitions between biological and information-technological discourses, interfaces of scientific and cultural activities, genderstudies.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

196 197 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — » — — Bill Vorn

Bill Vorn has been working on robotic art projects since 1992. His installation work involves robotics and motion control, sound, lighting, video and cybernetic processes. He teaches Electronic Arts in the Department of Studio Arts at Concordia University (Canada). His work has been presented at many international events, including Ars Electronica, ISEA, DEAF, Sonar, Art Futura, EMAF and Artec. He was awarded the Life 2.0 award (in 1999, Madrid), the Leprecon Award for Interactivity (in 1998, New York) and the Prix Ars Electronica Distinction award (in 1996, Linz). Jutta Weber

Jutta Weber received her PhD with extinction from the University of Bremen with a comparative study on concepts of nature in postmodern philosophy and ALife research. 2004 - 2006 she was senior researcher conducting theoretical and empirical research in the field of robotics, computer science and Artificial Intelligence at the Department of Philosophy of Science at the University of Vienna. Her current research is in Science and Technology Studies, Epistemology, Philosophy of Science, Inter-/Transdisciplinarity, Gender Studies, Cultural Studies. Since 2006, Jutta Weber is visiting professor of Interdisciplinary Studies at the University of Duisburg-Essen (Germany). Gil Weinberg

Gil Weinberg attempts to expand and enrich musical expression, creativity, and learning through technology. His résearch interests include new instruments for musical expression, musical networks, robotic musicianship and sonification. His interactive musical projects have been performed with leading orchestras and have been presented in museums and festivals worldwide. Gil Weinberg received his Masters and PhD in media arts and sciences from MIT. He is currently the director of the Music Technology Group at Georgia Tech.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —