Semiotics and Visual Communication: Concepts and Practices 1443854689, 9781443854689

Table of contents :
TABLE OF CONTENTS......Page 7
LIST OF FIGURES......Page 10
LIST OF TABLES......Page 12
PREFACE......Page 13
ACKNOWLEDGEMENTS......Page 17
CHAPTER ONE......Page 19
EMBEDDED, INTROSPECTIVE AND POETICNARRATIVES IN 3-DIMENSIONAL DESIGN......Page 20
HOW TYPE CANMOVE US—TYPE IN THE ENVIRONMENT......Page 29
DOUBLE WRITINGIN ARCHITECTURAL DESIGN......Page 41
CHAPTER TWO......Page 51
THE RENAISSANCEOF ACADEMIC PUBLISHING......Page 52
MARKETING SEMIOTICS APPLIEDTO THE DESIGN OF INTEGRATEDGRAPHIC COMMUNICATION SYSTEMS......Page 63
(DE)CODING THE FABRIC OF THE EUROPEANYEARS’ VISUAL REPRESENTATIONS......Page 74
INTERSEMIOTIC TRANSLATIONIN ADVERTISING DISCOURSE......Page 90
THE GREEK-CYPRIOT DIALECT IN WRITING......Page 104
CHAPTER THREE......Page 117
SIGNS AT THE INTERFACE......Page 118
LUDIC AND SOCIAL MEDIA INTERACTIONDESIGN PRINCIPLES IN SMART CITYDEVELOPMENT......Page 128
TYPOGRAPHY AND LANGUAGE......Page 144
FILM AND NEW ARTMEDIA SEMIOTICS......Page 157
KINETIC TYPOGRAPHY......Page 168
CHAPTER FOUR......Page 181
100 THINGS......Page 182
VISUAL DIASPORAS......Page 195
THE DEVELOPMENT OF A VISUAL LITERACYCOURSE IN HIGHER EDUCATION......Page 211
“MY FIRST EXPERIMENT” “MY FIRST EX”......Page 225
DEVELOPING STUDENTS’ VISUAL DESIGNCOMPETENCE THROUGH SITUATEDLITERACY PRACTICES......Page 242
THE QUEST FOR “VISUAL THINKING”AND THE DOUBLE BIND OF EDUCATION......Page 257
PROJECTMY CITYMY PLACE......Page 274
THE RECEIVER IS THE MESSAGE?......Page 287
ACOURSE IN VISUAL COMMUNICATION......Page 302
CHILDREN ARE PAINTING INSCRIPTIONS......Page 313
CHAPTER FIVE......Page 327
MARKS, SIGNS AND IMAGES......Page 328
SHOWING SAYING......Page 340
CONTRIBUTORS LIST......Page 350

Citation preview

Semiotics and Visual Communication: Concepts and Practices

Semiotics and Visual Communication: Concepts and Practices

Edited by

Evripides Zantides

Semiotics and Visual Communication: Concepts and Practices, Edited by Evripides Zantides This book first published 2014 Cambridge Scholars Publishing 12 Back Chapman Street, Newcastle upon Tyne, NE6 2XX, UK British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library Copyright © 2014 by Evripides Zantides and contributors Book Cover design and copyrights by Theseas Mouzouropoulos All rights for this book reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior permission of the copyright owner. ISBN (10): 1-4438-5468-9, ISBN (13): 978-1-4438-5468-9

…to all the graphic warriors

TABLE OF CONTENTS

List of Figures.............................................................................................. x List of Tables ............................................................................................. xii Preface ...................................................................................................... xiii Acknowledgements ................................................................................. xvii Chapter One: Architectural and Spatial Design-Design for Three Dimensional Products Embedded, Introspective and Poetic Narratives in 3-Dimensional Design .... 2 Ralph Ball How Type Can Move Us—Type in the Environment: France .................. 11 Jeff Leak Double Writing in Architectural Design: A PhenomenologicalSemiotic Approach .................................................................................... 23 Theodora Papidou Chapter Two: Design for Print Applications The Renaissance of Academic Publishing: The Deconstruction of the Journal into a Pragmatic Manifestation of a Postmodernist Set of Discourses ....................................................................................... 34 Artemis Alexiou Marketing Semiotics Applied to the Design of Integrated Graphic Communication Systems ........................................................................... 45 Dora Ivonne Alvarez Tamayo (De)coding the Fabric of the European Years’ Visual Representations .... 56 Camelia Cmeciu and Doina Cmeciu

viii

Table of Contents

Intersemiotic Translation in Advertising Discourse: Plastic Visual Signs in Primary Function in Communication .......................................... 72 Evangelos Kourdis The Greek-Cypriot Dialect in Writing: Orthographic Conventions and Typographic Practices......................................................................... 86 Aspasia Papadima, Ioli Ayiomamitou and Stelios Kyriacou Chapter Three: Design for Screen Based Media Signs at the Interface: An Exploration of Semiotics and Interaction Design...................................................................................................... 100 Nikos Bubaris Ludic and Social Media Interaction Design Principles in Smart City Development ........................................................................................... 110 Patrick J. Coppock Typography and Language: A Semiotic Perspective ............................... 126 Jack Post Film and New Art Media Semiotics: On the Figural ............................... 139 Irini Stathi Kinetic Typography: A Semiotic Exploration ......................................... 150 Theo van Leeuwen and Emilia Djonov Chapter Four: Pedagogy of Visual Communication 100 Things: A Process for Foundation in Theory and Practice ............... 164 Law Alsobrook Visual Diasporas: Comics as Transcultural Phenomena ......................... 177 Holger Briel The Development of a Visual Literacy Course in Higher Education ...... 193 Anastasia Christodoulou and George Damaskinidis “My First Experiment” “My First Ex”: A Multimodal Tool Proposed in the Didactics of Literature ................................................................... 207 Symeon Degermentzides

Semiotics and Visual Communication: Concepts and Practices

ix

Developing Students’ Visual Design Competence through Situated Literacy Practices: The Case of the Erasmus IP “P.S.BoWMa” ............. 224 Catherine Dimitriadou and Androniki Gakoudi The Quest for “Visual Thinking” and the Double Bind of Education ..... 239 Miltos Frangopoulos Project My City My Place: A Cross-Cultural Collaboration in Graphic Design...................................................................................................... 256 Maryam Hosseinnia The Receiver is the Message? ................................................................. 269 Peter C. Jones A Course in Visual Communication ........................................................ 284 Tony Pritchard Children Are Painting Inscriptions: Pedagogy of Visual Communication in Local History ....................................................................................... 295 Evangelia Svirou, Ifigeneia Vamvakidou and Paraskevi Golia Chapter Five: Visual Arts Marks, Signs and Images: The Sense of Belonging and Commitment which Pre-Dates History but has now become a Powerful Global Language ................................................................................................. 310 Paul Middleton Showing Saying: On Speech Balloons .................................................... 322 Lizzie Ridout Contributors List ...................................................................................... 332

LIST OF FIGURES

Figure 1-1: Swan and Dolphin Hotels by Michael Graves/ Notch and Tilt Best Supermarkets by SITE Figure 1-2: light and shade collection Figure 1-3: the complete history of shelf supports Figure 1-4: One Day I’ll design the Perfect Paper Lampshade Figure 1-5: power tower Figure 1-6: Transparent Figure 1-7: The removal of the billboard has left this sign visible behind the later poster frame Figure 1-8: This gable end shows four advertising messages all overlaid and revealed by the passage of time Figure 1-9: These two signs for the alcohol brand Suze were photographed in different parts of France and go someway to illustrating the visual differences that painted signs wrought on brand marks Figure 1-10: Space of inscription: it is circumscribed by double writing and geo-graphy. Figure 1-11: Architectural design as semiosis. Figure 2-1: Snapshot fromTalk to Her by Pedro Almodovar Figure 2-2: Camera Obscura: feminism, culture and media studies Figure 2-3: Camera Obscura: feminism, culture and media studies Figure 2-4: Stages of the design process Figure 2-5: Logos of European Years Figure 2-6: Print advertising of Cyprus Bank Figure 2-7: Print advertising of the Greek mobile network operator Cosmote Figure 2-8: Print advertising of Piraeus Bank Figure 2-9: Consonants-vowels representation in words with the same meaning, both in GC and SMG. Figure 2-10: Motifs created by the counter-forms of characters in GC Figure 2-11: How young people write in GC Figure 3-1: The opening screen of the Ermou Street multimedia application Figure 3-2: Activating interactive objects Figure 3-3: Smart City promotion images from three incidentally selected web sites

Semiotics and Visual Communication: Concepts and Practices

xi

Figure 3-4: Connotative system Figure 3-5: Nameplate Le Monde Figure 3-6: Four different realisations of the letter d Figure 3-7: Triadic sign model Figure 3-8: Saussure Figure 3-9: A conceptual and a narrative image Figure 4-1: 100 Things Figure 4-2: An example of word :: image :: image :: word Figure 4-3: Student critiquing another student’s work Figure 4-4: Final poster examples from 100 Things Figure 4-5: Teaching scenario a comparison between Florina and Bitola Figure 4-6: The teaching scenario Arcturos Figure 4-7: The teaching scenario Christian Orthodox Religious symbols in Prespes Figure 4-8: Statement of the Eames Design Process by Charles Eames Figure 4-9: Laszlo Moholy-Nagy: Visual Representation of the novel Finnegans Wake by James Joyce, Figure 4-10: Grammar as “memory image” and “object alphabet” from Johannes Romberch Figure 4-11: AUK. Male Kuwaiti student, age 29 Figure 4-12: MSUM Male American student, age 22 Figure 4-13: AUK. Female Kuwaiti student, age 19 Figure 4-14: MSUM. Female American student, age 20 Figure 4-15: The research proposition as an example of the eclectic and indeterminate nature of the Design process. Figure 4-16: Grouping Figure 4-17: Density Figure 4-18: The T-ness of T Figure 4-19: Typographic hierarchy exercise Figure 4-20: A Vyzantis sign Figure 4-21: A children's sign-The doctors Figure 4-22: A children's sign–VOLT, a sport shop Figure 5-1: Tag created in The Graffiti Creator application Figure 5-2: Catherine Middleton’s (Duchess of Cambridge) coat of arms Figure 5-3: Catherine Middleton’s (Duchess of Cambridge) coat of arms Figure 5-4: This Nike advertisement is a clear reference to the nativity star Figure 5-5: Greenpeace’s ‘rebranded’ BP symbol Figure 5-6: Soliloquy [After Bakhtin] Figure 5-7: Soliloquy [After Bakhtin] Figure 5-8: Compilation of pages from Ways to talk and yet say nothing, or ways to not talk and yet say everything

LIST OF TABLES

Table 2-1: The cognitive flow beyond the European Years’ logos Table 2-2: Type visual deontic modality Table 2-3: Token visual deontic modality Table 4-1: An overview of the structure of the course Table 4-2: Multimodal factors Table 4-3: The semiotic practices comprising the IP P.S. BoWMa within the concept of “design” Table 4-4: Inventory of the resources used for the compilation of the teaching scenarios

PREFACE

This book is the result of selective research papers that were presented at the first international conference of Semiotics and Visual Communication at the Cyprus University of Technology in November 2011. The conference was built around the theme from theory to practice and brought together researchers and practitioners who study and evaluate the ways that semiotic theories can be analysed, perceived and applied in the context of various forms in visual communication. Within a Semiotic framework, the book explores research questions under five main thematic areas: Architectural and Spatial Design-Design for Three-Dimensional Products, Design for Print Applications, Design for Screen-Based Media, Pedagogy of Visual Communication and Visual Arts. It investigates Semiotics, not only from a theoretical and historical perspective, but also from an applied point of view, looking at how theory can be implemented into design and visual communication practice. A key feature of the book is the diversity of 25 essential contributions by 33 academics and practitioners that display their concepts and ideas on Semiotics within the interdisciplinary nature of Visual Communication. From Plato’s Cratylus to structuralism and post-structuralism, the presence and aspects of Semiotics as defined by linguists, have always been strongly present and applied in non-verbal languages. The selected authors that follow are a proof of the fascinating research and design opportunities that constantly emerge and enrich, at the same time, visual communication. Ralph Ball is concerned with the generation of artefacts, which deconstruct and reconstruct design axioms and ideologies. He presents an evolving series of conceptual artefacts, which act as visual reflections on Modern, Postmodern and Contemporary design culture, as well as reexamines typologies and generic forms with reference to the rhetorical themes and axioms specific to Modernism. Jeff Leak successfully juxtaposes parallels between the caves of Lascaux and the old painted roadside advertising that decorates the roadside in France. He suggests that the simplicity of their temporally altered messages implies a different kind of social interaction; a more civilised societal code and explores their Frenchness from a semiotic perspective. Theodora Papidou investigates the moment of first retaining an intention in lines and words during the process of architectural design as ushers in the activation of a mechanism,

xiv

Preface

fixing architectural thought in iconic and verbal signs. She looks at the traces of this first instance of retaining up to the emergence of form and concept, as well as the text of the special design writing to be regarded as a unit of signification. Artemis Alexiou re-examines the design and layout of academic print journals in order to reflect the nature of contemporary academic discourse more, as well as introduce a concentrated design conceptualisation and production into the sphere of academic journals to improve their visibility and promote academic ideas to a wider audience beyond the academic communities. Dora Ivonne Alvarez Tamayo focuses on the production of discursive systems aimed at promoting the brand positioning through the articulation of visual messages that are issued in different media, by which an organisation, company or product uses to contact their users. Camelia Cmeciu and Doina Cmeciu collaborate on the deep structure of unity with the EU communication campaigns for promoting European Years, common to all the member states, and look at their diversity with the implementation of the issues of each European Year within public communication campaigns of different member states’ organisations. Evangelos Kourdis presents selected cases of intersemiotic translation in advertisements adopting Groupe ȝ.’ s (1992) approach and he examines Greek examples whose intersemiosis is based primarily on the interpretation of the verbal system by plastic visual systems that co-exist with iconic visual signs. Aspasia Papadima, Ioli Ayiomamitou and Stelios Kyriacou investigate issues related to the interplay of typography and orthography design for a non-codified dialect. They engage in researching the orthographic representation of the nonstandard Greek-Cypriot dialect spoken by the Greek-Cypriots in Cyprus. Nikos BubarisI explores Semiotics and Interaction Design by proposing a synthetic model of four communicative functions of user-interface signs: modes of remediation, action-oriented representations, nodes in information maps and computational effects by reference to a multimedia application that a team of students produced as an assignment. Patrick J. Coppock focuses on how “ludic and social media interaction design principles” may be useful for mapping, planning, designing and realising people-friendly urban spaces in Smart(er) Cities. Jack Post proposes that typography can be considered as a poetical or aesthetic language that subverts the primary functions of the alphabet and written language. He recommends that a semiotics of typography is possible, and can be approached as a secondary poetic organisation of the planar written surface. Irini Stathi indicates that the heterogeneity and complexity of multimodal coded texts probably require new semiotic concepts and likely new methods of research in order to delineate the role of the figural in the

Semiotics and Visual Communication: Concepts and Practices

xv

screen-based arts’ signification. Theo van Leeuwen and Emilia Djonov explore kinetic typography as a fundamental change in the semiotic landscape and investigate the potential meaning of this new semiotic mode, what we can ‘say’ with it and how, as well as how this potential meaning comes about. Law Alsobrook unpacks some of the methods and means by which teaching sophomores the language of design is taking place, while they explore the design process. Holger Briel looks at comics as transcultural phenomena and analyzes how graphic novels can become effective teaching tools by investigating common challenges that come from students themselves. Anastasia Christodoulou and George Damaskinidis examine how literacy can be practised when analysing video as a new form of multimodal text. By employing this new concept of pedagogy, they aim to introduce a framework to describe the activities of individuals as they identify, read and create new texts using various semiotic codes. Symeon Degermentzides proposes an educational software that aims to register information about the choices a student makes concerning meaning-making resources, while surfing on the Internet. Catherine Dimitriadou and Androniki Gakoudi explore the ways that situated literacy practices can contribute to the development of student teachers’ semiotic awareness and their competence as educational agents. Miltos Frangopoulos negotiates the quest for “visual thinking” and the double bind of Education so as to assist students to move beyond mere transmission or ‘communication production’, towards invention, confronting the more significant issues related to the authorship of meaningful proposals. Maryam Hosseinnia looks at cross-cultural collaboration in design learning and she uses the semiotic approach to analyse, process and interpret messages/signs to understand their meaning and their influence on people’s interpretation in different parts of the world. Peter C. Jones outlines the early stages of a practice-based PhD into the effect on communication design methodologies and outputs, by substituting established types of market segmentation with theories and categories used by teachers to identify the styles or models by which people learn. Tony Pritchard presents a Visual Communication course from the London College of Communication that its postgraduate design sets out to demystify the theories and practices of visual communication. Through the presentation of a case study, he aims to show how semiotics and related theories are applied in a practical learning and teaching context. Evangelia Svirou, Ifigeneia Vamvakidou and Paraskevi Golia outline the outcome of a didactic proposal for children who learn about local history. Their theory and analysis are based on “reading” students’ products using social semiotics within the scope of visual communication.

xvi

Preface

Paul Middleton explores the emergence of primitive marks as signs which identified people long before a verbal language emerged. He builds on them as marks that followed a different evolutionary path, suggesting that they communicate with simplicity and economy; conveying far more through simple juxtaposed images than a complex series of words. Lizzie Ridout brings together theoretical, historical and practice-based research to examine the semiotic aspects of speech balloon. In part, she outlines a collection of ruminations, preoccupations, truths and tales examining the speech balloon and its dear, yet distinct relative, the thought balloon. Taking this publication as a starting point for action, combined with the constitution of Cyprus Semiotics Association in 2013 and the support of the Hellenic Semiotic Society, Cyprus begins to have a place on the international Semiotic map. I hope you enjoy this book and find it stimulating and useful in providing some answers on putting semiotic theory into visual communication practice. Evripides Zantides Lemesos www.svclab.com

ACKNOWLEDGEMENTS

This book, as a result of the first International Conference on Semiotics and Visual Communication that was held in Cyprus in November 2011, would not have been possible without the contribution and help of its scientific and organising committees, reviewers, speakers and volunteers who were involved throughout its process and accomplishment. Special thanks must go to Savvas Christodoulides, Anastasia Christodoulou, Antonis Danos, Miltos Frangopoulos, Matthew Hobson, Marianna Kafaridou, Evangelos Kourdis, Alexandros Ph. Lagopoulos, Jeff Leak, Theo van Leeuwen, Paul Middleton, Arafat Al Naim, Grigoris Paschalidis, Marios Phocas, Ifigeneia Vamvakidou and Lia Yoka. Many thanks also to Monika Herodotou, Eleftheria Iasonos, Christina Koutalis, Theseas Mouzouropoulos, Christina Nicolaou, Angelos Panayides, Aspasia Papadima, Ioanna Tymbiotou, Panayiotis Zaphiris for all their support, as well as to the Department of Multimedia and Graphic Arts at the Cyprus University of Technology, the Cyprus Tourism Organisation, +design magazine, the vcdc-visual communication designer’s club, the Hellenic Semiotics Society, the Semiotics and Visual Communication Lab at Cyprus University of Technology, the Cyprus Semiotic Association, Cambridge Scholars Publishing and all my colleagues, friends and students who respond positively in the struggle for contribution to semiotic knowledge, visual communication and graphic design practice.

CHAPTER ONE: ARCHITECTURAL AND SPATIAL DESIGN— DESIGN FOR THREE DIMENSIONAL PRODUCTS

EMBEDDED, INTROSPECTIVE AND POETIC NARRATIVES IN 3-DIMENSIONAL DESIGN RALPH BALL

My research is concerned with the generation of artefacts, which deconstruct and reconstruct design axioms and ideologies. The studies presented in this paper form part of an evolving series of conceptual artefacts, which act as visual reflections on Modern, Postmodern and Contemporary design culture. Typologies and generic forms characteristic of modern furniture and lighting are re-examined with reference to the rhetorical themes and axioms specific to Modernism. Axiom examples typical of the modernist canon include the following: Form follows Function (L Sullivan), Less is More (L. Mies van de Rohe), Decoration and Crime (A. Loos) often misquoted and interpreted as Decoration is a Crime; Starting from Zero and Continuous Revolution (W.Gropius) and Truth to Materials (C. Brancusi). Themes considered and examined include Transparency, Minimalism and Multifunctionality. The aim of the visual re-examinations is to simultaneously challenge and accommodate the above axioms in search of new, authentic forms of visual expression. Postmodernism, as understood with reference to design, challenged modernism’s formal purity claiming that modernism’s rational abstraction limits subjective and narrative expression. In ‘Complexity and Contradiction in Architecture’ (1), architect Robert Venturi famously paraphrased Mies van de Rhoe’s modernist mantra of “Less is More” with “Less is a Bore”. Venturi’s historical architectural analysis called for a replacement of modernism’s reductive abstraction with an intelligent, richly diverse, symbolic form in a contemporary context. He called for inclusiveness and ambiguity using terms like both-and rather than either–or. Whilst Venturi advocated complexity and contradiction, he was opposed to the incoherent and the arbitrary in architectural expression. However, his intensions were very often misinterpreted and corrupted and gave rise to much arbitrary and whimsical architecture: many so-called postmodern buildings employed an eclectic mix of architectural styles but often lacked any coherent, conceptual underpinning.

Ralph Ball

3

Fig. 1-1 Swan and Dolphin Hotels by Michael Graves (top of this image) Below are Notch and Tilt Best Supermarkets by SITE

For example, the arbitrary and decorative theatricality of the Swan and Dolphin Hotels by Michael Graves (2) (at the top of this image) is, I suggest, typical of postmodern architecture. Contrast this with the more unusual, ‘conceptual theatricality’ of architecture by SITE for Best Supermarkets. (3) These buildings present the supermarket authentically as a commercially pragmatic, big, simple box. Acknowledging the legitimacy of the big box allows entrances to be dramatically signified. Entrances are conceived and visually announced as inventive ways of ‘opening’ the box. My research similarly intends to filter the rationality of the modern through the contradictions and complexities of the postmodern without loss of conceptual authenticity. In these studies, modernism’s rational and reductive axioms are reframed or pushed to logical extremes in order to endorse the paradox and legitimise the invention of formal incongruities, rational irrationalities or poetic transgressions. Ironic iconics: the pieces are self-consciously introspective and are intended to reflect upon themselves and their inherent culture. The research studies presented use a method analogous to archaeology. The intention is to uncover ‘embedded’ visual potential. The aim is to expand or reinforce meaning in products by introducing or revealing latent narratives. Plausible forms are extrapolated in this manner by examining, and drawing from, a product’s formal, visual parameters and implicit contexts. This process introduces the concept of ‘design poetics’. Designs are materialised in ways that go beyond their function or even their symbolism, and playfully or critically reflect on a cultural meaning.

4

Embedded, Introspective and Poetic Narratives in 3-Dimensional Design

This ‘archaeological’ method is used to reinvest appreciation of that which becomes undervalued through familiarity. It intends to visually articulate areas, which are unregarded, celebrating the generic rather than the specific: it intends to find the extraordinary in the ordinary, re-seeing objects and functions as if for the first time: it intends to envision fresh possibilities in commonalties, to start with the given and find new ways of expressing this within and through the objects studied. The studies, therefore, involve the reconfiguration of familiar, archetypical products. These product types we have called ‘mature typologies’ and defined as objects, which generally have an agreed consensus on basic form and application. With this definition as a point of base reference, we now illustrate a series of visual studies together with their various rationales.

Light and Shade The ‘Light and Shade’ series explores the formal relationship between generic light bulb and lampshade. A reconfiguring of this relationship transforms the reading of the geometrically abstract form of the modernist lampshade, (a truncated cone) into a series of more specific concrete objects and meanings. The truncated cone, a formally abstract container of light, is turned into other, familiar container types. By reconfiguring the relationship of bulb to shade, the ‘light-shade’ can be variously read as fruit bowl, plant-pot, skirt or bucket. Each of these formal changes, in turn, explores further modernist reference.

Fig. 1- 2 ‘light and shade collection’ steel, stone, aluminium, glass, fabric various sizes © 1997Ralph Ball

Golden Delicious The inverted truncated cone becomes a container. This generic container is then more specifically identified by the choice of content. In the first object many light bulbs are piled into the container, over one single lit bulb, turning the bulbs into metaphorical fruit and, by association, the

Ralph Ball

5

container into a fruit bowl. The bulb, originally only the source of light, when used here ‘en masse’, also becomes the diffuser or shade element. The original shade becomes a frame to hold the new diffuser in place. The configuration produces an elegant paradox with reference to the modernist axioms of Brancusi’s ‘truth to materials’, Mies van de Rhoe’s ‘less is more’ and Adolph Loos’s ‘decoration and crime’. Here is the apparent conundrum. This artefact uses ‘pure’ form and material: the light bulbs are generic (truth to materials). The elements are functional and used efficiently: the bulb acts as both light source and light shade (conceptually less is more). Yet conversely, the result is excessive and decorative in the quantity of bulbs used and in their playful distribution. In effect, this construct breaks the rule of Loos’s ‘decoration and crime’ whilst simultaneously adhering to the ‘less is more’ and ‘truth to materials’ values of Mies van de Rohe and Brancusi.

Generations In the second example called “Generations”, the container is, again, more specifically identified by content. The truncated cone here contains a vertical tube from the end of which sprout two dichroic reflectors. The configuration turns these reflectors into metaphorical flowers and, by association, the container into a plant pot. Dichroic reflector bulbs are clearly a more recent generation of light source than the classic and generic ‘Edison’ bulb. The original term ‘light bulb’ was clearly coined with reference to its shape, resembling that of organic bulbs and tubers. In this construct, an upside down light-shade implies the containment of the classic Edison bulb within the cone. From this ‘now potted’ Edison bulb, newer generations emerge and flower. Here, again, Loos’s Decoration and Crime (variously misquoted but commonly interpreted as ‘decoration is a crime’) is subverted despite using only the essential, unadorned elements of light making. The configuration is essentially figurative rather than abstract.

Task Light (Wall-washer) In this third example, the truncated cone is inverted again and hung onto a ladder structure. This ladder frame, by association, turns the reading of the cone metaphorically into a window cleaner’s bucket. The aluminium ladder enables low voltage current to be conducted through the frame. This, in turn, allows the cone, now a ‘bucket of light’, to be lifted from rung to rung and to relight on contact with each of the rungs.

6

Embedded, Introspective and Poetic Narratives in 3-Dimensional Design

The modernist preoccupation under examination here is ‘adjustability’. However, instead of the sophisticated adjustability of the angle poise, dimmer switch or multi-track lighting, adjustability is expressed here in one of the most fundamental forms. The light source can only be height adjusted by the simple process of lifting it from rung to rung: a simple, visibly accessible, analogue process supported by a sophisticated and invisible low voltage application.

Switch The relationship of formal position is explored in the fourth object. The conventional relationship in a freestanding standard lamp is clearly that the shade covers the bulb at the top of a supporting column. In this study, the shade is dropped to the ground, leaving the bulb unshielded. An unshielded bulb is often referred to as a ‘naked’ light. The cone in this ground level position may now be perceived as a dropped or slipped skirt. The on/off positional relationship of skirt to bulb also acts as a switch for the electrical on/off. The skirt may be moved from the ground back to covering the bulb. The physical and electrical‘switch’ acts in opposition to each other, doubling the ‘switch’ concept. Putting the skirt on (covering the bulb) switches the light off, and taking the skirt off switches the light on.

Fig. 1-3 ‘the complete history of shelf supports’ timber paper card 200x250x900mm ©1998 Ralph Ball

“The Book of Sand” is a short story by the Argentinean writer Georges Luis Borges (4). In this story, Borges describes the existence of an infinite book: a book in which any page, once seen, can never be found again on the book’s re-opening. A book in which front and back pages can never be reached. Further pages always intervene no matter how hard the fingers try

Ralph Ball

7

to get between the pages and the covers. “The Complete History of Shelf Supports”, illustrated above, represents a similar work of infinite fiction. It is made up of two volumes, which perform the function that the title describes. This is an inversion in which the books support the shelf instead of the other way round. In this work of fiction, volume one catalogues all of the shelf supports that have ever been designed. Volume two contains all of the possible shelf supports, which will be designed in the future. Walter Gropius, director of the Bauhaus, called for ‘Starting from Zero’ and ‘Continuous Revolution’. The purist rhetoric of modernism’s agenda was a quest for ideal forms. Its aim, for any given functional artifact, was to optimise form and material and to distil functional and formal essentials. “The Complete History of Shelf Supports” represents an ironic realisation of that elusive ideal. Being an infinite catalogue of shelf supports, it must, by definition, contain the quintessential support! The irony of that irony is that, as with Borges Book of Sand, when we consult our infinite catalogue, this ideal form is still predestined to remain elusive.

Fig. 1- 4 ‘One Day I’ll design the Perfect Paper Lampshade’ chrome plated steel heat-proof paper and ink, 400 x 250mm dia. ©2000 Ralph Ball

8

Embedded, Introspective and Poetic Narratives in 3-Dimensional Design

This piece is called “One Day I’ll design the Perfect Paper Lampshade”. It represents, rhetorically, a similar quest for functional and formal perfection. In this piece, a wire frame 'wastepaper basket' contains a light bulb set in the centre. Crumpled sheets of paper surround the light source, each sheet containing discarded sketch ideas for paper lights. The rejected sketch sheets function as the diffuser or shade. Process and product are intermingled; the process becomes the product and the outcome is both a product and a narrative about the trial and error of idea generation. The object represents a kind of perpetually renewed, conceptual ideal. The quest for the perfect paper light shade becomes, in itself, an icon for perfection. Legitimate, rational protocols conspire to produce an informal and incidentally constructed form. This study, following the advocacy of Gropius, ends in playful irony.

Fig. 1- 5 ‘power tower’generic plastic electric power sockets, plugs, bulbs and flexes 2000x400x400mm © Ralph Ball 1998

The study illustrated above is called “Power Tower”. “Power Tower” is a work of earnest rationality and, in consequence, also a work of complete, pedantic madness. It takes the banal and generic components

Ralph Ball

9

associated with lighting (plugs, sockets and bulbs) and reconfigures them in a more ‘poetic light’. The tower makes play with the modernist ideals of ‘adjustability’ using a mechanistic rather than electronic idiom. An invented, historical possibility (something which could have existed before track lighting), the piece is an absurd extrapolation into the past. The result is a kind of retrospective track lighting system understandable in analogue, pre-electronic format. In order to achieve this level of analogue adjustability, more than 100 sockets are used. The sockets are set in four different directions on each level of the tower. This enables the lights to be plugged in with different orientations on any given level. This kind of excessive effort turns the tower into a rationally irrational artefact. Stripped down to rudimentary, pragmatic elements with no intrinsic artifice, it is nevertheless a highly elaborate confection, both ordinary and extravagantly decadent.

Fig. 1-6 ‘Transparent’ glass, glass cleaning bottles with fluid. 750x750x300mm © 1997 Ralph Ball

This is called “transparent” and is clearly a play on one of modernism’s thematic preoccupations-transparency. Taking the axioms ‘form follows function’ and ‘less is more’ to absurdly logical extremes, the meaning and function of this table is as explicitly ‘clear’ as can be. In the modernist canon, transparency represents both unadorned form and the elimination of visual weight. The legs (glass cleaning bottles) support and maintain the glass top. Glass tops are often a pretext for the display of legs. Here, the legs not only support the glass, but also ensure their own visibility. A contradiction or conflict with the term ‘clear’ is engendered. Whilst the view through the surface is ‘clear’, the physical surface is not ‘clear’, it is interrupted by the added, functional presence of the cleaning nozzle

10

Embedded, Introspective and Poetic Narratives in 3-Dimensional Design

heads. Here, multifunctionality gets in its own way. Integrating a secondary function partially compromises the full use of the surface. Nevertheless, the table has an explicit sense of self. Self-supporting, self-cleaning, selfsustaining, self-promoting and self-evident… a visual narrative entirely transparent! This paper has focused primarily on lighting concepts together with two examples from furniture studies in table and shelf typology. Further studies in table and chair typologies have also been developed and form an evolving series of experiments in visual narrative, rhetoric and polemic design. These form the material for further papers exploring the subject. Initial indicative samples of further and ongoing studies using table and seating typologies can be found in Form follows Idea (5). 1 Complexity and Contradiction in Architecture Robert Venturi Publisher Museum of Modern Art 1966 2 Swan and Dolphin Hotels 1990 Disney World Florida Architect Michael Graves 3 Tilt Towson MD 1978 Notch Sacramento CA 1977 BEST Supermarkets USA SITE James Wines 4 The Book of Sand Jorges Luis Borges 1975 Spanish 1977 English Translated by Norman Thomas di Giovanni Original publisher EP Dutton and Penguin Books 5 Form Follows Idea Ralph Ball and Maxine Naylor Publisher Black Dog 2005

HOW TYPE CAN MOVE US— TYPE IN THE ENVIRONMENT: FRANCE JEFF LEAK

As a private person, I have a passion for landscape, and I have never seen one improved by a billboard. Where every prospect pleases, man is at his vilest when he erects a billboard. When I retire from Madison Avenue, I am going to start a secret society of masked vigilantes who will travel around the world on silent motor bicycles, chopping down posters at the dark of the moon. How many juries will convict us when we are caught in these acts of beneficent citizenship? David Ogilvy, Confessions of an Advertising Man (2011).

Signs used as a means of communication within the environment today are almost omnipresent. We mostly pass by them without taking the time to notice, to read or to perceive them–something that many advertisers are aware of these days. As designers, we should try to make the time to see these signs. Mankind has been using signs since ancient times to convey messages before language developed into what we know today; for example, the capitalised letter A is derived from an early depiction of a horned animal’s head facing the viewer, that has simply been rotated by 180o. Today’s painted signs exist thanks to a happy accident; their makers unaware of the longevity of the message implied. French painted roadside advertisements, like cave paintings, survive in part, due to a number of factors: a lack of light, so that the pigments are not faded nor bleached; an obscure or difficult to reach location that ensures that visitors and those who might damage or develop the location have been deterred; a kind, atmospheric environment that has not corroded or washed the signs away, and perhaps lastly; that these images have ceased to be relevant to their audience and simply been forgotten. Perhaps parallels can be drawn between the caves of Lascaux and the old painted roadside advertising that decorates the roadside in France, and which this paper seeks to consider and discuss.

12

How Type Can Move Us—Type in the Environment: France

France France is a flourishing hub of creative design and advertising, with designers such as Grapus and Philippe Apeloig being notable and wellknown exponents of contemporary French poster design. French posters initially developed a reputation for being the byword for understated and powerful advertising. Although the United States is still perceived as the mecca of advertising innovation, French advertising has clearly developed its own unique style. (Angelini and Federico, 1998)

Poster design, with the earliest and most notable being the work by Toulouse Lautrec and Jules Chéret in the 1800s, was inherently French. This birthright was developed and built upon with avant-garde posters by Cassandre and Savignac continuing this tradition of innovation, wit and clarity mixed with contemporary art influences. More often than not, one did not need to speak French to understand the inherent semiology evident in the striking imagery produced. The character of Bibendum – the archetype for Michelin – is typical of this Gallic flair for visual communication shorthand. Today, France is also home to the JCDecaux Group, one of, if not the world’s largest outdoor advertising company. It is this link, between the need to communicate, creative design and site or environment, that is the trinity of French poster advertising. However, it is also perhaps this forward moving innovation that is overlaying part of the discarded and forgotten heritage of the French poster. This disappearing history is real and can still be found on the sides of buildings up and down the country’s roads. The fact that they have been neglected, if not forgotten, is part of their allure and charm. “Every painting tells something about past times. It shows us how society has changed”, (Bartolomeo, 2007).

This patina of age adds something to our contemporary understanding of the visual message that is additional to that which was intended when the sign was newly made. Kress and van Leeuwen (2006 p.35) state that it is difficult to understand the ‘true’ meaning of such visual communication without the benefit of cultural and chronological signifiers. Particular features and modes of communication should be seen in the history of their development, and in the environment of all the other modes of communication which surround them. The use of the visual mode is not the same now as it was even fifty years ago in western societies; it is not

Jeff Leak

13

the same from one society to another; and it is not the same from one social group or institution to another (Kress, G and van Leeuwen, T., 2006).

Fig 1-7. The removal of the billboard has left this sign visible behind the later poster frame

However, it is this sense of a time past, of a culture past that intrigues. Travelling around France, one cannot help but be impressed by its culture, its scenery, its history; the timeless nature of the landscape – both natural and man-made. Despite the wow-factor of the Millau Viaduct, La Defense and countless other public and private developments that might embarrass Britain with their efficiency and expediency, it is the old and crumbling, grand or antique that enchants. This research has involved recording these signs at a particular moment in time, in their own devolution. Whether this need to record is born out of an impending sense of loss – that these signs will not last forever and when they go, they will take a piece of French culture with them–or whether recording them is an act of preserving a moment in time that is contemporary and suggestive of the now – that point in time, what was seen and experienced; the signs acting as material signifiers of Gallic culture. Advertising moved beyond the boundaries of cities and towns where signs were predominantly seen by pedestrians. The earliest images of painted advertising hoardings can be found in the photographs of William Henry Fox Talbot (1800-1877). Street scenes of Paris show buildings that are decorated with fresh, vivid and legible painted advertising messages.

14

How Type Can Move Us—Type in the Environment: France

France is a large country and so the development of the car and the rise of French car manufacturers such as Renault, Citroen and Peugeot amongst many others, was inevitable and also necessary to give the French people mobility. This, in turn, led to an increased opportunity to advertise to people on the move, and so began the era of large roadside advertising hoardings. Advertisers turned to houses, garages, barns or indeed any building that had prominent gable ends or walls on which to promote their wares as a medium to be exploited. Locations were sought on prominent structures at the entry points to cities, towns, villages and even one-dwelling, blinkand-you’ll-miss-it farmhouses and barns. A longer journey also reveals advertisements painted onto buildings alongside long and winding country lanes, seemingly isolated from commercial influence. This was to target the motorist specifically, driving past with only the natural landscape as companion–what better way to interrupt such beautiful monotony? These spaces would have been highly valued by advertisers wishing to promote their goods and services to the public and such walls were often in demand; with their owners being courted by both advertisers and their agents. It is evident in some of the painted advertisements that there is more than one message. Time and erosion will often damage the topmost message, but then perhaps reveal the message underneath creating a strange amalgam of words, colours and meanings; a type of palimpsest. It is this practice of overpainting that has in some ways preserved the underlying message somewhat more than the later overpainted sign; creating a painted barrier against the elements.

Fig 1-8. This gable end shows four advertising messages all overlaid and revealed by the passage of time.

Jeff Leak

15

Before, between and after the wars, posters were carefully and skilfully hand-painted onto these large brick and rendered canvasses. Often these advertisements were created by local signwriters working to clear, and sometimes not so clear, guidelines. Other times, companies would commission teams of painters to travel around and paint these advertisements. As such, the signwriters were often given an amount of franchise to transfer the design to the wall. Often they were faced with irregular shapes, unforeseen windows, gables or pipes. The wall may not have corresponded to the shape of the logo or design or perhaps the material texture of the wall’s construction may have placed limitations on the design’s transfer or detailing. Signwriters, in many instances, had to adapt and interpret, as can be seen in some of the variance, to be seen across France. It resulted in the kind of brand variety that would cause contemporary designers and brand managers to smile in bemusement or wince in horror. However, this variety does give these messages a personality that is lost in today’s ‘one-size-fits-all’ society. Does it really matter that the letters are not quite aligned, or appear slightly different in their style? The audience knows what they mean, what they are trying to say, while brand signifiers today need to be clearly identified and legally protected, it might be considered to be at the loss of the idiosyncratic, the ingenuous and the human. To transfer the design onto the wall, a signwriter would most often use a grid-based technique: drawing equal-sized squares over the supplied artwork and then, using chalk or graphite, drawing this same square grid many times larger onto the wall or sometimes even a cloth or paper. The contents of each square was then carefully transposed and enlarged into the larger grid until the image was successfully copied and enlarged. If the image was drawn onto a cloth or paper on the ground, it was then hung in place against the wall; small holes were made at key points along the lines of the drawn image and then chalk dust pushed through and onto the wall to give a visual guide. Once the design was successfully transposed, the painter would then apply paint to the wall, following these guides. Often the paint used was lead-based, although this has been long-since banned. However, this paint gave the designs a strong opacity and colour transfer when originally painted and it has also been a factor in their longevity. Usually the painter would work to apply the lighter pigments first, building up to use the darker colours later to overpaint and ‘correct’ the design as it progressed.

16

How Type Can Move Us—Type in the Environment: France

One has to remember that this was long before the computer and cheap digital printing revolutionised the billboard, ensuring our visual senses are now bombarded into passive submission. These painted ‘posters’ were meant to last from a time when life seemed slower. Products that regularly change, that espouse the ‘nouveau’ and improved today, were not as important then. The idea of change serving to regularly maintain, grow or revive sales in the face of everchanging and aggressive competition was an idea seemingly without place. Complexity in their layout and design was shunned in favour of clear, easy-to-read messages focussing primarily on the advertiser’s logotype. This made them easy to read both at a distance and at speed from a car. Occasionally, a simple sentence was added to serve as a witty mnemonic or companion to a strong image. The simplicity of many of these early painted signs might suggest that the sophisticated readings and semiological layers that modern audiences are able to understand, were not needed nor valued. It is true that widespread literacy was less developed in the latter part of the nineteenth and early part of the twentieth centuries. Perhaps advertisers simply wanted their audience to remember a word’s shape or colour so that they might recognise it in a bar or shop. Perhaps it was to establish the product’s name into the collective conscience at a time when there was no need to differentiate similar products in a crowded market. Over time, these posters became a bright commercial counterpoint to the often rural landscape of France; with vivid reds, greens, yellows and blues enhancing and contrasting with the colours of nature. Manufacturers such as Suze, Michelin, Dubonnet, Pernod, Igol and more, shouted their sales pitch from painted posters that appeared on buildings all around France. Brands more readily associated with, and for, the motorist – those who sold drinks, food, lubricants etc. – were seemingly the most widely promoted on these built hoardings. Perhaps most surprisingly in a modern context, is the prevalence of alcohol brands that seem clearly targeted at the motorist. Eventually, however, these posters came to be seen as a blight on the urban environment. Unlike today’s regulated environments, these images sprang up without planning permission and invaded the streets; covered and cloaked the local architecture in a host of messages that clamoured for attention in a discordant visual feud. While we now see these signs as being, perhaps, charming, the government of the time sought to control them and rein in their spread, dubbing them “the leprosy of the road” (Combier M., 2009).

Jeff Leak

17

Fig 1-9. These two signs for the alcohol brand Suze were photographed in different parts of France and go someway to illustrating the visual differences that painted signs wrought on brand marks.

After the Second World War, various Acts were passed in France to control the permissions needed to allow painted advertising onto buildings and also to limit these newly ‘endorsed’ images to a maximum size. Further, these laws controlled advertisements and walls already in existence and denied them being automatically repainted without these new controls being enforced. Law 217 placed aesthetic control into the hands of local administrators or wardens and non-compliance could mean a criminal sentence for the advertiser or signwriter. As a result, local municipalities were, and are, able to regulate advertising beyond existing national frameworks by using their own local courts and by enforcing zoning restrictions. This had an immediate effect on the cities, where the repainting of new advertisements had already been greatly affected by the Second World War. Signs that had commanded highly visible positions on buildings were neglected. It was too regulated or too expensive to overpaint them without any obvious or ready commercial gain.

18

How Type Can Move Us—Type in the Environment: France

Those signs in the big cities, such as Paris, have slowly disappeared as the city has developed and remodelled its buildings; and architecture has been changed to reflect contemporary use. These signs have been built over, knocked down or simply covered up. There is nothing to prevent developers from doing this. These signs seem not to have any importance, whether as an urban landmark, being part of local social history, or being regarded as important historical artefacts. “A true palimpsest on the walls, these graphic works are unfortunately being lost”, (Combier M., 2009). This change in the law and the moving away from painted advertising was not so dramatic however outside of the cities, but eventually the economics of this kind of advertising versus the new paper-based, printed posters and officially sanctioned ‘temporary’ poster sites meant that their demise was inevitable. Over the ensuing years, these painted signs became neglected. Modern motorways bypassed the small single carriageway roads on which these advertisements originally flourished. Radio and later television became the advertising media of choice and these beautiful and semi-permanent statements slowly fell into disrepair. Who needed these archaic messages espousing long obsolete products and who even saw them now?

Life became faster Fortunately, many of these signs have been neglected rather than overpainted, refaced or bulldozed. Part of the charm of France lies in its visual appearance; that of slow, care-worn, timelessness. Has that door been painted in the last forty years? How long has that chimneystack been close to collapse? Often, when it comes to houses, it is a studious look that belies modern and wellmaintained interiors. However, many of these painted posters seemed to have survived thanks to neglect. A lack of maintenance or the DIY spirit, of the kind which Britain is currently enthralled, has ensured that, thankfully, the only real enemy of these posters has been the elements. Over the years, these images have been the victims of blistering heat and sunshine, fierce winds and rain – the full gamut of France’s climate and its variance from north to south, east to west. As one might expect, the red colours in particular, in these posters, have been the worst hit victims of the elements, being bleached and degraded by the sunlight. Differing surface materials are also a factor when considering the longevity of the images painted onto them. The paint has reacted or been absorbed into the surface, or the texture of the wall has

Jeff Leak

19

caused the paint to peel where it has to flow over corners and around brick or granite blocks. But if you are careful you can still see the vibrancy of colour and form that has been there for 50, 60, 70 and more years. But should these signs be preserved, renovated or even restored? Not too long ago, these painted advertisements were more often viewed as an indecorous blight on buildings, defacing their beauty; their modern and crass messages serving to vandalise the building’s historic and aesthetic values. With modern eyes, used to sophisticated advertising messages, we are beginning to regard these messages as being connotative of a more erudite and civilised time and as an integral part of our urban environment rather than as an interloper. We can understand them by the virtue of their continuity with our own experience of being consumers of messages and yet we are often too distant in time to be able to relate to them in the way that Kress and van Leeuwen posit. The position the signs hold within individual and collective memories relates in some way to the survival of the decaying original rather than any attempt to restore it (Roberts S., 2010).

The decay and neglect of these signs might be seen as having value in defining our own discontinuity of experience and in helping us to construct a personalised fiction of history. It is this very tangible materiality that is central to the appeal of these advertisements. There has been some widely documented renovation of some of these painted signs, most notably the renovation of the “Savon Cadum” painted sign in Paris on the Boulevard Montmartre. This has sparked a heated debate among Parisians, some of whom are delighted that the mural was saved and not destroyed, and others who see the newly restored image as lacking ‘authenticity’ and making the social history of the city seem unreal. The ongoing debate around the social significance of the fading painted walls of France and the lack of any planning controls to protect them has led to the formation of a host of organisations being formed, as well as interested individuals campaigning for them to be recorded and respected. As a response to this lack of protective policy to compel building owners to maintain or restore their murals, CONPER (Conservatoire des Publicités Extérieures et Routières) is a relatively recent voluntary group

20

How Type Can Move Us—Type in the Environment: France

whose mission is to raise awareness of, document, record and even preserve the many painted advertising sites and locations across France. Meanwhile, in French speaking Quebec, there recently seems to be an enlightened approach to the issue of fading painted advertisements, Since they are eligible for maintenance support as part of the external appearance of a building, we ask to encourage their conservation and to classify them as cultural objects, witnesses of social, urban evolution (Brunel S. and Beaudet J., 2004).

The renovation of these posters would be, however, a retrograde action. The charm and fascination that they hold is inherent in their age, however this has degraded them; their originality. Recording and protecting, rather than preserving these signs, should be made part of the way that local municipalities act to preserve French culture; not only for its own residents, but for the cultural wealth of the nation. However, we can be comforted by the resurgence of interest in this almost lost art form. In the US, companies such as the Colossal Media Group, create new painted advertisements for enlightened clients. A modern variation that explores the physicality of the medium involves clever overpainting to create new or developing narratives, such as the BBDO campaign for Gillette "The World's Biggest Shave" (YouTube, 2011). In France, these historic painted advertisements, these faded and overgrown murals and palimpsests, denote the sense of a time past, of a culture past, that intrigues and engages the imagination. It causes one to imagine the past in the context of imagined understandings. The simplicity of their temporally altered messages implies a different kind of social interaction; a more civilised societal code. They denote Frenchness without any predetermined commission to do so. They accumulate in the mind as you travel and create fascinating narratives, disjointed in their message but unified in their visual approach and sharing the bleached and deconstructed degradation bought about by the passage of indeterminate periods of time and cultural history. Advertising signs within the environment today are almost omnipresent. We mostly pass by them without taking the time to notice, to read or to perceive them but we should not underestimate their cultural significance. Taken out of time, their meaning is made more apparent by our dislocation and displacement; we notice them because they are no longer advertising the brands or using the medium that we expect – they have become unusual and therefore a significant and integral part of the French countryside.

Jeff Leak

21

Bibliography Angelini, Eileen and Salvatore Frederico. 1998. “Understanding French Culture Through Advertisements”. Global business Languages. Accessed September 09, 2007. http://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=1044&context= gbl Archeologue.over-blog. 2009. “Le retour de Bébé Cadum, tout propre, tout neuf”. Accessed September 08 2011. http://archeologue.overblog.com/article-33650129.html Brunel Suzel. and Joances Beaudet J. 2004. “La Murale Urbaine :Pratique Et Fonctions”. Accessed May 09 2011. http://www.cbcq.gouv.qc.ca/murale.html Collingridge Vanessa. 2010. “Making History; Ghost Signs”. BBC Who do you think you are? Magazine, August. Colossal Media Group. 2011. Colossal Media. Accessed April 12 2010. http://colossalmedia.com/ Combier, Marc. 2009. “la lèpre de la route” (tr. ‘the leprosy of the road’). Accessed April 12 2010. http://culturcafe.blogspot.com/p/archivesexposition.html CONPER, 2008. “Le site officiel du CONservatoire des Publicités Extérieures et Routières”. Accessed 24 September 2009. http://conpermursreclames.uniterre.com/page2/&thisy=&thism=&thisd= Ghostsigns UK. 2010. “Ghostsigns”. Accessed April 15 2010. http://www.ghostsigns.co.uk/ Ghostsigns Blog. 2007. “Ghostsigns blog”. Accessed April 20 2010. http://brickads.blogspot.com/2007_06_01_archive.html Klein, Naomi. 2000. No Logo. London: Flamingo. Kress, Gunther and Theo van Leeuwen. 2006. Reading Images. The Grammar of Visual Design. 2nd ed. Oxford: Routledge. Mecanico, Bartolomeo. 2007. “Old Painted Roadside Advertisements”. Accessed September 09 2007. http://www.elve.net/padv/home.htm Ogilvy David. 2011. Confessions of an Advertising Man. London: Southbank Publishing. Roberts, Sam. 2010. “Ghostsigns: saving our hand painted advertising”. The Ephemerist 148, spring 2010. YouTube. 2011. “The World's Biggest Shave”. Accessed March 17 2011. http://www.youtube.com/watch?v=U05yZFoiyjE

22

How Type Can Move Us—Type in the Environment: France

Images Figures 1, 2 and 3 are photographs taken by the author between 2005 and 2007

DOUBLE WRITING IN ARCHITECTURAL DESIGN: A PHENOMENOLOGICAL-SEMIOTIC APPROACH THEODORA PAPIDOU

Architectural design is regarded as what can be rendered as a text, and the starting point for its study is the written traces, the first records of architectural thought in the form of lines and words. In this way, we shall explore the semiotic economy of a special iconic as well as verbal text, which is circumscribed, the design text. Instead of the standard distinction between depiction and verbally articulated expression as two fields which by definition oppose each other, for architectural design, iconic and verbal signs are elevated to a unit of signification. Depicting in lines and putting into words represent two ways of enunciating the same gesture, the gesture of writing: they constitute the double writing of architectural design. The concept of the sign, as described by the German philosopher Edmund Husserl, is what allows for a conscious shift from the distinction between iconic and verbal signs to a joint approach towards them with regard to design text, irrespective of their material status. Design text points beyond itself and at something that is out there, rather than expressing its creator. Because of that, by assuming spatial dimensions, the written signs of the design text open up an interstitial – as it will be argued – space, that of inscription, a space where writing prevails. Let us consider architectural thinking. By that I don’t mean to conceive architecture as a technique separate from thought and therefore possibly suitable to represent it in space, to constitute almost an embodiment of thinking, but rather to raise the question of architecture as a possibility of thought, which cannot be reduced to the status of a representation of thought. […] one considers architecture as a simple technique and detaches it from thought, whereas there may be an undiscovered way of thinking belonging to the architectural moment, to desire, to creation (Derrida, 1986, p. 17).

24

Double Writing in Architectural Design

The written traces of architectural thought Architectural design: the process of transforming and translating a main idea, which gradually assumes a specific form and which is finally materialised in the work of architecture. In its general form, the definition above conceals the role that the written trace plays in the aforementioned seemingly linear process of transforming and translating. Whether it involves the lines formulating some initial sketches, or the first statements with single words and phrases about the intentions of the architect-designer, the appearance of the trace, iconic or verbal, defines the seemingly constant and abstract flow of transformations and translations, as it sets the conditions for retaining meaning. Before the final formulation of the work of architecture, and of the piece of writing supporting the choices in design that have led to it, architectural thought is initially sustained in lines and single words: lines which seek a direction, and words which cannot yet configure a clear conceptual framework for design. That moment of first retaining, in lines and words, an intention that has not yet been stated ushers in the activation of a mechanism ‘fixing’ architectural thought in graphic signs.1 This paper discusses the semiotic origin of this mechanism,2 which demonstrates the special nature of architectural thought.

From the graphic signs of architectural design to double writing Exploring the mechanism that fixes architectural thought in graphic signs has rendered writing as the special viewpoint from which architectural design is approached. Considered within a broader framework, and as defined by the French anthropologist André Leroi-Gourhan, writing is the ability of fixing thought in signs.3 In the case of architectural design 1

The term “fix” (Feststellen) is employed in this paper in the sense provided by Martin Heidegger in his paper “The Origin of the Work of Art.” For Heidegger, fixed is something which is “outlined, admitted into the boundary,” which, in turn, “does not block off, but rather […] first brings what is present to radiance. […] The boundary which fixes and consolidates is what reposes, reposes in the fullness of movement” (Heidegger, 2002, p.53). 2 Research into this mechanism was also the topic of my doctoral thesis titled Architectural Design and its Space of Inscription (El proyecto arquitectónico y su espacio de inscripción), School of Architecture of Barcelona, UPC, Spain. 3 See Leroi-Gourhan, 1964, p. 261.

Theodora Papidou

25

specifically, writing is fixed in graphic signs but these are presented in two ways: as iconic signs (sketches, diagrams, designs) and as verbal signs (single words, phrases, texts). What is iconic and what is verbal, what is spatially formulated and what is verbally articulated concur, therefore, in the special writing of architectural design, although, in any case, they appear as graphic signs, that is, as signs of writing. Depicting in lines and putting into words represent two ways of enunciating the same gesture, the gesture of writing. Instead of the standard distinction between depiction and verbally articulated expression as two fields which, by definition, oppose each other, for architectural design iconic and verbal signs are elevated to a unity or, in other words, to a unit of signification. That is, iconic and verbal signs are regarded as graphic signs irrespective of their material status or, to put it in different terms, irrespective of their signifying body. In this way, a special text, iconic as well as verbal, is circumscribed, that is, the design text, through which architectural design is approached and whose semiotic economy we shall explore. Therefore, architectural design is considered here with respect to what can be rendered as a text, and the starting point for its study is the first records of architectural thought in the form of lines and words. Its study begins with what we have defined as design text, which is eventually elevated to an agent of signification for the work of architecture itself. Besides, the terms ‘line’ and ‘word’, instead of ‘form’ and ‘concept’, specifically underline the quality of design text as manageable material, they underline the material and written nature of language and depiction and, thus, the written nature of architectural design itself. Finally, it should be noted that a study regarding writing in architectural design does not seek the code which can transcribe the intellectual workings of the architect-designer clearly and thoroughly, but looks for those functions that remain unaltered during the process of design.

The definition of the sign by Edmund Husserl We shall ponder over this: do, after all, the linear writing of language and the spatial writing of depiction jointly constitute the writing of design or is our consideration of them as a double writing, as the special writing of design, legitimate? The answer lies in the definition of the sign. The concept of the sign is what allows for the conscious shift from the distinction between iconic and verbal signs to a joint approach towards them in design text, irrespective of their material status. The concept of the sign, in particular, as this is

26

Double Writing in Architectural Design

described by the German philosopher Edmund Husserl, is what clarifies the semiotic function of design text. According to Husserl, “[e]very sign is a sign for something […] [b]ut not every sign has a ‘Bedeutung,’ a ‘sense’ (Sinn) that the sign ‘expresses’” (Derrida, 2011, p.20). That is, there are signs which express something on their own, and signs whose meaning is distant, spatially and temporally, from them. In fact, in the second case, meaning results from that exact transfer/(in)consistency. More specifically, Jacques Derrida offers in his treatise an explanation for the problem of the sign in the Husserlian phenomenology. Derrida explains that, according to Husserl, the sign comprises two heterogeneous concepts: expression (Ausdruck) and indication (Anzeichen). While expression is exclusively related to the discursive sphere and, in a way, voices an internal ‘soliloquy’4 indication, Derrida writes that it does not express anything, because it does not transmit anything that could be named “Bedeutung or Sinn” (Derrida, 2011, p. 15). Nonetheless, obviously the fact that it does not transmit any signification does not mean that it is a sign without signification. Indication is defined by some “motivation” (Motivierung), in the sense of a movement, of a transition from “actual knowledge” to “non-actual knowledge” (Derrida, 2011, p. 24). However, it is not just that. What indication succeeds in doing, and this is important for this paper, is signaling the exit from any conceptual space that the signs open up, that is, the exit towards the empirical world. In indication, therefore, the body of the sign has a “physical side” and what is indicated is “an existence in the world” (Derrida, 2011, p. 33; p. 28). After taking into consideration this intrinsic dichotomy of the sign, would it be possible to safely define the semiotic function of design text? To firmly state, that is, that graphic signs are the expression of the idiomatic design language of the architect-designer, as well as of the widely discussed architectural language. Or, do graphic signs, by contrast, constitute an indication of what preceded transcription, as well as an indication of what has not yet been fixed?

4

“It will be very quickly confirmed that, for Husserl, the expressivity of expression – which always assumes the ideality of a Bedeutung – has an irreducible link to the possibility of spoken discourse (Rede)” (Derrida, 2011, pp 15-16).

Theodora Papidou

27

The function of indication in design text and the semiotic space that design text opens up Before taking a stand on the question above, we ought to follow the reasoning of Husserl, who does not just settle for establishing this dichotomy within the sign. At some point in “The Voice and the Phenomenon” which is worth citing, Derrida explains the difference between indication and expression as a difference that is more functional than substantial. Indication and expression are functions or signifying relations and not terms. One and the same phenomenon can be apprehended as expression or as indication, as a discursive sign or as a non-discursive sign. That depends on the intentional lived-experience that animates it. […] Two functions can be interwoven or entangled in the same concatenation of signs, in the same signification (Derrida, 2011, p. 17).

Any phenomenon, therefore, any group of signs is characterised by the fact that expression and indication are interwoven or entangled. The fact that expression and indication are interwoven legitimises us to consider the writing of design – although double – as one. Besides, the concept itself of the text as ‘textum’ includes the aspect of interweaving: in the case of architectural design, in particular, what is spatially formulated is interwoven with what is verbally articulated. ‘One and the same phenomenon’ can, therefore, be regarded as expression or as indication, depending on the ‘experience it animates’, Derrida writes. Design text, thus, may firstly be related to thoughts, the imagination of the architect-designer, and, secondly, it may refer to the work of architecture. In the case of the latter, in fact, it refers to it in a twofold way: as an expression that is stated thanks to resemblance and symbolism, and as an indication of what-wants-to-be-fixed, first on a piece of paper or a computer screen, and, later, beyond these. In the case of the former, where design text is related to the thought of the architect, voicing the thoughts and experiences of the architect during the designing process is not expression, that is, a descriptive discourse, a narration of some internal workings, but it rather resembles a gesture, it takes the form of an indication of an experience, even though this is sustained in words. Words “act like gestures”,5 and along with depictions, they do not state explicitly the intention of the designer, but they imply it, they indicate it. In the case of the second link mentioned above, design text 5

See Derrida, 2011, p. 32.

28

Double Writing in Architectural Design

refers to the work of architecture, expressing, with the help of resemblance and symbolism, what will be inscribed in real space. Design text, therefore, expresses the work of architecture ensuring, as an accurate design or a narration, its illustration and the way it is rendered in words. By adopting recognisable symbolisms, it also expresses a wide range of agreed-upon significations, like typologies, styles, terminologies, rules, etc. Nonetheless, design text refers to the work of architecture without necessarily representing it and without symbolising something beyond that. Instead of illustrating or narrating, and instead of adapting to a hyperencoded architectural language, design text is regarded as a written act that opens up a special semiotic space facing, as we shall later explain, the empirical world. The function of expression which manifests itself with resemblance and symbolism is, therefore, consciously suspended, in order for the function of indication6 in the design text to emerge, as the index in Charles Sanders Peirce’s semiotics “marks the junction between two portions of experience” (Peirce, 1932, 2.285): in the case of architectural design, more specifically, insofar as it marks the junction between the act of thinking and the written trace, between writing and design. This special hermeneutic space7 which the graphic signs of design open up is defined as space of inscription8 (Fig. 1-10): facing the empirical world, it aims at inscribing in it what-wants-to-be-fixed during the process of architectural design. Design text, therefore, rather points at something outside and beyond itself, and because of that, its graphic signs, by assuming spatial dimensions, open up an interstitial space between the ideational linguistic-geometrical space and the existential-architectural space. The space of inscription opens up by graphic signs and attempts to reinstate the unity among the torn apart – due to being represented – ‘fragments of experience’ between expressing, demonstrating, pointing at, illustrating and constructing. It is this unity that geo-graphy – this

6

Besides, Martin Heidegger defines signs as primarily “means whose special character consists in indicating,” and what is indicated is nothing but the thought that wishes to be preserved (Sini, 1989, p. 23; my translation). The function of reference, of indicating, to use Heideggerian terms, is, thus, traced in the concept of the sign itself (Sini, 1989, p. 23). 7 Besides, “the spaces [of the world] are hermeneutic spaces, that is, they are connoted by signs” (Lobo, 1999, p. 26; my translation). 8 See chapter entitled “Lugar” in Papidou, 2013.

Theodora Papidou

29

fundamental architectural gesture of marking the ground,9 as Ferrán Lobo used to say – declared by definition, where verbal articulation, graphic gesture and construction concurred during the ritual of founding a place, a city. In contrast, writing has the ability to attain a similar unity, as long as it is related, not with what is (positively) sustained in its signs, but primarily with what-wants-to-be-fixed, which keeps what will be fixed and what will be eventually suppressed by the act of (in)scribing itself united.

Fig. 1-10. Space of inscription: it is circumscribed by double writing and geo-graphy.

Yet, what is what-wants-to-be-fixed in architectural design? We can briefly answer: design text refers to the work of architecture by showing what-wants-to-be-fixed as locus.

Locus: the peculiar interpretability of architectural design According to the “three-dimensional” semiotic model of Charles S. Peirce, what-wants-to-be-fixed as locus constitutes the foundation (“ground”) of semiosis, that is, the foundation for the triadic relation among design text, architectural design and locus (Fig. 1-11). In correspondence with the well-known definition of the sign,10 design text constitutes the representamen whose object is the work of architecture, while locus plays the role of regulating the relation between the two, taking into account the sense that the interpretant assumes in Peirce’s theory. Acting as a kind of

9

See Lobo, 2000, p. 67. See Peirce, 1955, p. 99.

10

30

Double Writing in Architectural Design

interpreter,11 locus is what guarantees a semiotic relation of another type, beyond resemblance and symbolism, between design text and the work of architecture.

Fig. 1-11. Architectural design as semiosis.

What-wants-to-be-fixed as locus constitutes the foundation which summons design text and the writing of design to unite with the future inscription of the work of architecture in real locus. As an idea or abstraction, what-wants-to-be-fixed as locus insists on remaining active during the process of design, during the constant referencing of the graphic signs to the work of architecture. Carlo Sini characterises this constant referencing between representamen and interpretant as a game “in which the nature of the sign emerges in its most profound essence” (Sini, 1989, p. 49; my translation). To what extent, however, is it legitimate to apply the triadic model of semiosis by Peirce to architectural design? Semiosis is regarded “as a symbolic configuration of the dynamic multiplicity that experience is characterised by and which the sign intents to capture or retain” (Bentolila, 2008; my translation). Interpreting architectural design with writing as the starting point aims at integrating the multiplicity of experience that runs through the phenomenon of design – illustrating, describing, inscribing, naming – to a general form. And it is possible for this interpretation to achieve some kind of classification, inasmuch as it has at its disposal the material marks of this multiplicity, the written indications, as unaffected as possible from the prevailing representational codes and symbolisms.

11

See Tordera, 1978, pp 105-06. Moreover, in Foundations of the Theory of Signs Charles Morris marks the distinction between Interpretant and Interpreter, whom he regards as the fourth factor of semiosis (Morris, 1944, p. 3).

Theodora Papidou

31

Synopsis That moment of first retaining an intention in lines and words during the process of architectural design ushers in the activation of a mechanism, fixing architectural thought in iconic and verbal signs. From the traces of this first instance of retaining up to the emergence of form and concept, the text of the special design writing is regarded as a unit of signification. Bypassing dialectics as a hermeneutic tool between the iconic and the verbal, design text is considered to be a written act, whose signs are regarded as indications facing the external world of the architect-designer, and the empirical world. As the function of expression – the orders of resemblance and symbolism – of the design text is suspended, the semiotic process is disengaged from imperatives of compulsory codes, so as for that Firstness of the Interpretant to emerge, that is the possibility of the design text to be interpreted before something or somebody interprets it based on a specific code. Peirce has written that “[e]very sign has its own peculiar interpretability before it gets an interpreter” (Peirce, 1987, p. 146; my translation). For design text, this peculiar interpretability is the locus. The essence of design can emerge within the constant referencing between the text and the locus.

Bibliography Bentolila, Héctor. “Signo y movimiento en el pensamiento de Charles S. Peirce.” In Peirce en Argentina, III Jornadas GEP (2008). http://www.unav.es/gep/IIIPeirceArgentinaBentolila.html Derrida, Jacques. “Architecture Where the Desire May Live” [interview]. Domus 671 (1986): 17-25. —. Voice and Phenomenon: Introduction to the Problem of the Sign in Husserl’s Phenomenology. Evanston Illinois: Northwestern University Press, 2011. Heidegger, Martin. Off the Beaten Track, edited by Julian Young and Kenneth Haynes. Cambridge: Cambridge University Press, 2002. Leroi-Gourhan, André. Le geste et la parole. Vol. I, Technique et langage. Paris: Albin Michel, 1964. Lobo, Ferrán. “Signo, arquitectura, habitación”. In Pensar, construir, habitar: Aproximación a la arquitectura contemporánea, edited by P. Soláns, 55-69. Palma de Mallorca: COAIB, 2000.

32

Double Writing in Architectural Design

—. Teoría del conocimiento: Ética y Estética [Lectures for the postgraduate program “Teoría e Historia de la Arquitectura,” ETSAB, UPC, Barcelona], 1999. Morris, Charles. Foundations of the Theory of Signs. Chicago: University of Chicago Press, 1944. Papidou, Theodora. El proyecto arquitectónico y su espacio de inscripción (doctoral thesis). ETSAB, Barcelona, 2013. Peirce, Charles Sanders. Collected Papers of Charles Sanders Peirce. Volumes I and II: Principles of Philosophy and Elements of Logic. Edited by Ch. Hartshorne and P. Weiss. Cambridge MA: Harvard University Press, 1932. —. “Logic as Semiotic: The Theory of Signs.” In Philosophical Writings of Peirce, 98-119. New York: Dover Publications, 1955. —. Obra lógico-semiótica. Madrid: Editorial Taurus, 1987. Sini, Carlo. Pasar el signo. Madrid: Mondadori, 1989. Tordera, Antonio. Hacia una Semiótica Pragmática: el signo en Ch. S. Peirce. Valencia: Fernando Torres, 1978.



CHAPTER TWO: DESIGN FOR PRINT APPLICATIONS





THE RENAISSANCE OF ACADEMIC PUBLISHING: THE DECONSTRUCTION OF THE JOURNAL INTO A PRAGMATIC MANIFESTATION OF A POSTMODERNIST SET OF DISCOURSES ARTEMIS ALEXIOU

Ferdinand de Saussure has stated that: “Language and writing are two distinct systems of signs. The second exists for the sole purpose of representing the first” 1 . This project, partly influenced by the above principle and in line with the Marxist idea restated by Neil Postman: […] that the press [is] not merely a machine but a structure for discourse, which both rules out and insists upon certain kinds of content and, inevitably, a certain kind of audience2

was developed on the hypothesis that academic research and journals are two distinct systems of signs, whereas the second system of signs exists for only one reason: to disseminate the first system of signs. The study had philosophical and practical objectives, which (although substantially different in nature) overlapped in many respects and were all directly relevant to art and design sectors and academic disciplines. The philosophical objectives were to highlight the importance of academic reading and writing amongst art and design practitioners (including art and design students) and promote the engagement with academic journals. The practice-based objectives were to reassess the design and page architecture of the academic multidisciplinary feminist journal Camera Obscura: Feminism, Culture and Media Studies3 (Fig. 2-1) in order to reinvent a design that would

 1

Saussure, Ferdinand De. Course in Linguistics. London: G. Duckworth, 1983. Postman, Neil. Amusing ourselves to death: Public discourse in the age of show business. New York: Penguin Books, 2005. 3 “Camera Obscura provides a forum for scholarship and debate on feminism, culture, and media studies. The journal encourages contributions in areas such as 2



Artemis Alexiou

35

reflect the nature of contemporary academic discourse more accurately and would improve the journal’s visibility, so as to succeed in the dissemination of academic ideas to a wider audience, beyond the traditional disciplinary academic community, and amongst multidisciplinary; interdisciplinary; trans-disciplinary audiences.

Fig. 2-1. Snapshot fromTalk to Her by Pedro Almodovar. 2002. Front Cover Page. 148mm x 210mm. From: Camera Obscura. Durham: Duke University Press, 2008.

In regard to the concept of ‘discourse’ (whether academic or otherwise), Michel Foucault proposed that: whether it is the philosophy of a founding subject, a philosophy of originating experience or a philosophy of universal mediation, discourse is really only an activity, of writing in the first case, of reading in the second and exchange in the third. This exchange, this writing, this reading never involve anything but signs. Discourse thus nullifies itself, in reality, in placing itself at the disposal of the signifier.4

 the conjunctions of gender, race, class, and sexuality with audiovisual culture; new histories and theories of film, television, video, and digital media; and politically engaged approaches to a range of media practices.” Duke University Press. www.dukeupress.edu. 2013. www.dukeupress.edu/Catalog/ViewProduct.php?productid=45605 (accessed August 12, 2013). 4 Foucault, Michel. Archaeology of Knowledge. Translated by A M SheridanSmith. New York: Pantheon Books, 1972.



36

The Renaissance of Academic Publishing

Thus, the above research concept of reflecting the nature of contemporary academic discourse refers to the academic discourse as not being merely a modernist, hierarchical structure of Author communicating to Reader, but a more nuanced, postmodernist, equally balanced set of discourses moving back and forth between Author(s) and Reader(s) and Reader(s) as Author(s) in the academic community (as is frequently the case in academic conferences and on research websites) and eventually, the text that is the product of such discourse is: [...] a multidimensional space in which a variety of writings, none of them original, blend and clash. The text [becomes] a tissue of quotations. [...] the writer can only imitate a gesture that is always anterior, never original. His only power is to mix writings, to counter the ones with the others, in such a way as never to rest on any one of them.5

Research Problem An initial brief investigation showed that the majority of academic presses publish journals with conservative appearance, conventional structure, standard page layout and traditional typefaces. Thereafter, following a rather extensive investigation, it was evident that even the academic journals that are debating matters on art and design (which are targeted predominantly towards relevant audiences) are equally deficient in design integration as most of their scientific counterparts. Concurrently, the majority of academic journals almost always accommodate lengthy, heavily intellectual content with reduced levels of integrated design. Due to this apparent circumvention of design integration, there is a considerable amount of art and design practitioners (especially art and design students), who do not read academic texts by choice. This is predominantly because they find these publications unattractive externally and monotonous internally, which lead to a difficulty in retaining concentration and accomplishing comprehension. Visual communication has been a significant aid towards the evolution of print publishing. In the late 19th century, periodicals radically changed (both in terms of design6 and marketing) and new technologies for printing enabled mass production, causing new markets to expand. As a result, the majority of periodicals – including weekly illustrated newspapers (i.e. “The Graphic”, 1869-1932) and weekly feminist periodicals (i.e. “The

 5

Barthes, Roland. The death of the author. London: Fontana, 1977. The term is used anachronically for the purposes of this article, given that design was not widely used as a term during the 19th Century.

6



Artemis Alexiou

37

Woman’s Herald”, 1891-1893) – were actively seeking new material methods to progress and stand out amongst their counterparts. In the 1960s and 1970s there was a rise of underground magazines (i.e. “Oz”, 19671973), which were using rather elaborate graphics and designs in order to promote messages on world peace and sexual liberation, and during the 1980s, graphic design practice for periodicals reached exceptional levels of sophistication for few renowned discipline specific magazines (i.e. “Émigré”, 1984-2005). It was then that: academics argued that graphic design was more than the mere study of technique and technology, more than form and function - it was an intellectual pursuit that demanded philosophical fluency. […] during the late 1990’s in part as a way to counterbalance the perceived primacy of style, theory branched into a new rigor called ‘authorship’. […] more importantly, authorship was always about designers expanding their influence as creators rather than mere packagers of content.7

Nonetheless, the majority of academic presses have suppressed graphic design assimilation in their journals, which has also discouraged graphic design practitioners and practice-based researchers on that matter, who, as a result, have shown a significant neglect towards publications of that nature as graphic designers as well as readers and authors. Consequently, graphic design research on periodicals has focused mainly on magazines and almost exclusively within a contemporary context; graphic design studies8 are usually general overviews of design practices that focus primarily on magazines and, although new academic journals are being introduced frequently and on many different subjects, they have yet to be assessed as visual communication products and in terms of their ability to communicate the message they aim to promote through their exterior and deliberate within their interior.

Methodology Since the beginning, this research project was to be practice-based in nature, but it seemed essential to employ a theory-based qualitative methodology for its initial stage in order to reassess the hypothesis (that academic research and journals are two distinct systems of signs and the

 7

Heller, Steven, and Audrey Bennett. Design Studies: Theory and Research in Graphic Design. New York: Princeton Architectural Press, 2006. 8 i.e. Jeremy Leslie’s, Magculture: New Magazine Design, published by Harper and Collins in 2003.



38

The Renaissance of Academic Publishing

second exists for the sole purpose of disseminating the first) and also potentially reconstruct the research question and sub-questions, based on the primary and secondary data collected. The first stage of the theory-based qualitative data collection included: an informal focus group session; an online survey; two interviews; two case studies. The focus group consisted of four participants. They were presented with one issue of each of three different periodicals and they were asked about three to four open questions. The online survey was a list of 31 questions 9 (five of which were accompanied by visuals) and was completed by a multidisciplinary group of 19 participants10 consisting of art and design students, academics and practitioners. The two case studies were based on the profession-specific magazine Blueprint11 (1983-present) and the popular feminist magazine Spare Rib12

 9

Twelve closed questions, eleven open questions, six demographical questions, and two commentary questions. 10 The group of participants consisted of: two BA design students, 15 MA design students; three academics and one architect. The younger participants were 18 years of age and the older was over 48 years of age. Two were between 18-23 years old; nine were between 23-28 years old; one was between 28-33 years old; three were between 33-38 years old; two were between 43-48; one was older than 48 years. Overall, five participants were male and 14 were female. The group also consisted of nine British citizens; one Portuguese; one North American; three Norwegian; two Colombian; one Indian; one Dutch; one Greek. The chosen mother languages were as follows; ten spoke English as their native language; two Greek; two Portuguese; two Spanish; four other. 11 “ Launched in 1983, Blueprint was the first magazine to cross the boundaries between design and architecture. It was established by Peter Murray and Deyan Sudjic with the financial backing of leading architects and designers including Norman Foster and Rodney Fitch. Today, it continues to be revered by architects and designers around the world for its fresh and unconventional approach.” Blueprint. Blueprint Magazine. 2013. http://www.blueprintmagazine.co.uk/index.php/about/ (accessed August 12, 2013). 12 “ Spare Rib emerged from the underground press (Time Out, Oz etc.) in 1972, started by women from these papers who ‘still found they were always making the tea'. The underground press (in particular Ink now itself defunct) generously loaned money to finance the enterprise. There were then many scattered women's liberation groups, but with little contact; and women's workshop literature was not stocked on public bookstalls. A need was felt for a central magazine, publicly available, which would cater for women's repression and form a link between groups. In the face of huge costs, minimal encouragement and collapsing magazines around, Marsha Rowe and Rosie Boycott contrived to get Spare Rib established (and with howsplendid a title) so that it has now reached its 36th



Artemis Alexiou

39

(1972-1993). Both studies included visuals, text and interview recordings. The two interviews were arranged and conducted through a similar procedure, though they were both unstructured. Vicky Richardson (who was then the editor of Blueprint) and Marsha Rowe (co-founder and coeditor of Spare Rib) were the interviewees. The two sets of questions were different, but there were specific structural and conceptual objectives for both, in order to be able to assemble a group of answers relevant to the research project questions. The second stage of practice-based data collection was divided in four sub-sections. The first sub-section was “Observation” and consisted of a choice of short studies of periodicals that were somehow successful examples of visual communication practice and the aim was to collect enough visual material to form a comprehensive idea of former and current practices. The second sub-section was “Information” and consisted of an interview with Dr. Dipti Baghat, the Chair of Design History Society and member of the editorial board of Design History Journal13. The aim was to collect information about the practicalities of producing an academic journal, including financial limitations and protocol procedures. The third sub-section of data collection was “Evaluation & Testing” and consisted of a formal focus group of four, who were presented with a reconstructed version of the multidisciplinary feminist academic journal Camera Obscura and were asked to comment on whether or not it was visually communicating the proposed design concept. The final subsection of the practice-based methodology was “Action & Reflection”, which also included self-evaluation and self-reflection in a form of a diary/journal that was a physical record of the practice-based work.

Results The results showed that for a journal to be appealing to an art and design audience, it would have to embrace a distinctive appearance and present the content in a comprehensive manner, while the materiality of the publication was found to be a significant element towards the engagement of wider audiences.

 monthly issue and seems here to stay.” Bell, Hazel K. "Spare Rib." The National Housewives Newsletter, Autumn 1975: 10-11. 13 “Journal of Design History is a leading journal in its field. It plays an active role in the development of design history (including the history of the crafts and applied arts), as well as contributing to the broader field of studies of visual and material culture.” Design History Society. Journal of Design History. 2013. http://jdh.oxfordjournals.org (accessed August 12, 2013).



40

The Renaissance of Academic Publishing

I. Exterior Design & Audience Attraction Form and texture can be significant tools for graphic designers and in this particular case, a larger scale format with rectangular proportions in a portrait arrangement, was found to be the most preferred choice amongst the subjects participating in this research. These findings were in great accord with the philosophical concept by Gerard Genette, who was one of the first to adopt the concept of the bricoleur 14 and bricolage 15 (both initially introduced by Claude Levi-Strauss), and who had previously paid particular attention to the fact that: […] one must bear in mind the paratextual value, which can belong to other types of expression: iconic (illustrations), material (everything which proceeds, for example, from sometimes very significant typographical choices made in the composition of a book) or purely factual […].16

Furthermore, considering the materiality of the publication, it was also found that size is a highly influential factor when it comes to product purchase or collectability but (in contrast to the general perception that most readers prefer a small, easy to transport publications) this study discovered that amongst art and design audiences a large scale format is preferred. There was also a marked preference for the front and also for the back cover pages17 to be in accordance with the content. In other words, the subjects were expecting to find the same level of design implementation for both the exterior and the interior content of the publication (Fig. 2-2). Furthermore, the two composition elements that were most important

 14

“The bricoleur works with signs, constructing new arrangements by adopting existing signifieds as signifiers and ‘speaking’ ‘through the medium of things’- by the choices made from ‘limited possibilities’[...].”Levi-Strauss, Claude. The savage mind. London: Weidenfield & Nicolson, 1974. 15 “ […] ‘The first aspect of bricolage is […] to construct a system of paradigms with the fragments of syntagmatic chains’ (Levi-Strauss 1974), leading in turn to new syntagms […]. ‘Authorship’ could be seen in similar terms.” Chandler, Daniel. Semiotics: The Basics. London: Routledge, 2002. 16 Genette, Gerard, and Marie Maclean. "Introduction to the Paratext." New Literary History (Probings: Art, Criticism, Genre) 22, no. 2 (1991): 261-272. 17 “ The cover page is the surface which allows the designer to visually communicate with the audience in a direct manner and therefore, persuade them to adopt a particular viewpoint, which could possibly mean persuade them to accept particular information or data.” Tyler, Ann. "Shaping Belief: the role of teh audience in visual communication." Design Issues 9, no. 1 (1992): 21-30.



Artemis Alexiou

41

about the cover pages were line, which, according to Downer, is the visual path that enables the eye to move within the page, brought together by inventive page architecture18, and colour19.

Fig. 2-2. Artemis Alexiou. Camera Obscura: feminism, culture and media studies, Issue 68, Vol 23, N. 2 [Original Reconstructed Design]. 2009. Alternate Cover Pages. Adobe InDesign. 420mm x 597mm. London Metropolitan University. 2009.

Lewis and Walker have defined typeface as the capacity of a typestyle to connote meaning over and above the primary meaning, which is linguistically conveyed by words20. During this study, this concept was supported by the participants: the majority agreed that the typeface of the brand name of an academic journal was a substantial component for attracting new audiences and, based on the results, the typeface of the brand name is usually expected to reflect the history and legacy of the journal, as well as implement elements of the visual language which is specific to the audience(s) the journal is aimed at and discipline(s) the journal is debating about.



18 Downer, Marion. Discovering Design. Boston, Massachusetts: Lee, Lothrop & Shepard Co., 1965. 19 Please note that colour was deemed an essential component, based on the results of this study. Figure 2-1 is presented here accurately, however figures 2-2 and 2-3 were originally in full colour in the final reconstructed printed volume. 20 Lewis, C, and P Walker. "Typoraphic Influences on reading." British Journal of Psychology 80, no. 2 (1989): 241-257.



42

The Renaissance of Academic Publishing

II. Interior Design & Text Comprehension Unity between the components on a page is significant for the comprehension of the text, but the available space for page architecture is limited, so the designer ought to be careful with elements in close proximity and the relationship they appear to be communicating. Alignment, size, colour and shape are very important elements for an appealing composition, but they can also be misleading if used with carelessness, so if they are used repetitively to create a sense of rhythm, and essentially create a sense of unity, readers tend to prefer an identical or a theme-based repetition. Visual continuation is also essential in creating a sense of hierarchy, and it can be achieved by constructing a two dimensional architecture that exists within the notional boundaries of the page and in analogical distances from the edges. The findings also revealed that size, colour and shape of all elements on a page and their positioning in relation to the reader’s eye level are essential tools with which the designer can create ascendancy. This intentional structured hierarchy would then guide the readers along the page and direct them from the more, and towards the less, significant content of the page.

Fig. 2-3. Artemis Alexiou. Camera Obscura: feminism, culture and media studies, Issue 68, Vol 23, N. 2 [Original Reconstructed Design]. 2009. Pages 26-27. Adobe InDesign. 420mm x 594mm. London Metropolitan University. 2009.

Balance was also found to be critical and could be achieved in a variety of ways, which could reflect different styles and motives in harmony with the design concept of the journal. For instance, equal spaces and gaps would traditionally reflect classicism and formality, whereas an



Artemis Alexiou

43

asymmetrical composition with various sizes, styles and shapes would customarily reflect a contemporary style. In any case, page architecture has to include a combination of text and visuals within the notional grid (Fig. 2-3), which, according to Williamson, translates as the compositional design matrix for controlling the placement of typography and imagery21, in order to achieve successful comprehension of the content.

III. Colour and Visual Communication Regarding the visual communication of a journal, colour was found to be the one element that can revive or relegate a journal. The formation of a colour strategy during the early stages, which is followed throughout the design development and production, can complement the visual identity of the journal and also emphasize the importance of consistency throughout the design. A colour strategy can also contribute towards the hierarchical and authoritarian principles of the design, by emphasising or drawing away the reader’s attention or simplifying complex data for an enhanced digestion of the information and ideas projected by the authors. For the purposes of making this reconstructed model of Camera Obscura more accessible to the reader, an innovative structure in accordance with a distinctive colour strategy was applied on the content in order to introduce a range of original components, which act as complementary entries to the main body of the text. More specifically: i. an abstract is printed at the beginning of each article22; ii. all references mentioned in the articles are located on the same page, on the right or left side of the body of text23, iii. general information mentioned in the articles (essential for understanding the content) are also located on the same page, on the right or left side of the body of text24, iv. any visuals appearing amongst the main body of text is accompanied by commentary information including a source reference 25 , v. any supplementary information found inside the body of the text is tinted with a low impact colour26 and vi. all quotations mentioned by the authors are clearly highlighted to stand out of

 21

Williamson, J H. "The Grid: history, use and meaning." Design Issues 3, no. 2 (1986): 15-30. 22 Colour(s): (text) black. 23 Colour(s): (text) deep purple. 24 Colour(s): (text) cherry red [all biographies are also accompanied by a portrait photo]. 25 Colour(s): (text) black / (background block) bright yellow. 26 Colour(s): (text) light grey.



44

The Renaissance of Academic Publishing

the main body of the text27.

Conclusion One might argue that this revolutionary idea of transforming academic journals into avant garde publications (with a distinctive colour strategy for the whole generation of each journal; set number of authors writing for each issue; specific theme for each issue; numerous implemented commentaries and visuals; a larger format; lavish printing and binding accompanied by quality paper; twin reverse cover pages, which allow the reader to read the journal from both sides; an inside cover spread of an artwork; complementary quotes) is an economically and practically unrealistic concept, but one should bear in mind that there is a lot of space for experimentation in print academic publishing. This reconstructed model of the feminist academic journal Camera Obscura is a lone, modest effort to contribute to the formation of the concept of a complete (but not necessarily absolute) vision of what manifests as the action of academic discourse(s); thus, the experimental concept of including authors coming from different disciplines for every issue should be seen as an attempt to favour the crossing and collusion of visions, all brought together simultaneously in the realm of a discourse in order to create a platform that would welcome multidisciplinary, interdisciplinary and (most importantly) trans-disciplinary discussion. Thus, this practice-based study challenged the concept of graphic design assimilation by academic journals and its essential value in the engagement of wider audiences (especially from the art and design disciplines) and also emphasised that graphic design incorporation could eventually allow new transdisciplinary and trans-sectoral audiences to benefit and progress by incorporating academic knowledge into their practice and potentially cultivate an interest towards authorship.

 27



Colour(s): (text) black / (background block) light green.

MARKETING SEMIOTICS APPLIED TO THE DESIGN OF INTEGRATED GRAPHIC COMMUNICATION SYSTEMS DORA IVONNE ALVAREZ TAMAYO

Introduction “Graphic design is a service profession, contemporary, which is produced industrially by a creative process that is focused on the user” (UPAEP 2009). As a rhetorical action, its production requires the generation of argumentative strategies to respond to persuasive communication through the articulation of the functional, formal and symbolic characteristics of a message given in a context. Its exercise requires interacting with other disciplines that provide information necessary for a process of user-centred design. Such is the case of marketing, whose tools of marketing research and understanding of consumer behaviour can develop the analytical stage of this process, and semiotics, a discipline which is oriented to the study of signs and signification processes. This article focuses on the production of discursive systems aimed at promoting the brand positioning through the articulation of visual messages that are issued through the different media, by which an organization, company or product use to contact their users. These kinds of systems have been called integrated graphic communication systems. The operational relationship between semiotics, marketing and graphic design, will serve as the basis of the approach of the design process. This implementation requires the integration of knowledge offered by people with different training in order to constitute interdisciplinary powerful teams. So, there is a proposal for a methodological model based on interdisciplinarity, used to apply marketing semiotics and solve positioning problems.

46

Semiotics Applied to the Design of Graphic Communication Systems

Integrated graphic communication system The concept of integrated graphic communication system is my own term taking in consideration the already-known concept of integrated marketing communications (IMC) as a reference. In addition, the high impact of visual stimuli was used to define the model programme. This was the product of the planning process, which lets the visual messages work together through contacts by the various sub-systems of communication of an organisation. All of the messages must be consistent with the objectives of the marketing and communication of an organisation. As a result, the integrity of the messages guarantees the positioning in the minds of the audience. The integrated graphic communication system requires a global and holistic view because the communication specialists must be trained to understand the consumer, the market and the media. In the development of persuasive speech, the person who has the tone of the interaction is one that has the following: clear intent, knowledge of the topic, the receiver, the context, the circumstances, the interpretation system and one who recognises his or her own production process. Effective communication requires to be carefully planned, focusing on the consumer so that the messages are formulated around the personality of the brand (Schultz, Tannenbaum and Lauterborn 2007, 98). These types of integrated communication programmes require creative thinking to send congruent messages and produce the appropriate arguments for specific audiences. Integration is the strategy that sets the course of the production of messages in different ways to contact the user, and it is based on the agreement of who the customer is, what he wants and how he or she wants and how to obtain it. Schultz, Tannenbaum and Lauterborn (2007) explain that "The need for integrated communications is the result of what we have learned about the way humans process information and their experiences, and use it to make purchasing decisions” (54). So, the knowledge of these passageways of information processing becomes a prerequisite for strategic approach to communication by an organisation, and therefore, the closeness with the user is essential, it is our most creative source. Ries and Trout (2002, 4) emphasise that the solution to a problem associated with the positioning is not resolved by focusing on the product, nor in the mind of the sender of the communication, but in the mind of the prospective customer. It is also necessary for the strategist to understand that the user is exposed to many messages that put up a competitive fight to win an advantaged position in the users’ minds. In a highly competitive market,

Dora Ivonne Alvarez Tamayo

47

the unique products or services a company offers their target market, is what can potentially give them a competitive advantage over their competitors, in what the consumers think about the brand. (Schultz et al. 2007, 81). The consumer is the one who categorises and decides what is relevant as a message through their interaction and experience with the organisations and the signs associated with them. For this reason, in a commercial context, organisations invest in programmes, resources and communication mechanisms with the hope of succeeding. For Ellis and McClintock (1993, 124) communication is a process of negotiation in which each person involved seeks a common ground where they can reach an agreement. The consumer´s own belief is what constitutes the true brand value. By understanding this position, we can understand why semiotics becomes an essential tool for the design of integrated graphics communication systems; and, if one considers that 80% of the information that a consumer receives comes from the sense of sight, then graphic design plays a fundamental role in the design of the positioning strategy. These kinds of programmes must define the desired brand positioning, have clarity of personality traits based on identity and style, the argument competitive and consumers’ benefits, and despite of it being a carefully planned programme, its structure must allow flexibility. Chaves and Belluccia (2003) state "what the public thinks about an organization is the result of the interaction established by the organization with the public" (26). They define two types of interactions between organisations and the public, they are the following: the first one is through the use of products and/or services generating a degree of satisfaction associated with quality. And the second interaction is through a communication system, through which the audience can understand the profile of the organisation. Accessing the consumer's mind involves an immersion into his or her mental categories and the way characteristic features of the product in question in relation to their identity and style is organised. Chaves and Belluccia (2003, 16-17) explain that an identifying sign (whether verbal, visual, hearing, to name a few) behaves as a mark surrounded by a series of semantic references that enrich the purely verbal function (who is it?) with descriptive and evaluative functions (what and how) to expand the meaning. This semantic load occurs inevitably as a result of the social positioning of the entity and the mere effect of spontaneous interaction with their audiences. The key of success in these projects is planning; this is oriented to articulate the messages and generate a systematic communication production. The result is a type programme ready to be operated. These programmes are designed to ensure that commercial communication will be highly effective in such a

48

Semiotics Applied to the Design of Graphic Communication Systems

way that the consumers will get the message sense defined by the company.

Relationship between graphic design, marketing and semiotics The meeting point between marketing, semiotics and graphic design is that their process is focused on the user; therefore the relationship helps address situations of positioning. Ries and Trout (2002, 4-5) define the battle to win a place in the mind of the consumer as positioning, where the user organises what gets into his mind in different categories. If we understand the basic rule for winning positioning is not to prioritise the position we want to win, but to recognise the position we currently have in the consumer´s mind, then we could open strategic possibilities to create a competitive effect. According to Yves Zimmermann (1998) designing is "the choice of signs constituting a signal assigned" (98) consistent and aligned to the communicative intention, which has the role of guiding all decisions made during the configuration process. This means the design and intention are very involved in the design process. Additionally, graphic design has a rhetorical vocation; it means communicative intention is persuasive and it involves changing from an initial mental state of a user to a desired mental state that leads to actions that the consumer raises. From this perspective, the design process requires an excellent knowledge of the audience and the internal and external conditions that determine the interaction between the situation and visual messages that are proposed. Responding to the dimensions involved, a visual message will be successful when its syntactical, semantical and pragmatical dimensions are well established and able to respond to the communicative intention corresponding to the linguistic demands of a client, based on knowledge and understanding of the user. On the other hand, the American Marketing Association (2007) defines marketing as a “set of actions, institutions and processes aimed at creating, communicating, and distributing the exchange offers that have value for customers, partners and society in general”. To meet the needs of a customer, marketing has a number of functions aimed at specific purposes and that collectively make up its field of intervention: market research, consumer behaviour, strategic management, distribution and logistics, e-marketing, relationship marketing sales and after sales, etc.

Dora Ivonne Alvarez Tamayo

49

Consumers perform a series of actions to satisfy their needs such as to identify, evaluate, search, buy, dispose and judge, and therefore to carry out marketing plans, the strategist has to understand not only the consumers’ actions of consumers, but the reasons why they behave in certain ways. This information is fundamental for companies to be able to find the most effective strategies to achieve their target market. Finally, semiotics provides the base for designing instruments to collect and to analyse information from the users. Human communication involves cognitive processes to access the recipient's inner world, therefore it is necessary for strategists to know the categories from which consumers organise their thinking, which can be reached through market research. This kind of research allows identifying denotative and connotative meanings, as well as contextual factors that determine significance. To understand the audience mindsets, it is required to make a systematic inquiry for recovering their codes and recognise their belief system by which the interpretive exercise is performed. Umberto Eco (2005a, 69) explains that semiotics is concerned with the signs understood as social forces; it means each sign is a cultural unit. The person who uses signs to communicate is occupying the position of a receiver or a sender; he or she decodes the messages considering his or her culture background, therefore social life is developed based not on things but in cultural units instead of things. Within a social group, these cultural units are organised according to rules that are built into systems called codes, which work thanks to agreement and that humans learn as part of the social dynamics. Trying to respond to the question “how to get into the mind of the audience?” semiomarketing could be a great tool in two ways: the first is as an analytical and production of signs tool, and second as a marketing research tool to get data from the users through qualitative and/or quantitative methods. Marcelo López (2002) explains at the beginning, semiotics was used as an assessment instrument after the design process (a posteriori), but now semiotics starts to be used before the design process as an analytical tool (a priori). Semiotics begins to take major importance in business to evaluate their potential for innovation in the space of the phenomena of consumer behaviour, marketing and advertising. This exploratory mode of works is called semiomarketing.

Design process Every effort to communicate through visual signs is an opportunity for graphic design to intervene, especially in integrated graphic communication

50

Semiotics Applied to the Design of Graphic G Comm munication Systeems

systems, andd from this peerspective, sem miotic techniqu ques should bee included in any proceess of this disccipline. Accordinng to Sánchezz (2002, 233), a process parradigm consid ders three key elementts: inputs, trannsformation ag gents and outp tput elements. To carry out a designn process, whhen one think ks of it as a set of concep ptual and methodological operationnalised aspectss, it is possiblle to create insstruments that determiine the rules of o thinking an nd processing information needed n to build the vissual communiication strategies. A designn process has three fundam mental momennts (see fig. 2--4) in the solution of problems: thhe analytical, the strategiccal and the executive e stages.

Fig 2-4: Stagees of the designn process. Sourcce: Alvarez, 20005

In the analytical sttage, the strrategist pays special atteention to understand aand to define the problem based b on interrviews with cllients and the recognitiion of the fieldd of interventiion. The audieence offers infformation about their belief system m (previous agreements) and operativ ve codes. During the strategic stagge, the designer works in tthe constructio on of the argument tto achieve the t persuasiv ve intention, and sets the signs considering their syntactiical, semanticcal and pragm matical levels to design the contacts with the userr. Finally, duriing the executtive phase, thee designer develops thee technical proocesses of pro oduction and eestablishes the rules of implementattion. Followinng Chaves (1990, 107), the executionn of a prog gramming exercise exiists between the t detection of o the need too intervene in n the field of image andd communicattion, and the production p of concrete interrventions. This program mming exerciise is a techniical process, w which produces the set

Dora Ivonne Alvarez Tamayo

51

of specific requirements to be fulfilled by concrete actions. The result of this work must be recorded in a model programme which makes an explicic trait of the name´s sub-system, basic identification signs and the communication subsystem.

Methodological model for the design of integrated graphic communication systems The methodological model is not a rigid structure, but it is the understanding of the design process which shows that marketing semiotics provides key-inputs in crucial moments: ANALYTICAL STAGE 1. Understanding the controversy over design (a problem). 2. Analysis of the text from the client, making an interpretive exercise based on an understanding of the contexts. Aim to understand the intention of communication. 3. Verification of the organisational goals. 4. Description of profile (s) user (s). 5. Analysing the features of identity and style that the organisation wishes to declare in a box positioning (See annex 1). This analysis implies a symbolic mapping exercise based on categorical structures. 6. Audit of existing communications system (if any) through semiosis levels of syntactic, semantic and pragmatic (diagnostic). 7. Design of tools to access the user's belief system and access their mental categories. 8. Analysis of the data collected using semiotics. Mapping the user´s representational system, identifying the level of significance, the denotations and connotations, and the connections that come from inferential processes. STRATEGIC STAGE 9. Establishment of guidelines for message articulation and defining the central arguments. 10. Definition of communication channels describing the discursive axis. 11. Development of the graphic concept from semantization processes. 12. Production of prototype applications of graphic communication. 13. Evaluation of prototypes through instruments based on semiomarketing techniques.

52

Semiotics Applied to the Design of Graphic Communication Systems

EXECUTIVE STAGE 14. Definition of an implementation programme and its manual. 15. Monitoring and evaluation. This process is flexible because it is a general strategy; it should be tailored to the needs of each case of intervention. The choice of semiotic resources for the implementation of the project also has some flexibility. For example, it is suggested to establish systems for evaluation of the product by taking as reference the levels of semiosis developed by Morris and retaken by Eco (2005a), syntaxis, semantic and pragmatic. Access to user codes can be analysed based on the theory of codes by Umberto Eco (2005b, 81-119) through componential tree configuration. The categories of firstness, secondness and thirdness described by Peirce (2004, 69-118) can be put in practice and his studies of cognitive processes regarding the meaning and the argument leading to action: induction, deduction and abduction. The signs of the production process can use the box of signs´ production of Umberto Eco (2005b), the design game rules described by Roman Esqueda (2003) and/or treatment of iconic and plastic signs raised by Groupe μ (1992).

Findings and conclusions An integrated graphic communication system will articulate the organisation issuing statements, encouraging the perception of their audiences. This model has been monitored in a specific positioning project. Thus, preliminary results have allowed us to reach the following conclusions. Consumer behaviour is a complex network of components that disrupts the entire apparatus of cognitive and emotional human being. This interlocking network of belief systems of motivation, values, experiences, conscious and unconscious introjects, archetypes, pressures, prejudices, customs, ideals, aspirations... makes such a deep study of consumer behaviour that requires an interdisciplinary team able to develop the tools to make the qualitative and/or quantitative research high effective. When we accept that graphic design participates in solving positioning problems, the marketing concept could be the basis of visual resources to establish a series of images as signs. The relationship between the design, semiotics and marketing shows great potential to promote the persuasive communication objectives. To access the inner world of the audience, semiomarketing offers highly effective tools.

Dora Ivonne Alvarez Tamayo

53

Semiotics has been applied as a tool to analyse the process of signification in different areas, and to gain a higher importance in the field of business from which it begins to evaluate a semiotic potential for innovation in the areas of the consumer behavioural phenomena, marketing and advertising. The understanding of cultural codes becomes an important tool to learn why users behave the way they do. The criterion chosen for semiotic framework depends on the goals of each case and on participants’ profiles. The design process could be a meeting point to integrate interdisciplinary work teams through the interacademic work to develop a combined project.

Bibliography Alvarez, Dora. Modelo metodológico para el diseño de sistemas de identidad visual corporativa basado en el Modelo DHP. In Annais do 3er. Congresso Internacional de Pesquisa em Design. Río de Janeiro: ANPED-Univercidade, 2005. American Marketing Association AMA. Definition of marketing. Accessed January 4, 2011. http://www.marketingpower.com/AboutAMA/Pages/DefinitionofMark eting.aspx Chaves, Norberto. La imagen corporativa: Teoría y metodología de la identificación institucional. Barcelona: Gustavo Gili, 1990. Chaves, Norberto and Belluccia, Raúl. La marca corporativa. Buenos Aires: Paidós, 2003. Eco, Umberto. La estructura ausente: Introducción a la semiótica. México: DeBolsillo, 2005a. Eco, Umberto. Tratado de Semiótica general. México: DeBolsillo, 2005b. Ellis, Richard and McClintock, Ann. Teoría y práctica de la comunicación humana. Barcelona: Paidós, 1993. Esqueda, Román. El juego del diseño: Un acercamiento a sus reglas de interpretación creativa. Designio: México, 2003. Groupe μ. Traité du signe visual: Pour une rhétorique de l’image. Paris: Seuil, 1992. López, Marcelo. La semiótica mete la cuchara. De cómo la semiótica salió del aula y entró al salón del directorio. (August, 2002) Retrieved from: http://www.razonypalabra.org.mx/anteriores/n28/mlopez.html Peirce, Charles. Écrits sur le signe: Rassamblés, traduits et commentés par Gérard Deledalle. Paris: Éditions du Seuil, 2004.

54

Semiotics Applied to the Design of Graphic Communication Systems

Ries, Al and Trout, Jack. Posicionamiento la batalla por tu mente. México: Mc Graw Hill, 2002. Sánchez, Margarita A. De. Desarrollo de Habilidades del Pensamiento: Procesos directivos, ejecutivos y de adquisición de conocimiento. México: Trillas, 2003. Schultz, Don, Tanenbaum, Stanley and Lauterborn, Robert. Comunicaciones de marketing integradas. Cómo lograr una ventaja competitiva. Buenos Aires: Granica, 2007. UPAEP. Documentos fundamentales del programa académico de diseño gráfico. Facultad de Diseño Gráfico UPAEP. Disponible para consulta en red interna DADA, 2009. Zimmermann, Yves. Del Diseño. Barcelona: Editorial Contrapunto, 1998.

Dora Ivonne Alvarez Tamayo

55

Annexes Annex 1: positioning box The positioning box is an instrument that helps build the features of a brand´s organisation, company or product. The positioning box includes information from the analytical stage of the design process. The box displays the course to be taken by the efforts of the design based on the communicative intent and the user, and can serve as reference for the evaluation of products designed as part of the integrated graphic communications strategies. The component elements are: TARGET: direct and indirect audiences IT IS: identify characteristics WHICH: offer BECAUSE: arguments that support the offer BRAND´S CHARACTER: style features BRAND EXTENSION: description of style features (synonyms) and their opposites (antonyms) Target (user) It is Which Because Therefore the brand character is: Concept

Brand extension

Adjectives

Synonyms

Antonyms

Example: Actualized

Modern, contemporary

Old

Example: Competitive

Effective, qualified

Unqualified

(DE)CODING THE FABRIC OF THE EUROPEAN YEARS’ VISUAL REPRESENTATIONS CAMELIA CMECIU AND DOINA CMECIU

Insights into the concept of Europeanization “A reversible fabric, one side variegated, the other a single colour, rich and deep”1 is the definition of Europe’s culture that Leonard Orban, the first Romanian European Commissioner for Multilingualism used in his speech on October, 3, 2007, in front of the European Parliament. Europe’s culture as a fabric can be interpreted as the metaphorical representation of the well-known syntagm “unity in diversity”.

Whereas diversity lies in the variegation of different colours (the sourceconcept for European member states), unity is the deep structure bearing the unifying colour. Despite the promotion of the European Union as the embodiment of unity, the shift of power to supranational European institutions may bring forth a public and a democratic deficit. Two different dimensions of the Europeanization of national public spheres2 may be linked to this deficit: (a) a vertical Europeanization (instances of top-down), and (b) a Europeanization through synchronisation (operationalised by the reporting of EU topics). The pessimistic perspectives claim that the average media coverage on the EU mainly provides a negative image, Brussels being perceived as “the synonym for bureaucracy, regulation, and weak

1

This definition of Europe’s culture belongs to Alberto Moravia, an Italian novelist. Leonard Orban’s speech may be found at http://europa.eu/rapid/pressReleasesAction.do?reference=SPEECH/07/590&type= HTML, accessed 20 September, 2008. 2 Marcel Machill, Markus Beiler and Corinna Fischer, “Europe-Topics in Europe’s Media. The Debate about the European Public Sphere: A Meta-Analysis of Media Content Analyses,” European Journal of Communication 21(1) (2006): 78, accessed 20 March, 2009, doi: 10.1177/0267323106060989.

Camelia Cmeciu and Doina Cmeciu

57

compromises”3. The democratic deficit linked to the limited powers of the European Parliament is considered to be the main reason for the absence of a truly European public sphere4. The literature5 on Europeanization highlights three main aspects at a discursive level: (1) the salience of an expert-like discourse focusing on the ‘high diplomacy’ pattern and the political elites; (2) a discursive representation of a symbolic distance between Central and Eastern Europeans and Western Europeans; (3) the sign of emptiness that Europe is associated with in Central European campaigns. The counterpart of these instances of the menace approach on Europeanization focuses on a horizontal Europeanization model6 which lies on the reporting of other EU states. This horizontal type of communication, which is mainly made visible through national mass media, has a twofold implication: on the one hand, the EU is no longer perceived as a centre; on the other hand, decisions are desirable to be reached through a negotiation between the governments of the member states and the neighbouring states.

European Years (EY) – the EU official visual representations Besides the democratic deficit that the above-mentioned studies on Europeanization revealed, there has also been pinpointed a symbolic deficit7 that was officially recognised for the first time in 2004 by Romano Prodi. The syntagm “unity in diversity” has implications at the visual level as well. Besides the 2004 attempt to change the European flag (Blue Europe)

3

Adreas Pribersky, “Europe as a Symbol in Political Image Constructions,” Semiotica 159 (1/4) (2006): 146, accessed 15 April, 2011, doi: 10.1515/SEM.2006.025. 4 Donatella della Porta and Manuela Caiani, “The Europeanization of the Public Discourse in Italy: A Top-Down Process?” European Union Politics 7(1) (2006): 78, accessed 10 February 2009, doi: 10.1177/1465116506060913. 5 Ágnes Kapitány and Gábor Kapitány, “Symbols and Communication of Values in the Accession to the EU (Hungary),” Semiotica, 159 (1/4) (2006): 111–41. Accessed 15 April, 2011. doi: 10.1515/SEM.2006.024; Machill, Beiler and Fischer, “Europe-Topics”; Pribersky, “Europe”; della Porta, and Caiani, “The Europeanization”. 6 Machill, Beiler and Fischer, “Europe-Topics”, 63. 7 Priberksy, “Europe,” 146.

58

(De)coding the Fabric of the European Years’ Visual Representations

into a barcode8, or the competitions for the European Capital of Culture9 and for the EU birthday logo10, European Years constitute another attempt to reduce the visual deficit associated with the EU. Since 1983, the EU has been increasingly promoting different social, cultural or economic issues through European Years. The European authorities have chosen a different topic annually. The main purpose11 is to educate the widest possible audience, to attract the attention of the member states’ governments on the respective issue, to foster intra- and interEuropean dialogue and to change the citizens’ attitudes or behaviours. The verbal and visual strategies used in the promotion of European Years come as a solution to the “lack of visual presence [of the EU] in the public sphere” and consequently “the lack of symbolic representation of the EU”12. Thus, European Years focus on a public sphere where European citizens speak about the same issue (for example, volunteering in 2011), they become aware of the other citizens’ verbal and visual representations of the respective issue and try to use in their representations of the European issue, the criteria of relevance specific to their national, regional and local environments.

European Years’ visual guidelines – instances of a vertical Europeanization The acknowledgment of a visual deficit and of a negative perception on two famous images of the EU, namely the ‘family photos’ representing the meetings of the European Council or the image of a reconstruction site, made the EU change its visual embodiments of power. The discursive

8

Priberksy, “Europe,” 146-47; Giorgia Aiello, “The Appearance of Diversity: Visual Design and the Public Communication of EU Identity,” in European Union Identity: Perceptions from Asia and Europe, ed. Jessica Bain, and Martin Holland (Baden-Baden: Nomos, 2007), 148-50. 9 Giorgia Aiello and Crispin Thurlow, “Symbolic Capitals: Visual Discourse and Intercultural Exchange in the European Capital of Culture Scheme,” Language and Intercultural Communication 2(6) (2006): 148-162, accessed 17 August, 2012. doi: 10.2167/laic234.0. 10 Giorgia Aiello, “All Tögethé® Now: The Recontextualization of Branding and the Stylization of Diversity in EU Public Communication,” Social Semiotics 22(4) (2012): 459-86, Accessed 10 August, 2012, doi: 10.1080/10350330.2012.693291 11 http://en.strasbourg-europe.eu/european-year,27569,en.html, Accessed 13 May 13, 2012. 12 Priberksy, “Europe,” 146.

Camelia Cmeciu and Doina Cmeciu

59

strategy for each European Year is to verbally and visually represent different issues that may concern the European citizens. Coming back to Albert Moravia’s definition of European culture, our analysis will focus not on the “reversible fabric, one side variegated”, but on the other side, the “single colour, rich and deep”. Our aim is to provide an analysis of the vertical Europeanization at a visual level. Our empirical data will focus on the way in which four European issues are visually framed within the visual guidelines that can be found in the communication toolboxes that four European Years imposed on each member state: ƒ European Year of Intercultural Dialogue (2008); ƒ European Year of Creativity and Innovation (2009); ƒ European Year for Combating Poverty and Social Exclusion (2010); ƒ European Year of Volunteering (2011). When presenting European discourses in terms of inclusion and exclusion, “a European nexus”13 may be identified. It implies the ongoing negotiation of meanings of, and belongings to Europe in many different public spaces occurring in a whole range of genres, and in many languages14.

There are two instances15 of approaching the European nexus: (1) from a low intensity point of inclusion focused on a horizontal Europeanization, which implies reciprocal participation: national, regional and local bodies may provide their own insights into the European Years’ issue; (2) from a high intensity point of inclusion focused on a vertical Europeanization, which implies all types of (visual and verbal) restrictions imposed on member states. In the visual communication of Europe through European Years, we will analyse the double layers of the high intensity point of inclusion: - the European Years’ branding process; - the visual deontic modality within the visual guidelines of each European Year communication toolbox. 13

Ruth Wodak, “Discourses in European Union Organizations: Aspects of Access, Participation, and Exclusion,” Text and Talk 27 (5/6) (2007): 659, accessed 10 November, 2010, doi: 10.1515/TEXT.2007.030 14 Wodak, “Discourses,” 659. 15 Camelia M. Cmeciu, “Insights into the European Years’ Communication Toolboxes,” Styles of Communication 4 (1) (2012b): 40.

60

(De)coding the Fabric of the European Years’ Visual Representations

Beyond the EY branding process Having a temporal development, branding implies the adding of “a layer to an already existing meaning”16. Beyond the mere representation of letters, lines and colours, which form a logo, brands become semiotic resources producing discourse17 and investing objects with an added value. The link between a logo and a brand may be represented as a flow throughout three layers18: -the layer of metaforms implies an experiential abduction, focused on an association-by-inference process; -the layer of meta-metaforms should be understood in culture-specific ways, implying indexicality in reference; -the layer of metasymbols focuses on traces to a culture’s historical past, being governed by conventions and culture-specific ways. The European Years’ logos are actually signifiers/semiotic resources of an already existing meaning/signified, namely the European syntagm, “unity in diversity”. Using Marcel Danesi’s cognitive flow and interpreting lines and colours as semiotic modes19, we will provide the analysis of the three layers embedded within the shaping of the EY logos (Fig. 2-5). The interpretation of the EY logos (Table 2-1) lies on the understanding of the three layers within the cognitive flow of meaning: the layer of metaform, the layer of meta-metaform and the layer of metasymbols.

16 Torkild Thellefsen, Bent Sørensen, Mikael Vetner and Christian Andersen, “Negotiating the Meaning of Artefacts: Branding in a Semiotic Perspective,” Semiotica, 162 (1/4) (2006): 374, accessed 24 February, 2009, doi: 10.1515/SEM.2006.085. 17 Carlos Scolari, “Online Brands: Branding, Possible Worlds, and Interactive Grammars,” Semiotica 169 (1/4) (2008): 170, accessed 23 February, 2009, doi: 10.1515/SEM.2008.030. 18 We will interpret the EY logos in terms of Marcel Danesi’s layering theory (2002) on metaphor. 19 Gunther Kress and Theo van Leeuwen, Multimodal Discourse. The Modes and Media of Contemporary Communication. 2nd edition (London, New York: Bloomsbury Academic, 2010), 57-9.

Camelia Cmeciu and Doina Cmeciu

61

Fig. 2-5. Logos of European Years (2008-2011)

Table 2-1. The cognitive flow beyond the European Years’ logos Layers Layer of metaforms

EY 2008 curved lines, colours

Layer of metametaforms

humanlike figures holding hands

Layer of metasymbols

EY 2009 curved & diagonal lines colours sparks, synapses

EY 2010 straight lines colours

EY 2011 diagonal lines colours

bricks

holding hands

dancing together Ļ “joie de vivre” Ļ unity

synapses Ļ links between different fields Ļ unity

position of bricks Ļ building together Ļ unity

lines Ļ working together Ļ unity

differently coloured figures Ļ diversity

differently coloured sparks Ļ diversity

unequal form of bricks Ļ diversity

differently coloured hands Ļ diversity

62

(De)coding the Fabric of the European Years’ Visual Representations

The layer of metaform. Lines and colours are the formal semiotic devices which provide “cohesion and coherence”20 to the four EY logos. At this layer, the curved, straight or diagonal lines and the various colours are modes without any reference to any object, thus reminding us of Ch. S. Peirce’s level of firstness. As observed in Table 2-1, curved and diagonal lines and red, blue, yellow and green colours are more salient in the production of EY logos. The layer of meta-metaforms. A semiotic device becomes a mode if it turns into a resource for making signs. By indexicality in reference21, the signifier-material can be used to carry the signifieds of sign-makers22. The semiotic devices (lines and colours) in the layer of metaform turn into semiotic modes for the design of EY logos since they become resources for making signs. The straight, curved and diagonal lines and the various types of colours are the signifier-materials which combine in order to provide an indexical reference to distinct European Years’ signifieds: human-like figures (EY 2008), stars and synapses (EY 2009), bricks (EY 2010) and hands (EY 2011). The layer of metasymbols. The cognitive flow from signifier-materials to semiotic modes reaches the last layer. The lines and colours acquire a meaning potential through materiality and interactivity. As observed in Table 2-1, the layer of metasymbols embeds the visual representations of the European syntagm “unity in diversity”. The lines and colours as semiotic modes are meant “to represent aspects of the world as it is experienced by humans”23. Thus, the EY logos are European experiences of interculturality (2008), creativity and innovation (2009), combating poverty and social exclusion (2010) and volunteering (2011). Four unifying conventional processes of interaction may have the significance of unity: human figures dancing (2008), stars and synapses connecting different fields (2009), bricks being placed one upon the other in order to build together (2010), hands holding together and helping each other (2011). The four pervasive colours (red, yellow, blue, green) have the meaning potential of diversity at a double level: - at the interpersonal level: colours as indexical embodiments of different European member states or competences24; 20

Kress and Van Leeuwen, Multimodal Discourse, 58. Danesi, “Abstract Concept,” 6. 22 Kress and Van Leeuwen, Multimodal Discourse, 58. 23 Gunther Kress and Theo van Leeuwen, Reading Images: The Grammar of Visual Design. 2nd edition (London, New York: Routledge, 2006), 42. 24 In the case of the European Year of Creativity and Innovation, the stars signify several competences: communication in mother tongue (pink), communication in 21

Camelia Cmeciu and Doina Cmeciu

63

- at the intrapersonal level, colours as four instances of representational processes25: red as a material process – our experience of the material world, doing and happening; yellow as a relational process – a means of characterisation and identification; blue as a mental process – our experience of the world of our consciousness, sensing; green as a verbal process – creating narratives. The cognitive flows beyond the EY logos highlight the ways in which the semiotic devices of lines and colours are shaped throughout the three layers in order to visually represent European Years as embodiments of unity through unifying social practices and of diversity through various actions (doing, being, sensing, telling) within a European body.

Beyond the visual deontic modality of European Years’ visual guidelines Despite the freedom of implementing the EY issues at a micro-level, the restrictions to be found in the visual guidelines may be interpreted in terms of a visual deontic modality26. The EU institutions provide the visual regulations for each European Year. The member states’ national, regional and local organisations have to perform their activities within the visual context imposed by the European institutions if they want to become a member of the EY “brand discourse community”27. Thus, MS organisations become ‘the prisoners’ of the images of the European power and of the power of EY images28. Our choice of ‘deontic modality’ for the analysis of EY visual guidelines lies on the scale of authority intensity (Palmer, 1990, p. 16) that this concept implies. In our case, the two discursive participants to be found on the scale of authority are:

foreign languages (green), mathematical competence and basic competences in science and technology (red), digital competence (yellow), learning to lean (light blue), social and civic competences (purple), sense of initiative and entrepreneurship (blue), cultural awareness and expression (orange). 25 Michael A. K. Halliday, An Introduction to Functional Grammar. 3rd edition. (London: Hodder Arnold, 2004), 197, 210, 252. 26 The concept of “visual deontic modality” was first presented in Cmeciu, “Insights,” 45-6. In this article the analysis focused on EY 2011 and EY 2012. 27 Thellefsen, Sorensen, Vetner and Andersen, “Negotiating,” 373. 28 We adapted William J. T. Mitchell’s syntagm (1994), “we are prisoners of images of power and the power of images”, to the context of European Years.

64

(De)coding the Fabric of the European Years’ Visual Representations

(a) the European bodies (the Directorates-General29) which provide the visual guidelines – the speaker/the writer/the participant providing some instance of permission or demand; (b) the MS national, regional, local organisations promoting the EY issues – the addressee/the participant who is capable of producing the suggested/ordered act. Through Visual Guidelines, the European bodies ‘impose’ a representation of reality on the MS organisations, which do not have the freedom of choice if they want their social actions to be under the auspices of the European Years. Thus, the visual deontic modality embeds two instances of modality from social semiotics30: ƒ an ideational modality (“provides through its system of modality markers an image of the cultural, conceptual and cognitive position of the addressee”31): the visual markers mentioned in each EY Visual Guideline which are to be inserted in the MS organisations’ promotional items; ƒ an interactive modality (“a system of social deixis which ‘addresses’ a particular kind of viewer, or a particular social/cultural group”32): the stipulations regarding the interaction of EY visual markers in order to provide coherent and cohesive objects of promotion. Content analysis was used to analyse how visual deontic modality was framed in the Visual Guidelines of three European Years33: ƒ the European Year of Intercultural Dialogue 2008 Style Guide; ƒ the European Year of Creativity and Innovation 2009 Corporate Design Manual; ƒ the European Year of Volunteering 2011 Visual Guidelines. ƒ The coding procedure focused on two aspects: ƒ the operationalisation of each Visual Guideline in terms of six visual markers of deontic modality34: logo size, logo colour, logo brand space,

29

The EY 2008 and EY 2009 communication toolboxes were prepared by the Directorate-General for Education and Culture. The EY 2011 communication toolbox was prepared by the Directorate-General for Education and Culture, the Directorate-General for Employment, Social Affairs and Inclusion, DirectorateGeneral for Enterprise and Industry etc.). 30 Kress and Van Leeuwen, Reading, 172. 31 Kress and Van Leeuwen, Reading, 172. 32 Kress and Van Leeuwen, Reading, 172. 33 The communication toolbox for the European Year 2010 (prepared by the Directorate-General for Employment, Social Affairs and Inclusion) did not include any visual guidelines.

Camelia Cmeciu and Doina Cmeciu

65

layout colour, logo position and typography. We identified the visual references to the proper use of these six visual markers. The coding was performed by two coders and the inter-coder reliability was 0.91 (pi value). ƒ the coding of EY visual markers in terms of a type modality and of a token modality. For this operationalisation, we associated the type and token modality with two processes that form the conceptual representations35 in social semiotics: analytical processes and classificational processes. Analytical processes “relate participants in terms of a partwhole structure”36. Within the EY logos, the Carrier (the whole) is the layer of meta-metaform of the logo (figures, stars, bricks, hands) and the Possessive Attributes (the parts) are the size, colour, position of the respective Carrier. Classificational processes “relate participants to each other in terms of ‘a kind’ of relation, a taxonomy”37. In our analysis, we identified the participants as visual embodiments of the European layers of meta-metaforms/Carriers for each member state. This difference between analytical processes and classificational processes determined us to code the type visual deontic modality in terms of analytical processes (standard EY images) and the token visual deontic modality in terms of classificational processes (taxonomies of EY images for different member states). Our study focused on two research questions: RQ1: What is the salience of the visual markers of deontic modality? RQ2: What is the salience of the type versus the token visual deontic modality? Table 2-2 and Table 2-3 illustrate that the EY Visual Guidelines embed 247 references regarding the use of visual markers of deontic modality: 168 references for type modality (68%) and 79 references for token modality (32%). Thus, standard EY images as instances of type visual modality are more salient than EY images adapted to member states (token visual modality). This dominance may be linked to the EU tendency of standardising the visual representations of European Years.

34

These visual markers for EY 2011 and EY 2012 are also presented in Cmeciu, “Insights,” 47-8. 35 Conceptual representations “design social constructs” (Kress and Van Leeuwen, Reading, 79). 36 Kress and Van Leeuwen, Reading, 87. 37 Kress and Van Leeuwen, Reading, 79.

66

(De)coding the Fabric of the European Years’ Visual Representations

The hierarchy of visual markers for type visual deontic modality (Table 2-2) is the following: logo position (35%), logo colour (29%), logo size (18%), typography (11%). The hierarchy of visual markers for token visual deontic modality (Table 2-3) is the following: logo colour (51%), logo position (23%), typography (10%). As observed from these two hierarchies combined, there is a salience of logo colour (89 visual references – 36%) and of logo position (76 visual references–31%). Table 2-2. Type visual deontic modality Visual markers Logo

Size

EY 2008 21

EY 2009 1

EY 2010 0

EY 2011 9

Colour

16

16

0

17

Brand space

1

2

0

1

Layout colour

3 2 (Don’ts) 46

0

0

1

5

0

7

1

2

0

1

6

7

0

6

94 2 (Don’ts)

31

0

41

Logo position Background Typography Total

Theme fonts/Type face

Total 31 (18%) 49 (29%) 4 (2%) 6 (3%) 58 (35%) 4 (2%) 19 (11%) 168 (100%)

The type visual deontic modality may be associated with structured analytical processes, showing a coherent visual Carrier with all its Possessive Attributes fitting together38. Two important aspects may be included in structured analytical processes: brand space and logo size. The brand space is the protective zone around the Carrier and its purpose is to rule out any visual competition with other design elements39. The logo size 38 39

Kress and Van Leeuwen, Reading, 87. European Year of Intercultural Dialogue 2008, Style Guide, 4.

Camelia Cmeciu and Doina Cmeciu

67

is significant for visibility. Each European Visual Guideline mentions the minimum recommended size of the logo for good visibility (2008–35%, 2011–45 mm). The Visual Guidelines of the European Year 2009 were more permissive, the optimal logo size being “determined for individual media, depending on the area of application. It is recommended to use the long and horizontal version of the logo (...)”40. The token visual deontic modality may be associated with two taxonomies as part of classificational processes: a) taxonomies of the logo designs and language adaptations for 23 member states (2008 and 2009), and b) taxonomies of colour contrasts (nine possible combinations in 2011). The colours or typefaces are shown as parts, without visually rendering the way in which these parts fit together to make up the Carrier. Table 2-3. Token visual deontic modality

Visual markers Logo

Typography Total

Size

EY 2008 0

Colour

23

Logo position

0

Background

0

Theme fonts/ Type face

0 23

EY 2009 4 (Don’ts) 5 2 (Don’ts) 6 7 5 1 (Don’ts) 23 7 (Don’ts)

EY 2010 0

EY 2011 1

0

12

0

10

0

1

0

2

0

26

Total 5 (6%) 40 (51%) 18 (23%) 8 (10%) 8 (10%) 79 (100%)

When we presented the cognitive flow within the EY branding process, we mentioned the salience of four colours in the European Years’ logo, namely red, blue, yellow and green. We have analysed these semiotic modes in terms of a functional perspective on representation. We believe that two more logo possessive attributes are important in the visual deontic modality, namely position and typography. 40

European Year of Creativity and Innovation 2009, Corporate Design Manual, 4.

68

(De)coding the Fabric of the European Years’ Visual Representations

The type and token EY images can be interpreted as “socially constructed knowledges of (some aspect of) reality”41. The logo position, through its strict distribution of visual elements, makes reference to the aspect of reality in terms of the power relations visually created between three generic participants: the European Year, the European Union and member state. The 76 visual references (31%) to logo positions in type and token visual deontic modality emphasise the importance laid on the position of the EY, the EU, MS logos within the image space. The Visual Guidelines provide covert taxonomies on the specific position of the three participants. Within the visual distribution of the EY, EU and MS logos, the 2008, 2009, 2010 and 2011 EY logo is always the last one in the distribution line, being placed on the right-hand bottom position, thus being the last visual item to be remembered. Besides colours, typography is another important semiotic mode since it has textual, ideational and interpersonal meaning and it is multimodal and systemic42. Each European Year is assigned a typeface and a theme font: Arial for EYID 2008, Myriad Pro and Tahoma for EYCI 2009, and ITC Lubalin Graph and Interstate for EYV 2011. These five typefaces are, firstly, presented as a medium, their provenance (designers, release years), applications and possible variants being highlighted. The choice for these typefaces lies in their humanistic characteristics as it is mentioned in every European Year Visual Guideline. We will analyse these humanistic characteristics in terms of curvature and orientation, two distinctive features within the grammar of typography43. Opposed to angularity, which signifies “technical and harsh”, curvature stands for roundedness which implies smoothness, softness and organic. Whereas horizontal orientation suggests “heaviness, solidity”, vertical orientation stands for “lightness, upwards aspiration”44. These meanings can also be found in the explanations45 provided for the Arial and Myriad Pro typeface: a) “The overall treatment of curves [in Arial typeface] is softer and fuller than in most industrial style sans serif faces. The terminal strokes are cut on the diagonal which helps to give the face a less mechanical appearance.” (p. 10)

41

Kress and Van Leeuwen, Multimodal Discourse, 24. Theo Van Leeuwen, “Towards a Semiotics of Typography,” Information Design Journal +Document Design, 14(2) (2006): 154. 43 Van Leeuwen, “Towards,” 148. 44 Van Leeuwen, “Towards,” 149. 45 The European Year of Intercultural Dialogue 2008 Style Guide, 10; The European Year of Creativity and Innovation 2009 Corporate Design Manual, 10. 42

Camelia Cmeciu and Doina Cmeciu

69

b) “Myriad is easily recognised due to its special “y” descender (tail) and slanting “e” cut, and rounded curves.” (p. 10) Besides the humanistic characteristics, the choice of these typefaces lies in legibility. Since all EY visual representations are the official standpoint of the European Union, reading and formality should be two significant connotations to be transmitted. Alongside with pictures and logos, the EY visual items also include titles, subtitles and texts. We consider that weight, expansion, slope and connectivity46 are the four features of typography which carry the meaning potential of a formal style that the European Union wants to impose upon the member states through Visual Guidelines. Titles and subtitles are provided with a bold weight, wide expansion, upright slope and disconnection in order to highlight the important aspects of the respective information to be sent. Unlike the text which has a regular weight and condensed expansion, titles and subtitles also become visible through another semiotic mode, namely colour, thus emphasising the importance of multimodality in the visual representations of European Years.

Conclusions The choice of annual issues as European Years to be implemented at a national, regional and local level, has been a means of shifting the attention from the centre (European Union) unto the margins (Member States). Despite this intention, the European Union and the DirectoratesGeneral show their power at a visual level by imposing a visual standardisation and uniformity. Our analysis focused on the vertical Europeanization, namely on the EY fabric, “the single colour rich and deep”, as it was discursively embedded in the visual guidelines of each EY communication toolbox (2008-2011). The discourse of inclusion promoted in the EY visual guidelines has a high intensity level point, which is attained by the EY branding process and the type and token visual deontic modality. The logo position, colour and typography are the most salient visual markers highlighting the European authority. The instances of the standard EY images (type modality) and of the MS visual taxonomies (token modality) reveal the power relations among participants (EU, EY, and Member States) where the indexical signs of the EU and the EY should prevail in the promotional items of national, regional or local organisations. 46

Van Leeuwen, “Towards,” 148-9.

70

(De)coding the Fabric of the European Years’ Visual Representations

Bibliography Aiello, Giorgia, and Crispin Thurlow. “Symbolic Capitals: Visual Discourse and Intercultural Exchange in the European Capital of Culture Scheme.” Language and Intercultural Communication 2(6) (2006): 148–62. Accessed 17 August, 2012. doi: 10.2167/laic234.0 Aiello, Giorgia. “The Appearance of Diversity: Visual Design and the Public Communication of EU Identity.” In European Union Identity: Perceptions from Asia and Europe, edited by Jessica Bain, and Martin Holland, 147–81. Baden-Baden: Nomos, 2007. —. “All Tögethé® Now: The Recontextualization of Branding and the Stylization of Diversity in EU Public Communication.” Social Semiotics 22(4) (2012): 459-486. Accessed 10 August, 2012. doi: 10.1080/10350330.2012.693291 Cmeciu, Camelia M. “Insights into the European Years’ Communication Toolboxes.” Styles of Communication 4 (1) (2012): 25–35. Danesi, Marcel. “Abstract Concept-Formation as Metaphorical Layering.” Studies in Communication Sciences 2 (1) (2002): 1–22. della Porta, Donatella, and Manuela Caiani. “The Europeanization of the Public Discourse in Italy: A Top-Down Process?” European Union Politics 7(1) (2006): 77–112. Accessed 10 February 2009. doi: 10.1177/1465116506060913 Halliday, Michael A. K. An Introduction to Functional Grammar. 3rd edition. Revised by C. Matthiessen, London: Hodder Arnold, 2004. Kapitány, Ágnes, and Gábor Kapitány. “Symbols and Communication of Values in the Accession to the EU (Hungary).” Semiotica, 159 (1/4) (2006): 111–41. Accessed 15 April, 2011. doi: 10.1515/ SEM.2006.024. Kress, Gunther, and Theo van Leeuwen. Reading Images: The Grammar of Visual Design. 2nd edition. London, New York: Routledge, 2006. Kress, Gunther, and Theo van Leeuwen. Multimodal Discourse. The Modes and Media of Contemporary Communication. 2nd edition. London, New York: Bloomsbury Academic, 2010. Machill, Marcel, Markus Beiler, and Corinna Fischer. “Europe-Topics in Europe’s Media. The Debate about the European Public Sphere: A Meta-Analysis of Media Content Analyses.” European Journal of Communication 21(1) (2006): 57–88. Accessed 20 March, 2009. doi: 10.1177/0267323106060989 Mitchell, William J. T. Picture Theory: Essays on Verbal and Visual Representation. Chicago, London: University of Chicago Press, 1994.

Camelia Cmeciu and Doina Cmeciu

71

Palmer, Frank R. Modality and the English Modals. 2nd edition. London, New York: Longman, 1990. Peirce, Charles S. The Collected Papers of Charles Sanders Peirce. Vols. IVI. Ch. Hartshorne & P. Weiss (Eds.). Cambridge, MA: Harvard University Press 1931-1935), Vols. VII-VIII. A.W. Burks (Ed., 1985). All eight volumes in electronic form – J. Deely (Ed). Charlottesville, VA: Intelex Corporation, 1994. Pribersky, Andreas. “Europe as a Symbol in Political Image Constructions.” Semiotica 159 (1/4) (2006): 143–50. Accessed 15 April, 2011. doi: 10.1515/SEM.2006.025. Scolari, Carlos. “Online Brands: Branding, Possible Worlds, and Interactive Grammars.” Semiotica 169 (1/4) (2008): 169–88. Accessed 23 February, 2009. doi: 10.1515/SEM.2008.030. Thellefsen, Torkild, Bent Sørensen, Mikael Vetner, and Christian Andersen. “Negotiating the Meaning of Artefacts: Branding in a Semiotic Perspective.” Semiotica, 162 (1/4) (2006): 371–81. Accessed 24 February, 2009. doi: 10.1515/SEM.2006.085 Van Leeuwen, Theo. Introducing Social Semiotics. London, New York: Routledge, 2005. —. “Towards a Semiotics of Typography.” Information Design Journal +Document Design, 14(2) (2006): 139–55. Wodak, Ruth. “Discourses in European Union Organisations: Aspects of Access, Participation, and Exclusion.” Text and Talk 27 (5/6) (2007): 655–80. Accessed 10 November, 2010. doi: 10.1515/TEXT.2007.030

INTERSEMIOTIC TRANSLATION IN ADVERTISING DISCOURSE: PLASTIC VISUAL SIGNS IN PRIMARY FUNCTION IN COMMUNICATION EVANGELOS KOURDIS

Introduction In recent years, more and more advertising campaigns have been using synthetic elements that represent universal human values such as ecology, good health and respect for differences, which are based on contrastive rhetorical forms (pollution/ecology, illness/good health, etc.). Advertisers, we have noted, have not focused their efforts on key semiotic systems such as language (slogans) and images (photographs, paintings, drawings), but have, instead, upgraded their use of plastic visual signs, in particular colours, graphics and typography, which, up until the late 20th century, had been considered as supplementary semiotic systems of verbal and visual iconic signs. It is particularly interesting to say that for Eco (2001: 221) advertising is considered as a mass-communication text, a syncretic text, which often involves more than one semiotic system and moves across linguistic and cultural boundaries. According to him, this kind of text is also useful for dealing with cases of intersemiotic translation. Bearing in mind that language is considered to be a primary semiotic system, I will show that secondary semiotic systems, as are plastic visual signs, sometimes play a central role in communication through advertising, despite being part of a broader semiotic system, that of iconism. I will also examine how intersemiotic translation could be an easy interpretative procedure, and at the same time a more complex constructive procedure, and how it depends on cultural knowledge of the verbal message’s connotative meaning.

Evangelos Kourdis

73

Visual signs in advertisements Jean-Marie Klinkenberg (2001: 110), one of the founders of the Belgian Mu Group, or Groupe ȝ, mentions that visual semiotics aims to endow the reading of images (photos, cinema, paintings, designs, posters) with the same precision that textual semiotics was able to develop in literary, political and other speech. This precision was associated with the use of the semiotic system of language, which leading linguists and semioticians (Saussure 1986 [1916]: 9; Hjelmslev 1943: 109; Barthes 1964: 43-44; Jakobson 1970: 511; Lotman 2001 [1990]: x; Eco 1979: 174) described as a primary sign system. In contrast, where the visual system was concerned, such views were indeed expressed, but only where this system was accompanied by the verbal system (Barthes 1964: 43; Klinkenberg 2001: 110) with the aim of tackling the ambiguity of the visual system. In the case of plastic visual signs, their value was noted from very early on, especially in the field of advertising. One of the first systematic works in semiotic studies in advertising to talk about plastic signs (however, without using this term) was Barthes’ essay (1964: 42) “The rhetoric of the image”. Barthes, in his study of the French advertisement of Panzani pasta, classified the semiotic systems or messages into two main types: verbal and non-verbal messages. He then classified non-verbal messages into codified iconic and non-codified iconic messages, plastic signs being the codified iconic messages. It is worth mentioning that in Barthes’ (1964: 43-44) classification, the verbal message is premised in relation to the iconic messages, since "writing and speech are always complete terms of informational structure" and because it confronts the polysemic character of the image. Guidère (2000: 39), commenting on this trilateral division of advertising signs, states that it has turned advertising language into an advertising giant, a cluster of disparate signs1. Twenty years later, Groupe ȝ (1992), in their famous work “Traité du signe visuel” (1992)2, elaborated on Barthes’ classification, criticising linguistic imperialism and emphasising the specificity of the visual sign (Vandeloise1995: 423) since, while any visual sign can be verbalised, a visual sign does not correspond to any particular word. Groupe ȝ categorised non-verbal semiotic systems into iconic visual signs and plastic visual signs such as colour, form and texture. Groupe ȝ (1992:

1

My translation Göran Sonesson said that this work was to visual communication what Saussure’s Cours de linguistique générale was to linguistics. 2

74

Intersemiotic Translation in Advertising Discourse

361) defines the relationship between iconic visual and plastic visual signs in the following way: the plastic, being phenomenologically the signifying of the iconic signs, enables the identification of the iconic. In turn, the iconic, once identified, enables one to attribute a content to the plastic elements which don’t belong to the iconic type.

According to Groupe ȝ, signifiers of an iconic entity coincide as a rule with signifiers of a plastic entity, and vice versa. But is there any relation between the two types of visual signs? For Groupe ȝ (1995: 597), there is an iconoplastic relationship between iconic visual signs and plastic visual signs, and this relation: […] is evidence that the plastic element is autonomous from the iconic representation. In fact plastic and iconic elements complement each other. Because it is the phenomenological signifier of the iconic sign, the plastic element allows viewers to identify the iconic, while the iconic element thus identified makes it possible to discover a content in the plastic elements that do not belong to iconic types.

Another difference between the two types of visual sign is that iconic visual signs create a triadic relationship between the signifying, the type3 (object) and the referent.

Plastic visual signs and intersemiosis In contrast to iconic visual signs, plastic visual signs are independent of types. One of the reasons supporting the autonomy of plastic signs is that they can serve as one of the two poles in intersemiotic translation. The autonomy of signs is of primary interest for intersemiotic translation. According to Aguiar & Queiroz (2009: 205): [...] intersemiotic translation can be described as a multi-hierarchical process of relation between semi-independent layers of description. The layers of organisation do not act independently but they are autonomous in functional and descriptive terms.

Plastic visual signs include signs such as colour, form and texture, but only to the extent that they refer to a signified and that we can approach 3

For a definition of type, see Klinkenberg (2001: 111).

Evangelos Kourdis

75

them as a symbol or index (Klinkenberg 2001: 111). We should point out that some plastic signs are important because they exploit a plastic rhetoric (Klinkenberg 1996: 91) and that they are semiotic in that they associate forms of expression with forms of content (Groupe ȝ 1995: 584). Vandeloise (1994: 438) argue that: [l]ike the distinctive features by which Jakobson represents linguistic signifiers, plastic signs are grounded in a system of oppositions. Pertinent contrasts are light/dark, simple/complex, vertical/horizontal, etc.

I will show later that these polarised elements are often used to create intersemiosis as an interaction between semiotic systems. In fact, intersemiosis characterises the whole semiotic phenomenon, which is based on what Jakobson (2001 [1959]: 139) calls intersemiotic translation or transmutation, that is ‘‘[…] the interpretation of verbal signs by means of signs of non-verbal systems’’. However, many researchers claim that intersemiotic translation should not necessarily include verbal semiotic systems. This position gives a dynamic dimension to intersemiotic translation which advertisers take into account, since, as Gorlée (1994: 167) remarks: (the) strong points (of intersemiotic translation) are not information nor thought, but novelty and creativity, --in short, variance. Variance meaning openness and possibility, it permits, and even encourages, multiple interpretations.

For Torop (2003: 273): [t]he understanding of intersemiotic translation starts from the realisation of text processuality, on the one hand, and coexistence of diverse sign systems, i.e. semiotic heterogeneity, on the other hand.

As we will see below, plastic signs can serve as intersemiotic translations of verbal signs, in this way increasing the number of non-verbal signs that produce intersemiotic translations.

Analysing the material of the study This paper presents three selected cases of intersemiotic translation in Greek advertisements adopting the Groupe ȝ (1992) approach. The intersemiosis takes place between the verbal semiotic system and the nonverbal semiotic systems, mainly the plastic signs, in the advertisements.

76

Intersemiotic Translation in Advertising Discourse

The three advertisements analysed below are print advertisements that were placed in Greek newspapers and magazines. The first advertisement (fig. 2-6) studied presents a banking product offered by Cyprus Bank. The advertisement is divided into two parts: the iconic message, situated above, takes up 3/5 of the composition, while below we have the verbal message, which consists of seven separate verbal messages. The verbal messages are placed in distinct positions within the space provided and are written in letters of different sizes, colours and thicknesses. However, two important pieces of verbal information have been placed in yellow boxes: the verbal message “ȜȠȖĮȡȚĮıȝȩȢ ȝȚıșȠįȠıȓĮȢ extra” [extra payroll account] and the message directly below it, which is none other that the bank’s logo. It is particularly interesting to see that the intersemiotic translation in the advertisement is achieved through two non-verbal signs found in the first of the seven verbal messages, namely the 5% interest rate offered to any civil servant who opens a payroll account at this bank. The sign “5%” comes under the semiotic system of mathematics and essentially consists of two signs, “5” and “%”, which, in this context, do not make any sense on their own. It is precisely this sign that the advertisers have translated intersemiotically in the advertisement’s iconic message, since it is the benefit that the depositor stands to gain, and it is this that the advertisement focuses on. It is, however, also interesting to note in what way this translation has been achieved. The sign “5” is represented by a road with a broken dividing line, and the sign “%” by means of two bushes and a river. Here, the polarised elements are linked through intersemiosis, since the colours used in this advertisement are contrasting: the bright green surrounding the road contrasts with the light grey colour of this road, but is still not as strong a colour as the dark green of the bushes representing the zeros in the percentage symbol. In addition, the river is a bright blue that is in stark contrast to the green surrounding the road. Thus, as Van Leeuwen (2011: 11) remarks, ‘‘colour (is) used ‘textually’, to create coherence between the different elements of a larger whole and/or to distinguish between its different parts’’. Moreover, the use of the proxemic sign in the graphic and chromatic representation of the sign “5%” ensures that the latter can be decoded and translated. The fact that the shadow of the two bushes is included in the iconic message allows us to claim that the transmutation of semiotic systems does not preclude the use of realistic means of portraying reality. This is also supported by the bushes scattered around the remaining iconic message and not affecting intersemiosis but rather adding a sense of

Evangelos Kourdis

77

realism to the iconic message. Furthermore, even though three colours, namely blue, green and grey, have been used to create the iconic message, it is the colour green that dominates, which indicates that the advertisement follows the current trend towards green and ecological products. Of course, the specific banking product has nothing to do with ecology, but the advertisers preferred to use a colour that represents a universal value and which advertisers have used ad nauseam in the last decade. In other words, they are banking on the positive connotations of the colour green. The verb “įȡȠȝȠȜȠȖȒıIJİ” [put under way], found in the first verbal message in the second person plural, is also rendered intersemiotically by the road. We can, therefore, claim that the graphic and chromatic representation of the sign “5%” in the iconic message conveys the same signified, but with a different signifier. We thus have a differentiation, not of content, but rather of form4, which allows us to speak of the transmutation of signs or intersemiotic translation. Although Jakobson (2004 [1959]: 139) defined the two synonymous terms as the ‘‘interpretation of verbal signs by means of signs of non-verbal sign systems’’, translation semioticians such as Petrilli (2003: 18) clarify that: intersemiotic translation or transmutation […] consists of interpreting verbal signs by means of nonverbal signs and vice versa, as well as nonverbal signs of a given sign system with nonverbal signs of another sign system.

Interestingly, the non-verbal sign “5%” found in the verbal message is translated intersemiotically with the assistance of the additional non-verbal signs of graphics (display typography), colours and proxemics. The second advertisement (fig. 2-7) forms part of the advertising campaign of Cosmote, a mobile network operator. Through this advertisement, Cosmote endeavours to show that it is able to see the world through its customers’ eyes, that it understands their different needs and meets them with even greater flexibility and vision5. We note that the operator’s logo (“Cosmote”) and the standard verbal message that always follows it (“Ƞ țȩıȝȠȢ ȝĮȢ, İıȪ”, meaning “our world is you”) has been 4

This formulation is seemingly in contrast to that of the Russian formalists (Erlich, 1980: 197), for whom content was determined by form and hence each different form had a different meaning. However, in advertisement we have fixed signifieds, semantically reduced, as they refer to a commodity. So, in the end in advertising the signified meaning is reduced to simple connotation.  5 See http://www.myphone.gr/forum/showthread.php?t=236992.

78

Intersemiotic Translation in Advertising Discourse

placed in the bottom right-hand section of the composition, just as in the previous case (fig. 2-6). In this advert, the advertisers have also made use of the rhetorical practice of abduction, moving from the general “our world” to the specific “is you”, and employing the second person singular so that the reader may identify with the use of this particular product.

Fig. 2-6: print advertisement of Cyprus Bank

Unlike the previous advertisement, the principal verbal message here (slogan) – “ȉȫȡĮ IJȠ 3G ıİ ĮțȠȜȠȣșİȓ ȩʌȠȣ țĮȚ ĮȞ ʌĮȢ”, meaning “Now 3G goes wherever you go” – has been positioned in the middle of the composition. The slogan comprises three semiotic systems, two verbal (Greek, English) and one non-verbal (number), and has been made white in order to stand out from the blue (and other hues) of the sea. The advertisers have placed the additional verbal message at the bottom of the composition and have used black and white letters for contrasting purposes. The green background on which this message has been placed is the mobile network operator’s trademark colour. As far as the colour system is concerned, we find that green and blue and their various shades have been used in this advertisement, too. The

Evangelos Kourdis

79

message “3G” found in the slogan and in the additional verbal message has been graphically and chromatically reproduced at the top of the composition. The number “3” has been reproduced in green as vegetation growing on a cliff, while the letter “G” is represented in the form of a cloud above the sea. For this portrayal to acquire semantic and semiotic content, the advertisers have also made use of the proxemic sign, positioning these two signs next to each other. This advertisement, unlike the previous one, also has a purely visual iconic message, showing a beach with sunbathers and the surrounding mountain. It is on this visual iconic sign that the plastic visual signs forming the message “3G” have been added, which enables us to claim that an imaginary sign can be constructed on a real sign.

Fig.2- 7: print advertisement of the Greek mobile network operator Cosmote

The third advertisement composition (fig. 2-8), titled “ȈțȑȥȠȣ ȆȡȐıȚȞĮ” [“Think Green”], is part of a new project launched by Piraeus Bank in the social media, on the benefits of an environmentally friendly way of life6. According to McCarthy & Mothersbaugh (2002: 671) “one inference found repeatedly in persuasion research is ‘length implies 6

See http://www.econews.gr/2011/02/28/skepsou-prasina-trapeza-peiraios.

80

Intersemiotic Translation in Advertising Discourse

strength’ (and that) the logic is that the more a brand has to say about itself, the better the brand must be’’. We argue that this position is not applicable to this advertisement, where the principal verbal message is too short (only two words). We also note that green is the dominant colour in this advertisement. The verbal slogan (“Think Green!”) is here, too, found in the middle of the composition and has been written in the second person singular in an effort to establish direct communication with the reader. The principal advertising composition is placed within a coloured frame; around it is a second frame of a different colour containing the additional message “ȂȚĮ ʌȡȦIJȠȕȠȣȜȓĮ IJȘȢ ȉȡȐʌİȗĮȢ ȆİȚȡĮȚȫȢ” [“A Piraeus Bank initiative”]. The advertising composition as a whole and the two verbal messages appear to be in initial contrast with the business environment promoting them, since ecological consciousness is being advertised by a bank dealing in a starkly different environment – the financial world, an environment not known among different cultures for its promotion and respect for human values7.

Fig. 2-8: print advertisement of Piraeus Bank

What distinguishes this advertisement from the previous one is the double occurrence of intersemiotic translation: the verb “ıțȑȥȠȣ” [“think”] is written in white, the same colour as its intersemiotic translation 7

This is not a new location. As Corner (2004: 236) observes ‘‘it is the belief that advertising does indeed work in a ‘dispersed’ way to encourage certain values and beliefs, as well as in a ‘concentrated’ way to sell goods, that has generated so much controversy about advertising as a communicative practice’’.

Evangelos Kourdis

81

provided in the form of the (human) brain, which has replaced the tree’s foliage in the iconic message. Directly underneath, the adverb “ʌȡȐıȚȞĮ” [“green”] is rendered iconically by the tree trunk supporting the (human) brain/foliage. It is interesting to see that different colours are used to distinguish between the two intersemiotic translations and that the translations include techniques of rhetorical expression. Thus, by intersemiotically translating the verb “think” through the (human) brain, the advertisers have chosen to translate/render the concept of thought, the whole, by means of the (human) brain, a part of the whole, using the expressive form of synecdoche. The same applies to the intersemiotic translation of the adverb “green”, which is used to express flora as a whole by means of the tree trunk, which represents part of the whole. There is no intersemiotic translation for the exclamation mark accompanying the slogan “Think Green”.

Cultural values, advertising and intersemiotic translation For Ugo Volli (2000: 220), advertising belongs to a rich culture genre of texts closely related to intersemiotic translation. The three advertisements we have studied promote cultural values8 that are of primary interest for the consumers. This is a standard practice. According to O’Guinn, Allen & Semenik (2012: 187), ‘‘advertisers try to either associate their product with a cultural value or to criticise a competitor for being out of step with one’’. According to them: [v]alues are enduring expressions of culture and they cannot be changed quickly or easily. They are thus different from attitudes, which can be changed through a single advertising campaign or a single advertisement. Attitudes are in turn influenced by values as well as by many other sources.

The three advertisements tried to influence consumers’ attitudes (to become clients of the two banks and of the mobile network operator), based on their cultural values (financial profit, easy communication and environment protection) and intersemiotic translation serves this effort through the use of plastic signs. It is worth mentioning the cultural dimension of plastic signs, especially that of colours. As Fowles (1996: 8 Even the direction of the letter's justification is a vehicle of cultural values. McCarthy & Mothersbaugh (2002: 673-674) state that the ‘‘direction of justification […] affect legibility […], and is likely culturally bound (e.g. right justification being most legible for those Asian readers who read from right to the left’’. In our case, the first two advertisements are written from the left to the right.

82

Intersemiotic Translation in Advertising Discourse

159) observes, ‘‘advertising distils from the variety of human appearances the few that will be accepted as apotheosis and returns them in perfected form to an audience desiring to see such singular rendition’’.

General remarks The study of the three advertisements confirms Van Leeuwen’s (2011: 92) remark that ‘‘the structure of texts in magazines, websites and other modern media is now often signaled, not by means of words, but by means of layout, colour and typography, so much so that without layout, colour and typography many of these texts would be incomprehensibl’’. Plastic signs play a major role in this effort, particularly when they signify something connotatively. In our three cases, no matter what the product advertised, green was used to enhance the advertisement since it has positive connotations and is an often-used and recognised advertising option in the Greek market. Colour differences and form (graphics) are plastic elements that dominate, compared with verbal and visual iconic signs, even though all the semiotic systems employed here have worked well to convey the message successfully. In fact, we could say that they have been upgraded in the compositions to the extent that they may have gained certain autonomy in the advertisements through the intersemiotic translation that has taken place with these particular signs. It is, however, worth noting that the semiotic systems of proxemics and display typography9 are almost always used to assist in the intersemiotic translation of these signs. Display typography involves the interpretative and illustrative use of letterforms, providing opportunity for the associative values and the formal characteristics of letters to be explored and exploited to deliberate effect. The display typography of the number in the first advertisement and of the number and letter in the second advertisement is intericonic. In other words, their traditional, expected design combines – by incorporating into their form and composition – other icons that assist further in conveying the message. This visualtypographical game often appeals to viewers and enhances the advertisement due to the witty way in which it combines two ideas in one. 9

According to Baines and Haslam (2005: 48) by the time of the Industrial Revolution there was a growth in many kinds of printing to meet the demands of commerce and these new ephemeral uses and needs required new typeforms whose principal aim was to attract attention. Because of their scale and intended use, they are sometimes referred to as display faces. 

Evangelos Kourdis

83

I believe that the advertisers’ choice to employ intersemiotic translation between verbal and plastic visual signs is a successful choice, since it helps them to break free from the visual iconic sign, whose interpretation is more complex, and it also brings into play methods of interpretation that are based on a cognitive procedure whose principal characteristics are quick decoding and wittiness.

In lieu of a conclusion Plastic visual signs often participate to expressive forms of rhetoric, such as is synecdoche. Their polarised elements (for instance, light/dark) play a central role in intersemiosis where plastic visual signs are always present. Plastic visual signs, especially colours, can be used as symbols and indexes: green colour for nature and ecology, grey for roads, white for clouds. Thus, plastic visual signs in our study have all the characteristics described by Groupe ȝ, plus – and it is something important for our purposes – they can serve as one of the two poles in intersemiotic translations because of their status as autonomous signs. This remark shows us the importance the advertisers place upon them and that plastic visual signs can be considered as a new and growing field in advertising based on cultural knowledge and cognitive procedures.

Bibliography Aguiar, Daniella, and João J Queiroz. ‘‘Towards a Model of Intersemiotic Translation,’’ The International Journal of the Arts in Society 4, no 4 (2009): 203-210. Baines, Phil, and Andrew Haslam. Type & Typography. London: Laurence King, 2005. Barthes, Roland. ‘‘La rhétorique de l’image,’’ Communications 4 (1964): 40-51. Corner, John. ‘‘Adworlds.’’ In The Television Studies Reader, edited by Robert Allen and Anette Hill, 226-241. London, New York: Routledge, 2004. Eco, Umberto. La structure absente. Introduction à la recherche sémiotique. Paris: Mercure de France, 1972. —. A Theory of Semiotics. Bloomington: Midland, 1979. Eco, Umberto, and Siri Nergaard. ‘‘Semiotic approaches.’’ In Roultedge Encyclopedia of Translation Studies, edited by Mona Baker, 218-222. London and New York: Roultledge, 2001.

84

Intersemiotic Translation in Advertising Discourse

Erlich, Victor. Russian formalism: History-doctrine. The Hague: Mouton, 1980. Fowles, Jib. Advertising and Popular Culture. Thousand Oaks, London, New Delhi: Sage, 1996. Gorlée Dinda. Semiotics and the problem of translation. With special reference to the semiotics of Charles S. Peirce. Amsterdam and Atlanta: Rodopi, 1994. Groupe ȝ. Traité du signe visuel. Pour une rhétorique de l'image. Paris: Seuil, 1992. —. ‘‘A rhetoric of visual statements.’’ In Advances in visual semiotics: the semiotic web 1992-93, edited by Thomas A. Sebeok and Jean UmikerSebeok, 581-599. Berlin and New York: Mouton de Gruyter, 1995. Guidère, Mathieu. Publicité et traduction. Paris: L’Harmattan, 2000. Hjelmslev, Louis. Prolegomena to a Theory of Language. Madison: University of Wisconsin, 1943. Jakobson, Roman. ‘‘La Linguistique.’’ In Tendances principales de la recherche dans les Sciences Sociales et Humaines, edited by Unesco, 504-556. Paris, La Haye: Mouton, Unesco, 1970. —. ‘‘On linguistic aspects of translation.’’ In The Translation Studies Reader, edited by Lawrence Venuti, 138-143. New York and London: Routledge, 2001 [1959]. Klinkenberg, Jean-Marie. Précis de sémiotique générale. Bruxelles: De Boeck, 1996. —. ‘‘Qu’est-ce que le signe?.’’ In Le Langage: nature, histoire et usage, edited by Jean-François Dortier, 105-112. Auxerre: Science Humaines, 2001. Lotman, Juri. The Universe of the Mind. A Semiotic Theory of Culture. London: Tauris, 2001 [1990]. McCarthy, Michael, and David Mothersbaugh. ‘‘Effects of Typographic Factors in Advertising-Based Persuasion: A General Model and Initial Empirical Tests,’’ Psychology and Marketing 19, no 7-8 (2002): 663691. O’Guinn, Thomas, Chris Allen and Richard Semenik. Advertsising and Intergrated Brand Promotion. Mason: South-Western Cengage Learning, 2012. Petrilli, Susan. Translation, Translation. Amsterdam and New York: Rodopi, 2003. Saussure, Ferdinand De. Course in General Linguistics. La Salle, Illinois: Open Court Publishing Company, 1986 [1916].

Evangelos Kourdis

85

Torop, Peeter. ‘‘Intersemiosis and Intersemiotic Translation.’’ In Translation, Translation, edited by Susan Petrilli, 271-281. Amsterdam and New York: Rodopi, 2003. Vandeloise, Claude. ‘‘Cognitive Linguistics and Prototypes.’’ In Advances in visual semiotics: the semiotic web 1992-93, edited by Thomas-A. Sebeok and Donna-Jean Umiker-Sebeok, 423-442. Berlin and New York: Mouton de Gruyter, 1995. Van Leeuwen, Theo. The Language of Colour. An Introduction. Abingdon: Routledge, 2011. Volli, Ugo. Manuale di semiotica. Bari: Laterza, 2000.

THE GREEK-CYPRIOT DIALECT IN WRITING: ORTHOGRAPHIC CONVENTIONS AND TYPOGRAPHIC PRACTICES ASPASIA PAPADIMA, IOLI AYIOMAMITOU AND STELIOS KYRIACOU

This paper investigates issues related to the interplay of typography and orthography design for a non-codified dialect. Specifically, it deals with the orthographic representation of the non-standard, Greek-Cypriot dialect (henceforth GC) spoken by the Greek-Cypriots in Cyprus with a focus on the unconventional and highly controversial orthography of the distinctive phonological features of the GC dialect, all representing consonantal variation (Schneider & Wagner, 2006)1. The analysis and interpretation of the study’s findings revealed that traditionally the representation of the GC dialect in written discourse has been characterised by non-systematicity. In most cases the choice of spelling conventions has been underpinned by contradictory language ideologies regarding the different types of orthographic systems (Sebba, 2007)2. As Halliday stated, language has a semiotic value, through language we construct to a great extent our identities, our ideologies and experiences3. In the case of Cyprus, language has indeed obtained the central and almost exclusive role in indexing the national and cultural identity of Greek-Cypriots (Goutsos & Karyolemou, 2004)4. In addition, the study shows that a general confusion regarding the

1

E. W. Schneider and C. Wagner, “The Variability of Literary Dialect in Jamaican Creole - Thelwell's the 'Harder they Come',” Journal of Pidgin and Creole Languages 21, no.1 (2006): 45-96. 2 Mark Sebba, Spelling and Society : The Culture and Politics of Orthography Around the World (Cambridge, UK: Cambridge University Press, 2007). 3 Michael A. K. Halliday, Language as a Social Semiotic. The Social Interpretation of Language and Meaning. (London, UK: Edward Arnold, 1978). 4 Dionysis Goutsos and Marilena Karyolemou, “Introduction,” International Journal of the Sociology of Language 168, (2004): 1-17.

Aspasia Papadima, Ioli Ayiomamitou and Stelios Kyriacou

87

‘correct’ orthography of the dialect prevails, raising numerous debates among linguists and lexicographers.

Introduction In this article we present a part of the research that is being conducted by the Language and Graphic Communication Research Lab, which is operated by Cyprus University of Technology’s Department of Multimedia and Graphic Arts. Our research focuses on how the GreekCypriot dialect (henceforth GC) manifests itself in the written word, an issue that is both multifaceted and complicated. Specifically, we will look closely at orthographic conventions and typographic practices followed in visual representations of the GC. We should mention, by way of introduction, that there is a great variety of orthographic conventions for the representation of sounds in the dialect that linguists, writers and researchers have not yet implemented, and thus do not follow a common practice as to how the dialect should appear on paper. This study argues that, through the years, the absence of a single, complete, systematic and commonly accepted orthographic system created the need for different typographic practices, which in turn created a discontinuity in the method of representing the GC in written texts. The fact that the state encouraged the use of Standard Modern Greek (henceforth SMG) language and orthography, in many instances at the expense of dialectical sounds, reinforced this practice. Such a situation can be understood only if one researches the historical underpinning of the state that created the need for a continuous effort on the part of GreekCypriots to preserve and protect SMG in both its spoken and written forms, which confirms Halliday’s assertion that the official language of a state has semantic value and reflects and manufactures identities and ideologies.5 Furthermore, the absence of Unicode characters and a properly designed font containing a number of separate characters to render the GC created a number of problems for publishers, who used alternative, but at the same time time-consuming and amateur practices that did not conform to basic principles of microtypography. In this study we will start by describing different aspects of the sociolinguistic situation in the Greek-Cypriot community. Relevant information regarding the method of analysis used and the data sources which support our assertions will follow. In the following section we will present our main findings concerning the dialect’s written language. Specifically, we 5

See note 6 above.

88

The Greek-Cypriot Dialect in Writing

will present a system for classifying writing systems for the GC, the typographic conventions and practices used in writing it and issues related to microtypography. Finally, we will look at the orthography of the dialect as a common practice and uncover the ideologies hidden behind the choice of one orthographic system over another.

Socio-linguistic underpinning The Greek-Cypriot community is characterised by the phenomenon of social diglossia. Two varieties linguistically related, the SMG and the regional GC dialect, co-exist in a single continuum, each serving different social functions and carrying a different weight. SMG is the official language of the state and it is used for all official communication, including that of the courts, mass media, education, and generally for written texts. On the other hand, the GC is the mother tongue of Cypriots. It is used in everyday oral communication and is considered to carry less authority than SMG. The contemporary GC does not have a normalised orthography, despite the fact that we find it in written form as early as the 14th century in the legal text “The Assizes”. The written representation of the dialect is based on the orthography of SMG. However, due to the different phonetic systems used by the GC and SMG, the Greek characters cannot accurately accommodate the distinct dialectical sounds of the GC. Furthermore, the dialect has additional consonants that sometimes function as allophones and sometimes as independent phonemes that do not fit into the SMG’s phonetic system. Below are the palato-alveolar sounds that exist in the GC but not in SMG: 1. [Ȓ], voiceless, palato-alveolar, fricative. /x, s, sk/ when followed by /e, i, j/ 2. [ࣝ], voiced, palato-alveolar, fricative. /z/ when followed by /j/ . 3. [ȶ], voiceless, palato-alveolar, affricate. /k, ts/ when followed by /e,i, j/. 4. [nȳ], voiced, palato-alveolar, affricate. /g/ when followed by /e,i/.

Research purpose The purpose of this research is to explore the following questions: a. What orthographic models and typographic conventions have been used throughout the GC’s written history?

Aspasia Papadima, Ioli Ayiomamitou and Stelios Kyriacou

89

b. How do the means of writing and the receiver influence the way the dialect appears in written form? c. How do orthographic practices of Cypriots reveal their ideologies and feelings towards the two linguistic varieties?

Methodology Originally, we gathered data from published GC texts, dictionaries, literature, academic books, blogs and websites, as well as from unpublished GC texts, such as notes, text messages and email. GC words that include GC-specific sounds were gathered into a single electronic archive and classified for analysis. At the same time, we carried out an experiment to examine whether or not the written dialect is influenced by the medium it is written in and by the receiver of the message. For the purposes of exploring the questions mentioned above, five texts were dictated to subjects between the ages of 18 and 24 who happened to be students at the Cyprus University of Technology. This specific age group was chosen due to the fact that individuals on the threshold of adulthood are more spontaneous and more familiar with contemporary communication media, as well as to the fact that it would be easier to compare them with the other age groups. We asked them to compose one message for their parents and one for a friend, based on the dictations, using the same means of communication (i.e. text message/email and Post-It note). The five dictations included GC-specific sounds that pose problems when written. The dictations were: Dictation 1: ȆĮʌȐ İȞ ȑıİȚ ȖȐȜĮȞ ıIJȠ ȥȣȖİȓȠȞ ʌȒĮȚȞİ ıIJȠ ʌİȡȓʌIJİȡȠȞ IJȗĮȚ ʌțȓĮİ șțȣȠ țȠȣșțȚȐ [Dad, there isn’t any milk in the refrigerator. Go to the convenience store and pick up two bottles.] Dictation 2: ȀȩȡȘ İȞ ȞĮ 'ıİȚȢ ȫȡĮ ȞĮ ʌțȚȐıȦ IJȠ IJȐȕȜȚ ʌȠȣ ȐijȘıĮ ıʌȓIJȚ ıȠȣ İȤIJȑȢ IJȗĮȚ ijȦȞȐȗİȚ Ƞ șțİȚȩȢ ȝȠȣ [Girl, will you have time for me to pick up the backgammon board that I left at your house yesterday that my uncle is asking for.] Dictation 3: ȀȩȡȘ İȓʌİȞ ȝȠȣ Ƞ ǻȘȝȒIJȡȘȢ ȩIJȚ İȞ ȝȠȣ ȑıİȚ İȝʌȚıIJȠıȪȞȘ IJȗĮȚ șȑȜİȚ ȞĮ ȤȦȡȓıȠȣȝİ. ȆȐȝİ ʌȩȥİ ȞĮ ʌțȚȠȪȝİ ʌȠIJȩ ȞĮ ıȠȣ IJĮ ʌȦ; [Girl, Dimitris told me that he doesn’t trust me and wants to split up. Can we go for a drink tonight so I can tell you about it?] Dictation 4: ȆĮʌȐ İʌȒȡİȞ Ș ȝȐȝȝĮ ȝȠȣ IJȠȞ ııȪȜȠȞ ıIJȠȞ țIJȘȞȓĮIJȡȠȞ IJȗĮȚ İȞ ȑıİȚ ȫȡĮȞ ȞĮ ʌȐİȚ ȞĮ IJȠȞ ʌțȚȐıİȚ. ǼȞ ȞĮ ȝʌȠȡİȓȢ ȞĮ IJȠȞ ijȑȡİȚȢ İıȪ; [Dad, mom had to take the dog to the vet and doesn’t have time to go and pick him up. Will you be able to get him?]

90

The Greek-Cypriot Dialect in Writing

Dictation 5: ȀȩȡȘ țȐʌȠȚȠȢ ȑʌțȚĮıİȞ IJȘȞ IJȠȪȡIJĮ ıȠțȠȜȐIJĮȢ ʌȠȣ IJȠ ȥȣȖİȓȠ țĮȚ İȞ ȑȤȦ ȞĮ IJıİȡȐıȦ IJȠȣȢ ȟȑȞȠȣȢ. ĭȑȡİ șțȣȠ ʌȓIJıİȢ ıĮȞ ȑȡțİıĮȚ ıʌȓIJȚ. [Girl, someone took the chocolate cake that was in the fridge and I don’t have anything to offer the guests. Bring two pizzas on your way home.] Later, the data we collected was analysed with the concordance software Monoconc 2.2 to verify the frequency with which different forms of the dialect were used.

Results Synopsis of orthographic conventions used by speakers of the GC On the basis of the information that we archived at the original stage of collection, we tried to piece together a visual language of the dialect. To do this, we classified the typographic conventions of the GC by separating them into categories/orthographic systems according to the system that was followed: In the first system, Greek characters are put in bold type. This system was first implemented at the beginning of the 20th century and was based on the orthography of SMG, while using bold type for the phonemes of SMG to indicate the different pronunciation of dialectical allophones. It was found mainly in literature but is no longer in use. The second system combines Greek characters and diacritical marks. This system was based on the Historical Dictionary of the Academy of Athens, and different types of diacritical marks such as the hatchek, the brève or the apostrophe were used to indicate allophones. It sometimes appears more etymological and at others just the opposite, according to the ideology of whoever happened to have created it. We come across this system in dictionaries, scholarly manuals and in literature. The third system combines Greek letters with the letter ‘Ț’. To indicate the Cypriot pronunciation of the word ‘țȣȡȐ’ on paper, for instance, the combination of the letters -IJ, -ȗ, -Ț is used for the allophone /ț/. This system is used in official and unofficial texts, textbooks in elementary school and on the web. Finally, the fourth system includes: a. Latin characters slipped into Greek texts, a practice that was used in the past in glossaries but which has fallen into disuse, and b. “Greeklish”, which we encounter in communication via computers and in text messages.

Aspasia Papadima, Ioli Ayiomamitou and Stelios Kyriacou

91

The GC’s visual language: Microtypography Based on the dominant writing system used, which is a combination of Greek characters and diacritical marks, the GC’s written form conceals an amazing wealth of letter and word forms and contains a great variety of combinations of letters that do not exist in SMG: words with many consonants, with many double consonants, and words with an unequal proportion of consonants and vowels within the same word. If we try to illustrate this proportion using some words from the GC as examples and compare them with words in SMG with the same meaning, some interesting conclusions arise6. It is obvious that consonants are overrepresented in the GC, while the visual patterns, created in relation to the consonants-vowels in the same word, are more balanced in terms of distribution and alternation in SMG than in the GC. The total number of letters in one word is greater in the dialect than in SMG, thus creating more visual signs at the level of texts. (fig. 2-9)

Figure 2-9: Consonants-vowels representation in words with the same meaning, both in GC and SMG.

The same phenomenon also occurs in words that are equally common to both SMG and the GC. When these words are used in the GC, in many cases we encounter double consonants not only in the middle of a word but also at the beginning, as in the addition of the letter ‘Ȟ’ to the end of neuter nouns, something that is foreign to SMG. When we analyse letterforms and counterforms at the level of microtypography, we discover interesting interrelated forms and symmetries 6

GC dialectal words are spelled according to the orthographic system followed in the GC dictionary by ȀȦȞıIJĮȞIJȓȞȠȢ īȚĮȖțȠȣȜȜȒȢ, ĬȘıĮȣȡȩȢ ȀȣʌȡȚĮțȒȢ ǻȚĮȜȑțIJȠȣ (ȁİȣțȦıȓĮ: Theopress, 2009).

92

The Greek-Cypriot Dialect in Writing

that derive from the corresponding combinations, double consonants and consonantal clusters. This wealth is reflected graphically in a plethora of new forms that are created in the negative space between characters and that enrich the visual diversity of signs within the space. The new forms are sometimes dynamic and at others graceful, according to the font used, creating motifs reminiscent of dancing figures. (fig.2-10)

Figure 2-10: Motifs created by the counter-forms of characters in GC

The visual language is enriched still further by the addition of diacritical marks above or below the letters that highlight the pronunciation of the sounds. New visual diversity is thus created along the horizontal axis of the text, in the counterforms in between the ascenders and the descenders of the letters, when combined with the additional forms created by the diacritical marks that appear in the spaces in between lines. Gunther Kress, commenting on the multimodality of language, notes that: There is here a specialization of tasks between image and writing. Writing is used to tell what happened, it informs about the events; image is used to show what there is or was, it informs about content. Language serves one function, image another. Language is not the full carrier of all meaning, not even of all ‘central’ or ‘essential’ meaning.7 7

Gunther Kress, “Sociolinguistics and Social Semiotics,” in The Routledge companion to Semiotics and Linguistics, ed. Paul Cobley (London, UK: Routledge, 2001), 69.

Aspasia Papadima, Ioli Ayiomamitou and Stelios Kyriacou

93

Assigning the role of the image to that of typographical design, we can grasp what it means for the rendering of meaning and content in the written word. As Danesi observes: Alphabetic writing has become the norm in many cultures. But in every alphabetic sign, there is a pictographic history and prehistory similar to the one described above for the letter A. The pictographic content of our letters goes unnoticed because our eyes are no longer trained to extract pictorial meaning from them.8

Beyond all of this, the choice of font, the basic design features, as well as size, weight, slant and the negative space along the horizontal and vertical axes of the text, suggest, alter and finally dictate the form and means of transmitting a complete message. Bringhurst, commenting on visual communication within typography and specifically on the use of accents and diacritical marks, mentions that: Simplicity is good, but so is plurality. Typography’s principal function (not its only function) is communication, and the greatest threat to communication is not difference but sameness. Communication ceases when one being is no different from another: when there is nothing strange to wonder at and no new information to exchange.9

Looking back at Cypriot publications through the centuries, we can understand better not only the significance of the visual polyphony that derives from the dialect’s writing systems and typographic design, but also the visual evolution of the written word at the design level. From the embellished letter design and amazing diversity in the design of ligatures during the Middle Ages and Renaissance, we gradually end up at the simplicity and restraint of letter design in modern typography, which expresses itself through the simplification of character forms in completely basic shapes and reflects the evolution and simplification of the language.

Orthography as social practice Research into the history of writing unveils the way medium influences writing content as well as its characters. Writing evolved over many

8

Marcel Danesi, Messages, Signs, and Meanings : A Basic Textbook in Semiotics and Communication (Toronto, Canada: Canadian Scholars' Press, 2004), 113. 9 Robert Bringhurst, The Elements of Typographic Style (Point Roberts, WA : Hartley & Marks, Publishers, 2005), 89.

94

The Greek-Cypriot Dialect in Writing

thousands of years and within its history, the history and evolution of human culture is reflected. Every culture’s writing developed not only according to the needs for which it was created, but also according to its geopolitical position, its available resources and materials, the writing surfaces used, the type and angle of the writing implement, the skill of the writer and even the position he wrote in. Katsoulidis notes that writing surfaces dictated not only the type of writing implement, but also the form the characters would take. Hard surfaces used at the birth of writing (such as rock, metal and pottery shards) as well as the instruments used for engraving characters into these materials, gave spare, geometric letters. As these materials yielded to softer and more pliable writing surfaces (such as animal skins, wax tablets and paper) and experimentation led to the appropriate writing implements for each surface, there was the possibility to create more fluid characters with a greater freedom in their form and direction.10 Technology played a defining role in the evolution of typographic letter forms, as the formulaic and geometric nature of contemporary fonts shows, the design of which is preordained by the austere nature of digital pixel clusters11. Technology transformed not only the content and writing materials but also its very space. Bolter, commenting on the refashioning of the writing space in the late age of print, mentions that: …Each writing space is a material and visual field, whose properties are determined by a writing technology and the uses to which that technology is put by a culture of readers and writers. A writing space is generated by the interaction of material properties and cultural choices and practices.12

The new writing spaces in which the dialect is encountered in its written form offer a wide field for typographic experimentation and implementation of practices that differentiate between, or dictate, its written form according to the technology and medium used, the ideology of the writer and the receiver of his message. On the basis of the information we collected from the experiment we carried out for this research, we observed that the method of writing the

10

ȉȐțȘȢ ȀĮIJıȠȣȜȓįȘȢ, ȉȠ ıȤȑįȚȠ IJȠȣ īȡȐȝȝĮIJȠȢ (ǹșȒȞĮ: ǼțįȠIJȚțȒ ǼȜȜȐįȠȢ ǹ.Ǽ., 2000), 21-24. 11 The first printers that appeared in the 1980s could print only in pixel clusters. This restriction led to very characteristic, geometric font designs. 12 Jay David Bolter, Writing Space: Computers, Hypertext, and the Remediation of Print (Mahwah, N.J.: Lawrence Erlbaum Associates, c2001), 12.

Aspasia Papadima, Ioli Ayiomamitou and Stelios Kyriacou

95

dialect is influenced by the knowledge of orthography of SMG, by the medium and by the receiver of the message. The manner of writing differs according to who receives the text since in cases where young people are jotting a quick message to their parents, they prefer Greek letters for the most part and whole words, so that their writing can be more easily read by their elders. Likewise, as we confirmed, there was a higher frequency of English punctuation marks, like the question mark, than Greek. When young people leave a quick note for their friends or neighbours, they use short hand and Latin characters more often.

Figure 2-11: How young people write in GC

On the other hand, as far as medium is concerned, when they send a text message to their parents with their mobile phones, while they try to write whole words in order to be more easily understood, they nearly always use Greeklish. At the same time, they sometimes forget and introduce characters they use in their everyday speech with friends on the Internet into their writing, such as, for example, when they substitute the number “4” for the Greek letter “ȥ”. Written communication between young people mediated by computer or mobile phone typically features Latin characters and short hand. (fig. 2-11) Generally, in written communication, when young people use the dialect when making notes on paper, they avoid using the more readily identifiable dialectical forms. This probably owes to the fact that as long as they are using Greek characters, they want to preserve the mental image they have of proper orthography which comes from their knowledge of

The Greek-Cypriot Dialect in Writing

96

SMG. On the other hand, when the dialect is used on computers and mobile phones, dialectical forms are used because in this case they are using Latin characters with which the speakers believe they can more easily render the sounds of the dialect. Indifference to orthography and the principle of least effort were also noted. For example, in monosyllabic words they use only the first consonant (mou=m, sou=s), and at the same time they avoid changing languages on both mobile phones and on computers.

Hidden social messages in orthographic conventions of the GC The third goal of our research was to explore the hidden social messages concealed by Cypriot orthographic practices and their meanings. For this goal we adopted the theoretical framework introduced by Sebba, who dealt extensively with the orthographic conventions followed in different countries, their practice and what these practices indicate13. Sebba considers orthography to be a social practice. Briefly, he considers orthography itself as a social practice and notes that it is not a neutral process but a symbolic action that carries social messages. Orthography is intertwined with the culture and letter forms are shaped within social practices in order to transmit messages/meanings. Members of a society choose a specific way of writing out of a variety of likely choices. Especially important in such a choice are the parameters that involve: a. how different the writing is from the official variety and b. the degree of recognisability, which, incidentally, must be close enough to the norm/official language as to be recognisable.14 While analysing the data, we found that in the case of the GC, the variety in orthographic conventions reflects the ideological position of the user, which connects language with national identity. We classified the interviewees into three categories: In the first category, the supporters of etymologically correct writing want an orthographic system based on the orthography of SMG where dialectical sounds appear in bold or with symbols above the phonemes of the SMG. In this case, orthography reveals national identity since it establishes the connection between Greece and Cyprus. Sometimes, however, in their attempt for a more historically accurate etymology they are driven to extreme lengths, such as, for example, the word 13 14

See note 5 above. Ibid.

Aspasia Papadima, Ioli Ayiomamitou and Stelios Kyriacou

97

ȤǿȤǿȪȜȜȠȢ=ıțȪȜȠȢ [dog], (written with a double ‘Ȥ’ and diacritical marks) which is not used in SMG. In the second category, the proponents of an intermediate solution, always having the orthography of SMG as a model, designate dialectical sounds with diacritical marks and/or combinations of Greek characters such as, for example, the word ıǿɹɏɇɋ=ȤȑȡȚȞ [hand]. They show the ‘distinctness’ of the GC from SMG sometimes more and at other times less clearly. In the third category, the proponents of non-etymological writing use Latin characters slipped into Greek/Greeklish and generally Latin characters. They believe that they are representing the oral flavour of the dialect more faithfully this way. They stress the Cypriot identity more and keep a distance from SMG. To answer the question “Why is Greek orthography used?” we must keep in mind that SMG plays an important role in Greek-Cypriot society and constitutes a state linguistic policy. This is relevant to the history of Cyprus and to the intense politicisation of Greek-Cypriots, while the language is irrevocably connected to national identity and constitutes proof of the Greekness of the island.

Conclusions The absence of an official orthographic system led to the creation of multiple systems of writing that reflect the ideologies and feelings of the speakers of the GC. The absence of a properly designed digital font creates typographic problems when writing the dialect via electronic media. There is a clear need for a complete and systematised writing system for the dialect that will be based not only on linguistic criteria but also on the needs of speakers, its use and its recognisability (Sebba, 2007)15. Many studies have been conducted on the GC but none has focused, until this moment, on typographical issues. This research is the beginning of further research on the dialect from another visual angle.

15

See note 5 above.

98

The Greek-Cypriot Dialect in Writing

Bibliography Bringhurst, Robert. The Elements of Typographic Point Roberts, WA: Hartley & Marks, Publishers, 2005. Bolter, J. D. Writing Space: Computers, Hypertext, and the Remediation of Print Mahwah, N.J.: Lawrence Erlbaum Associates, 2001. īȚĮȖțȠȣȜȜȒȢ, ȀȦȞıIJĮȞIJȓȞȠȢ ī. . ĬȘıĮȣȡȩȢ țȣʌȡȚĮțȒȢ įȚĮȜȑțIJȠȣ: İȡȝȘȞİȣIJȚțȩ, İIJȣȝȠȜȠȖȚțȩ, ijȡĮıİȠȜȠȖȚțȩ țĮȚ ȠȞȠȝĮIJȠȜȠȖȚțȩ ȜİȟȚțȩ IJȘȢ ȝİıĮȚȦȞȚțȒȢ țĮȚ ȞİȩIJİȡȘȢ țȣʌȡȚĮțȒȢ ȁİȣțȦıȓĮ: Theopress, c2009; 3Ș ȑțį, 2009. Danesi, Marcel. Messages, Signs, and Meanings: A Basic Textbook in Semiotics and Communication. Toronto: Canadian Scholars' Press, 2004. Goutsos, Dionysis, and Karyolemou, Marilena.. “Introduction.” International Journal of the Sociology of Language 168, (2004): 1-17. Halliday, Michael A. K., Language as Social Semiotic: The Social Interpretation of Language and Meaning. London: Arnold, Edward, 1978. ȀĮIJıȠȣȜȓįȘȢ, ȉȐțȘȢ. ȉȠ ıȤȑįȚȠ IJȠȣ īȡȐȝȝĮIJȠȢ. ǹșȒȞĮ: ǼțįȠIJȚțȒ ǼȜȜȐįȠȢ ǹ.Ǽ., 2000. Kress, Gunther. “Sociolinguistics and Social Semiotics.” In The Routledge companion to Semiotics and Linguistics, edited by Paul Cobley, 66-82. London: Routledge, 2001. Newton, Brian. Cypriot Greek: Its Phonology and Inflections. Paris; The Hague: Mouton, 1972. Schneider, E. W. and Wagner, C. “The Variability of Literary Dialect in Jamaican Creole - Thelwell's the 'Harder they Come'.” Journal of Pidgin and Creole Languages 21, no.1 (2006): 45-96. Sebba, Mark. Spelling and Society: The Culture and Politics of Orthography Around the World. Cambridge, UK: Cambridge University Press, 2007.

CHAPTER THREE: DESIGN FOR SCREEN BASED MEDIA

SIGNS AT THE INTERFACE: AN EXPLORATION OF SEMIOTICS AND INTERACTION DESIGN NIKOS BUBARIS

Semiotic theories have contributed constructively to the study of humancomputer interaction, both with respect to the semantic structures of functional, aesthetic and interactive features of user interfaces and to the interpretative practices of programmers, designers and users. Semiotic theorists have concentrated mostly on symbolic representation as a common element of user interface features and on the use of language as a common tool of the people involved in the signification of user interfaces. Some theorists consider semiotics to be a unifying theoretical method (Nadin 1988) that encompasses not only the ‘soft’ but also the technical aspects of computer systems (Andersen 1997), and thus they are concerned with the relation between designers and users mainly in the context of studying the construction of user interfaces (de Souza 2005). However, other semioticians argue for a substantial difference between human and computers in the way they process data (Nake and Grabowski 2001) and for the need to transcend the structural epistemological foundations of semiotics in order to address novel features of digital media, such as interactivity (O’ Neil 2008). In a sense, the variety of semiotic approaches to the study of humancomputer interaction reflects the transformation of the role of computers: from machines used predominantly for calculation to popular digital media. The introduction of graphical user interface and the widespread use of multimedia applications in various social, cultural and economic contexts, among other developments, have rendered the layers of the computational system and the associated practices highly specialised and discrete. Indicatively, a commonly used client-server model of software architecture defines three separate tiers: the user interface or presentation tier, the application or logic tier, and the database or data tier (Eckerson 1995). This specialisation means that, for example, a visual designer may have no knowledge of programming, since the logic of the visual language on the screen differs substantially from the formal language and the

Nikos Bubaris

101

control/command speech acts of the software that underlies the visualisations. However, recent research in new media and software studies stresses the need to reconnect the analytically distinct layers of a computational system, both with one another and with their cultural context, since they are all effectively interrelated (Fuller 2008; Manovich 2011). To contribute to this direction, semiotics needs to move beyond the application of a universal theoretical framework and to encompass diverse conceptual tools that have been developed through the interaction of semiotics with other epistemological traditions. The structures of signification of multimedia screens are clearly distinctive from those of other layers of a computational system. For example, the semantics and the pragmatics of the user interface are potentially ambiguous, polysemic and context-bound in contrast to the decontextualised and formal logic of programming languages. At the same time, the user experience in interacting with multimedia signs on the screen is considerably shaped by the affordances of the computational system. Hence, the ways that the cultural logic of computation (i.e. to abstract, to formalise, to calculate, etc.) permeates the user interface is of particular importance in studying the signs involved in multimedia displays. Taking all the above into consideration, I propose in this paper a synthetic model of the following four communicative functions of userinterface signs: a) modes of remediation b) action-oriented representations c) nodes in information maps d) computational effects I will illustrate these functions by reference to a multimedia application that a team of students produced as an assignment for an undergraduate course I teach on Cultural Representation. The application is about what each of the four students identify as the most significant landmarks in their experience when they go about their everyday activities in Ermou Street, the main commercial street in the city of Mytilene, the capital of the island of Lesvos in Greece.1 Henceforth, I refer to it as the Ermou Street application.

1

For an online version of this application: http://www.cubimension.net/flash/ermou.html (retrieved January 24, 2012).

102

Signs at the Interface

Fig. 3-1. The opening screen of the Ermou Street multimedia application

Fig. 3-2. Activating interactive objects

The opening screen of the application introduces its general topic "Portraits of People in Ermou street" (Fig. 3-1, top left) and information about the students, such as their names, years of residency on the island and one-word description of their personality (Fig. 3-1, bottom right). An animated visual image of the students moving along a dashed line, which represents the street, is constantly at the centre of the screen (Fig.3-1 & 32). By clicking on a student-sign, three images appear that correspond to the most important landmarks of the student's personal experience (Fig. 32). Hovering the mouse over an image causes a short text to appear conveying the student's thoughts about the importance of the corresponding landmark. For example, as one student said: For a reason I can’t explain, the first thing that comes to mind when I think of Ermou street is the carpenter’s workshop across the site of the old

Nikos Bubaris

103

mosque. I love the smell of wood and woodchips. The traditional character both of the place and of the carpenter's craft is typical of the 'Old Market'.

The other two markers of the Ermou Street experience for this particular student are his favourite restaurant (Fig.3-2, top right) and the occasional smell of the sewage system (Fig.3-2, bottom right). The form and the structure of the information presented follow the same pattern for the other three students.

User Interface Signs as Modes of Remediation The visual signs that comprise the on-screen interface of this application are to a large extent remediated. In new media studies, the term “remediation”, introduced by Bolter and Grusin (2000), describes the double logic of immediacy and hypermediacy. Immediacy refers to the sense of transparency of the medium in conveying meanings, which creates an illusion of unmediated reality. Ǿypermediacy refers to the opaque presence of media in constructing meanings and realities. Immediacy conceals the special properties of each newly developed medium by re-introducing features from older media that people are already accustomed to. In the case of multimedia, older forms of visual culture (i.e. paintings, photographs, moving images, texts, etc.) are assimilated and sometimes enhanced in the design of visual interface displays that guide the interpretation of the digital content. Several principles of the visual grammar of older media can be found in the screen design of the Ermou Street application. First, the Gestaltderived background/foreground relation: for example, the permanence of the background throughout the use of the application creates a sense of remaining within the same information space and thus enables the user to focus his attention on the significant foreground information. Second, the spatial organisation of images as codification of their differentiated informational value: for example, the semantic relation of the students to their personal landmarks is represented through the spatial structure of centre/periphery, and their ‘like’ or ‘dislike’ for these landmarks through the spatial structure of high/low, which corresponds to the ideal/real distinction in Kress and van Leuween's (1996) terms. Third, the semantic relation of multimodal visualisation to the narrative structure: the use of generic pictograms representing the students, the use of pictures and drawings as symbols and metonyms of their landmarks and the use of texts for presenting the students' intimate feelings and thoughts form a threefold navigation path, moving from the nondescript to the increasingly personal, that invites the user to discover the personalised landmarks of the four

104

Signs at the Interface

students and their reasons for selecting them. Finally, the function of address: the placement of the student pictograms in relation to the stylised building façades of the Ermou street, combined with the informal tone of the text, casts the user in the role of an interlocutor or an interviewer who collects information. These remediated principles as applied to on-screen signs provide an initial framework for analysing the semiotics of the interface design.

User Interface Signs as Action-Oriented Representations Multimedia signs, however, are not mere reproductions of older media images in new media environments. What is more, the user of multimedia is not simply a version of the users of the ‘older’ media in a new ‘viewerlistener-reader’ combination. ȉhere are new techno-cultural features and properties at work. Along with digitisation, interaction is commonly considered a qualitatively new feature. The images that constitute the graphical user interface are designed to impel the user to do things with them and not only to ‘read’ them. In his book “A Theory of Computer Semiotics”, Andersen (1997) takes a first step in theorising interaction as a meaning-making process by considering the properties of the actionoriented representations on the computer screen. Andersen identifies three classes of properties: handling features, referring to the input of user actions through devices such as mouse, keyboard and touch screens; permanent features, referring to the constant properties of a sign; and transient features, referring to the internal changes of the state of the sign caused by its use. Among the various types of screen signs, interactive signs are, according to Andersen, unique to the computer medium. Interactive signs have permanent and transient features. The latter can be activated through their handling features, which make them responsive to the actions of the user. Further, interactive signs can act upon other signs, for example by bringing new signs on the screen or by changing the transient features of signs already present. In the Ermou Street application, the main interactive signs are the pictograms of the students. The student-sign has permanent features (such as its size and its movement along the horizontal line) and transient features activated by the user (such as motion and stasis, opaque and transparent colour). On activation, a student-sign stops moving, reveals the signs of the corresponding personal street landmarks and affects the transient features of the other student-signs (by turning their colour transparent). All these events are designed to suggest particular meanings. Simulating the real-world experience, the immobilisation of the student-

Nikos Bubaris

105

signs signifies an encounter of the student with the user in the virtual street of the multimedia application and the availability of the student to show and describe his or her personal landmarks. This relation becomes the centre of the user attention, while the other students recede into the background, forming a subordinate relation that is signified by the faded, transparent colour of the corresponding student-signs. Thus, permanent, transient and handling features are all combined in the formation of an onscreen sign as a communicative unit. However, the correlation between transient and handling features is the most crucial in interaction design. At this point it should be noted that even when humans and computers operate in concert, they remain two distinct performative systems, since humans communicate at the level of sign and computers operate at the level of signal. Computational systems operate with the highest possible abstraction, relying on binary code and processing the difference between sets of digital signals regardless of their reference. The founding father of the mathematical approach to communication, Claude Shannon, notes (1948) that signals are devoid of meaning and unrelated to content variation. If screen signs were identical to digital signals in communicative operational terms, the experience of interaction would be extremely dull and confusing. Conversely, if digital signals bore an analogy to screen signs, the communicative medium would be radically different from a computer. This communicative gap between humans and computers is bridged by software that provides, according to Nake and Grabowski (2001), a sense of pseudo-communication by blending the formal structuralism of the computational system with the designer's and the programmer's objectives at any given instance. Software, through its algorithms, makes the content of the presented cultural data abstract, so that they can be processed in a way that is compatible with both the deterministic logic of the computer and the potential polysemic interpretation of the user. Consequently, a semiotic approach to software as an interface between the user and the computer, is faced with two additional issues that relate to the communicative functions of userinterface signs. The first issue has to do with modes of relating screen signs to the software and the communicative function of on-screen signs as nodes in information maps. The second concerns the computational effects of on-screen signs, namely modes of relating the software-driven signs to computer signals.

106

Signs at the Interface

User Interface Signs as Nodes in Information Maps In interface design, information architecture outlines the semantic and syntactical setting of human-computer interaction. The cultural data selected for multimedia presentation are first categorised analytically and then associated within non-linear structures that can be translated into software relations. The act of analytically selecting, ordering and connecting the previously undifferentiated cultural data into information maps is a profound cultural practice of structuring meaning. We tend to assume that the structural logic of the resulting information maps is essentially identical to the algorithmic logic of the software and the binary logic of computation. This is partly because among the various forms of mapping cultural data, the tree structure is most commonly used. According to Martinec & van Leeuwen, the tree structure is a particular hierarchical model of knowledge representation based on “the semantic principle of classification from general to specific” (Martinec and Leeuwen 2008, 6). Besides the tree structure with its componential logic of a whole-part hierarchy, the authors present other modes of structuring cultural data. Most notable for our purposes here is the “nucleus-satellites” relation between a central information element and attributes corresponding to different modes of its ‘being’ and ‘having’. In the Ermou Street application, the principal model of information architecture is that of the nucleus-satellites. The personal landmarks of the students are not conceived as components but as modes of experiencing the city. At the same time, the internal classification of the personal landmarks into likes and dislikes follows the binary logic of a tree structure. This blend of nucleus-satellites model and tree structure is represented in the screen layout and translated into acts of interaction and into navigation paths. In the screen layout, the student-signs are placed at the centre and are surrounded by the personal landmark signs arranged at the top and bottom of the screen. Two of the three analytical categories represented in the nodes of the information map (i.e. students, personal landmarks and students’ quotes) are translated into interactive objects of different importance: the superordinate student-signs are activated on click, a haptic experience of control through grabbing; the subordinate images of personal landmarks are activated when the mouse hovers over them, an act that simulates a rather loose and unstable contact. Finally, the design of navigation paths for the user translates the information map into traceable connections among conceptual nodes. For example, the user needs to first interact with the student-signs in order to follow the path from personal landmarks to their corresponding text. The syntagmatic

Nikos Bubaris

107

structure of this navigation path is shaped by connotations of anthropocentrism and individualism. Students are chosen as the principal information unit; alternatively, it could be the landmarks, or even the students’ quotes. Further, information is presented in the form of monologues; there are no links between the personal landmarks or quotes of each student and the other students, their personal landmarks or their quotes.

Computational Effects in User Interface Signs In the Ermou Street application, there is one particular transient feature of screen signs that enhances the experience of interaction beyond the logic of ‘click-and-get-info’. This feature changes how much and how fast the student-signs move. The occurrence of these changes is not derived from either the information map or the user actions. Rather, they are the manifest effects of changes in the internal state of the sign, based on calculations using a random function. The algorithms executed to produce these changes are, in turn, enabled by the complex calculative properties of the computational system. In semiotic terms, these properties operate in the plane of myth. They are mythic because they over-code the structures of representation on the user interface layer. In other words, binary code seems to function beyond human action and perception. The pragmatics of this experience partially undermine the established principle of user control that is central to the usability model of interface design. However, this design engages users’ fascination and effect as they interact with the environment of the application. This experience of a dynamic and partly random mode of interaction with on-screen signs, which is facilitated by computational processes, provides a hyper-mediated sense of being-in and being-with the multimedia application. The user knows what clicking on the student-sign signifies, but the act of chasing the mobile sign or waiting to click on it at the right moment adds a playful act of synchronisation with the random computational processes at work. The playfulness of the user experience is driven by blending perceptual experience (i.e. following the movement of the sign) with proto-semiotic experience (i.e. inventing ways to click). Drawing on Peircean semiotics, we could argue that this mode of humancomputer interaction, which is based on the randomness of the system and the corresponding playfulness of the user, intensifies the dialectic between ‘firstness’ and ‘secondness’, that is, between the experience of nondifferentiation of the user from the computer object and the presence of the

108

Signs at the Interface

computational system as ‘other’, which forces the user to become aware of his or her limitations and capabilities.

Conclusion In this paper, I have illustrated how semiotics can contribute to the study of user interaction with multimedia as a meaning-making process by outlining four communicative functions of on-screen signs. First, onscreen multimedia signs remediate established codes and other conceptual tools drawn from the visual grammar of older communication media. They thus provide recognisable, commonly accepted cognitive references for designing interfaces that convey layers of meanings to the user. Second, on-screen computer signs are not designed only to be ‘read’ but primarily to be ‘acted upon’. Interactive signs have permanent, transient and handling features, whose synergy embodies and shapes the interpretative practices of the user. Furthermore, as humans and computers operate differently in terms of semantics and syntactics, there are two more ways of establishing communication through semiotic practices. The first involves the use of interactive signs as nodes in the non-linear structures of organising knowledge in multimedia. The second is related to the capacity of computers as calculation machines to produce signs endowed with a perceived quality of idiosyncratic behaviour keeping the user at the threshold of semiosis. This last function in particular carries great potential for enriching a semiotic approach to the study of human computer interaction and for gaining a deeper insight into the emerging communicative tropes of multimedia interfaces.

Bibliography Andersen, Peter B. A Theory of Computer Semiotics: Semiotic Approaches to Construction and Assessment of Computer Systems. Cambridge: Cambridge University Press, 1997. Bolter, Jay D., and Richard Grusin. Remediation: Understanding New Media. Cambridge, MA : The MIT Press, 2000. De Souza, Clarisse S. The Semiotic Engineering of Human-Computer Interaction. Cambridge, MA : The MIT Press, 2005. Eckerson, Wayne W. "Three Tier Client/Server Architecture: Achieving Scalability, Performance, and Efficiency in Client Server Applications." Open Information Systems 10, no.1 (1995): 3(20). Fuller, Matthew, ed. Software Studies. A Lexicon. Cambridge MA: The MIT Press, 2008.

Nikos Bubaris

109

Kress, Gunther and Theo van Leeuwen. Reading Images: The Grammar of Visual Design. London: Routledge, 1996. Manovich, Lev. Cultural Software. http://manovich.net/DOCS/Manoich.Cultural_Software.2011.pdf (accessed March, 25, 2013). Martinec, Radan, and Theo van Leeuwen. The Language of New Media Design: Theory and Practice. London: Routledge, 2008. Nadin, Mihai. “Interface Design: A Semiotic Paradigm”. Semiotica 69 (1988): 269-302. Nake, Frieder, and Susanne Grabowski.. “Human-Computer Interaction Viewed asPseudo-Communication”. Knowledge-Based Systems 14, no.8 (2001): 44 – 447. O’ Neil, Shaleph. Interactive Media: The Semiotics of Embodied Interaction. London: Springer, 2008. Shannon, Claude A. “A Mathematical Theory of Communication”. The Bell System Technical Journal 27 (1948): 379–423, 623–656.

LUDIC AND SOCIAL MEDIA INTERACTION DESIGN PRINCIPLES IN SMART CITY DEVELOPMENT PATRICK J. COPPOCK

Introduction This paper focuses on how “ludic and social media interaction design principles” (Murray, 2012) may be useful for mapping, planning, designing and realising people-friendly urban spaces in Smart(er) Cities. This latter concept has been in focus for some time, as evidenced in 2011 by inserts and articles1 in the “International Herald Tribune” highlighting a cluster of cities: Auckland, Berlin, Barcelona, Cape Town, Copenhagen, Curitiba, Montreal, Santiago, Shanghai and Vilnius all envisioning future development policies in Smart City terms. That same year, the prominent Italian economic broadsheet “Il Sole 24” dedicated a special number of its weekend innovation supplement “Nòva”, to ideas and agendas for Smarter Cities.

Smarter City Players in Europe and Beyond Amongst the principal business and political actors in Europe are American global technology giant “International Business Machines” (IBM), and the European Union. The field is in rapid evolution: in 2011, MIT established a “City Science Research Centre”2, and in 2011 and 2012, Amsterdam, London, Budapest, Milan and other European cities hosted Smart Cities conferences. The following sections sum up some relevant issues raised at these initiatives. This is not intended as a comprehensive representation of all the relevant issues, however, website links proffer further information. 1 2

http://nyti.ms/oPpJHY http://cities.media.mit.edu

Patrick J. Coppock

111

IBM IBM’s Smarter Planet Initiative3 involves about 2000 cities worldwide that signed agreements with IBM to develop short- and long-term Smart City solutions. Amongst these are the cities mentioned above and others like Seoul, Oulo, and the Province of Reggio Emilia, seat of the University of Modena and Reggio Emilia Department of Communication and Economics, and an experimental E-Learning Center. A pilot agreement in 2009 between IBM and Reggio Emilia, focused on education, training and employment, through the creation of digital networks to facilitate datasharing among interested parties, as outlined in the following excerpt from a press release announcing the agreement: Initial projects within the strategic initiative in Reggio Emilia are expected to include a pilot called “Classroom 2.0” in several schools around the city. Students, teachers, parents and local companies will have a collaboration platform to interact and exchange information about classroom management activities, students' interests and results, workgroup activities at specific schools, and job openings.4

The Classroom 2.0 collaboration is not yet implemented, so further reference to it here is not possible. In what follows, we interrogate the notion of Smarter Cities, offering considerations of how ludic and social media interaction design principles, allied with site-sensitive ludic initiatives based on these, may serve as strategic tools for involving citizens in rethinking, reconceptualising, mapping, discovering and evaluating – together with planners, designers and developers – hidden, often unutilised, potentials that their own urban environments possess for becoming ‘smarter’. In particular, we are interested in ways to further the emergence of innovation processes rooted in nascent, or already existent, social networks and their value systems.

3

http://www.ibm.com/smarterplanet/; http://www.ibm.com/smarterplanet/global/files/us__en_us__smarter_cities__wsj_o p_ad_final.pdf ; http://www.ibm.com/smarterplanet/us/en/overview/industries/index.html ; http://www-03.ibm.com/innovation/us/thesmartercity/ ; http://www-01.ibm.com/software/solutions/soa/innov8/cityone/index.html 4 http://www-03.ibm.com/press/us/en/pressrelease/27281.wss

112

Ludic and Social Media Interaction Design Principles

The European Union The European Union launched its “Smart Cities and Communities Initiative” (SCCI)5 at an international Brussels conference on 21 June 20116. Three thematic domains, supported by funding for innovation, research and development from the EU’s “Seventh Framework Program” (FP7) and “Europe/Horizon 2020”7, focus on Buildings, Energy Networks and Transport8: Buildings: New buildings with net zero energy requirements or net zero carbon emissions when averaged over the year by 2015, thus anticipating the requirements of the recast Directive on the energy performance of buildings (EPBD). This requirement could be anticipated (e.g. 2012) for all new buildings of the local public authority (city). Refurbish existing buildings to bring them to the lowest possible energy consumption levels (e.g. passive house standard or level of efficiency that is justified by age, technology, architectural constrains) maintaining or increase performances and comfort. This would include innovative insulation material (solid insulation, vacuum insulation, vacuum windows, cool roofs, etc.). Energy networks Heating and Cooling Innovative and cost effective biomass, solar thermal and geothermal applications. Innovative hybrid heating and cooling systems from biomass, solar thermal, ambient thermal and geothermal with advanced distributed heat storage technologies. Highly efficient co- or tri-generation and district heating and cooling systems. Electricity Smart grids, allowing renewable generation, electric vehicles charging, storage, demand response and grid balancing. Smart metering and energy management systems. Smart appliances (ICT, domestic appliances), lighting (in particular solid state lighting for street and indoor), equipment (e.g. motor systems, water systems). 5 http://ec.europa.eu/energy/technology/initiatives/20110621_smart_cities_ conference_en.htm 6 http://scic.ec.europa.eu/str/indexh264.php?sessionno=dda04f9d634145a9c68d5 dfe53b21272 7 http://ec.europa.eu/europe2020/reaching-the-goals/flagship-initiatives/index _en.htm 8 Cited from http://ec.europa.eu/energy/technology/initiatives/smart_cities_en.htm

Patrick J. Coppock

113

Foster local RES electricity production (especially PV and wind applications). Transport 10–20 testing and deployment programmes for low carbon public transport and individual transport systems, including smart applications for ticketing, intelligent traffic management and congestion avoidance, demand management, travel information and communication, freight distribution, walking and cycling. Sustainable mobility: advanced smart public transport, intelligent traffic management and congestion avoidance, demand management, information and communication, freight distribution, walking and cycling.

Semiotically speaking, the term Smarter Cities evokes an optimistic, positive vision of better, healthier, more meaningful urban lifestyles for all in the future. It is easy to envision that, if all public service instances function optimally; if pollution and energy waste in buildings and industry reduced to zero; if public and private transport perfectly coordinated, with low pollution levels; and if cars, transporters, buses and trains guaranteed to function smoothly and arrive on time – how fantastic this would be: almost like magic! Indeed, the following three images (Fig. 3-3), symbolically and rhetorically, denote a few of a veritable multitude of models, visions, perspectives and understandings of Smarter Cities, all portrayed, implicitly or explicitly, as offering carefree, happier, healthier futures for all – in relation to the very diverse living conditions people face today in their everyday lives all over the world.

Fig. 3-3 Smart City promotion images from three incidentally selected web sites9

However, images are images, and everyday life is something else. We must ask ourselves in this context how will we all – in spite of our diversities and divergences of priorities, interests, opinions, cultures, traditions, languages, abilities, life environments – experience living, 9

http://vator.tv/news/2010-12-24-cityville-surpasses-farmville-in-popularity ; http://www.spheru.ca/research_projects/projects/working-upstream-schk.php; http://aecotic.org/?p=2938

114

Ludic and Social Media Interaction Design Principles

studying, working, relaxing and playing in such Smarter Cities? We are all well aware that greater or lesser disparities always exist between futuristic, technology-driven visions like those represented above, and their realisation, development and day-to-day functioning. So, it is reasonable we begin asking ourselves, for example, questions like: ƒ will Smarter Cities really be pleasing, aesthetic, comfortable places to live in? ƒ will they function equally well for friends, families, neighbours etc. irrespective of respective income levels, ethnicities, social and occupational profiles etc.? ƒ will they offer easy access to necessary information and intellectually, aesthetically, emotionally stimulating urban spaces? ƒ will they function equally well for immigrants and casual visitors as for us who already live there? ƒ how can we balance and optimise ‘top-down’ and ‘bottom-up’ decision-making in planning and implementation? ƒ how can we map, document and utilise existent physical, human and cultural resources that will contribute to successful Smart City development? No quick and easy answers to the above questions exist. Indeed, a lot of effort is already being made to involve citizens, political and cultural organisations, businesses and their social networks in large- and smallscale conceptualisation, planning, design and innovation processes. This is a fundamental presupposition for developing liveable – in Smarter Cities – which could also be envisioned in combination with “Smarter Town, Countryside and Regional” projects. One ongoing global effort to involve citizens in innovation and planning processes is the “United Cities and Local Government (UCLG) World Council”, hosted from December 9-11, 2011 by the City of Florence, here in Italy. There, 500 local and regional representatives from over 40 countries engaged in discussion of current and future governance and development models, with special attention to strategies for involving citizens, community associations and businesses through social networking and national, regional and local public meetings, workshops and forums. Indeed, Bologna is currently hosting the initiative “Laboratorio Urbano”10, inviting citizens to participate in open workshops, seminars and discussions with politicians, architects, planners, IT experts and researchers involved in urban innovation projects. Future visions for the city of “Urban Democracy”; 10

http://www.laboratoriourbano.info/

Patrick J. Coppock

115

“Gender and Difference”; “City and Territory”; “Welfare Institutions”; “Work”; “Digital City”; “Culture”; “University and Innovation”; “Mobility”; “Green Economy” and “Local Development” are now being drafted. In addition, there are some more informal settings where ludic interaction and participation design principals are being tried out in different environments. These reach and involve people and groups who may not appreciate, have time to participate in, or feel attracted by, ‘formal’ opportunities for participation in innovation and design processes like those mentioned above. Before we look at alternative interaction and participation design models, a few words on the concept of “Transcultural, Transmedia Gameplay Space” (TTGS), a metaphor useful for understanding digital cultural spaces and places as environments for explorative ludic behaviours, and as globally linked ‘gateways’ to new ways of seeing, thinking about and understanding less well-known aspects of urban space and activities taking place there now, or which may do so in the future.

Transcultural, Transmedia Gameplay Space (TTGS) Digital global networks and social and ludic media instruments offer a powerful potential for creating global-local (glocal) places and spaces where at-a-distance interaction facilitates easy sharing of ideas, experiences, evaluations, values and own- or other-created digital artefacts with people in cultures quite different from our own, creating a hybrid cultural zone I often refer to as “transcultural, transmedia gameplay space” (Coppock, 2010). Digitally mediated cultural spaces derive from, and facilitate, innovation based on “convergence processes” (Jenkins, 2006, 2008), by recycling digital key content matter in “social media ecosystems” (McLuhan, 1964). Convergence processes tend to merge ‘old’ and ‘new’ media technologies: their content, formats, codes, languages and value systems. This offers potential for ‘win-win’ solutions through participation in other ways of seeing and being, through openness, transparency, creativity and collaborative action by individuals and communities elsewhere, coupled with ecologically sound11 emergent practices for managing, sharing and recycling natural, human, cultural, technological and economic resources.

11

The term “ecological” includes both natural resources, human, cultural, technological resources and social systems supporting activities, institutions and lifestyles we rely on for survival and life quality.

116

Ludic and Social Media Interaction Design Principles

Cultural places and spaces – material or immaterial, analogue or digital – can be conceived of as potential sites for shared forms of ludic action. Philosophical (Caillois, 2001; Wittgenstein, 1953/2001), anthropological (Huizinga, 1971/1938; Sutton Smith, 2001), historical (Frost, 2009) and other studies assert the central historical role of play and playfulness (Dekoven, 2002; De Jong, 2009, 2010) in fostering logical reasoning, problem-solving, experimentation, innovation and, more specifically, key cultural institutions like the arts, science, technology, social and interpersonal infrastructures, networks and relationships. Our quotidian experience of TTGS literally ‘extends’ us as actors far beyond, and transcends, our conceptions of historical ‘local’ conventions, habits and limitations associated with: ƒ physical and digital space and time ƒ our societies and cultures ƒ our language styles and local variants ƒ established and emergent media forms and systems ƒ environments we live, work and play in In its heyday, around 2000, the science fiction notion of ‘Cyberspace’ carried with it strong cultural connotations of something powerful, mysterious, vague, evocative and ephemeral: an ‘alien’ technologically mediated space inhabited by artificial intelligence forms or ‘Cyborgs, far distant from our everyday socio-cultural realities. This notion is now seen as rather outdated, especially by younger generations: the ‘digital natives’. Our physical and digital realities, and personal and collective identities associated with these12, are becoming increasingly interdependent. Today’s digital natives never knew any other media context, so for them, TTGS is merely an omnipresent, quotidian, creative resource to be taken for granted and made the most of. Henry Jenkins, who coined the terms convergence culture and participatory culture, sees global/local media convergences as changing the ways we think about ourselves, our cultures, the world we live in, and how we interact with one another and with our environment. A Macarthur Foundation Report on “media education for the 21st century” (Jenkins & al. 2009), characterises participatory culture as offering: ƒ low barriers for artistic expression and civic engagement; ƒ strong support for creating and sharing creations with others; ƒ informal mentorship whereby what is known by the more experienced is passed along to novices; 12

http://redsocial.uimp20.es/events/nuevas-identidades-culturales

Patrick J. Coppock

117

ƒ members who believe their creations matter; ƒ members who feel some degree of social connection with one another (at least, they care what other people think about what they have created). (pp. 5-6). This “shifts the focus of literacy from individual expression to community involvement” (p.6). “We are”, the authors note, “moving away from a world in which some produce and many consume media toward one in which everyone has a more active stake in the culture that is produced.” (p.12). Here, it is easy to envision that with the advent of the “Internet of Things” paradigm our possibilities of conceptualising, creating and distributing both intangible media artefacts and tangible cultural artefacts, will increase over time, as intersections between experienced digital space and lived physical space become increasingly fluid and porous. Collaboration with remote others, whom we may never meet face to face together, following innovation pathways from initial concept development, to production and distribution of both tangible and intangible cultural artefacts in places and spaces once considered unthinkably distant from us, will be possible. Aesthetic conceptualisation, visualisation and interaction design techniques and tools will craft ‘ludic interfaces’ that allow us to access, chart, visualise, conceptualise and utilise innovation potential in a hybrid digital space that is gradually permeating and transforming our familiar lived spaces and places. We can envision interacting over vast distances (also cultural) via ludic interfaces with agile and complex digital information sources, physical tools, devices and machines we can manage creatively, as we know exactly what and how they can do things for us, even though we might never have seen or touched them.

Feeding Innovation with Ludic Interaction Design Principles To exemplify how ludic interaction design principles may inform and stimulate new ways of thinking in urban and global innovation processes, we shall briefly present some “Augmented and/or Alternative Reality Games” (A/ARGs) that embody ludic interaction design principles, challenging and facilitating players in moving back and forth across the borders of physical and digital cultural spaces they may or may not know in advance. This design principle can evoke a liberating sense of ambiguity as players cross borders between, and act in concord with, both rule

118

Ludic and Social Media Interaction Design Principles

systems of game ‘worlds’, and those of the ‘real’ world. Johannes Huizinga’s notion of the “Magic Circle” has often been cited by game theorists since its introduction in the 1930s. He used it to characterise the experiential divide between the fictional worlds and rule systems of games, and the rule systems governing everyday life ‘outside’. Dave Szulborski (2005) and Jane McGonigal (2011), take players’ subjective experiences in playful settings as a starting point, positing a distinction between two cognitive frames: “This-Is-A-Game” (TIAG) and “This-Is-Not-A-Game” (TINAG). In a recent article (Coppock & Ferri, in press), game designer and theorist Gabriele Ferri and this author note that players of traditional games are always well aware of activities they are taking part in and where these are taking place. The TIAG frame establishes player expectations and acknowledgements as participants in play. Interactions with the game system generate a ludic discursive universe in the TIAG layer, acknowledging the fictionality of gaming interactions. As player focalization (Genette, 1972/1980), switches to the ludic universe, TIAG interpretive rules are activated, setting aside players’ encyclopaedic real world knowledge – so they are no longer surprised, for example, by the height of Super Mario’s jumps. However, many games also harbour a secondary system of expectations nested in the TIAG layer: an intradiegetic frame where players adopt “a habit of pretending to believe this is not a game” (Eco, 1989) – necessary for maintaining the TINAG frame. Typically pervasive13, A/ARGs use these mechanisms to enhance player engagement, by creating ambiguity regarding existential delimitations of the TIAG and TINAG frames. The following final section examines five examples of A/ARGs designed for play in different cultural spaces and places that are specifically local, or generically global, or a hybrid of both. All demonstrate in different ways how ludic interaction design principles can engage players in the learning processes by revealing for them new dimensions of cultural spaces and places they already believe they know well, and interact with daily.

13

Pervasive games are played in public environments like cities, shopping malls, workplaces, museums, art galleries, schools etc., “played intentionally alongside everyday life, often throughout the day, and serve as an activity [that] is allowed to carry over into other areas of life” (Montola, Stenros & Waern, 2009: 226).

Patrick J. Coppock

119

Five Examples of Ludic Interaction Design The Beast (2001) “The Beast” was a transmedia-based ARG created by the Puppetmasters, a programming team at Microsoft, to promote Steven Spielberg’s film “A.I.: Artificial Intelligence”. It ran for 12 weeks in spring and early summer 200114. Set in the year 2142, 16 years after events chronicled in “A.I.” (located in 2126), it had three entry points to the game: “rabbit holes” in ARG parlance: 1. Trailers and posters for “A.I.” with a credit for Jeanine Salla, Sentient Machine Therapist among the main credits. 2. One trailer encoded a telephone number in the promotional text; if a player called this number and followed instructions he/she would receive the email: “Jeanine is the key” - “you've seen her name before”. 3. An “A.I.” promotional poster in technology and entertainment media outlets had this simple clue: “Evan Chan was murdered. Jeanine is the key”. Based on these clues, players had to solve the crime. Each rabbit hole led to questions about Jeanine Salla. Googling her name brought up web pages set in the fictional gameworld including Salla's employer, Bangalore World University. Salla's bio page showed her phone number and a link to the page of her granddaughter, Laia. These clues lead to the homepage of Evan and Nancy Chan, family friends of the Sallas. Jeanine's phone message reveals Evan’s death in a boating accident on his A.I.-enhanced boat, the Cloudmaker. At this point, the player joins the investigation into Evan's death. To tackle the informational complexity of the game, players created a Yahoo group called “Cloudmakers”. At its peak, its thousands of members produced over 40,000 messages. The game was continually under development as it was being played. The Cloudmakers constantly challenged the game developers and influenced the plot. After the game, the Puppetmasters admitted they relied on the vast knowledge base of the Cloudmakers and other player groups to meet puzzles the designers had created. For instance, a puzzle near the end of The Beast required players to understand lute tablature, and sure enough, some Cloudmakers could 14 The most intensive play period was April 2001-May 2002, c.f. http://games.groups.yahoo.com/group/Cloudmakers/

120

Ludic and Social Media Interaction Design Principles

solve it. This game is often cited as demonstrating how globally networked ludic and social media evoke growth of specialised knowledge communities, or “collective intelligence”. (Levy, 1999; Rheingold, 2002).

Urgent EVOKE (2010) “EVOKE”15 is “a ten-week crash course in changing the world”, “free to play and open to anyone, anywhere”. Open to participants of all ages, from a base age of 13 and up, it was developed by the World Bank Group Learning and Knowledge arm. The ARG Master was researcher and author Jane McGonigal of the “Games for Change Initiative”. The game goal is to empower people to come up with creative solutions to the perceived as the most urgent social problems where they live. The first EVOKE season ran from March 3rd, 2010 to May 12th, 2010. It is still possible to request to join the game. After the first game season, successful participants became the first graduating class in the “EVOKE network”. A few top players earned online mentorships with experienced social innovators and business leaders, seed funding for new ventures, and travel scholarships to share future visions at the EVOKE Summit in Washington DC. McGonigal evaluated the project with members of the design and management teams, and the results are online in a dedicated section of the project weblog16. The project was a fair success in relation to expectations. Over 4693 active players (target: 700) took part, and 37 EVOKATION prizes: 15 scholarships, 22 mentorships and 10 seed funding awards were made. It has so far not been repeated. An A/ARG is of interest due to its complexity, thoroughness of organisation and global ambitions, and due to its rootedness in players’ own social realities; it is quite likely to lead to significant social and economic developments in the long-run.

Mettiti in Gioco17 (2011) “Mettiti in Gioco” (MIG) was designed during a workshop led by game designer Gabriele Ferri, at the Department of Communication and Economics at the University of Modena and Reggio Emilia, with the objective of encouraging multicultural encounters and dialogue in urban 15

http://www.urgentevoke.com/ http://blog.urgentevoke.net/2010/07/26/what-went-right-what-went-wronglessons-from-season-1-evoke1/ 17 Trad: “Bring Yourself into Play”. A workshop to design MIG was supported by grants from the Province of Reggio Emilia. 16

Patrick J. Coppock

121

settings. A first trial of MIG was organised18 in Castelnovo ne' Monti, in the Province of Reggio Emilia, in April 2011. The design process began by selecting key features from Mary Flanagan’s “Massively Multiplayer Soba”19 (MMS) series, to adapt it to function in a small Italian municipality. The core mechanics of MIG are similar to its American counterpart: the game is held in an urban area, participants are divided into groups, each receiving a sealed envelope containing a mission to complete. A modification relative to MMS was that MIG players were given hand-held video cameras and were asked to document experiences of the game. The first mission required the translation of a recipe into Italian from a language20 unknown to players, who had to ask for help from passers-by fluent in these languages. The second mission required players to collect a story set in the town where the game was being played and retell it to a fixed video camera at a “Story Bank”. While game organisers had some degree of control over the first mission, the selection of which stories to collect, and who to approach to ask for them was up to the players. The missions were designed to be complementary, requiring players to interact with senior citizens to collect stories from the past, and with immigrants to translate multilingual clues. The third and fourth missions required participants to shop for recipe ingredients and to prepare and cook the food, sharing it with other participants, those they had asked for help. The Story Bank element represents a significant difference between the MIG and MMS game series. Only by ‘depositing’ their story in the Story Bank could players complete the second mission and advance in the game. Only the Bank could award, in exchange for stories, coupons needed to shop for ingredients in the next mission. The fact that retelling a story is recorded on video in exchange for a prize adds value to narration (a good story is precious), serving to build personal, direct connections between players, passers-by and the game environment. Participants are no longer ‘just players’; they also take responsibility for small but significant, placespecific story-telling, collection and archiving practices.

18 “Co-organized with L'Ovile–Cooperativa di Solidarietà Sociale, with the support of UNAR (Ufficio Nazionale Antidiscriminazioni Razziali), Province of Reggio Emilia, Municipality of Castelnovo ne' Monti, and Rete Contro le Discriminazioni. 19 http://www.tiltfactor.org/massively-multiplayer-urban-games 20 Eg. Russian, Arabic, Urdu and Reggio Emilia regional dialects.

122

Ludic and Social Media Interaction Design Principles

Falkland Ghost Hunt (2011) This A/ARG was designed by Mads Haahr of “Haunted Planet Studios”21, a National Digital Research Centre22 spin-off in Dublin, Ireland, in collaboration with The National Trust for Scotland23 and ZOLKc24. Falkland Royal Palace was built between 1501 and 1541, in the heart of a medieval village, as a country residence and hunting lodge for eight Stuart monarchs, including Mary Queen of Scots, who spent some of the happiest days of her life there: “playing the country girl in the woods and parks”. Falkland Palace is surrounded by a collection of beautiful gardens housing the Real Tennis Court (1539), Britain's oldest tennis court and a royal tennis club. The Falkland Ghost Hunt game used satellite positioning and A/AR technologies to bring ghosts, banshees and Mary Queen of Scots ‘back to life’ in the palace gardens. Players were required to use a “radar-enabled paranormal detection device” – an Android-based mobile phone application, allowing them to visualise and ‘hunt down’ ghostly apparitions, and prove their existence by capturing images and recording disembodied voices from the screen. Samples of what players experienced via their phone screens and headphones is available on YouTube25. During the Halloween 2011 trial period more than 300 people took part. Some were video-interviewed and their impressions can be heard on YouTube26. One of the most often repeated comments was that the game made the place and its history ‘come alive’ in an exciting, entertaining way.

Immaterials: Light painting WiFi (2011) “Immaterials Light painting WiFi” is an online video27 by Timo Arnall, Jørn Knutsen and Einar Sneve Martinussen, participants in the “YOUrban Project”28, at the Oslo School of Architecture and Design (AHO), in collaboration with InterMedia, both of the University of Oslo, Norway.

21

http://www.hauntedplanet.com/ http://www.ndrc.ie/ 23 http://www.nts.org.uk/Home/ 24 http://www.zolkc.com/ 25 http://www.youtube.com/watch?v=sIqaTHnDprU&feature=channel 26 http://www.youtube.com/watch?v=p0M7U65RO5M 27 http://yourban.no/2011/02/22/immaterials-light-painting-wifi/ 28 http://www.aho.no/no/Arena/Forskning/UL/YOUrban/ 22

Patrick J. Coppock

123

The background for this aesthetic visualisation project is the fact that city spaces host an invisible landscape of WiFi networks that are rapidly becoming an integral part of daily life. These and other sensor networks, our increasingly sophisticated mobile phones and portable digital devices, influence how we experience and understand urban environments. The video explores and reveals what the immaterial ‘terrain’ of WiFi signals looks like and how it relates to the city. It is essentially about discovering and contextualising WiFi networks through specialised visualisation tools, in a continuation of earlier explorations29 of intangible phenomena with further implications for design and production choices, affecting how cities are experienced in the future. Matt Jones of BERG30 has characterised ‘Immaterials’ using sociality, data, time and radio as typical examples. Radio and wireless communication are a fundamental part of today’s life in networked cities, generating what William Mitchell (2004) refers to as an intricate and invisible ‘electromagnetic terrain’ hinted at only by the presence of antennas. To understand spatial and material qualities of wireless networks, the Immaterials team built a WiFi measuring rod that visualises WiFi signal strength on a vertical bar of light-emitting diodes (LEDs). When moved through urban space, the rod displays a changing WiFi signal strength. A strong signal activates more LEDs so a greater rod length is illuminated. Long-exposure photographic sequences of the rod as it is carried through the WiFi field allow visualisation of network signal strength as light pattern cross-sections.

Conclusion A/AR projects adopt a ludic stance to explore convergences between imagined fictional worlds and the real world. Innovation processes depend on imagination to envision possible worlds differing in significant, relevant ways from what we conceive of as reality. This article argues that use of A/AR Game interaction design principles to evoke ambiguity in players as they move between TIAG and TINAG frames, is a positive strategy to encourage ‘stepping outside’ normative sociocultural constraints of reality in familiar settings, enabling fresh, innovative ‘rereadings’ of them. This opens up for a re-envisioning of familiar spaces and places, generating ideas for their redesign as more humane, functional,

29 30

http://www.nearfield.org/2009/10/immaterials-the-ghost-in-the-field http://berglondon.com/

124

Ludic and Social Media Interaction Design Principles

pleasant, stimulating, ecologically balanced places for us to live, work and play in, within future Smarter City contexts. Bibliography Caillois, R. (2001), Man, Play and Games, University of Illinois Press. Coppock, P.J. (2009), “Here, There and Everywhere: Glocalising Identity in Transword Transmedia Genius Loci” in K. Sandvik (ed.) MedieKultur: Journal of Media and Culture Research, Volume 47: 722 Coppock, P.J. (2010), “Amodal perception in hybrid forms of experienced agency in shared multimedia gameplay space”. In G.P. Storari, E. Gola (eds.), Forme e Formalizzazione. Atti del XVI Convegno Nazionale della Società Italiana di Filosofia del Linguaggio, Società di Filosofia del Linguaggio 07, CUEC, 137-156. ; Coppock, P.J., Ferri, G. (In Press), Serious Urban Games. From Play in the City to Play for the City, Position Paper for the ECREA Media and the City Temporary Working Group Workshop TWG ‘12, Milan, February 10th 2012. Paper online: http://twg.ecrea.eu/MC/?p=247 De Jong, M. (2009), All tomorrow’s parties: what is reflexive playfulness? UTA Games Research Lab & Nokia Research Center pdf document (pers. comm.). De Jong, M. (2010), “Paranoid, not an Android: Dystopic and Utopic Expressions in Playful Interaction with Technology and everyday surroundings”, in Aljas, A. et al. (eds.), Transforming culture in the digital age, Estonian National Museum, University of Tartu, Tartu 2010, 50-60. Dekoven, B. (2002), The Well-Played Game: A Playful Path to Wholeness, Lincoln (NE): iUniverse. Eco, U. (1989) (Trad. W. Weaver), Foucault’s Pendulum, London: Secker & Warburg: 386. Frost, Joe L. (2009), A History of Children's Play and Play Environments. Toward a Contemporary Child-Saving Movement, London: Routledge. Genette, G., (1972), Figures III, Paris, Seuil; En. tr. Narrative Discourse, Ithaca, Cornell U. Press, (1980). Huizinga, J. (1971/1938), Homo Ludens: A Study of the Play-Element in Culture, Boston: Beacon Press. Jenkins, H. (2008), Convergence Culture: Where Old and New Media Collide, New York: New York University Press ; Jenkins, H. (2006). Fans, Bloggers and Gamers: Exploring Participatory Culture. New York: New York University Press.

Patrick J. Coppock

125

Jenkins, H., et. Al. (eds.) (2009), Confronting the Challenges of Participatory Culture; Media Education for the 21st Century, New York: MIT Press. Levy, Pierre, (1999), Collective Intelligence: Mankind's Emerging World in Cyberspace, New York: Basic Books. ; Rheingold, H. (2002), Smart Mobs: The Next Social Revolution, New York: Basic Books McGonigal, J. (2011), Reality is Broken: How Games Can Change the World, New York, Penguin Press. McLuhan M. (1964), Understanding Media: The Extensions of Man. New York: McGraw-Hill. Mitchell, W. (2004), Me++: The Cyborg Self and the Networked City, Cambridge (MA): MIT Press Montola, M. Stenros J., Waern, A. 2009, Pervasive Games: Theory and DesignNew York: Morgan Kaufmann: 226. Murray, J. (2012), Inventing the Medium. Principles of Interaction Design as a Cultural Practice, Cambridge (MA): MIT Press, especially chapter 13: “The Game Model: Scripting Interaction as Structured Play” Sutton Smith, B. (2001), The Ambiguity of Play, Cambridge (MA): Harvard University Press. Szulborski, D. (2005), This Is Not A Game: A Guide to Alternate Reality Gaming, Lulu.com Wittgenstein, L. (1953/2001), Philosophical Investigations, Oxford: Blackwell.

TYPOGRAPHY AND LANGUAGE: A SEMIOTIC PERSPECTIVE JACK POST

Alphabetic writing is traditionally seen as the logical endpoint of the evolution of all preceding writing systems. In this ethnocentric and teleological perspective, each form of writing would ‘naturally’ develop into an alphabet that visually represents the sounds of a language and its writing perceived as secondary to speech, as a mere instrument to represent spoken language. With the emergence of French semiology based on Ferdinand de Saussure’s “Course in General Linguistics” (1968, 1983) in the 1960s and 1970s, the relation between written language, typography and spoken language suddenly came to the fore. Most scholars on the semiotics of writing and typography were of the opinion that Saussure held a very traditional view on the relation between written and spoken language. Johanna Drucker, for instance, characterises in her seminal study on typography “The Visible Word” (1994), Saussure’s “Course in General Linguistics” as an important milestone in the emancipation of written language but contends that Saussure “held to the position that the linguistic sign, although conceptual and nonmaterial, was constituted in spoken language” (20). Roy Harris (2000, 63) states that Saussure considers writing as merely meta-signs that signify the signs of speech and Anne-Marie Christin (1999) argues that for Saussure the graphic and visual manifestations of the alphabet were without any semiological pertinency. They all refer to “Course in General Linguistics” in which Saussure seems to characterise written language as accessory and secondary to spoken language. Although Saussure acknowledged the fact that writing and speech are two distinct systems, he seems, according to Drucker, to denigrate writing at the same time by according primacy to speech. In other words, albeit Saussurian semiotics originates in linguistics, and written language and typography are closely linked to language, his theory was not seen as an adequate point of departure for the development of a semiotics of typography.

Jack Post

127

Roland Barthes’ essay “Elements of Semiology” (1967) was the first systematic introduction to, and overview of, semiological theory and constituted the point of departure for a whole generation of linguists and semiologists. Barthes’ interpretation of cardinal analytical concepts of Saussure’s theory (such as sign, signifier and signified) and Louis Hjelmslev’s theory (such as denotation, connotation, expression and content) is still the accepted standard in contemporary semiotic research on typography. Barthes predicted that “the future probably belongs to a linguistics of connotation” (90), and indeed the concept of connotation has become one of the central concepts in semiological studies on typography. It was the central concept in Gérard Blanchard’s extensive study “Pour une Sémiologie de la Typographie” (1980) and more recently Theo van Leeuwen (2005, 139, see also 2006) took Barthes’ definition of connotation as a starting point for his semiology of typography. Barthes’ interpretation of denotation and connotation, however, and in particular his description of connotative systems as ‘staggered systems’ (1967, 89) and his conception of the function-sign (1964, 106, 1990, 264-65), are at certain points at odds with Saussure’s and Hjelmslev’s teachings. Moreover, “Course in General Linguistics” was never published during Saussure’s lifetime, the critical editions were based on lecture notes of his students. The manuscripts, including the draft of a book on general linguistics that were discovered in 1996 in the orangerie of Saussure’s former residence,1 show, according to Simon Bouquet (1999, 2002), that the edited versions of “Course in General Linguistics” not only distort and occlude but even contradict Saussure’s thoughts on essential points. A critical assessment of Saussure’s approach to written language in the light of the recently found manuscripts and of Hjelmslev’s concept of connotative semiotics, shows that a semiotic of typography based on the ‘original’ editions (and translations) of Saussure’s “Course in General Linguistics” and Barthes’ interpretation of Hjelmslevian connotative semiotics is flawed because both Hjelmslev and Saussure contend, as is also stated by the Paris School of Semiotics, that typography and writing are expressions of the language system in their own rights. It would, therefore, be more promising to develop a semiotic approach of typography based on the premises of the semiotics of the Paris School.

1

The manuscripts were published in de Saussure 2002, English translation 2006.

128

Typography and Language: A Semiotic Perspective

Gérard Blanchard – Typography and Connotation Gérard Blanchard2 was the first to develop a comprehensive connotative analysis of typography in the tradition of Barthes. Blanchard defines typographical signs as material (technical) objects that depict the letters of the latin alphabet. Blanchard explains that when Gutenberg designed his typographical system to mechanically ‘duplicate’ the written language, he had to normalise and rationalise the handwritten signs in such a way that he was able to mechanically segment the handwritten language into a limited set of ‘moveable types’ (minimal signs of the written forms) (1980, 30-33, 42-43). The effect of his typographical system was the unification and standardisation of writing laid down in a set of technical rules and practices which we now call typography. Since typographical signs stand closer to material (technical) objects than to ‘immaterial’ linguistic signs, they are perceived as an imbrication of the technological and signification. Blanchard’s semiology of the typography is clearly indebted to Roland Barthes’ “Elements of Semiology” (1967) and in particular to “The Fashion System” (1990). Next to the concepts of denotation and connotation Blanchard refers extensively to that of the function-sign to account for the fact that typographical signs are perceived as material (technical) objects. Barthes defines function-signs as utilitarian and functional objects. A material, everyday life object like a fur coat is in itself “an inert entity which neither produces nor receives meaning, but merely transmits it (…) because the garment is not in itself a system of signification, as is language” (1990, 265-66). The moment these objects exist in society and are socially used, they convert into function-signs, which only get meaning when they are incorporated into the larger whole of linguistic signs. In Barthes’ semiology, the world and its material objects only become meaningful in and through language. The function of a fur coat (as an object) is to protect us from the cold (function-sign), the semantisation converts the function-sign into a linguistic sign ‘a fur coat that protects us from the cold’. This meaning usually shifts to the background to make place for what Barthes calls secondary or connotative meanings. In our Western society, a fur coat is more likely to be perceived as a luxury item or more recently as a sign of negligence of animal rights (secondary connotative meanings), than as a practical garment that protects us from the cold (its primary denotative meaning based on the function-sign). 2

With exception of Gérard Blanchard’s study, typography was only accidentally object of semiotic research Mounin 1970, Lindekens 1971, Lindekens 1976.

Jack Post

129

Fig. 3-4: Connotative system

The function-sign (object-sign) thus becomes the signified of a denotative system, which in turn becomes the signifier for a new connotative system of additional socio-cultural meanings, which grafts itself so to say onto the denotative system (Barthes speaks about a parasitical relation) (1990, 40). Barthes calls the connotative signifiers connotators and the whole of the connotators a rhetorical system. Blanchard considers the typographical form as a signifier of connotation and illustrates the workings of the typographical connotation with an analysis of the nameplate (logotype) of the French quality newspaper “Le Monde” (1980, 299).

Fig. 3-5: Nameplate Le Monde (1945)

The logotype forms the basis of a function-sign which denotes ‘gothic characters’ and refers to the characters used in the Gutenberg bible. The denotative system which is based on this function-sign simply denotes ‘gothic character’ and constitutes the signifier for a secondary connotative system. The signified of this connotative system refers to the “Gutenberg Bible” and creates the following rhetorical trajectory: the original Gothic type positions the journal on the one hand as a journal of the truth (because it refers to The Bible) and on the other hand as a figure of supreme printing (because it refers to the Gutenberg print revolution). The ideological form of the signified of the connotation refers to the myth of the origin of printing and to the stabilised time of The Bible in opposition to the ephemeral time of the daily journal. The rhetorical effect of these connotations is that the logotype functions as an ideological reassurance

130

Typography and Language: A Semiotic Perspective

against the fears that the daily fluctuations in the news provoked by its readers.

Louis Hjelmslev – Writing and language In Barthes’ interpretation of Hjelmslev connotative semiotics a connotative sign consists of two systems “which are imbricated, but are out of joint with each other, or staggered” (1967, 89). Blanchard’s analysis of the nameplate of Le Monde inserts indeed the function-sign into the signified of the denotative system and the denotative system into the signifier of the connotative system. Hjelmslev however, never talks about imbrication or insertion because the connotative semiotics represents in his theory a formalisation of the varieties of the denotative semiotics. Take for example four realisations of the letter “d” of the Latin alphabet:

Fig. 3-6: Four different realisations of the letter d

Hjelmslev would describe these realisations as projections of the pure abstract form “d” on the content plane upon the substance of the expression plane of the denotative semiotics. On the content plane is the “d” as a pure abstract form in opposition with the “b”, “p” or “q”. The projection of the “d” on the substance of expression manifests the form of expression /d/ which establishes the norm or the ‘bandwidth’ so to say, within all possible varieties of the letter “d” that can still be identified as “d”. To resume, on the level of the content of the denotative semiotics the “d” is an invariant which is independent from, but presupposed by, its possible social realisations (the varieties of the letter d) and material manifestations /d/ (variants) of the form of the substance of the expression. From the perspective of the content plane of connotative semiotics however, every variant /d/ is related to a different variety of the letter “d” and thus to a different form of the content. Changing the “d” from blackletter into Futura or from regular into italics remains from the perspective of the denotative analysis within the norm, since they are all varieties of the “d”. From the perspective of the connotative analysis, however, they represent different variants /d/ (connotators) and are as such related to different forms on the content plane of the connotative semiotics. They are evidently different letters and at the same time the very same letter “d”.

Jack Post

131

The connotative analysis thus takes care of the overwhelming variability of the fonts produced by usage, i.e. the acts of the individual users and the habits adopted by a society (what Saussure called parole). Hence, before a connotative analysis can formalise the varieties of the denotative semiotics, an inventory of the invariants of the denotative semiotics (what Saussure called langue) should be made up: “after the analysis of the denotative semiotic is completed, the connotative semiotic must be subjected to an analysis according to the same procedure” (Hjelmslev 1969, 119). Thus, when Blanchard inserts the function-sign in the denotative sign and subsequently the denotative sign in the connotative sign, he actually ‘neglects’ the analysis of the denotative system itself. His analysis of connotation has, therefore, more in common with a traditional description of connotations as secondary, accessory or supplementary meanings that are added to the referential meaning. Just like typographers speak about the dress (connotation) and the body (referent) of the type (see Stoeckl 2005). The object of semiotic analysis is, however, the description of the formal schema of the denotative or connotative plane and not as Barthes argues a connotative analysis of the (material) substance of the connotative plane. Hjelmslev defines connotators as varieties of the invariants of the plane of content and not as additional meanings which are added to the denotative meanings or in the more politicised wordings of Barthes, as meanings that are naturalised by the denotation and which conceal the denotation. The task of the semiologist is not to ‘decipher’ the ‘real’ denotative significations buried under a crust of ideological connotations (Barthes 1967, 94), but to describe the usages on the basis of a description of the semiotic schema. Only a description of the denotative semiotics opens up the possibility of a formal and systematic study of the varieties and the variations of the usages, that is a connotative analysis (see also Badir 2000, 78-83).

Ferdinand de Saussure – Writing and speech By explicitly taking the substance of expression into account when analysing semiological signs or function-signs, Barthes’ semiology reintroduces the third term of the referent again in the binary couple of signifier and signified and departs thus from a ‘traditional’ triadic interpretation of the sign as for instance Umberto Eco (1988) presents in his “Le signe”:

132

Typography and Language: A Semiotic Perspective

Fig. 3-7: Triadic sign model

For Eco, comments Badir, “the signifier horse for example, has (in the real world or the possible world) the animal as referent in function of a signified–idea, concept, or something that takes its place - which relates both” (2000, 174). It is exactly this triadic model of the sign that becomes, inspired by Barthes’ publications, dominant in French semiological research in the 1960s and 1970s. Saussure, however, insisted on the binary relation of the signifier and the signified on the one hand and of the sign and the extra-linguistic reality on the other. Merging both conceptions in a single triadic relation is, according to Badir, impossible because the relation between signifier and signifier is established on the level of the langue whereas the relation between sign and ‘referent’ is situated on the level of the parole (175).

Saussure states repeatedly in his “Course of General Linguistics” that the combination between the signifier and signified produces a form and not a substance. By defining the function-sign (or semiological sign) in terms of its usage and substance, Barthes places himself in the tradition of functional linguistics of the Prague Linguistic Circle and André Martinet who define the signs in terms of their substance (or matter) (see for the discussion between Martinet en Hjelmslev Arrivé 1985). Human language contends Martinet is not only defined by the principle of the double articulation but also and primarily by its phonic substance. Hence, only languages with a double articulation and a phonic substance are human languages. Systems not based on phonic substance, such as written language and ipso facto typography, are what Barthes calls semiological systems. In the case of fashion, the signifiers of the function-signs are “always part of the physical world which is the clothing content, the fragment of the bodily space occupied by the clothing item (a woman’s suit, a pleat, a clip brooch, gilt buttons, et cetera)”. In Barthes’ function-sign signifier and signified collapse because “every object is also a sign” (1990, 264), which is indicated by the

Jack Post

133

fact that in “Elements of Semiology” the bar between the signifier and signified of the function-sign in the schema of the connotation is omitted. Hjelmslev takes explicitly stance towards the functionalist linguistics of the Prague Linguistic Circle and André Martinet by excluding not only the phonematic but also the graphematic substance from the linguistic analysis, although in his opinion both are co-existing systems with equal rights (1973a, 271). Writing and speaking are actually not two separate systems but “the same system manifested by two different substances, respectively the phonic and the graphic substance” (1973a, 271; see also Arrivé 1983). The fact that we express language through sound is, according to Hjelmslev, not due to the nature of language, as argues Martinet, but the consequence of our anatomical-physiological constitution. In other words, the alphabetic system is not, as is traditionally believed, based on a phonetic analysis of language: not the articulation of the pronunciation, but the articulation of the language itself has been recognised or attempted to be recognised by the invention of the alphabet. Not the phonetic substance has been transposed into a graphic substance but the linguistic form of expression has been imprinted directly into the graphic matter to take hold of it (1973b, 229).

Hjelmslev objects to the traditional view on writing as a derivation of the pronunciation or as a transposition of the spoken language into the visual order. Language can be expressed in substances other than sound, such as gestures or graphical symbols for instance. Although scholars of the history and theory of typography and writing, like Johanna Drucker, reproach Saussure to make written language subservient to spoken language, Saussure never puts the domains of the written language and the spoken language into a relation of dependency. He excludes matter, because the combination of the signifier and the signified produces a form and not a substance (1986, 157), just like Hjelmslev excludes material criteria from the definition of language. Sound is just material and not necessary for the definition of language itself (164-165). Because he never defines sound elements in language positively but always negatively in terms of oppositions and relations, language is pure form and signifiers are always immaterial and incorporeal. Curiously enough, Saussure explains the question of the material aspect of language with a discussion of the writing system. Firstly, he argues that “the signs used in writing are arbitrary. The letter t, for instance, has no connexion with the sound it denotes” (165). It seems as if Saussure here, when he links the letter to the sound, is at odds with his

134

Typography and Language: A Semiotic Perspective

own theory. The editors of “Course in General Linguistics” have probably adjusted the text to the traditional view on the relation between written and spoken language. However, in the notes of his students, which formed the basis of the “Cours de Linguistique Générale”, it is stated that Saussure said “la chose à désigner” [the thing it indicates] and not “le son qu’elle désigne” [the sound it denotes] (see critical edition by Rudolf Engler, de Saussure 1968, 269). In other words, as remarks Michel Arrivé in an analysis of this passage, Saussure refers here to the incorporeal signifier (2007, 71-72). In the found manuscripts, Saussure remarks that “there is never in any way a link between a certain sibilant sound and the shape of a letter S, and similarly it is no harder for the word vache than the word vacca to refer to a cow” (2006, 147), which is in total agreement with the next step in Saussure argumentation.

Fig. 3-8: Saussure 1986, p. 165

The same person can write a “t” in many different ways, because the letters are not defined positively and materially, but negatively, differentially and incorporeally. The only thing that matters is that the t is not confounded with other letters such as the “l” or “d”. In a third step, Saussure explains that because the letters are dependent on their mutual oppositions within a defined system of a limited number of elements, the graphical sign is arbitrary and thus the forms of the letters are of no importance. The last point Saussure discusses is that the means of production of the sign is indifferent to the system: Whether I write the letters in black or white, engraved or in relief, with a pen or with a chisel–none of that is of any importance for their meaning (1983, 118).

This is exactly the passage on the basis of which Anne-Marie Christin concludes that graphical and visual elements are for Saussure without any semiological pertinency (1999; see also Christin 2002, 12). As is argued above, the question is more complex. When Drucker asserts that it is not possible to generate “a concept of materiality out of Saussure’s theory of the sign” (1994, 23) because Saussure in his will to establish a “purely differential system of linguistic function” rejects “the materiality of the

Jack Post

135

signifier as substantive, as having a role to play in the production of meaning” (1994, 26), she is right and wrong. She is wrong in the sense that a semiotic theory of the substance of the signifier is not possible. It is possible but only on one condition: that it studies the typographical form of the manifestations (usage) and not the substance matter itself. The means of production of the written signs are of no importance for the language system (denotative plane) itself but they are relevant for the analysis of the connotative plane. She is right because an analysis of the connotative plane excludes the typographical substance or matter as such, which are in Hjelmslev’s opinion the objects of other sciences: in the case of typography for instance perception psychology (readability research), design history and theory, or informatics. Indeed, as Drucker herself explains, her research moved from “the role of the visual manipulation of the signifier” to “the way in which materiality of signifying practices (…) is inextricably bound up with the production of history and subjectivity in artistic practices” (1994, 2). The first question is of course a semiotic problem, the second more like the object of art and design historical research (maybe informed by semiotics).

Conclusion After the initial euphoric phase of the 1960s and 1970s many scholars turned their back on semiology, partly because of the fact that a connotative analysis à la Barthes turned out to be a very laborious construction of a traditional inventory of supplementary meanings. As A. J. Greimas remarked somewhat harshly semiological analysis of a connotative nature could only succeed in producing a redundancy of commonplaces, unless it where to seek a foundation elsewhere. Either in a certain form of psychology (at which time the object-semiotic system becomes the ‘signifier’ for psychoanalysis) or in a certain sociology (at which time semiology becomes the post-facto justification of a theory of ideology). (Greimas and Courtés 1982, 283).

Semiology in the tradition of Barthes, argues Greimas, lets the signifieds choose their own signifiers and thus does not take into account one of the basic postulates of semiotics, namely that the signifier and the signified presuppose each other reciprocally. Semiotics as developed by Greimas and the Paris School (see Greimas and Courtés 1979, 1982, 1986) approach writing differently, namely as an expression of the system of the natural language on the one hand and as a graphic system that is part of the

136

Typography and Language: A Semiotic Perspective

figurative (visual) semiotics on the other. The graphic materiality manifests a plastic signifier that can conventionally be interpreted as the compact objects of figure-letters. Moreover, it also has other possibilities of signifying, because it “could give rise to a subarticulation of letters into their constitutive features, thus revealing an underlying graphematic organisation” (Greimas 1989, 637). The segmentation of the graphematic organisation into plastic units differs fundamentally from the double articulation in that it breaks the plastic manifestation of the signifier down into plastic formants, figures and topological, eidetic and chromatic categories of the plastic form (638-644). Greimas explains that the analysis of the plastic signifier3 does not just proclaim that plastic objects signify, but wants to understand how they signify and what they signify (644). It does not restrict itself to the analysis of the meaning of the individual typographical characters, but also to how they give form to larger and more complex organisations such as lines, paragraphs, chapters, type pages and concrete poems, calligrammes or credit sequences in cinema (see also Zaganelli 2008, 89-111). Depending on the point of view taken, the question whether typography is a language can be answered differently. Typography is not a natural language but neither is speech, both are expressions of the langue in the sense of Saussure. Typography is at the same time inextricably interwoven with the natural language and should therefore always be analysed in relation to the langue. Typographical signs can thus be analysed in a twofold way: either as letters of the Latin alphabet or as elements of an independent plastic organisation with its own meaning production. Greimas even speaks about the autonomous organisation of the plastic signifiers and its signifieds as a secondary, poetical language (1989, 64647). Typography can, therefore, also be considered as a poetical or aesthetic language that subverts the primary functions of the alphabet and written language as the pertinent analysis of futurist poetry by Giovanni Bove (2009) illustrates. A semiotics of typography is thus possible and especially the idea that typography can be approached as a secondary poetic organisation of the planar written surface suggests that a radical different alternative to Roland Barthes’ project of a connotative semiotics exists.

3

The analysis of the plastic signifier is according to Greimas the task of a semisymbolic semiotics (1989, 644-47).

Jack Post

137

Bibliography Arrivé, Michel. “Les Danois aux Prises avec la Substance de l’Encre.” Langue Française (1983): 25–30. —. “Hjelmslev Lecteur de Martinet Lecteur de Hjelmslev,” In Nouveaux Essais, edited by Louis Hjelmslev, 195–207. Paris: PUF, 1985. —. À la Recherche de Ferdinand de Saussure. Paris: PUF, 2007. Badir, Sémir. Hjelmslev. Paris: Les Belles Lettres, 2000. Barthes, Roland. “Éléments de Sémiologie.” Communications 4 (1964): 15–84. —. Elements of Semiology. London: Cape, 1967. —. The Fashion System. Berkeley [etc.]: University of California Press, 1990. Blanchard, Gérard. “Pour une Sémiologie de la Typographie,” Troisième cycle, Sociologie et Sémiologie des Arts & des Littératures, École des Hautes Études en Science Sociales, Paris 1980. Bouquet, Simon. “La Linguistique Générale de Ferdinand de Saussure: Textes et Retour aux Textes.” Texto! [en ligne] (1999). —. “Après un Siècle, Les Manuscrits de Saussure Reviennent Bouleverser la Linguistique.” Texto! [en ligne] (2002). Bove, Giovanni. Scrivere Futurista. La Rivoluzione Tipografica tra Scrittura e Immagine. Roma: Nuova cultura, 2009. Christin, Anne-Marie. “Le Signe en Question.” Degrés 100 (1999): 1–12. Anne-Marie Christin, ed. A History of Writing. From Hieroglyph to Multimedia Paris/London: Flammarion/Thames & Hudson, 2002. de Saussure, Ferdinand. Course in General Linguistics. London: Duckworth, 1983. —. Cours de Linguistique Générale. Paris: Payot, 1986. —. Écrits de Linguistique Générale. Paris: Gallimard, 2002. —. Writings in General Linguistics. Oxford/New York: Oxford University Press, 2006. —. Cours de Linguistique Générale. Wiesbaden: Harrassowitz, 1968. Drucker, Johanna. The Visible Word. Experimental Typography and Modern Art, 1909-1923. Chicago [etc.]: University of Chicago Press, 1994. Eco, Umberto. Le Signe. Histoire et Analyse d’un Concept. Bruxelles: Editions Labor, 1988. Greimas, Algirdas Julien. “Figurative Semiotics and the Semiotics of the Plastic Arts.” New Literary History 20 (1989): 627–49. Greimas, Algirdas Julien, and Joseph Courtés. Dictionnaire Raisonné de la Théorie du Langage. Paris: Hachette, 1979.

138

Typography and Language: A Semiotic Perspective

Greimas, Algirdas Julien, and Joseph Courtés. Semiotics and Language. An Analytical Dictionary. Bloomington: Indiana University Press, 1982. Greimas, Algirdas Julien, and Joseph Courtés. Dictionnaire Raisonné de la Théorie du Langage. Tome II (Compléments, Débats, Propositions). Paris: Hachette, 1986. Harris, Roy. Rethinking Writing. Bloomington: Indiana University Press, 2000. Hjelmslev, Louis. Prolegomena to a Theory of Language. Madison, Milwaukee, and London: The University of Wisconsin Press, 1969. —. “Introduction à la Discussion Générale des Problèmes Relatifs à la Phonologie des Langues Mortes en l’Espèce du Grec et du Latin (1957),” In Essais Linguistiques II; Travaux du Cercle Linguistique de Copenhague. Vol. XIV, 267–78. Copenhague: Nordisk Sprog- og Kulturverlag, 1973. Hjelmslev, Louis. “Über die Beziehungen der Phonetik zur Sprachwissenschaft,” In Essais Linguistiques II; Travaux du Cercle Linguistique de Copenhague. Vol. XIV, 223–38. Copenhague: Nordisk Sprog- og Kulturverlag, 1973. Leeuwen, Theo van. “Typographic Meaning.” Visual Communication 4 (2005): 137–43. —. “Towards a Semiotics of Typography.” Information Design Journal & Document Design 14 (2006): 139–55. Lindekens, René. Éléments pour une Sémiotique de la Photographie. Bruxelles: Didier-Aimav, 1971. —. Essai de Sémiotique Visuelle. Le Photographique, le Filmique, le Graphique. Paris: Klincksieck, 1976. Mounin, Georges. “Quelques Observations sur la Notion d’Articulation en Sémiologie,” In Introduction à la Semiologie, 135–48. Paris: Les Éditions de Minuit, 1970. Stoeckl, Helmut. “Typography: Body and Dress of a Text - a Signing Mode Between Language and Image.” Visual Communication 4 (2005): 204–14. Zaganelli, Giovanna. Itinerari dell’immagine. Per una Semiotica della Scrittura. Milano: Lupetti, 2008.

FILM AND NEW ART MEDIA SEMIOTICS: ON THE FIGURAL IRINI STATHI

Introduction Many presemiotic discussions of the cinematic medium can be traced back to the early years of film history. Most of them were mainly interested in film syntax instead of the effects the figures produced in the image. But it was Christian Metz, the semiotician, who initiated in 1964 the semiotic discussion of film as language on a linguistic basis. Metz’s provocative statement that the film is “langage sans langue” (1964) gave rise immediately to a heated debate about the semiotic status of films. If filmic language cannot be related to langue in the Saussurean sense of system because it does not consist of a finite number of elements organised according to the rules of a specific syntax, then we must ask the crucial question of how this language can be described. Several Italian semioticians were stimulated by this problem. For Umberto Eco, film language cannot possess a ‘double articulation’, as natural language does. Instead, because of the complexity of filmic signs, it must be conceived of as a visual articulation, described in terms of three dimensional coordinates of kinesic figures. This register of the filmic signification is the basis of iconic signs that are located in the synchronic and diachronic dimensions. In the wake of this discussion, Emilio Garroni (1968) raised the issue of which level of formalisation can be obtained in film semiotics. Rather than being grounded on Metzian syntagmas, such formalisation could start from different filmic codes. These codes were later elaborated by Eco in his writings about the semiotics of the audiovisual in general (Eco 1980). Eco’s view gives priority to a natural iconicity based on perception, but he seems to understate the symbolic regulation of iconic meanings in abstract-iconic forms. This ‘interleaved’ description of iconic and symbolic forms is an advanced aspect of the conceptions of iconicity and diagrammatic reasoning explored by Peirce. On the other hand, in the

140

Film and New Art Media Semiotics: On the Figural

Soviet Union, Juri Lotman's “Semiotics of Cinema” (1973) did aim at the creation of some further semiotic reflections on this film language. Its detailed analyses, which bear upon the filmic illusion of reality, the filmic shot, forms of filmic narration, the question of film as a synthetic art, and the development of modern cinematography, make a foray into the ideological and aesthetic functions of cinema as the dominant mass media art of the 20th century. In Lotman’s study, the cinematic language is related to other media and other codes, thus setting an agenda for further research on the relationship between different codes or filmic materials (linguistic, figurative etc.). Semiotics gives a foundation for the classification of signs according to different forms of iconicity, ranging from the concrete-iconic forms of filmic images, over the abstract-iconic forms of graphs and diagrams, to the symbolic forms of symbols and languages (May 2007). These forms correspond to three underlying similarity measures analysed by Peirce: the concrete-iconic forms rely on a similarity of properties, the abstract-iconic forms rely on a similarity of relations and the symbolic forms rely on an ‘induced’ similarity of conceptual structures. These types of similarity correspond to systematic differences in the interpretation of the main iconic forms. Thus, images are interpreted as referring to their objects through a similarity of properties and languages and symbols are interpreted as referring to their objects through a metaphorical similarity of conceptual structures (figurativity).

From parole to figure The discourse that distinguishes between a linguistic and a figural approach on film semiotics can be extended and cover a multitude of other expressions related to images. What is hiding behind the expression ‘to claim the figural’ or ‘to see the figural’ is primarily that we will use a loose limitation to art that somebody looks at on a screen, and most importantly video and computer screens, which is what some might call new media art. Ideas connected to the experience concern the figural as an aesthetic concept and a reading of the concept in relation to moving images and new media art, may prove the importance of the term in understanding film and new media aesthetics. Thus, we will speak from a theoretically film-oriented perspective, but it is well known that the figural is a concept initially used when talking about static pictures, paintings, but nevertheless some kind of process or dynamic event in the images: something that triggers interpretation.

Irini Stathi

141

The sense of the figural can be developed in three steps. First, by making some remarks on the concept, its contemporary versions and different methodological aspects; secondly, we will try to establish some connections with the film theories of/on Jean Epstein and the figural in relation to moving images, and thirdly we will hint at some possible connections with digital media. An attempt to detect some kind of origin in the use of the concept to Jean-François Lyotard and his dissertation “Discours, figure” (1971) will demonstrate the importance of the term’s significance in the figurative representation. For Lyotard, the figural is an inherent dynamic in images, but something that is not primarily a representation. Lyotard opposes the reduction of art to linguistic units and he tries to position himself against some linguistic theories that have been very influential and were so indeed when Lyotard wrote this book. “A painting is not something you read, or understand”, he argues, and writes that standing inside the representation one “seeks plastic and libidinal events”, (Lyotard 1971). Lyotard, in the introducing chapter, calls his book a protest against the idea of text. The figural as such proposes, in this sense, that the image in itself is a form that thinks. Philippe Dubois emphasises three orders within the figural: order of (an) event, detail and intensity (Dubois 1998). The figural event in an image is related to and emerges through fulgurance, rupture and presence. The rupture is manifested through the effects of otherness and modification/distortion/alteration. Thus, the figural, as a visual event in an image is a ‘moment in process’, lies between “something unpredictable and necessary” and is the “interplay between these two extremes”, (Dubois 2004, 245). Lyotard stresses three characteristics related to the figural: its opacity, its relationship to the truth and its quality of an immediate expression. The first characteristic refers to the figural as to something in the image that is not manifest, thus that does not belong to the part that is transparent in the represented. The figural, as if stricto sensu, does not really represent something, it ‘disturbs’ the represented; it displaces it, runs over the representation, defies it and thereby defines its aspect. The figural is primarily a visual phenomenon; it is a “constant unpredicted event that is imposed to the work of art through the changing composition”, (Faure 1922/2003, 10). We can also think of the figural as a threshold, as the limit of the visible. This brings us closer to the notion of the “aesthetics of confines” (G. Didi-Huberman 1990). Painters who were working, having in mind the concept of the aesthetics of the limits, were endlessly trying to capture the non-representable, the ‘something’ that was always already eluding, evading the image. Therefore this aesthetics is also called

142

Film and New Art Media Semiotics: On the Figural aesthetics of l’image à venir. In our notion of the figural this is exactly where the figural resurfaces. Even though Didi-Huberman marks that this aesthetics shows us its imperfection and is about approximating something that cannot be attainable and despite the fact that the figural can be accessible (as an act) in its imperfections–in its surpluses, overabundance, excess, absences-these two as if borderline “concepts” overlap in an intensive and extensive register; both try to make us see the invisible (either from the side of the unattainable the question is how do we represent the divine) or from the side of the fleeting, changeable, inconstant and almost capricious (the exclamation mark is-how do we catch and experience the figural). As Olivier Schefer observes, the figural is “the expression of reality in excess, overflowing the discursive order” (Schefer 1999).

Figural and aesthetics It is quite obvious that we could say that the figural is an aesthetic concept we use to discuss dynamic fields in images where the focus of the aesthetic experience is connected to the materiality or plasticity of the images. It may also be related to a breaking up of the relationship between the plastic and the linguistic, a blurring of the categories of text and image. Despite this being an extreme simplification, it is necessary here, and we can consider it as an operational one that might facilitate the conciliation between images and meanings. It is not necessary to go into any conceptual history of the word figural, or figure which is a word with many interesting meanings and uses; the term ‘figure’ denotes a form or shape, often that of the human body in particular. The most obvious one is the Latin origin in fingere, which connotes for example modeling, but a more thorough history of the word would be a completely different discussion. Erich Auerbach has written an interesting genealogy of the word and its biblical context in his short book, called “Figura” (1993). Lyotard also draws attention to movement and rhythmical aspects of the figural, that one can see as a step towards the figural in a context of moving images. In cinema studies as well as in art theory, the concept of the figural has gone through a small revival o v e r the last years. Mostly in Europe with film scholars like Jacques Aumont (1996), Philippe Dubois (1998) and Nicole Brenez (1998) and art historian Georges Didi-Huberman, and recently D.N. Rodowick (“Reading the figural or philosophy after new media”, 2001). Rodowick's book is a good introduction to the figural, and in it he develops also from Lyotard, more political and social aspects of the concept, which explain a significant part of the figural’s appearance in several expressions of cultural life.

Irini Stathi

143

These different figural approaches are mostly about the re-articulation and re-evaluation of certain aesthetic and philosophical aspects that have been with us in different forms throughout most of the history of art and philosophy. There are probably different reasons why the figural is put forward today, but one common denominator is the desire to focus on the image as image (and not the image of something which is the Peircean thought) and what might be some casual dynamic fields in these images. The intention here is not about finding some kind of figural essence or specificity in new screen media, but simply to make some connections that could help develop a more aesthetically oriented discussion on new media art. Many artists and scholars say that what matters with digital technology, from an artistic point of view, is that manipulation, experimentation and access have become easier, more free, and varied. Just to mention one significant example, this is the case with Malcolm Le Grice who has been working with video and digital media since the 1960s and more recently has been writing quite a lot about the relationship between film and new technologies (Le Grice 2001). It is obvious that all possibilities of manipulation and modeling of moving images make the figural an interesting concept, when discussing new media aesthetics. This approach as a counterbalance to a lot of the theoretical writings on new media that have been focusing on aspects of textuality, narration, or to speak with Lev Manovich’s words: language (Manovich, 2001). There are also writings on the interrelationship between art, information and science, for example Stephen Wilson’s book about what he calls “information arts” (Wilson, 2001). Yet another category of writings might be the interrelationship between different media: intermediality: most known is perhaps Bolter’s and Grusin’s book “Remediation” (1999). But in a l l these writings, and many others, we have the feeling that the question of information tends to shadow, and even contaminate, the aesthetic discussion. There are quite a few serious and interesting aesthetic approaches, if we propose the figural as one possible way to take towards an interpretation of the image’s meaning. Despite the many faces of the figural in contemporary theory, it is quite easy to identify some fairly clear methodological aspects. There is one basic starting point in the idea that the figural comes from the images and not from something the analysis itself brings to them. The analysis starts from the images, which tends to get the status of subjects (not only and exclusively objects). For example, the French theoretician Jacques Aumont writes about images that ‘think’, images create problems and constantly ask questions. He writes for example in his book, “A quai pensent les films” (1996), which is the book where he develops the sense

144

Film and New Art Media Semiotics: On the Figural

of figural, that: “The object of film analysis [...] is the problems that the films create”, (Aumont 1996, 150). He also argues that “[...] Le sens d'une œuvre d'image soit dégagé, le plus possible, a partir de l’œuvre ellemême”, (Aumont 1996, 88)1. Nicole Brenez seeks a similar film analysis “with a starting point from the questions that they create”, as she argues (Brenez 1998, 11). And Philippe Dubois promotes the idea that the figural operates “[...] selon les modes associatifs ouverts et multiples de la matière visuelle en elle-même”, (Dubois 1998, 270)2. Nicole Brenez starts her discussion on the figural by saying that it is difficult, if not totally impossible, to have a method for figural analysis. In a strict sense, aspects related to the figural are perhaps too vague and too ambiguous to be organised in a method, but that is a methodological standpoint, and she, in her text, by establishing some principles for figural analysis is creating a method for that purpose. Maybe it is more appropriate to call it simply a figural approach. Without any further development, this approach seems to share many ideas with a contemporary version of phenomenology (for example through the attentiveness to the images and the kind of knowledge when looking at them).

Seeing the figural At this point we will consider some of the many aspects of Jean Epstein’s writings and films that could be mentioned in relation to the figural. What concerns our analysis is perhaps most evident in the movie, “Le Tempestaire” (The Storm Healer, 1947), where he draws to their extreme his ideas on temporality and speed variations. This film is the most obvious example of what Epstein calls a “temporal perspective” (Epstein 1946, 16). In the film the sea outside Bretagne is continually filmed in slow motion and the sound is equally manipulated in order to create an effect of ‘incorrect’. Epstein writes for example: “Entre l'eau et la glace, entre le liquide et le solide, il se crée une matière nouvel1e, un océan de mouvements visqueux [...]”3, (Epstein 1975, 45). The use of slow motion here is a way to work and present the very figural aspects of the film medium (Dubois 1998, 267-8). The speed variations of the filmstrip become a way to visualise the film matter itself, 1

“[...] meaning of a work should as much as possible be sought in the work itself.” 2 “[…] by the associative, open and multiple modes that exist in the visual matter itself.” 3 “Between water and ice, between the liquid and the solid, a new matter is created, an ocean of viscous movements [...]”

Irini Stathi

145

which means, to visualise the real flow of time, which is constantly imperceptible in real life. Epstein’s perspective is a temporal one and more precisely it is about different effects of temporal elasticity. This is an elasticity of space and time that in the real flow of life is impossible to capture. For Epstein, film is not primarily a way to reproduce movement, but to create a new movement based on variations of the image matter. This movement is the visualisation of an invisible dimension of reality. Generally, in Epstein’s films and writings, from the 1 9 20s and on, there are several other techniques and aspects related to the figural: multiple exposures, dissolves, the dissolution and stretching out of movement, kaleidoscopic images; all aspects related to the technology which is central to Epstein’s thought, writings and the art of film. In this context, technology constitutes an aesthetic approach based on the figural and the most important thing is materiality itself, and even if there are technological conditions behind, for example slow motion and other special effects, it is the image event itself that relates to the figural. Epstein’s most famous book called “L’intelligence d’une machine” [The intelligence of a machine, 1946], is not about thinking of machines in the sense of a computer (a means able to automatically compose not realistic movement), but more about creating something new with technology, to let machines ‘think’ and, to some extent, create their own forms and figures. This is probably one reason why Epstein’s thought is interesting in relation to new media art. The figural is perhaps a process, something primarily described in terms of a process; thus it is not an object. It has to do with something that ‘happens’ to the image as image. This is linked to the idea of/on the image as presence, not as form or representation, and this presence is nothing else than the presence of the material and different effects due to the plasticity and the materiality of moving images. This is only more interesting since moving images are often regarded as immaterial, as a flow which is impossible to touch. Light, time and sound are also often thought of as immaterial but through manipulations they move towards a more obvious presence of their material, towards a tactile or haptic dimension. This is of course easily seen and represented in a considerable number of experimental film and video. The most interesting example is Bill Viola’s work. These aspects also exist most significantly in nonexperimental films or art works, and often an interesting tension is registered there. Except Epstein, whose films are mostly quite popular narratives, we can think of the films of Wong Kar-Wai, for example “Ashes of Time” (1994).

146

Film and New Art Media Semiotics: On the Figural

Figural and new screen media To develop the figural in relation to new media art, there are possibly at least two major ways to follow: the first one is the montage where overlapping and different image-relations become more flexible and potentially dynamic, with interesting manipulations of time and space. The second is the facilitation of working with matter, working with pixels, the radically increased degree of manipulation (especially in relation to moving images). The special effect in motion pictures and animations that changes (or morphs) one image into another through a seamless transition (morphings), for example, could lead to discussions not only of timematter but also of space-matter and it is clear that experimentation and manipulation becomes easier, it becomes easier to model the image: a fact that has, as we know, influenced the idea on an indexical image of reality. The idea of indexicality has lost a great deal of its importance and, the figural could be a mode that in a certain sense assumes the role to promote some of this importance. At this point we consider significant to hint at one example from new media art that is quite explicitly about materiality and screen plasticity. It comes from the “artintact” series and aptly titled “The Subdivision of Electric Light” made by Perry Hoberman in 1996. The work is made up of a series of screens and projections where the stability and the two-dimensionality of the screen are continually manipulated. It is a work that testifies about the need or desire to integrate the projection in a threedimensional space. And the tool to do it is the subject matter of the screen itself. There are screens as objects (where the projection is made on different objects), screens as rooms (the projection is made on walls in a room), moving screens (the projection is made on moving objects) and screens as crease or fold (the projection is made on a creased screen, like an accordion-screen). The different types of screens often go into each other but the interesting thing is that Hoberman works with the figural because he works with the very materiality of the screen and with plastic aspects. In that sense he is actually representing (here in the sense of materialising though screening) the figural. A figural approach could be a way to drive the discourse on new media and new technologies from information to aesthetics. If collage or montage is the basic 20th century technique for artistic expression, as Clement Greenberg (1971) puts it, it seems that new media is the prolongation and reinforcement of these practices. A higher degree of manipulation and easier experimentation can also bring us closer to the details, to the materials of the media, and here the figural is an aesthetic

Irini Stathi

147

concept that can provide an interesting parallel discussion and/or development of certain aspects. There is a tendency to mention examples that work with time and light or the relation text/image as subject matter, but there are other possibilities, for example with colour and sound that would also fit into what we might call aspects of materiality in the digital realm of immateriality. Finally, the figural could be a concept and an approach that can be useful in order to discuss aspects of aesthetic displacements related to experimentation and manipulation of moving images in film and new media art. Even if we give a very simplified and operational definition of the figural, this as a way of reducing the concept to one specific meaning, that would be to violate its dynamic and potentiality. Instead, if we consider this approach as a free variation to test some conceptual qualities, then we prepare and let the concept meet the images, to confront them in some sense. It is not unproblematic to draw a line from conceptual ambiguity and mobility to complex and ambiguous images, but this could be one very interesting way to develop the figural as an aesthetic concept in relation to both film and new screen media.

Conclusion Semiotic research has become increasingly aware of the intertextual and intermedial statuses of film, moving images and digital images as well, recognising that the making of meaning depends on, among other factors, processes of multimediality and intermediality, but also of new figurativity, which imposes a new way to consider the digital images from the point of view of the figural as approached before. Ernest W.B. HessLüttich, tackles this problem by analysing multimedia semioses, the function of transfer processes and code changes in the mental activities of the spectator. But the heterogeneity and complexity of multimodal coded texts require probably new semiotic concepts and probable new methods of research in order to delineate the role of the figural in the screen-based arts’ signification.

148

Film and New Art Media Semiotics: On the Figural

Bibliography Auerbach, Erich. Figura. Paris: Editions Belin, 1993. Aubral, François and Dominique Chateau. (Eds) Figure, figural. Paris: L'Harmattan, 1999. Aumont, Jacques. A quoi pensent les films. Paris: Nouvelles Editions Seguier, 1996. Bolter, J. David and Richard Grusin. Remediation: Understanding New Media. Cambridge: MIT Press, 1999. Brenez, Nicole. De la figure en général et du corps en particulier. Paris: De Boeck, 1998. Didi-Huberman, Georges. Fra Angelico - Dissemblance et figuration. Paris: Flammarion, 1990. Dubois, Philippe. “La tempête et la matière-temps, ou le sublime et le figural dans l’œuvre de Jean Epstein.” In Jean Epstein - cinéaste, poète, philosophe, edited by Jacques Aumont, 267-323. Paris: Cinémathèque Française, 1998. —. (1999). “L’écriture figurale dans le cinéma muet des années 20." In Figure, Figural, edited by Francois Aubral and Dominique Chateau 243-252. Paris: L’Harmattan, 1999. Eco, Umberto. “Towards a Semiotic Inquiry into the Television Message.” In Communication Studies: A Reader, edited by John Corner and Jeremy Hawthorn, 131–149. London: Arnold, 1980 Eisenstein, Sergei M. Eisenstein: Selected Writings. Bloomington: Indiana University Press, 1996. Epstein, Jean. “La Feerie réelle.” Spectateur, 21, January 1947, in Ecrits sur le cinéma 1921-1953, Tome II : 1946-1953, Paris: Seghers, 1975, p. 45. —. L’Intelligence d’une machine. Paris: Editions J. Melot, 1946. —. Le Tempestaire (The Storm Healer), DVD. Directed by Jean Epstein. France: France Illustration-Film Magazine, 1947. Faure, Edgar. “De la cinéplastique.” in L’Arbre d’éden, Crès. Online: http://classiques.uqac.ca/classiques/Faure_Elie/fonction_cinema/cinema plastique/Faure_cineplastique.pdf, Bibliothèque Paul-Émile-Boulet de l’Université du Québec à Chicoutimi, 2003. Garroni, Emilio. Semiotica ed estetica: L'eterogeneità del linguaggio e il linguaggio cinematografico. Bari: Laterza, 1968. Greenberg, Clement. “Collage.” In Art and culture: critical essays, edited by Clement Greenberg, 70-83. Boston: Beacon Press, 1971. Hess-Lüttich, Ernest W.B. “Codes, Kodes, Poly-Codes.” In Semiohistory and the Media. Linear and Holistic Structures in Various Sign Systems,

Irini Stathi

149

edited by Ernest W.B. Hess-Lüttich and Jürgen E. Müller, 111-122. Tübingen: Gunter Narr, 1994. Hoberman, Perry. The Sub-Division of the Electric Light, from artintact 3, (ZKM Zentrum für Kunst und Medientechnologie Karlsruhe 1996). Le Grice, Mlcolm. Experimental Cinema in the Digital Age. London: BFI, 2001. Lotman, Juri. Semiotics of Cinema. Ann Arbor: University of Michigan Press, 1976. Lyotard, Jean-François. Discours, figure. Paris: Klincksieck, 1971. Manovich, Lev. The Language of New Media. Cambridge: MIT Press, 2001. May, Michael. "A Semiotic Framework for the Semantics of Digital Multimedia Learning Objects". 14th International Conference

on Image Analysis and Processing Workshops, 2007. ICIAPW 2007. 33-38. Los Alamitos, CA: IEEE Computer Society Press. 2007. Möller Nass, Karl Dietmar. Filmsprache: Eine kritische Theoriegeschichte. Münster: MAKS, 1986. Müller, Jürgen E. (ed.) Towards a Pragmatics of the Audiovisual. 2 vols. Münster: Nodus, 1995. Odin, Roger. Cinéma et production de sens. Paris: Armand Colin, 1990. Rodowick, David Norman. Reading the Figural, or, Philosophy after the New Media. Durham & London: Duke Univ. Press. 2001. Schefer, Olivier. “Qu’est-ce que le figural?” In Figure, figural. Critique, n. 630, edited by F. Aubral and D. Chateau, 912 -925.Paris: L'Harmattan, 1999. Wilson, Stephen. Information Arts. Intersections of art, science and technology. Cambridge: MIT Press, 2001. Wong Kar-Wai (on a Louis Cha novel). Ashes of Time, DVD. Directed by Wong Kar-Wai. Hong-Kong: Jet Tone Production, 1994.

KINETIC TYPOGRAPHY: A SEMIOTIC EXPLORATION THEO VAN LEEUWEN AND EMILIA DJONOV

1. Kinetic typography has existed for some time, for instance in film titles and television commercials. Early observers, such as typography historian Beatrice Warde, immediately grasped its significance. After watching an animation film by Norman McLaren projected on the gigantic Animated Electric Screen in Times Square, New York, in 1961, she commented (quoted in Bellantoni and Woolman, 2000: 5): I saw two Egyptian A’s walking off arm in arm with the unmistakable swagger of a music-hall comedy team. I saw base serifs pulled together as if by ballet shoes, so that the letters tripped off literally sur les pointes. I saw words changing their mind about how they should look even more swiftly than a woman before her milliner’s mirror. After forty centuries of the necessarily static alphabet, I saw what its members could do in the fourth dimension of Time, ‘flux’, movement.

More recently kinetic typography has moved into a new stage. It is no longer restricted to professional designers and animators, but has become available to anyone with access to a computer, potentially transforming everyday forms of writing. A fundamental change in the semiotic landscape has taken place in front of our eyes. And as always, the semiotician asks: ‘What is the meaning potential of this new semiotic mode?’, ‘What can we ‘say’ with it, and how?’, and ‘How does this meaning potential come about?’.

2. The literature on the subject is still emerging. Richly illustrated books by design academics (e.g. Bellantoni and Woolman, 2000) trace the history of kinetic typography and show the work of key designers, captioning the

Theo van Leeuwen and Emilia Djonov

151

examples with brief descriptions and indications of their meaning potential, as when a moving logo is said to “capture the pace and energy of a younger age group in vibrantly coloured actions that use water paint, fire, lens flares and electric sparks”, (ibid: 35). Research papers begin to map the new territory. Kinetic typography is likely to be used in email, digitally based expressive communication such as books, poetry or essays on CD-ROM, agent representation (using kinetic typography to communicate personality), information design on pagers and PDS, and motion graphics (Ford et al., 1997).

It will be able to “convey a speaker’s tone of voice” (Forlizzi et al., 2003: 377). Pitch, for instance, can be transcribed by upward and downward motion, loudness by increasing size, weight and sometimes colour or contrast, and tempo by modifying letter tracking and spatial stretching to stretch time (ibid: 380). And it will be able to convey “qualities of character, and affective (emotional) qualities of texts” and “explicitly direct or manipulate the attention of the viewer” (ibid: 377). In ‘kineticons’ the pixels of icons rotate, stretch, etc., in ways that “relate to real world events, objects or actions to convey meaning” (Harrison et al., 2011: 1999) and are understood on the basis of our “innate understanding of how the world operates physically. Metaphors such as mass, rigidity and inertia are readily portable to digital domains” (ibid: 2001). In technical literature software, designers describe systems and interfaces for kinetic typography programs (e.g. Möhler et al., 2004; Minakuchi and Tanaka, 2005; Uekita et al., 2000) in ways discussed below.

3. The move to introduce kinetic typography, first in artistic experiment, and more recently in everyday writing software such as PowerPoint is not, it seems, driven by immediate practical necessities, but by broader cultural trends that manifest themselves also in other domains. Four can be discerned.

Pictorialization Early writing used simplified pictures. More recently such pictures have returned, mingling with letterforms. In the early 20th century, Otto von Neurath began to develop his international picture language, Isotype, and

152

Kinetic Typography: A Semiotic Exploration

Futurist poets like Marinetti began to mix expressive letterforms and simple pictures in their ‘concrete poetry’. In today’s writing practices images and letterforms happily mingle, e.g. ‘emoticons’ in emails, and letters and icons in logos. Ten years ago, McLean, in the “Thames and Hudson Manual of Typography” (2000: 56), could still write “When rope is coiled to form the word ‘Ship Ahoy’ or branches writhe into ‘Our Trees’, that is illustration, not calligraphy”, but in today’s kinetic typography illustration is common. The word ‘circular’ may go round and round. The word ‘long’ may be stretched. The word ‘loud’ may expand to become big and bold. And “adding a small vibration to the word ‘help’ can convey a sense of fear”, (Ford et al., 1997: 269).

Informalization Many writers see kinetic typography as restoring the expressiveness of speech that comes from tempo, rhythm, intonation and voice quality. Möhler et al. (2004: 1505) call it “a technology to enhance text with speech-like expressiveness”. Writing is to lose its lofty formality and its impersonal tone, and to become more like everyday informal speech. Historians have documented how, in the 1920s, writers of radio commercials began to write lines like “Thank you (GASP OF RELIEF). My gawd, I’m glad that’s over” (Barnouw, 1966: 168) and radio speakers were encouraged to speak in a low-key, conversational manner, introducing deliberate hesitations, repetitions and slips in order to sound more ‘natural’, all of which were meticulously scripted (Cardiff, 1980: 31). Even Joseph Goebbels (quoted in Leitner, 1980:26) recommended that radio speakers should sound “like the listener’s best friend” and use local dialects. Soon, such informalities became more common in print as well. This is a significant shift. Writing made it possible to separate words from their speaker, to make the text, rather than the speaker, ‘say it’. Much of the authority of the written word rested on this. Today we are moving to forms of writing, which, like informal speech, rely on non-verbal information and shared context for their completion and full understanding (cf. Joos, 1967).

Emotivization Contemporary culture, following a trend begun in advertising, seeks to invest formerly formal and impersonal genres of writing with affect, to make it appeal to our emotions, so that it will be ‘owned’ and believed and

Theo van Leeuwen and Emilia Djonov

153

felt, rather than just understood. Yet the computer screen impoverishes writing’s affective qualities. Writing loses the individuality of handwriting, with its strong traces of personality and mood. Print loses the smell and texture of books and paper. All this must be compensated for by what can be visually and aurally transmitted – colour, visual texture… and movement.

Dynamicization The adjective ‘dynamic’ was already a keyword in the Futurist and Constructivist art movements of the 1920s, and continues to be a repository of positive values today. Whatever is still static must become dynamic, typography included – occurring in real time, like speech, evermoving and ever-changing. As Lewis Mumford (1939) showed more than 70 years ago, innovation rarely results directly from technological inventions. It is set into motion by a period of ‘cultural preparation’, taking the form of often apparently peripheral ideas, science fiction fantasies purveyed in fiction, toys and games. Long before rigid timing was introduced into everyday life, the clock was already an object of fascination. Long before film was invented, people were fascinated by toys that created the illusion of moving images, from flipcharts to complex kinetoscopes. Long before software such as Adobe After Effects made kinetic typography accessible to every computer user, artists tried it out in experimental films that often required painstakingly slow manual work. The drivers behind such innovations may not always be consciously understood, but the direction in which they are moving is irreversible and eventually they become an integral part of the fabric of social life. Until recently, animation was a rather peripheral cinematic genre. Today, it has moved centre stage, not just in cinema, but in all visual and audiovisual communication.

4. Kinetic typography developed in the context of film titles, though there have been other early forms such as the Animated Electric Screen on Times Square which so impressed Beatrice Warde, a 720-square-feet signboard made up of thousand of light bulbs showing the news ‘coming in as it happened’ – just as today at the bottom of the screen in Bloomberg business news and similar types of TV news. In early film titles the words were not usually animated. Movement came from ominous shadows or light flashes cast over the title and credits, especially in horror films. In the original “King Kong” in 1933, large

154

Kinetic Typography: A Semiotic Exploration

leaves formed the wipes between the titles, as though the viewer was wading through a dense jungle. In the 1950s and 1960s title designers such as Norman McLaren, Saul Bass and Pablo Ferro began to animate the titles themselves. In Norman McLaren’s animated titles and experimental films of the 1940s and 1950s letters sprung to life and moved around until they spelled words and names. Saul Bass’ famous title sequence for Hitchcock’s “Psycho” mixed abstract graphic elements and letterforms. Grey bars aligned to suggest a curtain, raised to reveal the names of the stars, or the shower in which Janet Leigh will be murdered, or the bars behind which Norman Bates, Anthony Perkins’ character, will eventually be imprisoned. The title and some of the credits split horizontally into two halves, which were disaligned to suggest the split personality of Norman Bates, then aligned, then split again. Such titles used kinetic typography to tell a story and began to create the elements of a ‘language’ of kinetic typography. In this early stage of the development of that language of kinetic design, it was a growing collection of individual inventions, usually based on metaphor. Serifs could suggest shoes, as in Warde’s example, because in common with shoes, they are elongated horizontals on which something stands. Vibration could suggest fear because when we are afraid we tremble. The splitting of words could suggest the splitting of the mind in schizophrenia. Here, meaning came directly from the material properties of the forms and movements the designer worked with in the same way that sculptors may see the potential for sculpting a human figure in the natural form of a rock or rusted metal. Once discovered, such inventions became part of a developing lexicon of clichés. ‘Dictionaries’ of visual language list them alphabetically and illustrate them with examples. In Thompson and Davenport (1982:55), for instance, we read under the heading “clockwork”: Mechanically motivated, the childhood fascination of winding up clockwork toys with a KEY is shared by designers. Sometimes the symbolism is concerned with the manipulation of people.

Four pictures illustrate how the clockwork motif has been used in book covers, advertisements, posters, etc. Jim Martin (1992) calls this kind of ‘language’ “lexese” – a language which has a vocabulary but no grammar, a language which expresses everything through the equivalent of word meaning, without being able to combine these word-level meanings into larger wholes to form the equivalent of phrases and clauses. It is the kind of language Roland

Theo van Leeuwen and Emilia Djonov

155

Barthes referred to when he wrote about what he called the “posing of objects” (1977: 23). The objects are accepted inducers of ideas, or, in a more obscure way, veritable symbols. Such objects constitute excellent elements of signification: on the one hand they are discontinuous and complete in themselves…while on the other they refer to clear familiar signifieds. They are thus the elements of a veritable lexicon.

What decides whether a particular semiotic mode will be a ‘lexese’, an inventory of individual and separate signs, or a ‘language’ with a grammar and a vocabulary? Some years ago, in London, Theo van Leeuwen conducted a series of seminars with PhD students on the subject of smell. They read an article by Brian Eno, who argued that smells are, and can only be, unique individual experiences, often associated with specific, individual memories, and evaluated in highly subjective ways. There can be no ‘grammar’. Smells will forever defeat the semiotician’s quest for codification. Then, they had a guest lecture by an aromatherapist who explained how aromas are put together. There are about 50 basic smells, she said. These combine to form complex smells consisting of three elements, a ‘head’, a ‘body’ and a ‘base’. Basic smells have to have specific qualities (e.g. degrees of volatility) to be able to function as ‘head’ or ‘body’ or ‘base’. The complex smells, in turn, are combined to form aromas. Everyone was stunned. That is how language is described. An average of 50 phonemes combine to form words according to certain rules, and these words, in turn, combine to form sentences, complete messages. The same semiotic mode can apparently be a ‘lexese’ for the consumer and a language with a grammar for the producer. But when consumers become producers, as is the case with kinetic typography now that it is available to all computer users, things change and a language may develop that is not restricted to specialists.

5. In Reading Images: The Grammar of Visual Design, Gunther Kress and Theo van Leeuwen (2006 [1996]) devised a ‘grammar’ of images. Until then, the language of images had been described as a ‘lexese’ by most semioticians and art historians. More recently, Leao (2012), in her work on film titles, has been developing a ‘grammar’ of animated movement. The principle behind such ‘grammars’ can be explained by means of an example. Linguists have generally made a distinction between ‘stative’ and ‘dynamic’ verbs (see e.g. Quirk et al, 1972: 39), though not always in

156

Kinetic Typography: A Semiotic Exploration

the same way. Halliday’s systemic-functional grammar (Halliday & Matthiessen, 2004), for instance, distinguishes between ‘processes’ which signify some kind of permanent or quasi-permanent state (usually realised by verbs such as be and have) and ‘processes’ which signify some kind of action, whether material (as realised by verbs such as jump, break, etc), verbal (as realised by verbs such as say, ask etc) or mental (as realised by verbs such as know, hear, fear etc). These different processes combine with different kinds of ‘participants’ (realised by names, nouns or noun phrases) to form clauses or complete messages. Quite similar content can either be formulated as a ‘relational’ (stative) or a ‘mental’ (dynamic) process. Consider, for instance: He Carrier

is Relational process

curious Attribute

He Senser

wondered Mental process

what it was Phenomenon

Reading Images suggested that the distinction between ‘stative’ (or ‘conceptual’, as it was called there) and ‘dynamic’ (or ‘narrative’) can also be realised by the compositional structure of images (and other visuals, for instance diagrams). In other words, Kress and Van Leeuwen argued that the structures described in Halliday’s grammar are not unique to the English language, but underlie culture as a whole, and can in principle also be realised by other semiotic modes (and other languages). They are semantic structures. What is unique to language is only the way they are realised – through nouns and verbs. To make this point, Gunther Kress and Theo van Leeuwen used a page from an Australian primary school social studies textbook, which contained two pictures, printed side by side. The left picture showed three Aboriginal artefacts, an axe, a basket and a wooden sword, symmetrically arranged against a blank background. Even though in reality they differ in size, the picture gives them the same size and the same orientation towards the horizontal and vertical axis. In this way it suggests that the axe, the basket and the wooden sword belong to the same category. It is a ‘conceptual’ image. It classifies the three objects, indicating a supposedly permanent characteristic of them (although it does not indicate the category to which they all belong – the caption simply reads “Stone axe, bark basket and wooden sword”). Such ‘classificatory’ pictures can readily be found elsewhere. Advertisements, for instance, often have symmetrical,

Theo van Leeuwen and Emilia Djonov

157

or almost symmetrical, arrangements of the range of products of a brand, or the variety of consumers who use it.

Fig. 3-9: A conceptual and a narrative image

The right hand picture is an early 19th century engraving showing two ‘British’ (as they are called in the caption) pointing their guns at a group of Aboriginals seated around a fire. ‘The British’ and the group of Aboriginals, the most salient ‘volumes’ (Arnheim, 1982: 154), realise the ‘participants’. They are connected to each other by means of a vector (ibid), formed by their outstretched arms and the guns, which realizes a narrative visual ‘process’. This results in a structure, which could have been realised linguistically by nouns and verbs, but here it is realised visually by volumes and vectors. It is, of course, striking that Aboriginal technology should be depicted as ‘static’ and British technology (the guns) as ‘dynamic’. A similar distinction is made in Leao’s (2013) work on animated movement. She recognises two broad types of processes, those involving a change of state or a change of identity, and those involving the displacement of participants or parts thereof. In the former, participants can appear or disappear, change colour or size, or transform into some other kind of participant (e.g. a letter can change into a picture, or vice versa). These are, therefore, akin to conceptual processes, though of course in a dynamic way. The latter expresses actions through the physical displacement of participants or parts thereof, actions like falling, bouncing or pinning, or transactions involving two participants, with one participant bouncing off another or squashing another, or fusing with another, and are therefore similar to narrative processes. A presentation created with the software Keynote by one of Theo van Leeuwen’s students, for example, had curly brackets open up like sliding doors to reveal text which plunged down, raising a little dust cloud before settling in position.

158

Kinetic Typography: A Semiotic Exploration

Leao (2013) distinguishes many sub-categories of both types of process, showing that kinetic typography, or rather animated movement generally, can be described as a language with a lexicon and a grammar. Kinetic typography is, therefore, not just an emotional overlay on words, but a means of expression in its own right, and just one aspect of a language of animated movement in which the participants can be letters or parts of letters, words, sentences, paragraphs, pictures or parts of pictures, or abstract graphic elements, or some combination of all of these, and in which these participants are related to each other to form meaningful wholes through animated movement.

6. Describing the semiotics of animated movement in this way effectively changes a ‘lexese’, a repertory of creative inventions that have become “accepted inducers of ideas” to use Barthes’ (1977) term, into a language with a grammar and a lexicon. But semioticians are not the only ones engaged in turning kinetic typography into a language. Software designers, too, lay down ‘what can be said’ with kinetic typography and how. For Möhler et al. (2004: 1505), for instance, kinetic typography serves two communicative functions, “expressing a desired emotion” and “emphasizing words that are important for the user”. Users enter text, which is automatically broken down into lines, and then choose which words they want to emphasise and which ‘animation scheme’ (e.g. ‘hesitant’, ‘assertive’) they want to apply, but the software decides how ‘hesitancy’, ‘assertiveness’ etc. will be expressed: Users should not need to think about how to achieve a coherent aesthetic output. This is the task of the visual framework that defines the overall look, but at the same time enables the user to choose from a large variety of animations although in some case it is beneficial to let the user manipulate certain aspects of the visual appearance (ibid: 1506).

Other software designers take a different road. The approach of Minakuchi et al. (2005: 222), for instance, is closer to Leao’s, distinguishing two broad types of movement, separate “physical motion that mimics natural phenomena” such as ‘falling’ and ‘bouncing’ and ‘physiological phenomena’ that express emotions such as ‘turning red’ and ‘shedding tears’. In this kind of work the semiotics of kinetic typography is no longer a theoretical proposition. It becomes a reality which in the case of ubiquitous software such as Adobe After Effects and Flash will deeply influence what users actually say and do with kinetic typography, as a scan

Theo van Leeuwen and Emilia Djonov

159

of examples on the Web will quickly verify. In an interview with Lawrie Hunter, Ellen Lupton (in Hunter, 2006: 135) said that “we do have a language of vision now, but it was created by corporate software designers”. We could paraphrase: “We do have a language of kinetic typography now, but it was created by corporate software designers”. Some writers are concerned that this constrains the creativity of earlier, manual forms of kinetic typography. In an article on writing in Flash, Sorapure (2006: 423) comments: Software that does not allow access to the underlying code is more restrictive and positions writers more narrowly in line with what corporate planners and programmers intend.

But writing has always had an underlying code, and writers have always had to follow authoritative formats. The difference is that the rules are now imposed by software, rather than by school teachers, subeditors, house styles and so on. On the other side, writing has also always allowed creative and poetic expression, despite the confines of its rules of spelling and grammar. Contemporary software, likewise, can be used creatively, as David Byrne has shown with the much maligned medium of PowerPoint in an exhibition of works entirely created with PowerPoint and subsequently turned into a book and DVD, titled Envisioning Emotional Epistemological Information (Byrne, 2003). In one part of this work, titled “Architectures of Comparison”, arrows wander meaninglessly across the screen, accompanied by slow, dreamy music. In terms of visual grammar, these arrows are processes. But here, they do not connect participants. They come from nowhere and go nowhere, thus becoming an end in themselves. As Byrne explained (2003): Goal oriented behaviour is like sleepwalking. It is easy and purposeful, but what is its purpose? Its purpose is itself.

In another work, titled “Self-Exemplification”, time words, from ‘seconds’ and ‘minutes’ to ‘eternity’ and ‘forever’, rise slowly from the bottom of the screen, stopping in the centre of the screen where they begin to overlap each other until at last they form the indeterminate chaos which was once dissected and ordered by the very words that constitute it. Here, Byrne uses the language of kinetic typography within PowerPoint, a language that has often been decried as trite (e.g. by Tufte, 2003), to create insightful ideas and powerful poetry, giving us a glimpse of a future of creative writing which has kinetic typography at its very centre.

160

Kinetic Typography: A Semiotic Exploration

Acknowledgement The paper is part of a larger project, “Towards a Social Theory of Semiotic Technology: Exploring PowerPoint’s Design and its Use in Higher Education and Corporate Settings”, which is supported through an Australian Research Council Discovery Grant.

Bibliography Arnheim, R. The Power of the Center. Berkeley and Los Angeles: University of California Press, 1982. Barnouw, E. A Tower in Babel:A History of Broadcasting in the United States to 1933. New York: Oxford University Press, 1966. Barthes, R. Image-Music-Text. London: Fontana, 1977. Bellantoni, J and Woolman, M. Type in motion: Innovations in digital graphics. London: Thames and Hudson, 2000. Byrne, D. Envisioning Emotional Epistemological Information. New York: Steidl Pace-MacGill, 2003. Cardiff, D. The Serious and the Popular: Aspects of the evolution of style in the radio talk 1928–1939. Media, Culture and Society 2(1) (1980): 29-47. Chao, C. M., & Maeda, J. Concrete programming paradigm for kinetic typography. Proceedings of the 1997 IEEE Symposium on Visual Languages, (1997): 446-447. Ford, S., Forlizzi, J., & Ishizaki, S. Kinetic typography: Issues in timebased extended abstracts. Proceedings of CHI97 Conference Extended Abstracts, (1997): 269-270. Forlizzi, J., Lee, J., & Hudson, S. The Kinedit System: Affective Messages Using Dynamic Texts. Proceedings of the CHI 2003 Conference on Human Factors in Computing Systems (2003): 377-384. Halliday, M. A. K., & Matthiessen, C. M. I. M. An Introduction to Functional Grammar (3rd Edition). London: Arnold, 2004. Harrison, C., Hsieh, G., Willis, K. D. D., Forlizzi, J., & Hudson, S. E. Kineticons: using iconographic motion in graphical user interface design. Proceedings of the 2011 annual conference on Human factors in computing systems, (2011): 1999-2008. Hunter, L. ‘Critical form as everyday practice: An interview with Ellen Lupton’, Information Design Journal 14(2), (2006): 130-138. Joos, M. The Five Clocks of Language. New York: Harcourt, Brace & World, 1967.

Theo van Leeuwen and Emilia Djonov

161

Kress, G. and van Leeuwen, T. Reading Images – The Grammar of Visual Design. London: Routledge, 2006 [1996]. Leao, G. Movement in film titles: An analytical approach. Unpublished PhD, University of Technology, Sydney, Sydney, 2013. Leitner, G. BBC English and Deutsche Rundfunksprache: A comparative and historical analysis of the language on the radio. International Journal of the Sociology of Language 26, (1980): 75-100. Martin, J.R. English Text: System and structure. Amsterdam: John Benjamins, 1992 Minakuchi, M., & Tanaka, K. Automatic kinetic typography composer. Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology, (2005): 221-224. Möhler, G., Osen, M., & Harrikari, H. A user interface framework for kinetic typography-enabled messaging applications. CHI '04 extended abstracts on Human factors in computing systems, (2004): 1505-1508. Mumford, L. Technics and Civilization. London: Routledge and Kegan Paul, 1939. Sales, R. (Ed.). Alphabet 1964. International Annual of Letterforms. Birmingham: The Kynoch Press, 1964. Quirk, R, Greenbaum, S., Leech, G. and Svartvik, J. A Grammar of Contemporary English. London: Longman, 1972. Sorapure, M. Text, image, code, comment: Writing in Flash. Computers and Composition, 23(4), (2006): 412-429. Thompson, P. and Davenport, P. The Dictionary of Visual Language. Harmondsworth: Penguin, 1982 Tufte, E. The Cognitive Style of PowerPoint (2nd ed) Cheshire, CT: Graphics Press, 2003. Uekita, Y., Sakamoto, J., & Furukata, M. The method of kinetic typography communication. Proceedings of the 2000 IEEE International Conference on Systems, Man, and Cybernetics, 1, (2000): 432-436. Yeo, Z., & Hudson, S. E. KTE2: an engine for kinetic typography. Proceedings of the 27th international conference extended abstracts on Human factors in computing systems, (2009): 3413-3418.



CHAPTER FOUR: PEDAGOGY OF VISUAL COMMUNICATION





100 THINGS: A PROCESS FOR FOUNDATION IN THEORY AND PRACTICE LAW ALSOBROOK

Visual communication comes about through the manipulation of visible signs to create designed messages This is perhaps one of the basic tenets and core philosophies for the Graphic Design department of Virginia Commonwealth University in Qatar (VCUQatar). But to design messages is actually much more complicated than stated in the sentiment of this sentence. One must learn a great deal about signs and signifiers in order to use them well to create an effective visual message. As most students discover, this is not an easy task. While most students believe they know how to think, learning to design is learning to think about thinking and its relationship to communication. To facilitate this type of metacognition, many design education programmes teach semiotics. Through the investigation of signs, students tap into the realm of constructed meaning, which in turn assists them in formulating messages and visually designing their form. This is a challenge that requires students to observe many facets of their thinking and apply this learning to their own design process. Learning to communicate visually is challenging because in many ways it is like learning a second language; graphic design has its own finicky vocabulary and strange terminology. For many of our students at VCUQatar this will, in effect, be their third language, as many of our students already know Arabic, and must prove proficiency in English before joining our programme. To establish a foundation and provide models to build upon, we rely upon “Visible Signs” by David Crow. We find this book to be most in keeping with the departmental philosophy, one that employs fitting visual examples as well as coherent, easily understood information. At present, we also couple it with “How to Use Images” by Lindsey Marshall and Lester Meacham. We find the combination helpful



Law Alsobrook

165

in providing the basic theoretical framework for our class. This essay articulates the way we, graphic design educators at VCUQatar, try to teach visual communication through the manipulation of visible signs in a project called 100 Things.

Fig. 4-1: 100 Things (photo by author)

100 Things “100 Things” is an introductory project we use during the sophomore year of graphic design education at VCUQatar. The project is implemented within the class Design Methods and Processes, a class structured to deliver the basics of visual communication, visual semiotics and process. This class also furnishes a structural framework for the design process we promote. It is an intense project divided into four phases; each phase is tailored to cover various aspects of the design process while delineating critical features along the way using salient vocabulary within its application to design and design thinking. In essence, “100 Things” provides the application of theory in action. “100 Things” requires approximately seven weeks to complete and is probably the largest project most students will tackle up to this point in their educational career. It is designed to overwhelm both mentally and physically in order to push boundaries and explore thinking. Furthermore, it is a slow project — one that requires directed attention to the process of



166

100 Things: A Process for Foundation in Theory and Practice

individual thinking and how that affects message making. As in nearly any design programme, the VCUQatar design department feels it is paramount that students receive both theoretical models coupled with practical application; “100 Things” is a problem that attempts to constantly bridge these two arenas in a dynamic exchange of thinking and making. Initially conceived by John Nettleton and Paul Mazuka at Oregon State University, this project has been one that we have developed into a thorough, yet basic, understanding of visual semiotics and its application. The power of this project lies in its introduction to design thinking via the design process, with focus applied to each individual student. Process is a vital ingredient to design and in how meaning is made because it functions as both a laboratory of experimentation and the testing ground for these ideas. The approach this project employs creates a ‘slowing down’ of the students’ thinking and making, so they can begin to see what it means to communicate and how, by breaking down the design process into incremental states. This ‘slowness’ puts the student in a state of awareness that allows them to observe communication and how it goes from thought to action, idea to execution. The phases we employ correspond to the model we instill as our general design process: Phase I – Point of Departure — divergence Phase II – Editing & Refining — convergence Phase III – Making Meaning — verbal/visual equations Phase IV – Production — pulling it all together Equally important, though not technically an independent phase of the project, is the process book, a critical tool in both the teaching and learning aspects of any project we undertake at VCUQatar. We pay particular attention to the creation and development of the process book during this project because it serves as a template for all subsequent process books students encounter throughout their time at university.

Phase I – Point of Departure We begin the project, “100 Things”, by requesting that students bring in a tool that fits in their hand and performs one (perhaps two) basic operation(s). We explain that these tools are extensions of the human body. This object will become the locus for their own discovery in thinking and making; consequently, this exploration will model many aspects of the design process. We ask them not to agonise over this



Law Alsobrook

167

selection too much since there is no wrong or right answer in picking a tool; just keep it simple and mundane. We assure them that the journey is more important than either the beginning or the end of the project. Students have brought in expected objects like: staplers, keys, paperclips and the like, but even some unexpected objects turn up: eyelash curlers, kitchen whisks and screwdrivers. iPods, cameras, etc. are discouraged as being too complex, having too many functions or operations and often associated with too much cultural or personal baggage. As a point of departure, the tool a student chooses to explore functions as a concrete focal point for the generation of ideas. All the signs they create will begin with the contemplation of this simple tool. Additionally, the tool will act as an irritant around which a great deal of knowledge can be grown. This layering process of knowledge is one way to slow down the process of their thinking. We attach the specialised vocabulary of design and semiotics to the exercise so that they understand the connections and learn to identify it in their process. To begin, students are tasked with observing their tool and describing all they believe they know about it. We ask them to write about it, use it, etc. Every aspect of the tool, as they understand it, should be annotated and recorded. Phase I of the project, however, begins in earnest with the introduction of binary oppositions as a way to continue the students’ investigation of their tool. We use binary oppositions because it seems to be one of the simplest forms of enquiry available in the Western canon and one most easily observable within the academic context. The concept of observing binary oppositions began with Aristotle who noticed a preponderance of oppositional terms to describe human relationships between themselves and their environment. Because we are an American university, we use this feature of Western thinking as an introduction to the contemplation of meaning and its subsequent composition. We explain to the students that many things exist in antithetical states, and because of this it is relatively easy to name one thing and then describe its opposite. This is a powerful tool for designers because in exploring one idea, looking into what happens if you explore the opposite, often leads to new ideas and connections. The students’ investigation begins with a list of 40 pairs of common opposites (masculine:feminine, correct:incorrect, lie:truth, and so on) we give them as examples after which to pattern their explorations. The students supply the last 10 pairs. This activates their participation in the project so that they feel ownership in these explorations. The 50 pairs comprise 100 things.



168

100 Things: A Process for Foundation in Theory and Practice

To help the students understand the relevance of this task, we ask them to brainstorm about their tool, make associations and then make the signs. They are taught how to mind map, shown word association games and given other methods of generative production that designers use to create the raw material of their ideas. Students begin to step away from what the tool means and see that it comprises a much larger realm of associations. Because graphic design often relies upon making new connections, this new departure or divergence, is the students’ first foray into a more expansive way of thinking. Many practicing designers go through a similar generative phase when receiving a project brief. We underscore this for the students so that they can become comfortable with what divergence is and how to use it to create signs they might use in their own communications. Rather than ask the students to create a list of 100 disconnected things, the purpose of this initial activity is to imbue the process with inherent meaning. We endeavour to show the students that they are delineating boundaries they will later push against as they begin to formulate meaning using signs they generate. The tool is just a physical surrogate for the act of sign creation and crafting messages. The binary oppositions allow students to explore models of semiotic theory while developing a repository of images they can visit throughout the duration of the project. Concurrent to the generative, exploratory nature of phase one where students are physically making their signs, we begin the first in a series of lectures concerning semiotics and the science of signs. We examine types of signs they need to be aware of (icons, symbols and indices), explaining that these can take many forms and often have distinct meanings based on the context. We also discuss modes of communication and explain that there are at most three modes they need to concern themselves with as the basic forms of visual communication: the pragmatic, the poetic and the persuasive. At this point, students are given the first weekend to create 40 things. When we reconvene, the students are excited about their achievements and are eager to show each other their sets of 40 signs. Some have attempted to create a few oppositional pairs, but many have created solitary signs with no designated partner. Most signs the students have created only step away from the original tool in tiny departures. A few will have begun to experiment with the edges of where their tool has led them, but nearly all of them will assure you that they have exhausted the reaches of their tool. We point out to the students that they still have to make 60 more things before they have achieved 100. The true testing ground for whether or not an idea is working is the critique because ideas exist in the realm of cupcakes and unicorns, glitter



Law Alsobrook

169

and rainbows. In essence, ideas exist in an untarnished and perfected state. It is only in the execution of ideas – when they enter the realm of the tangible – that they show signs of viability and staying. We instruct the students to lay out all 40 of their signs. We invite them to look at each other’s work and observe how each individual approached their design objective. We encourage them to ask each other about methods or techniques used in making certain signs, and we allow them to revel in all that they have made. We then break the class into groups and invite individuals in each group to present an icon, a symbol and an index they have created. The other students in their group critique them based on what they know about semiotics. Surprisingly, they know more than they thought; only in the critique can they see what they know. Inspired, we spend the rest of the day with in-class sign making based on the explorations of binary oppositions and what they have learned from each other. We also meet with the students individually and guide them or offer advice on the various aspects of the project they may be struggling with. Two weeks after being assigned, 100 things are due; on this day, each student hangs all 50 of their binary pairs (100 things) in a meter-by-meter square in the halls of the Graphic Design department. It is a flurry of activity, but one filled with a sense of accomplishment. For many, this is the largest project they have ever contributed to (several thousand total pieces), but for most, it is also a realisation of just how much they are capable of when it comes to exploring an idea and trying to create signs that encapsulate it. We then introduce the students to a critique method we call a visual audit. We invest this one with more rigour and importance by disguising it as a test to gauge vocabulary and their understanding of semiotics. In this case, the visual audit is a handout comprised of three sections, each devoted to a particular area of semiotics we have covered. We ask the students to draw the requested terms as well as identify and explain its nature and implication. Students are paired, and together they audit each other’s work. We want the students to understand that they can, and should, rely on each other for support because the design process is more than just individual effort. Once the audit is complete the first phase of 100 things is done.

Phase II – Editing & Refining The second phase of “100 Things” concentrates on two aspects of the design process: convergence (editing signs to better emphasise a specific meaning) and the use of these signs in creating visual associations. With



170

100 Things: A Process for Foundation in Theory and Practice

convergence, raw ideas are winnowed, categorised and/or improved for later use in a designed message. It is the creation of these visual correspondences that pose the most interesting learning for students because this process entails the mechanics for how meaning is assembled. Commencing phase two requires each student to select from amongst their spectrum four signs they wish to explore as a way to extend and complete the following: Reduction :: Elaboration Concrete :: Abstract Literal :: Metaphorical Cultural :: Universal The chosen signs function as the fulcrum, or pivot point, by occupying the middle space between two new signs the student creates that fulfill the comparison in visual terms. This list is neither arbitrary nor without basis in design but, in fact, indicates some of the ways designers might manipulate signs in order to create units of meaning that reach a target audience. These pairs represent some of the interpretative ways designers tailor signs. This exercise proves especially challenging because maintaining the essential meaning of a sign, while refining its representation in order to manifest a particular and intentional significance, is not easy. Editing of this sort requires attention to detail and nuance of meaning that various visual forms can take. To help the students achieve this goal, we introduce the concept of sign manipulation. An idea is only as good as its execution. If “100 Things” constitutes the creation of raw signs, the students must exert representational control in order to communicate to a specific audience. Accordingly, this editing is known as sign operations and includes addition, subtraction and substitution. Put simply, this model introduces the students to a method of refinement. Indeed, this step should invariably lead to better signs students might use in obtaining their communication objectives. The students begin the exercise by placing their chosen signs into the middle of these word pairs. They may refine this initial sign, vis-a-vis the editing process of addition, subtraction or substitution, to arrive at a sign that more clearly corresponds to their intended idea. Each new sign they develop, for either side of this comparison, should maintain a semantic link to the initial sign while at the same time also maintain a visual link in some manner, too. This practice in manipulation of signs is aimed at showing how desired goals in communication can be achieved through the



Law Alsobrook

171

skillful manipulation of signs in order to precisely impart some aspect of a message visually. The challenge appears daunting at first to most of the students. Editing is difficult at the best of times, but when one is made aware of it, and how it happens, it is both a moment of clarity and frustration. Primarily because they have never been asked to think about how something means what it does. We inform the students that this is the nature of design thinking and the heart of how any message has been created. We assure the students there is no real way to get this wrong, if they at least attempt to do the exercise. However, we explain that there are perhaps more favourable visual solutions. We do this so that the students are obliged to create several compositions in order for them to see and take note of when and how certain configurations work, or when they do not and why. Revisiting earlier critique methods allows the students to understand when their intentions are working and when they are not. From what starts out as a series of trial-and-errors for most students ends with a deeper understanding because as they gain mastery of the edit, they begin to see how it allows them to visually say what they intend.

Phase III – Making Meaning Phase three of “100 Things” we call: word:: image:: image::word because in this exercise this is exactly what they must deal with – words and images. Again, revisiting their 100 things, the students are asked to retrieve three signs they wish to explore. This time, in addition to being allowed to edit these signs, vis-a-vis the previous sign operations they have learned, they are to pay particular attention to achieving a syntactic (formal) relationship between the new signs they will be producing. Producing the syntactic in signs is where many beginning graphic designers excel because it is a territory with which they are most familiar. Colours, shape, line quality, etc. are all qualities most artists recognise as vital to visual communication. No less is the case with design, but again, because they are being made aware of how and why this happens, design students find it harder only because they are not used to observing themselves in these moments. To underscore the importance of the syntactic to design, we introduce the students to the principles of Gestalt psychology, demonstrating how artists and designers use visual cues the brain is seemingly hard-wired for, to direct viewers to see and comprehend messages. We demonstrate to the students that they do not often know why they do certain things with regard to visual stimuli; we explain how this happens by discussing and



172

100 Things: A Process for Foundation in Theory and Practice

showing the students examples of Gestalt in action. Akin to revealing how a magic trick is done, we show them the example of the Fedex logo, and point out, to many for the first time, the arrow that has been hiding in plain sight for most of their lives. Once they see it in action, they never forget how the syntactic play of signs enforces visual communication.

Fig. 4-2: An example of word :: image :: image :: word (photo by author)

This portion of the project highlights the inherent dialogue that takes place when any text is paired with any image. It is the first time students are actively made aware of what the implications are for this and how it affects communication. As this is perhaps the first time students are requested to ‘play’ with language, the words the students choose should engage a flexibility that opens doors to interesting communication (a moment of hmm? – concentration or contemplation) as opposed to those that are closed (a moment of duh – boredom or apathy). The point being that to place the word “red” next to a red apple is virtually pointless, but placing the word “sin” next to it makes for a more meaningful, and thus, a more interesting exchange. For most, it is a struggle to try not to repeat what the images are already saying, but this is the point of most graphic design and a problem that they will face multiple times in their career.

Phase IV – Production Upon completion of phase three, the final phase of “100 Things” has led the students to their first concrete manifestation of a designed artefact – their first piece of visible communication. It has varied over the course of years, but the most successful results for this project usually come in a poster format – some form of large-scale communication. With this year’s assignment, the students were asked to create an explication poster – a poster that graphically explores the adventure of the previous seven weeks in investigating a tool of their choice. This final convergence brings with it



Law Alsobrook

173

a critical moment of reflection for many of the students because here they can see what a design process – their design process – leads to. Once again the students are tasked with exploring their 100 things. This time, however, they possess a critically enhanced facility towards choosing the signs they want to use as a means to communicate their experience, their learning, their process and design thinking. We reiterate that almost any project ends with the evolution of an idea into some cumulative outcome. Part of the impetus for “100 Things” is that it establishes a template for the design process that students can alter, improvise upon and/or return to as their needs arise and grow with their career. We believe it is important that they have a complete model of a normative design process, so that later, should they choose to explore and tinker with their various ways of making and thinking, they have a foundation upon which to build. To get them started with the poster, we explain the mechanics of its construction with a lecture about various aspects of poster design. We give them a preliminary presentation about systems and systemised thinking. Examples are hung about the room for students to study, and they are encouraged to research posters and poster designs so that they understand the medium they are about to attempt. At this juncture, students are once again encouraged to free-write about various aspects of their tool exploration as an aid to help access topics they might feel like discussing in a visual manner. We find that beginning each major project with this technique, and revisiting it again at various stages in a project, allows the students to articulate those things that only they see in their head. As we meet with the students and have them critique each other’s direction of thinking, one predominant poster idea usually emerges. This happens because as the students have been forced to re-imagine their message to activate the disparate modes, the new signs they use in these constructions forge new connections and visual solutions the posters can take visually. The poster is born from a ‘Frankenstein-like’ approach of selecting those features most interesting to each idea and combining them into a complete message. What students conceive of as isolated moments, we endeavour to show as flexible and supportive to each other. One mode will dominate, but it will do so supported usually by more meaningful, subordinated modes of communication. As the students’ ideas progress and evolve, they are continually called upon to employ the various convergent methods of refining and editing signs into the specifics of what they desire to communicate. As is often the case, some ideas are outpaced by technology while others expire under a relatively short shelf life. The students have learned, however, that ideas



174

100 Things: A Process for Foundation in Theory and Practice

are flexible and can be explored via other means of production and making. What they undertook as part of the generative, exploratory phase at the beginning of the project is still with them and should be continually revisited when seeking visual answers to their particular communication conundrum. As frustrating as it is in these moments, when students make this connection, the rewards of a strong design process becomes an electrifying moment of coalescence.

Fig. 4-3: Student critiquing another student’s work (photo by author)

As the students progress in creating their poster, but before the final full-scale version is due, we introduce a final and fairly powerful critique method. Students bring in full-sized roughs of their poster and hang them on the walls in the class. In turn, we explain that each student must write one criticism about every poster. This allows for two things: it prepares the class for the seriousness of the critique, but it also gives those more introverted students a moment to compose their thinking. What is most telling is in the amount of information students get in their criticism (some of which we, as teachers, may have missed). When the final due date arrives, we take the students through the design process one last time, indicating all the features they learned about and how they used these in the final production of their poster. We quiz them about the vocabulary they know and again go over how it has



Law Alsobrook

175

affected their thinking and making. For many, the end of “100 Things” is a release from something that seemed to go on forever. At the same time, for most of them, it is an immense feeling of accomplishment to know that all that they have learned and experienced has led them to this moment.

Fig. 4-4: Final poster examples from 100 Things (photo by author)

Conclusion “100 Things” is a massive project for a beginning student. Indeed, it is a project with a reputation that many hear about in their foundation year. For the many students who have endured it, the experience reveals to them what they are able to accomplish, while at the same time, gives them valuable skill sets they can continually use and improve upon. But “100 Things” is more than a test of design endurance. In earning these skills, the students have pushed themselves both mentally and physically as they explore the boundaries of their design process and thinking. Once these areas are stretched and expanded, they can never retain their previous shape. They have experimented with media and material, both familiar and unfamiliar, as a means to capture signs that visually declare just what they intend them to. From their initial incursion into semiotics and the production of signs, the students have learned a new language, a new vocabulary and a new way of thinking. Lastly, through various critique methods and strategies they have learned to better refine their ideas so that the signs they employ will converge on exactly what they mean to articulate.



176

100 Things: A Process for Foundation in Theory and Practice

Bibliography Crow, David. Visible Signs: An Introduction to Semiotics. New York: AVA Publishing, 2007. Marshall, Lindsey, and Meacham, Lester. How to Use Images. London, Laurence King Publishing 2010.



VISUAL DIASPORAS: COMICS AS TRANSCULTURAL PHENOMENA HOLGER BRIEL

Introduction In 2006 Andreas Hepp published his Transkulturelle Kommunikation, a comprehensive text on transnational and transcultural ways of communicating in an ever more digital world. His book has since gone on to become a mainstay of communication research in the German speaking academic community, and deservedly so. On the cover of his book there is a frame depicting the hero of one of Judith Park’s Manga, Y Square (2005), Yoshitaka Kogirei, rendered with a slightly stressed-out face in front of an urban background. Hepp comments on manga in his introduction, noting the specific connotation of manga reception in Europe and that “the globalization of media communication is not a one-way street (any more)” (Hepp 2006: 8). The interesting thing here is that the manga frame on the book cover is not of Japanese origin at all. Judith Park is one of Germany’s foremost mangaka and her work is published by Germany’s leading manga publishing house, Carlsen. With this sleight of hand, Hepp is requesting inductive reasoning from his readers, playing with their preconceptions and visually highlighting his central tenet, namely that transcultural communication has already and irrevocably permeated our lives, whether we know it or not. In the following, I would like to capitalise on this notion and broaden its implications especially for visual communication studies. Over the last 20 years or so, Japanese Manga have become ubiquitous the world over: they have helped to stabilise a downward Western comics market and infused new ways of drawing and thinking into standard Western comics practices. This being the case, the following questions require some answers though: what is the cultural basis for these picturesstories and how are they received? Why are they so successful? What are the contexts and the co-texts in which they appear and how are these

178

Visual Diasporas: Comics As Transcultural Phenomena

changing? And, how exactly do they participate in a transnational and transcultural flux of what Vilem Flusser termed TextImages (Flusser 1998), the inseparable interlacing of visual and textual material? It was the arrival of American comics into Japan after WWII which started modern-day manga in their westernised version in Japan. While the Japanese comic phenomenon is itself hundreds of years older than its American counterpart and a much broader social phenomenon, this arrival of Western comics would set into motion a cultural machine which, beginning in the 1990s, would come to haunt American and European indigenous comics’ markets, and here especially those of France and Italy. Judith Park is of course not the only non-Japanese to draw manga, there are many more, including many of their fans. An early example was Frédéric Boilet and his "Nouvelle Manga Manifesto" (Boilet, n.d.). Boilet lives and works in Japan, fusing Western and Eastern styles in his work. He makes clear in his manifesto that the time for rigid distinctions between “Western” and “Eastern” comics has come and gone. Manga have much more in common with Europe than their exoticising marketing might suggest. Quite a few of their features, such as their oftentimes “exotic”, i.e. Eastern and Western settings and their discursive practices and style elements, such as the characters’ over-sized eyes, allowed for the emergence of a globalised comics style that bears strong transnational traits, and exudes a strong artistic influence on other arts and even PR and advertising practices. Manga in the West then have begun to live up to their constitutive hybridity, in cultural contexts as well as in their combination of particular textual and visual elements and with the inherent semiotic challenges these pose to their respective audiences. One of these challenges is to problematise the perceived ease with which pictures can be viewed. As has already been underlined by scholars such as Jencks (1995), the bane of visual studies is the popular notion that images do not require any translation, as, supposedly, humans universally understand them in the same way. Much of the difficulties inherent in transborder image transport and (de)codification are due to their imagined simplicity. One could cite the example of the image of a man with his shirt not tucked into his trousers. Depending on its cultural origin, this image can symbolise very different things in different cultures: rebelliousness, casualness, formal wear, etc. In this way, such an image represents realities in very specific, even opposing ways. All its subtexts forcefully call into question the erroneous belief that images are easy to decode. Comics, and especially manga, are a convincing case in point. As I have demonstrated elsewhere (Briel 2007), readers take many shortcuts when decoding foreign comics and only use a limited number of

Holger Briel

179

tools on offer to decode them further. It is unfortunate that, unlike in Japan, when it comes to understanding a phenomenon as highly complex and influential as manga, their explication in and to the West has only recently been picking up speed. To date, there exists only a small number of studies, which put emphasis on the transculturality and transnationalism of manga and the many ways in which they are so compellingly expressed, as for example. In Henry Yoshitaka Kiyama’s The Four Immigrants Manga: A Japanese Experience in San Francisco, 1904-1924 (1999) or Miné Okubo’s Citizen 13666 (2004). After a short analysis of the social situation in which theses manga enter the Western markets, a small study conducted in 2009 in Cyprus on cultural elements in comics with special emphasis on manga will be introduced. Its analysis will engage the themes of hybridity, transnationality and readability more closely and the data collected will demonstrate how manga are exerting their particular influence on advertising styles, in cultural productions and in Hollywood films such as Quentin Tarantino’s Kill Bill (2003/2004), which carried an animated sequence within its main body. While there still exist exoticising tendencies within some Western reception strategies, by now an uncanny familiarity has tied the Western world to manga producers and consumers in the East in general and in Japan in particular and has begun to create new forms of glocal visual styles and reception modes.

Transnational developments, hybrid cultural products and identities If just a few years ago one could still work under the hypothesis that there existed a unilateral dataflow from the USA/the West to the rest of the world, this has decidedly changed with recent interventions and especially through the World Wide Web. And not only the USA has been challenged by the arrival of non-western-centred media, it is the West in general which sees its media dominance dwindle. Today there are more websites outside the USA, more bloggers in Japan than in the USA, more films produced in India and more Manga sold in France than native comics products. A cultural debordering has taken place, in which transnational companies produce and package cultural products for different markets (cf. Bourdieu 1984: 30 ff.). Much of these trends were facilitated by increased global transportation and telecommunication technologies, thus leading an increasing number of transnational (cultural) migrants to have several places to which they owe allegiance. This has had strong cultural repercussions, with social and geographic space oftentimes drifting apart.

180

Visual Diasporas: Comics As Transcultural Phenomena

At the same time, transnational agencies began to appear or receive more power. One could think of EU agencies and their accompanying philosophies, NGOs or transnational companies. While these were by and large economic developments, concomitant cultural changes took place as well. Also, while today there still remain postcolonial streams of communication in place(s), these are mainly holding on due to linguistically determined markets – the (Ex-) Commonwealth with its dominant English language culture, Francophone Africa and Spanish/Portuguese speaking Central and South America. But the manga phenomenon has transcended these culturally encoded streams of infotainment. As such, it challenges Hesmondalgh’s theory of geocultural markets defined by particular historical cultural spheres of influence (Hesmondalgh 2002), as this is decidedly not true for manga which have cut right across any of these geocultural straights. Several of the countries with a strong comic culture (e.g. France, Belgium, the USA), have had their markets taken over by the transnational and transcultural manga phenomenon. If one were to look for a similarly strong non-Western product in music, one could point to the music of Bob Marley, whose cultural reach easily transcends nations and national cultures. However, his reach not only exemplifies the growing hybridity of products, but also of their audiences. If a taxi driver in Africa or Asia typically puts on a Marley cassette the second s/he has a Western customer, because the Western customer will recognise it as part of his own canon and revel in the exoticness of this aural hybrid localisation, then the question of cultural positioning of both product and its reception is in need of much deeper probing. Manga display similar qualities, as they are typically consumed in a similar culturally hybrid setting. When thinking about where transcultural trends take place, one area most affected is the so-called mesolevel of cultural production. At the microlevel, local products tend to saturate the market, and at the macrolevel, typically transnational corporations dominate the markets. It is on the mesolevel where new audiences are formed, irrespective of their physical geographies and cultural histories. Furthermore, hybrid audiences can mostly be found in urban settings. (cf. Sassen (2006) and Saunders’ Arrival City (2010)), both of which discuss the hybridity of urban dwellers at the beginning of the third millennium). To reflect much of their audiences, manga oftentimes are staged in such urban settings. As such, they incorporate the avant-garde lifestyle of new cosmopolitans, who wish to consume cultural products exotic and local at the same time. As such they are a privileged but growing part of the global and transnational (cultural) migrant population, being offered hybrid narratives as an

Holger Briel

181

accompaniment to their own life circumstances. As part of the new internationalists, their mobility changes the way they perceive and consume media. In recent years, the mobile internet and the arrival of smart phones has once again drastically changed the way we represent the world to ourselves and us to the world. This process is far from over, with Augmented Reality (AR) in the guise of Google’s Glass and Social TV just around the corner. This mobility, then, includes not only comics, but also television and (satellite/internet) radio, which allow for a global consumption beyond the traditional, national airwaves. Thus the media are creating/feeding off of a growing steam of cultural remigrants who participate in their various localised cultures, even if their origins are geographically different. Hence, today identity is to be considered not as a finalising movement but rather as a series of identifications. In practice, this view has been expounded by a number of internet studies which suggest that rapid change of virtual identities has shaped the way we live offline as well. If our online identities are fluid, this will eventually also be played out in the offline world. Identity is never solid, but a never-ending identification process (cf. Nagy 2010). Or, in the words of Sherry Turkle (1997: 75), Online experiences challenge what many people have traditionally called identity: a sense of self is recast in terms of multiple windows and past.

This sliding scale of identities is then also used by the identities of comics products consumed. Neither globalisation nor its transnational implementation would have happened without the media. At first, local media would transmit images of the world, and later on, transnational/foreign media would bring us these or fairly similar images. Over the last 15 years or so, every major power has introduced an English language news channel, available anywhere in the world, which would allow it to describe its worldviews to a global audience. This global push is also evident in the cultural sphere. It was in Japan where the phrase for Glocalization (dochakuka) was coined, a term originally reserved for preparing agricultural products for different Japanese regions, then translated for industrial products and their exports and lastly introduced into cultural theory by British sociologist Roland Robertson. Thus, in a very telling way, this is related to the term culture, which originally also stems from agriculture. The (cultural) difference manga sell as manga is evident in the way they are marketed as exotic products. But such hybrid glocal consumption is not without its critics; it was criticised by Stuart Hall who, in a 1997 interview, had already developed the theory of how such cultural difference has become a

182

Visual Diasporas: Comics As Transcultural Phenomena

profitable good in itself. According to Hall, this is the case for all products which are sold with a “difference surplus” such as ethnic cuisines, an “exotic” vacation, or indeed “world music” (Hall 1997). While this was true then, his thesis would have to extend further today and incorporate the fact that contemporary audiences are of hybrid nature themselves. In the global village, homogeneity has been problematised and cultural differentiations are the norm. On the one hand, this might be a positive development, as this shows that large parts of the world are moving into Homi Bhabha’s “third space–an arena of hybridity allowing for multiculturalism and transnationalism as positive and formative forces. On the other hand, this fact which might be also described as Derridean différance might then have to be written forth into the future, and eventually go unnoticed, much to the detriment of the added exotic value peddlers in each “distinct” culture, who have build their empires on “pure” local cultures. If non-local products are consumed in the transnational world, the idea of representation needs to be addressed, as more and more phenomena are represented in foreign settings and as these products take on representative functions: of a genre, of a culture or of a place of origin. Here, representations can include things, concepts, and signs and it is their specific mixtures which generates specialised discourses. These discourses also include the exchanges of, and between, different kinds of cultural products. Some time ago already, such discourses have decidedly left the local and are opposing, interacting and cross-breeding with each other in the global market place; most markedly so again on the World Wide Web. If, as was claimed earlier, media have played a major part in transnationalism, this is certainly also the case for the distribution of manga. Without the Internet, their rapid and wide distribution would have hardly been possible as much of the cult status some manga attained was due to exchanges about them in Internet fora. Over the last two decades or so, both cultural homogeneity and the idea that a nation as a whole can solely be represented by its “own” national cultural artefacts have become a contested field. Here one might cite national cuisines as an example. Given the interlacing of ethnic restaurants and restaurateurs, these days one might just get the best Indonesian cuisine in Amsterdam or indeed the best curry in Bradford; this gastronomical exchange used to be strung along post-colonial lines. But given recent trends such as fusion food and ever increasing tourist streams, the best French food might also be served in New Caledonia or Cameroon. These hybridized cultural products then serve particular local and translocal consumers. Even the idea that there exists one national cuisine is

Holger Briel

183

a chimera. And this is not only true for food. As Sumita Chakravarty has already demonstrated almost 20 years ago (1993) for film, the national is too grand a theme to be properly re-presented or indeed representable. If at all, it can form a context in which individuals participating in its civilization can be portrayed in film or are performing some of its functions. To do justice to this fact, she created the fitting neologism “inperson-nation,” incorporating abstract national ideas into proper bodies. The national itself has also received a new contextualisation at the hands of the media, e.g. first with satellite TV, whose scope much extended national broadcasting limitations and then with the Internet, in which few things these days are national. This particular medial deterritorialisation can again be seen as further proof of the world moving away from American media paradigms towards more localised and nonWestern media structures and content. But the transnational itself creates its own new challenges. Theoretically, transnational media spheres are still considered problematic, especially when it comes to the public ones. The question, Is there a transcultural and/or a transnational public sphere? is oftentimes negated. Thus, Jim McGuigan views this belief in a transnational sphere as a “Western fantasy”, a last attempt at global hegemony maintained by the West. (1998: 95ff). In regard to transnationalism proper, an answer might be easier to postulate, as nations still have very clear geographical boundaries as do transnational companies which are still largely incorporated in Western countries. And it is consequently only this transgressive culture which continues to create trouble for states, especially those who want to defend their citizens against outside influences, for better or, usually, for worse. The move into digital media has changed this dramatically though. Internet distribution of cultural products is typically decentralised and multiple. The big advantage of the Internet over earlier media is that it gives instantaneous access to material previously unavailable. Much of manga reception continues to be dominated by the Internet and the computer rather through TV or in any kind of written form. Oftentimes, manga reception and local adaptation are achieved through a variety of sources – be it schoolyard talk, Internet fora or personal comparisons – and then through the incorporation of these images into one’s own cultural consciousness (cf. Gillespie 1996). If properly reflected, such consumption can become the basis for further and more sophisticated cultural considerations. Reflected consumption is typically underpinned by interest in, and knowledge of, the cultural co-text in which artefacts are created. The classic Liebes/Katz study with Dallas audiences in various countries demonstrated that the cognisance of the cultural background of a cultural

184

Visual Diasporas: Comics As Transcultural Phenomena

artefact is an indicator how well such material will be received in another culture. Further reflection will then also concentrate on the difference referential to one’s own life and how well this is bridged (Liebes/Katz 1993: 248-249). How successful, or rather how comprehensive, such dayto-day practices of such adaptations of culturally hybrid material are, will now be discussed below in relation to a comics study conducted in late 2009 at the University of Nicosia in Cyprus.

The study The study consisted of the analysis of the reception of manga in relation to Western comics and was conducted in October 2009 at the University of Nicosia in Cyprus. The student body at the university is mixed, with the undergraduate majority being of a Greek-Cypriot background. In its Master’s Programmes, oftentimes more than 50% of the students are nonCypriot. Both undergraduate and postgraduate courses are taught in English. The study was a small qualitative one with 30 participants: 12 undergraduates from the BA in Graphic Design’s 4th year cohort, 10 postgraduates from the MA in Media and Communication and 8 faculty members from the Humanities disciplines. The age range was 22-40 for the students and 27-46 for the faculty. In the first instance, a questionnaire was handed out, asking questions about the participants’ background and comics acculturation (see appendix I). Then, they were handed copies of three different comics and asked to comment on these. The comics were: 1) Battle Royale, a manga based on the controversial 1999 novel by Koushun Takami. The original manga print run was from November 2000 to January 2006. In 2000 the manga was made into a feature film, directed by Kinji Fukasaku. Its huge success prompted Hollywood to come up with its own take on this phenomenon, the immensely successful Hunger Games (dir. Gary Ross, 2012), although the author of the novel claims not to have heard about the Japanese original. In Japan, the original film was then followed in 2003 by Battle Royale II: Requiem. The plot is quickly told: in an alternate timeline, a Japanese Junior High School class is kidnapped and brought to an island where they have to kill each other; only one survivor will be allowed. The individual battles are broadcast live on TV. The students are wearing electronic collars which allow the director to view and hear everybody and also to kill them if they do not follow the rules. The series as a whole could be described as a combination of Golding’s Lord of the Flies and The Truman Show on Angel Dust.

Holger Briel

185

Particular technical details typical for this manga series include traditional frames, but with conjunctive writing. It also makes usage of filmic techniques such as extreme slow motion. In terms of content one can point to the political content, as the series describes and decries a dystopic Japan in which TV supervision, survival of the fittest and nationalism rule. Examples are the exoticisation of the villain, the hero’s portrayal in front of a large historic Japanese flag, and the graphic display of violence. 2) The second comic series handed out were Gilbert Shelton’s Fabulous Furry Freak Brothers. They first appeared in 1968, have sold over 40 million comic books worldwide and have been translated into 14 languages. The print run by San Francisco’s Ripp-Off Press lasted from 1971-1992. The subjects of the comic are Fat Freddy, Phineas Freak, and Freewheelin' Franklin, living hippie lives in 1960s Northern California. Much of their time is devoted to the procurement and consumption of drugs, and the comic highlights their parties and their battles with police and landlords. The series had a large cult following and continues to draw a mostly counterculture readership. A stop-motion film has been promised for some time but is still lacking funding. Technically, the comic is drawn in typical American underground style, with small frames and even smaller captions and barely legible text. 3) The last comic distributed was Tank Girl, a British comic largely set in a post-apocalyptic Australia, created by Jamie Hewlett and Alan Martin. It follows the adventures of the title heroine who lives in a tank and fights evil, more or less. While it adheres to traditional framing, the content of tank driving women and mutant kangaroos in the Australian Outback and the vivid psychedelic colours used in its drawings made Tank Girl a cult comic. Its print run lasted from 1988 to 1995, when the magazine it appeared in folded. Tank Girl itself was revived in 2007 though as a standalone series and was made into a motion picture in 1995. After a short introduction to the three comics, group discussions followed and then participants were asked to fill in Questionnaire II (see Appendix) to assess their ability to discuss in-depth culture related questions about comics. Finally, the answers given in both questionnaires were further analysed. Especially the answers to the culture-related questions in Questionnaire II were rich in yield. The following questions from Questionnaire II had been especially designed with such culture probing in mind: y What would you say is its specific cultural “other”? Be precise, mention particular frames, objects, attitudes.

186

Visual Diasporas: Comics As Transcultural Phenomena

y Do you think you understood the specific cultural references? How many? (in percent) y In your opinion, what are the Western influences on Battle Royale, if any? Be precise, mention particular frames, objects, attitudes.

Results The results of the study display the state of cultural in-between-ness which characterises much of cultural production and reception in the “developed” West. But they also pinpoint local differences which might be viewed as positive barriers to an all-too-facile taking-over of the cultural channels by big international media. To begin with, results confirm that today comics have become an acceptable part of cultural formation. There were no indications that the students’ parent generation had negatively influenced their children’s enjoyment of comics. Only two non-Western participants expressed some mild reservations about comics in general. Perhaps surprisingly, two thirds of the participants were not familiar with manga (20/30). This could be interpreted in at least two ways: first as an indication for the relative unfamiliarity with the term “manga” for certain comics, or alternately, that manga are not noticed any more as a distinct artistic format within the field of comics. When asked about their comics’ background, most cited Disney cartoons and magazines – demonstrating that there continues to be a large dominance by American comics and cartoons worldwide. This is especially true for the Mediterranean basin. Only three respondents mentioned Arkas, arguably the most famous Greek comic, as a decisive force in their comics formation. One of the main facets of Japanese comics is their “Cuteness Factor” – the way certain characters take over the narrative with their own cute whimsicalness. This could be compared to certain rote models in theatre – the old fogey, the young heroine, the smart servant, etc. The difference in manga is that this cuteness factor persists in almost all genres, no matter how gruesome some of the other facets of the manga are. As such, this “cuteness” is one of the main estrangement factors of Japanese manga, but it was not mentioned at all by any of the participants. This is surprising, not least because of the ubiquity of the phenomenon. Also, the participants almost completely reduced adult manga to violence, sex and erotics. Only three participants, two women and one man, thought they also dealt with women’s stories and were designed to “make you think”.

Holger Briel

187

Furthermore, only those participants having grown up in the US were aware of newspaper comics and the “Saturday morning ritual” of watching cartoons on TV. The others were not aware of these cultural practices, although today this format has taken over most Saturday morning TV station programming. It might be argued that a younger cohort would have probably had a different reaction, as global animated TV is only a recent invention. When it came to the relationship between comics and video games, this age differentiation in the study became apparent once again. By far the largest group of video game players were undergraduates. This, however, did not automatically correlate to heightened manga or comic usage; if at all, these participants were less aware of the cultural underpinnings of their practices than their older peers. Generally, in the participants’ age group, Japanese cartoons did not yet play a significant role; only Dragon Ball Z (1984 through 1995) was mentioned once, with the only other non-American animated series mentioned being Maja the Bee, based on a 1924 book by the Czech writer Waldemar Bonzel and originally aired in Japan from April 1975 to April 1976 and consequently adapted for the European market. However, most Europeans would not know that the series actually is a Japanese production. Otherwise, Mickey Mouse ruled supreme, closely followed by Spiderman. What was surprising was the fact that Tank Girl was largely ignored whereas the Fabulous Furry Freak Brothers were correctly interpreted and discussed at length, but then also dismissed as purely 1960s counterculture American comics. Apparently, Tank Girl’s psychedelic appearance and its own “weirdness” factor did not endear it to its audience. Lastly, manga’s cultural Other was not overtly commented upon nor realised in the Western readings. The participants, although overwhelmingly familiar with comics and cartoons, were not reading/watching these with an eye for cultural differences and did not comment upon them. The above results allow one to speculate a bit further. First of all, if the study had been conducted in France or in the USA, the results would have probably been different, as the majority of the participants would have been more knowledgeable about comics products. Much of their own cultural upbringing would have evidenced a higher degree of familiarity with comics and a more differentiated assessment of the different kinds of comics available. Furthermore, if one had taken younger individuals as participants, the results would have probably differed again. On the one hand, they would no doubt have been more acquainted with Japanese manga; on the other hand, their ability to correctly differentiate between manga and non-Japanese comics might have been more impaired than that

188

Visual Diasporas: Comics As Transcultural Phenomena

of the participants in the present study. Comics styles have become more Japanised over the last decade, and it is becoming more and more difficult to decide upon a comic’s geographical or cultural background. This is not necessarily a bad thing. However, it does evidence that cultural hybridity, for better or for worse, has become a day-to-day occurrence. This hybridity is further evoked by the production processes inherent in many of contemporary manga. Perhaps similar to Hollywood productions, whose main audiences are today increasingly localised outside of the USA, Japanese mangaka have also begun to “internationalise” their products. In line with Lu (2008), one can describe three main ways in which Japanese manga and anime producers have incorporated internationalisation in their products. These are: x De-politicised internationalisation, which primarily serves as a commercial tactic to attract international audiences; x Occidentalised internationalisation, which satiates a nationalistic sentiment; x Self-orientalised internationalisation, which reveals a cultural desire to establish Japan as an ersatz Western country in Asia. Lu claims that these products have adapted themselves to their expected audiences by toning down some of the more idiosyncratic Japanese cultural themes in order to make them more easily consumable outside of Japan. However, this strategy is one fraught with risk, because if the very reasons why Westerners were attracted to manga in the first place, i.e. their foreignness and exoticising moments, are reduced in order to make them more palpable to the Western eye, then this is a risky move; it potentially threatens to undermine the unique selling point these products had in the first place, and this especially at a time when Western comics themselves have begun to become more Japanese in style if not in content.

Conclusion By now, it has become apparent that many cultural products have set out on their victorious march across national and cultural borders and have evolved into new hybrid cultural manifestations in the process. The case of manga is an important one in this phenomenon as they have easily and quickly penetrated the global comics market and were the first comic genre to rely on the World Wide Web for primary transnational distribution. The transnational and transcultural have begun to interrelate in that undertaking in that manga production, dissemination and reception have become dependent on both phenomena.

Holger Briel

189

To continue, when analysing comics today, one has to take into account a quickly changing audience, which is used to consuming more and more globalised products. As this is becoming the norm and availability and accessibility have become less of a problem, what is still lagging behind though is the differentiability of these products in their readers’ minds. More and more, consumers have trouble recognising where these products originated from and which cultural streams they follow. It is still a bone of contention whether this really matters, as this is still a recent phenomenon, but it is clear that globalisation has successfully entered the comics market and can now account for having created a globalised audience. This audience is not sure anymore where the products they consume originate from and at least, according to the small-scale study above, they are increasingly unable to pinpoint cultural “trademarks” and conventions. Here, educators are clearly asked to increase their engagement with visual studies contents and methodologies in order to further educate students about the cultural products they consume. Accessibility can only be a first step, visual literacy and participation training has to follow if a more comprehensive understanding of diverse cultural artefacts is the aim. Lastly, it is becoming ever harder to distinguish between “local” audiences’ make up, individually or collectively, as large global urban centres have taken over the prime audience function from distinct national and linguistic markets. At the same time, the cultural products themselves have also been adapted for a global audience and these changes influence their very creation if not their cultural distinction and characteristics. Thus, it is not just manga but also their audiences who circulate around the globe. This includes not only frequent changes in geographical location, but also an emerging platform and device in(ter)dependence when it comes to accessing and storing (moving) image files, many of them migrating to mobile devices or some kind of cloud storage. The diasporas of erstwhile distinctive cultural productions have been multiplied and continue to appear in ever changing digital formations.

190

Visual Diasporas: Comics As Transcultural Phenomena

Bibliography Bhabha, Homi. The Location of Culture. London: Routledge, 2nd edition, 2004. Boilet, Frédéric. "Nouvelle Manga Manifesto." http://www.boilet.net/am/nouvellemanga_manifeste_1.html (Date of Access: 15 July 2013). Bourdieu, Pierre. Distinction. London: Routledge, 1984. Briel, Holger. “The Roving Eye Meets Travelling Pictures: The Field of Vision and the Global Rise of Adult Manga.” In: Berninger, Mark (Ed.). Comics as Nexus of Culture. Jefferson, North Carolina: McFarland, 2010: 187 - 211. Chakravarty, Sumita S. National Identity in popular Indian popular Film 1947 – 1987. Austin: U of Texas P, 1993. Flusser, Vilem. Kommunikologie. Frankfurt: Fischer1996. Gillespie, Marie. Television, Ethnicity, and Cultural Change. London: Routledge, 1995. Jenks, Chris."The Centrality of the Eye in Western Culture: An Introduction." In: Chris Jenks (Ed.). Visual Culture. London: Routledge1995. Kiyama, Henry Yoshitaka. The Four Immigrants Manga: A Japanese Experience in San Francisco, 1904 1924. Stone Bridge Press Berkeley, CA, 1999. Liebes, T, Katz, E. Export of Meaning. Cambridge: Polity1993. Lu, Amy Shirong. () The Many Faces of Internationalization in Japanese Anime. In: animation: an interdisciplinary journal. Vol. 3(2) 2008): 169–187. Martin, Alan and Jamie Hewitt. Tank Girl. Milwaukee: Dark Horse Comics, 1993. Martin, Alan and Jamie Hewlett. Tank Girl One. London: Titan, 2005 (originally in Deadline (1988)). Nagy Péter. “Second Life, Second Choice? The effects of virtual identity on consumer behaviour. A conceptual framework.” 2010. http://kgk.uni-obuda.hu/sites/default/files/pnagy.pdf (Date of Access: 15 July 2013). Okubo, Miné. Citizen 13666. Seattle: University of Washington Press, 2004. Park, Judith. Y Square. Hamburg: Carlsen, 2005. Sassen, Saskia. Cities in a World Economy, Thousand Oaks: Pine Forge Press, 2006.

Holger Briel

191

Sheldon, Gilbert. The Fabulous Furry Freak Brothers. San Francisco: Rip-Off Press, 1971ff. Taguchi, Masayuki. Battle Royale. Tokyo: Ohta, 2000-2006. Turkle, Sherry. “Multiple Subjectivity and Virtual Community at the End of the Freudian Century.” Sociological Inquiry, 67(1) 1997: 72-84.

Visual Diasporas: Comics As Transcultural Phenomena

192

Appendix I Questionnaire I y y y y y y y y y y y y y y

Age: Gender Nationality: Did you read comics as a child? If yes, which ones? What are your earliest memories of comics? Did you watch cartoons as a child? If yes, which ones? What are your earliest memories of cartoons? What were the reactions of your parents? Do you watch cartoons today? Which ones? Do you play video games? If yes, which ones? Are you a comics reader today? Do you use the Internet to find – read – download comics? (Circle the correct one(s). Which web pages do you use? Are you familiar with Manga? If yes, what do you think of them? Why are they so popular? Are you familiar with the concept of “adult” comics? If yes, what do you think it means? Do you think there are different cultural/national/linguistic centres for comics? If yes, which are they and why? (Problematic question, not many knew what to do with it) Do you think pictures need translating? If yes, why and how? (Problematic question again, not many knew what to do with it) Any other remarks you might have regarding comics:

Appendix II Questionnaire II y y y y y y

What did you think of Battle Royale? Why? What would you say is its specific cultural “other”? Be precise, mention particular frames, objects, attitudes. Do you think you understood the specific cultural references? How many? (in percent) In your opinion, what are the Western influences on Battle Royale, if any? Be precise, mention particular frames, objects, attitudes. How would you compare Battle Royale with the Fabulous Furry Freak Brothers? How with Tank Girl? Be precise, mention particular frames, objects, attitudes. Would you want to see the film of Battle Royale? Why/not?

THE DEVELOPMENT OF A VISUAL LITERACY COURSE IN HIGHER EDUCATION ANASTASIA CHRISTODOULOU AND GEORGE DAMASKINIDIS

The aims and scope of this study The Aristotle University of Thessaloniki in Greece has provided a curriculum for the postgraduate programme titled “Italian Language and Culture”. Our contribution to this curriculum was an innovative course titled “Visual Literacy in Language Teaching and Learning” for students specializing in Applied Linguistics. We have followed the “new pedagogies of multiliteracies”1, shifting from the dominant print text and examining how literacy can be practised when analysing video, as a new form of multimodal text, in the new millennium. By employing this new concept of pedagogy, we aim to introduce a framework consisting of two elements, a systemic functional (SF) approach and multimodal discourse analysis (MDA), to describe the activities of individuals as they identify, read and create new texts using various semiotic codes. This approach to literacy was introduced in a university setting in response to the call for higher education to adopt a strong commitment to a socially pertinent visual literacy2. While the programme’s curriculum offers a variety of courses to improve students’ language skills for this specialisation, its focus has remained relatively narrow. It is now apparent that changes primarily brought about by the use of video as teaching material in the language classroom have created new opportunities for video as a tool to promote and enhance the study of non-verbal semiotic modes of communication in multimodal texts. This course was developed because we realised the need to design an introductory course that would meet students’ basic requirements for 1 2

New London Group, 1996 Bleed, 2005

194

The Development of a Visual Literacy Course in Higher Education

understanding visual communication. Two conflicting aspects had to be balanced: firstly, there was the time constraint, since the course had to be completed in 12 lessons of three instructional hours each, and secondly, given the fact that visual literacy is a wide-ranging subject, we had to be very limited in scope and particularly focused. For that reason, we decided to use only one type of multimodal text, namely video, by adopting established theoretical concepts, teaching methods and tools. This paper reports briefly on the course’s first intake. Since we are still in the cyclical process of designing, applying, reflecting on and modifying the course, we now present an overall evaluation, with the intention of making the necessary amendments for a future version.

The pedagogic potential of the visual According to Turner3, we could analyse image composition and sequencing in ways similar to the vocabulary and syntax of verbal language. Such an analysis, we argue, could be expanded by substituting ‘verbal language’ for all ‘semiotic modes of communication’ with a structured form (e.g. images, colour, music, graphics). Bearing this in mind, we aim to demonstrate possible ways of analysing the video as a multimodal text in order to find out which semiotic resources of representation have been exploited to favour one viewpoint and render all other irrelevant. Our starting point has been the social and visual research methods used by Iedema4, Jewitt and Oyama5, and Tseng6. We followed Iedema’s work to analyse a documentary by employing a social semiotic analysis of telefilm based on the hypothesis that all meaning-making resources perform three overarching functions, or metafunctions7, that is, the “representational”, “interpersonal/interactional” and “compositional”. Teachers that understand these three meanings would be able to choose or create the right images and use them to teach English language skills and sub-skills. By adopting Jewitt and Oyama’s visual social semiotics approach as a tool for use in critical research, we aimed to help students identify and analyse possible relations between the verbal and visual elements of a multimodal text, and to bring to light initially unapparent contradictions. Tseng’s8 work on the interaction of co-occurring modalities and how they 3

Turner, 1994 Iedema, 2001 5 Jewitt and Oyama, 2001 6 Tseng, 2008 7 Kress and Van Leeuwen, 1996 8 Tseng, 2008 4

Anastasia Christodoulou and George Damaskinidis

195

combine and cohere to create meanings became a method for the students to investigate how meaning creation takes place. In addition, according to Jin and Boling9, teaching that integrates both verbal and non-verbal semiotic modes demonstrates better outcomes than does teaching with verbal-only or non-verbal visual modes. Similarly to other researchers, we investigate two key questions using the video as a data-multimodal-text: first, if the processes involved in watching videos develop different cognitive abilities than those required for reading and writing traditional print-based texts, and second, if these new modes of communication merely require traditional literacy skills to be applied to new types of texts. Curriculum documents and assessment requirements for reading and writing are based on established theories on the reading and writing of print-based texts. These theories have determined specific approaches and strategies for teaching reading and writing at different stages of learning. Yet, ongoing research is required to theorise the interactions that occur as learners read and process various visual, aural, spatial and textual modes, separately or simultaneously, in multimodal texts. Although the backbone of the course was the visual, we aspired to develop a classroom learning experience that would be appropriate for all forms of literacy. Thus, we needed to examine how new semiotic modes can be integral to classroom communication. For example, teaching English as a foreign/second language involves four skills: listening, speaking, reading and writing. If we were to consider visual literacy as an additional aspect of language teaching, then “viewing” could be added as a skill in the future. Moreover, this visual aspect is relevant to students and teachers alike.

Defining visual literacy A broad definition of visual literacy is the ability to understand and use visual images, including the ability to think, learn and express oneself by means of visual images. Such a definition has become common knowledge and might even be accepted in academia to the extent that references are omitted. However, the definition of literacy is still fluid. Given the existence of multiple levels and kinds of literacy, no single level of skill or knowledge could qualify someone as being literate. This does not mean that the term visual literacy is completely new. It was first used by

9

Jin and Boling, 2010

196

The Development of a Visual Literacy Course in Higher Education

Debes10, a pioneer in the field who contributed to the systematic, theoretical understanding of visual literacy skills. The term “visual literacy” originated from a variety of disciplines, with many conflicting definitions resulting. According to Hortin11, [d]isciplines such as art, education, English, linguistics, philosophy and psychology have contributed to our knowledge and understanding of visual literacy.

Another factor contributing to this status quo has been the fact that visual skills have been thus far acquired through experience rather than formally taught. Based on experimental studies, Avgerinou12 concluded that in the context of human, intentional visual communication, visual literacy refers to a group of largely acquired abilities, i.e., the abilities to understand (read), and to use (write) images, as well as to think and learn in terms of images.

This course’s design and the empirical work conducted are also seen as a continuation of these studies aiming at refining and validating the definition of “visual literacy”. As there are many definitions of visual literacy, each visual medium has its own characteristics, producing different visual literacies and requiring different skills. For the purposes of our course, we define visual literacy as the ability to understand and produce visual messages… a group of competencies that an individual can develop by seeing and at the same time having and integrating other sensory experiences…a nd the ability to interpret messages as well as generate images for ideas and concepts.13

The need for visual literacy in the language classroom Well before the 21st century, we were flooded with visual messages of all types in a wide range of media, both in personal and private spaces. The ability to decode visual expressions and to consider them critically has become an essential skill for researchers and teachers in education. It is high time this skill was passed on to students as a powerful learning tool in 10

Debes, 1968 Hortin, 1994: 21 12 Avgerinou, 2003: 36 13 Bleed, 2005: 5 11

Anastasia Christodoulou and George Damaskinidis

197

the development of creative, critical and independent thinking. We do not imply that teachers and researchers have fully mastered visual literacy. On the contrary, we believe that imparting this body of knowledge to students will better enable teachers and researchers to reflect on their practices and to propel the field of visual literacy one step forward. Bernhardt14 argues that [r]eaders of on-screen text interact physically with the text. Through the mouse, the cursor, the touch screen, or voice activation, the text becomes a dynamic object, capable of being physically manipulated and transformed.

Bernhardt stresses our need to begin using visual literacy in the composition class given students’ increasing interaction with these ‘dynamic’ texts in school and on the job. Following these arguments, rather than merely viewing the video, our students were instructed to use it interactively to convey ideas and to solve problems. Nevertheless, visual literacy-based activities in the language classroom should be included with great caution. In a writing programme at Virginia Tech, teaching visual literacy was believed to exceed first-year composition level competences.15 This concern was voiced because the addition of lessons to cover visual literacy moved the academic essay to more electronic or visual formats. Therefore, by shifting the core writing requirements, first-year composition would have to shift to match the change. In foreign language teaching, visually literate teachers that are capable of choosing or creating visual images could use them to enhance student learning. In addition to teaching visual literacy skills to language learners per se, the integration of a visual component in some of, or all, the stages of the teaching and learning processes, might enhance the teaching of the four traditional language skills: listening, speaking, reading and writing. Visual literacy may also help teachers to design more attractive teaching and learning processes that could better engage students in learning activities. Studies have consistently revealed that teaching with words and visuals has more favourable results than teaching using words or visuals alone.16 Moreover, separating language and image would create an unnatural boundary. In the visual and verbal dyad, one is not higher than the other;

14

Bernhardt, 1986: 154 Brizee, 2003 16 Jin and Boling, 2010 15

198

The Development of a Visual Literacy Course in Higher Education

in fact, they complement each other. This is pertinent to McKim’s17 argument that “the thinker who has a broad command of graphic languages… can find more complete expression for his thinking”. Thus, recognising the value of the theories surrounding visual thinking and visual language (and their pedagogical implications), composition instructors are now applying elements of visual literacy in writing classrooms.

The structure of the course Although initially designed for language purposes, the course was flexible enough to be adapted across the curriculum. An example of such an approach is mirrored in an undergraduate anthropology course, where a research paper in the form of a print assignment was replaced with the creation of digital movies.18 In our course, the students were given electronic assignments, in the form of slideshows consisting of visuals, audio and short written commentaries, to demonstrate the application of relevant theoretical concepts. Table 4-1 provides an overview of the course. The course was structured around a basic pedagogical framework: orientation, research and evaluation. First, the students were introduced to the study, they then applied the new knowledge, and finally they went through a process of evaluation. Given the aforementioned limitations, we divided the course into 12 lessons of three academic hours each. With this SF-MDA model-based course, students can communicate in social practice and investigate meaning arising from the integrated use of multiple semiotic resources. The creation of meaning could be explored through three metafunctions: representational (narrative and conceptual structures), interpersonal (visual acts, social distance and perspective) and compositional (information roles, salience and modality), all potentially valid and necessary. These metafunctions are filtered through the elements of MDA examination: content, design, production, expression, distribution and discourse. This paper’s limitation precludes an in-depth analysis of the relevant theories. Overall, the course introduces the grammar and expressive potential of visual forms. It applies methods developed by the social sciences and humanities to study perception and interpretation of the visual world. It requires students to be both creators and interpreters by producing visual statements, such as visual essays, and expressions using new media.

17 18

1980: 124 Bleed, 2005

Anastasia Christodoulou and George Damaskinidis

199

Finally, it prepares students to view and understand information presented in modes used in a variety of disciplines and areas. Table 4-1: An overview of the structure of the course Lesson

Description

1

Assessing students’ visual literacy skills.

2

Introduction to visual literacy and the SFMDA model.

3-5

Lectures on the three metafunctions and videobased practical applications. Literature review of Greek sources

6-7

8-9 10 11 12

Holistic application of SF-MDA. Discussion and reflexion on visual literacy. Practical applications of SF-MDA in multimodal texts. Students’ literature review on visual literacy. Re-assessing students’ visual literacy skills. Retrospection on the course through written assignment and focus group interview.

Design Premeasurement Contact with the visual

Increasing Visual Awareness Consolidating Visual Literacy Postmeasurement Self-evaluation

Pre-measurement In this first stage, we assessed students’ visual literacy skills by means of a questionnaire and focus group discussion. We argue that there are, as yet, no established methodological tools to assess visual literacy skills. Therefore, we narrowed down this stage by presenting only its practical considerations, leaving theoretical input for later stages of this research. First, we used a semi-structured questionnaire divided into four sections. These sections consisted of a set of questions corresponding roughly to the three stages of the SF-MDA model and to a fourth stage called “the intertextuality of the video”. The questionnaire was accompanied by a print-out of 50 selected still frames extracted from the video. The interview was semi-structured and conducted immediately after the students had seen the video. It was structured around our analysis, but

200

The Development of a Visual Literacy Course in Higher Education

we allowed the discussion to expand into other areas as well. We did not help the students to answer the questions so as to ascertain the extent to which these questions are helpful without any support. We chose a short video (eight minutes long) from the military field, whose provocative content was expected to stimulate students to answer spontaneously either by arousing their curiosity or by raising anti-war feelings. Produced in the mid 70s, the video presents “Stinger”, a manportable, shoulder-launched, guided, missile air-defence system, in a documentary-like style.19 Due to space limitations, we have provided only a summary of the students’ answers. As regards the representational metafunction, half of the students divided the video into roughly three stages, 00:00-00:30 (introduction), 00:31-08:30 (main part) and 08:30-08:40 (conclusion), without justifying their answers. These time periods coincide (almost) with our four-part split of the video based on the different music motifs. Interestingly, one student divided the video deductively from the outcome, through a reverse course of action. Occasionally, there were sincere efforts to support their choices with references made to particular frames. For example, a few students identified triangles or squares formed by various participants making a group of people. In the interpersonal metafunction, the sound element played a key role. The music’s intention, according to the students, was to attract the viewer’s attention, either by producing negative emotions or by creating excitement. Additionally, it was noted, but without further elucidation, that the narrator’s intonation varied throughout the video. Data on the third metafunction is limited because the students ran out of time, thus answering very briefly or not at all. Of particular interest are answers of the type “the rigidness of the participants’” bodies attracts attention” or “the lack of colour gives to the video a sense of robustness and solidity”. The few students who managed to complete the fourth stage related the video intertextually to the genre of the historical collection and documentary. In the focus group interview that followed, some students said that the particular questions were designed to guide their thinking, but they could not tell in which direction. Also, the number of attached video frames was, according to the students, small for the purposes of the activity. Lastly, the video clip of a popular song was used to demonstrate Windows Movie Maker, a program available on every computer running on a Windows operating system. The students were introduced to the basic 19

See also Christodoulou and Damaskinidis, 2011

Anastasia Christodoulou and George Damaskinidis

201

functions of this program so as to be able to edit visual and audio elements of videos.

Contact with the visual The students were given a basic bibliography in visual literacy including international and Greek sources, and were asked to locate additional Greek sources. Additionally, they were introduced to the SF-MDA model. This introduction consisted of basic principles of visual literacy, systemic functional linguistics and multimodal discourse analysis. For three consecutive lessons, a full lesson was devoted to each metafunction, consisting of a brief lecture followed by a practical application to a video advertisement about the Porsche 911.20 Bearing in mind that multimodal analysis is time-consuming, the advertisement’s short duration (3.5´) allows for an in-depth analysis of the whole video, and as an advertisement it is suitable for analysing the association between verbal and visual elements. In the lectures, we used actual pictures from magazines and textbooks rather than images projected on slideshows so that students would engage in active, ‘tangible’ discussion amongst themselves and with their instructors, rather than just passively watching slideshows. The application of the three metafunctions involved discussion around short pieces, static frames and audio elements (sound/music/dialogue) extracted from the video. The students encountered three problems in this phase. First, they lacked the necessary theoretical foundations of the visual, and of systemic functional linguistics, though it is difficult to establish what entails a basic theoretical introduction to visual theory. Second, they encountered difficulties with the ‘technical’ (according to their own statement) terminology used to describe visual material. Third, they struggled to locate relevant Greek references for these metafunctions. Despite these difficulties, all students responded positively and demonstrated a willingness to participate in the lessons.

Increasing Visual Awareness A core strategy was to give students the opportunity to explore visual literacy by preparing a short literature review of a limited number of relevant sources from the Greek bibliography. Thereby, they would

20

http://www.youtube.com/watch?v=Z-lVDfb5PTI

202

The Development of a Visual Literacy Course in Higher Education

assume responsibility for their own learning, while at the same time honing their general research skills. The students had to comment on some of the references they had found in the previous phase and to determine to what extent the three metafunctions were covered in the Greek literature. In order to achieve a holistic application of SF-MDA, we used the same video as in the previous stage so that they could follow the complex discussion more easily. However, some students found watching the same video again and again boring, while others considered repeatedly listening to the accompanying music ‘depressing’.

Consolidating Visual Literacy Here, the floor was given almost exclusively to the students. They had to choose a six- to eight-minute long video and apply the SF-MDA model. They could choose to apply the entire model, or a particular section, to short extracts, static frames and audio elements from the video. Although they could prepare their slideshows in their own way, all students followed the model we presented. The videos ranged from advertisements and movie trailers to environmental and social messages found on the Internet. Most subjects were relevant to students’ potential dissertations or professional background. Additionally, the students submitted a Greek literature review on visual literacy as a written assignment.

Post-measurement We repeated Stage 1, the “pre-assessment” stage, in exactly the same experimental conditions, to find out the differences in students’ answers before and after their introduction to visual literacy skills. We did not aim to examine the extent to which the course increased visual literacy skills, but simply to explore its effect on students in order to improve subsequent versions of the course. We believe that it would be premature to establish generic criteria of what constitutes a visually literate person. On the whole, the students now provided more extensive answers and were more willing to elaborate on them, however without being very critical. Most noticeably, various sounds (e.g. narrator’s intonation, music and missiles launching) were associated with different effects, such as attracting the audience’s attention, denoting solemnity or connoting triumphal tones. Some students related the scientist’s white shirt with a sense of formality, scientism and research. But where the music and the narrator’s intonation were concerned, the students felt as if they were

Anastasia Christodoulou and George Damaskinidis

203

watching a film. Other students ignored all sound elements and paid attention only to the moving pictures. Another instance indicating an awareness of the interpersonal metafunction is the human silhouettes’ ‘rigidness’ as an attention-attracting device. Several students noted the lack of diegetic sounds such as dialogues, moving vehicles and lab activity. If the students had to remake the video today, they would prefer electronic music, a narration with a different intonation and more frames/shots of the missile, rather than humans, since the purpose is to present technology. Finally, some students answered simply by giving the number of a frame.

Self-evaluation As homework, the students had to compare and contrast their answers in the pre-assessment and post-measurement phases. We then conducted a critical analysis of all aspects of the course through a focus group interview with the students in the classroom. Of particular interest is the students’ acknowledgement that in the post-measurement phase their answers were affected by the theories and practical applications learnt during the course. The fact that they had already seen the video enabled them to be more attentive this time. Also, some difficulties in understanding semispecialised vocabulary during pre-assessment had been solved by then. All students felt somewhat uncomfortable discussing non-verbal elements, which probably stems from a lack of contact with visual literacy. Some students said that although the video was in fact a documentary, the music gave it the feel of a film, and they described this contradiction as a conflict between the ideational and interpersonal metafunctions. Finally, the students said that had they known in advance that there was a post-measurement stage, their answers in the second questionnaire would have been biased.

Conclusion In this paper, we presented the design, application and evaluation of a postgraduate course in visual literacy. Despite limitations concerning the number of instructional hours and students’ lack of formal learning in visual literacy, we reached some very interesting conclusions. Most interesting was our difficulty explaining two important aspects: why we need visual literacy skills, both as individuals and foreign language

204

The Development of a Visual Literacy Course in Higher Education

teachers, and the concept of video as an autonomous type of (multimodal) text that is to be viewed, edited and analysed for pedagogical purposes. We frequently had to resort to parallelisms with other more traditional methods of teaching and learning in order to demonstrate this need. For example, the participants in a video were compared to the characters of a novel, and the different frames, shots, scenes and sequences to the unfolding of the plot. The SF-MDA model provides an effective tool for teaching visual literacy skills, yet is dependent on background knowledge and theoretical concepts of the visual as a semiotic mode of communication, as well as systemic functional theories and verbal-visual associations. The questions asked and the different stages remained the same, rather than being modified after taking into account the video as a channel of communication, or its particular subject. This reinforces the need to refine the model by developing different questions, and even stages, for different types of multimodal text. The students were initially unable to see the overall purpose of analysing visual material for teaching language skills. The literature review they prepared was poor in Greek references. Nonetheless, bearing in mind their limited background in visual literacy, they were relatively eager to identify possible areas of application for the SF-MDA model and became excited about choosing their own videos to present their application of the model in the classroom. Most of these presentations showed an understanding of multimodal principles, as demonstrated in the classroom, without however going into in-depth critical analyses. Their choice of video shows an understanding of the most appropriate video-text for their own teaching practices. Our initial intention was to provide a very specific, though sharply defined, point of view in visual literacy, namely that of the SF-MDA model applied to the analysis of a video. Hopefully, this approach will be supplemented by other approaches in verbal-visual education. The findings reported here point to the need for further research towards gaining an understanding of how to aid the development of teachers and students’ visual skills in language teaching and learning. Finally, we are faced with the dilemma of whether to offer this course again to postgraduate students or whether to design a modified one for undergraduates. Although still undecided, we are inclined towards the second option because we consider students who lack formal education in visual literacy ill-equipped to be introduced to the SF-MDA model.

Anastasia Christodoulou and George Damaskinidis

205

Bibliography Avgerinou, Maria. D. “A Mad-tea Party no More: Revisiting the Visual Literacy Definition Problem.” In Turning Trees, edited by Robert E. Griffin, John Lee and Vicki S. Williams, 29-41. State College, PA: International Visual Literacy Association, 2003. Bernhardt, Stephen. “Seeing the Text.” College Composition and Communication 37, no. 1 (1986): 66-78. Bleed, Ron. “Visual Literacy in Higher Education”, EDUCASE Learning Initiative, (2005) http://www.educause.edu/ir/library/pdf/ELI4001.pdf Brizee, H. Allen. Teaching Visual Literacy and Document Design in FirstYear Composition. MA Dissertation. Virginia Polytechnic Institute and State University, 2003. Christodoulou, Anastasia and Damaskinidis, George. “Documentary Readings: A Visual Social Semiotic Analysis of Stinger: A New Weapon with an Age-old History.” In Retorica del Visibile. Strategie dell’Immagine tra Significazione e comunicazione. 2. Comunicazioni, 2 (3), edited by Tiziana Migliore, 1431-1442. Roma: Aracne, 2011. Debes, John. “Some foundation of visual literacy.” Audiovisual Instruction 13, (1968): 961-964. Hortin, John. A. “Theoretical Foundations of Visual Learning.” In Visual Literacy: A Spectrum of Visual Learning, edited by David M. Moore and Francis M. Dwyer, 5-29. Englewood Cliffs: Educational Technology Publications, 1994. Iedema, Rick. “Analyzing Film and Television: A Social Semiotic Account of Hospital: An Unhealthy Business.” In Handbook of Visual Analysis, edited by Theo Van Leeuwen and Carey Jewitt, 183-206. London: Sage, 2001. Jewitt, Carey and Rumiko Oyama. “Visual Meaning: A Social Semiotic Approach.” In Handbook of Visual Analysis, edited by Theo Van Leeuwen and Carey Jewitt, 134-156. London: Sage, 2001. Jin, Sung-Hee and Boling, Elizabeth. “Instructional Designer's Intentions and Learners' Perceptions of the Instructional Functions of Visuals in an E-learning Context”. Journal of Visual Literacy 29, no. 2 (2010): 143-166. Kress, Gunther and Theo van Leeuwen. Reading Images - The Grammar of Visual Design. London: Routledge, 1996. McKim, Robert. Experiences in Visual Thinking. Monterey: Brooks/Cole, 1980. New London Group. “A Pedagogy of Multiliteracies: Designing Social Futures.” Harvard Educational Review 66, (1996): 60-92.

206

The Development of a Visual Literacy Course in Higher Education

Tseng, Chiaoi. “Coherence and Cohesive Harmony in Filmic Text.” In Multimodal Semiotics: Functional Analysis in Contexts of Education, edited by Len Unsworth, 87-104. London: Continuum, 2008. Turner, Graeme. “Film Languages.” In Media Texts: Authors and Readers, edited by David Graddol and Oliver Boyd-Barrett, 119-135. Clevedon: Multilingual Matters in association with the Open University, 1994.

“MY FIRST EXPERIMENT” “MY FIRST EX”: A MULTIMODAL TOOL PROPOSED IN THE DIDACTICS OF LITERATURE SYMEON DEGERMENTZIDES

Generals The educational software proposed – which I have called “My First Ex” – was used during a year-long (2008–2009) action research in two Gymnasiums (Thessaloniki, Greece), incorporated into flexible constructivist learning environments (Didactics of Literature): due to technical difficulties, the students mainly used it as a digital tool for their homework assignments. Structured in the tripartite synchronic signification of reality (material, social, semiotic), the theory of Hybrid Signs that I am proposing in my capacity as researcher is semiotic and fictional at the same time, being in connection with socio-cultural data characterising its practices (Degermentzides, 2011): “My First Ex” served as a tool for practical application of the basic structuring mechanism of the theory of hybrid signs, multimodal metaphor, i.e., copying/cutting and pasting of material, social and semiotic data and is based on Internet search for information, which the student must then organise with critical thinking, in order to write his/her own text. Therefore, since special techniques and digital data management skills are required by the student, we enter into more special issues of technological literacy. Apart from the evaluative criterion of the contents of writing – which presupposes a qualitative analysis of related data – we can detect certain factors and special features determining the technical, aesthetic and communicative philosophy of every website. My First Ex aims at investigating the degree of influence of such multimodal factors on the student using quantitative criteria. In other words, I seek to correlate the student’s interest in a website with more specific multimodal factors making that particular website attractive to the student, and I document my findings with statistically measurable results. The objective is to arrive at conclusions about different criteria and conditions for

208

A Multimodal Tool Proposed in the Didactics of Literature

selecting data in the construction of new knowledge, when the students browse the web to gather related information.

Brief presentation of the objectives and operation of My First Ex “My First Ex” is based on a constructivist design philosophy and has been tested for quite a long period of time, so as to be subject to the modifications required in order to make its use more functional.1 My principal objective is its usability; thus, it is characterised by simplicity in operation. Apart from the ease of transition from one step to another, special attention is given to cooperative learning [the software has been designed for cooperation among three students (Anthoulias, 1989:67-69)], which takes place in the classroom, whereas, in case this is not possible for different reasons, teamwork takes place in the classroom without the use of Educational Software and collection of digital material is done individually at home. Although the use of “My First Ex” is easy, the related software includes a very complex background for processing multimodal data: in particular, the student gives a code, in order to prevent others from having access to his/her personal data and, at the end of his/her search on the web sources, the first statistical data of multimodal factors are presented. Further processing of metadata, which are stored using special tools (Mathematica, Matlab), gives us both the mind diagram of the digital path of each student and graphs or associative representations of multimodal factors.

My First Ex ƒ Is based on constructivist foundations and, mostly, on the Vygotskian conception of the meaningful word, according to which the meaningful word is the microcosm of consciousness (Vygotski, 1988: 436). ƒ Serves as a tool for practical application of the basic structuring mechanism of the theory of hybrid signs, multimodal metaphor, i.e., copying and pasting material, social and semiotic data. ƒ Aims at developing the student’s meaning-making manipulations, thus rendering Literature a conceptual bridge between Myth and Reality.

1

I owe my grateful thanks to the analyst and programmer, Mr. Vasilios Iliadis, whose contribution has been invaluable in terms of the technical part of the software.

Symeon Degermentzides

209

ƒ The student browses the web using as conceptual coordinates the keywords he/she chooses for each of the three fields mentioned above (LITERATURE, MYTH, REALITY) in order to collect and record relevant information. The student’s final aim is to write an essay or a literary text, where he/she will compose any information drawn from all three fields: for example, he/she may cultivate his/her creative writing, creating a dialogue with a mythical hero, a character of a literary book and a real person or a real fact. The student may choose any word he/she wishes as a keyword for every field. The difficulty of the entire project lies in the fact that there is no rational assumption in advance for connecting the relevant data; as a result, its construction rests exclusively with the student’s learning ability2. The student will use the software to surf the web, based on the keywords of the search, which he/she enters himself/herself or after a dialogue with his/her classmates on the subject, if this has been preceded by cooperative learning. At the end, quantitative data on the student’s search path and on the websites the student has followed, will be gathered. In particular, the pieces of information that the software will record are: ƒ ƒ ƒ ƒ ƒ ƒ

the name of the website (url) the time the user spends on that site the background colour of the web page the number and size of images on every web page the number of videos on every web page the number of paragraphs, fonts and styles on every web page

In addition, general information will be recorded, e.g., which student asks for a website, as well as the motives of such search. Moreover, it is possible to record further personal data per student, while there will be a simple, though useful, Notepad available for the student, so that he/she may save any information of his/her essay he/she considers useful.

2

Cf. Brad Hokanson & Robert Fraher, (2008), where the prerequisite for placing the structure of a myth in educational planning is the ‘connection’ between ‘the content and the student’, while ‘the most effective structure for such a connection is achieved through a narrative form’ (31).

210

A Multimodal Tool Proposed in the Didactics of Literature

Operation of My First Ex The toolbar includes the well-known Word icons with the corresponding scrollbars, while the left and right arrows are a useful tool invented to help the student move forward or backward, until he/she finds his/her cognitive ‘step’. Below, there is a typical presentation of the basic steps of use of “My First Ex” through the presentation of the related displays, in order to make its operation clear. I must note that the related steps were given to the students and the teacher both in electronic and in printed form right after my intervention during the action research, so that they could answer by themselves any of their questions at any time. 1st screen: The user chooses to activate either the main software application, or word processing. 2nd screen: The initialisation process is activated. The program supports one to three students for teamwork. 3rd screen: The student enters his/her personal data. 4th screen: Every student enters a keyword per field. 5th screen: A window uses the all-powerful Google search engine as the main search mechanism of the software. Google search engine provides a list of websites with thematic content related to that of the keyword. 6th screen: The student may start from any field and use the corresponding keys of the software to move forward or backward. 7th screen: The student chooses “New”. 8th screen: The student chooses “Draft” and then “ȅȀ”. 9th screen: We have two active windows at the same time. The second window provides a plain word processor, so that the student may save any useful information he/she finds on the Internet. In addition, it is possible to save or print in a separate file. 10th screen: The student chooses the piece of information he/she is interested in and then “Copy” to copy the text. 11th screen: The student puts the cursor on the Draft Note and then chooses “Paste” to paste the text. The program automatically saves the information on the background. 12th screen: When the web search process is over, the student chooses “Save As”, to enter the notepad in the desirable storage unit as *.txt.file and process it subsequently during text writing. 13th screen: At the end of the entire process, the teacher/researcher presses the “ǹǹ” key and then enters the protection code for the student’s personal data, in order to ensure the confidentiality of his/her analysis through a code.

Symeon Degermentzides

211

14th screen: The preliminary data results have been created. In the meantime, the metadata, which require further specialised analysis using additional software tools (mathematica, matlab etc.), have been saved in another file.

Multimodal factors and mind diagrams: utilisation in teaching Multimodal factors as criteria of website attractiveness The multimodal factors, whose role I am examining, are: hyperlinks, images, video files/links, audio files/links, paragraphs, fonts, background colours, font colours, and font styles. Their categorization, according to the three levels of the theory of hybrid signs, results in the following classification: Table 4-2 Multimodal Factors

Categorisation of Multimodal Factors According to the Theory of Hybrid Signs Multimodal Factors of Hyperlinks Student’s Time Per Representational Web Page: Structures in Digital Hybrid Similarities Texts Under Construction Multimodal Factors of Images Student’s Time Per Interactive Resources in Video Files/Links Web Page: Digital Texts Audio Files/Links Hybrid Dialogue Under Construction Multimodal Factors of Paragraphs Student’s Time Per Synthetic Meanings in Fonts Web Page: Digital Texts Background Colours Hybrid Metafiction Font Colours Under Construction Font Styles

212

A Multimodal Tool Proposed in the Didactics of Literature

Mind Diagrams Navigation of the student through websites is not a linear process with a beginning, a middle part and an end, but it is rather characterised by its dynamic character and the synthetic construction of meanings. This evolutionary course of the student’s thinking during his/her web search may be illustrated through “My First Ex” in the form of a mind diagram, accurately recording the degree of complexity characterising the student’s reasoning according to the order of websites he/she has visited. The objective for the teacher is to form a clearer opinion of the quality of knowledge constructed by the student, with the help of a diagrammatic representation of the way the latter connects myth, literature and real facts.

Quantitative analysis and statistical measurements Introduction Use of the software results in data and metadata, which enable us to draw up the mind diagram of each student (that is, the entire course of his/her navigation through the web for drawing information based on the keyword of each field) and draw useful conclusions about the effect of multimodal factors on the formation of his/her thinking, using statistically measurable parameters3. These two mechanisms are particularly helpful for analysing the digital Zone of Proximal Development (Hedegaard, 1990) for every student, but in this paper a more specific analysis of such form is omitted, given that it does not constitute a substantial objective of my research action. Therefore, I shall confine myself to briefly presenting the general results drawn from the digital information search process by the students, with the help of “My First Ex”.

3 Cf. Somekh, B. (2007), regarding the ‘mapping of concepts as means of understanding the way students apprehend New Technologies in their world’ (167): “The method draws directly on Vygotsky’s conception of ‘instrumental’ psychology, by which ‘higher functions incorporate auxiliary stimuli, which are typically produced by the person himself (sic)’. In other words, our efforts to achieve any outcome are supported by cognitive tools which are an integral part of our skilled use of actual artifacts”. As an example, I am citing a similar case, where ‘the evaluation of the method of concept mapping was checked through a qualitative analysis’ and, as Somekh reports, this was done for a total of two thousand students, approximately (169).

Symeon Degermentzides

213

Example of a student’s mind diagram In the following example, we can see the typical representation of a mind diagram, i.e., the path that this particular student has followed during the pilot presentation of “My First Ex” in the computer lab of the school complex, in order to collect useful information for subsequent writing. I gave the students complete freedom to enter any keyword they wanted in every field, in order to rouse their curiosity more and stir their imagination. The choices made by the student called Odysseus excited the interest of all students: the anagoge of the literary character Captain Nemo from the novel “Twenty Thousand Leagues Under the Sea” by Jules Verne to the mythical archetype of Jason in the context of the Argonautic Expedition was related to reality through Greece’s triumph in the European Football Championship, Euro 2004! Example:

Myth: Jason (Argonautic Expedition) Literature: Captain Nemo (20000 leagues under the sea) Reality: Greece winning the European Football Championship What is evident from the above-mentioned typical example is the student’s cognitive feedback while collecting and processing information on the Internet, a condition that helps him/her construct new knowledge in terms of a dynamic access to digital data ƒ ƒ

both on a level of conceptual mixtures and on a level of meaning-making manipulations.

214

A Multimodal Tool Proposed in the Didactics of Literature

One can easily discern the dynamic character of the student’s navigation through websites, a process that gives him/her the necessary background to proceed to the synthetic construction of his/her notional correlations. With a similar mind diagram for every student at their disposal, the researcher and/or the teacher are able to ascertain the following, among other things: ƒ the exact course of the student’s digital path according to the order of web pages he/she has visited, ƒ the evolutionary course of his/her thinking while searching for sources of information on the Internet, and ƒ the degree of complexity characterising his/her reasoning. Such a diagrammatic representation of the way a student connects the fields of myth, literature and reality may serve as an auxiliary teaching tool, so that the researcher and/or the teacher may ascertain the quality of the student’s cognitive constructions and evaluate the writing process as a whole, forming a clearer opinion of the phase of collecting and processing digital data.

Overall quantitative data of multimodal factors Below I am selectively citing – given the large size a full presentation would occupy – the overall data of measurement of multimodal factors and characteristics concerning the technical, aesthetic and communicative ‘identity’ of websites visited by the students throughout the research period. My objective is to investigate the frequency of occurrence of such multimodal factors and draw conclusions about their role in the meaningmaking manipulations of the students. Considering the multimodal factors as criteria of website attractiveness for the students, it is deemed necessary to record their graded role with regard to the influence they have using quantitative criteria: on the one hand, the statistically measurable results allow one to investigate ƒ the criteria and ƒ the conditions of data selection during the cognitive construction, and, on the other hand, to deal with more specific issues of digital literacy. The way I have classified the multimodal factors in the three following categories, evaluating their role and function in digital texts based on the criteria of hybrid similarities

Symeon Degermentzides

215

(Representational Structures), hybrid dialogue (Interactive Resources) and hybrid metafiction (Synthetic Meanings), was described in the abovementioned theory of Hybrid Signs. Thus, the overall quantitative data, which resulted during the research period, are: ƒ ƒ ƒ ƒ ƒ ƒ ƒ ƒ ƒ ƒ ƒ

Total web pages: 522 Average number of hyperlinks (per web page): 111.53 hyperlinks Average number of images (per web page): 12 images Average number of audio files/links (per web page): 0.01 (or 1% of web pages contains audio files) Average number of video files/links (per web page): 0.03 (or 3% of web pages contain video files) Average time spent by the student (per web page): 75 seconds Average number of background colours: 2.6 (per web page) Average number of font colours: 2.1 (per web page) Average number of paragraphs: 8.3 (per web page) Average number of fonts used: 4.5 different fonts [bold/italics/ headings, or the font size, have not been included] Average number of font styles: 9.3 including different font sizes

An interesting element resulting from the above-mentioned data is the extremely small number of video and audio files/links per web page, which suggests that the designers of Greek websites mostly choose to communicate with their users ‘interactively’, i.e., through images.

Basic Diagrams Relation between time and web pages Remarks The time a student spends on each web page ranges from a few seconds to more than three minutes. The final conclusion drawn is that, on average, the students do not seem to feel obliged to read the entire page; on the contrary, their interest lasts for about two minutes maximum. During this period of time, they focus their attention on selected points, depending on the attractiveness of multimodal factors incorporated into the web page. This condition implies that the student/user usually accepts the multimodal ‘proposals’ of the designer of the site, who has formed it according to his/her own objectives. Thus, the short-term navigation of the student

216

A Multimodal Tool Proposed in the Didactics of Literature

through every web page presents increased probabilities of collecting the data that the designer wishes to set off.

Multimodal Factors of Representational Structures (Hybrid Similarities): Relation between web page and links Remarks The number of links per web page is rarely less than 10 and on most web pages it exceeds 100 links. In some cases, it may even exceed 200 links. The incorporation of a large number of links in the structure of the web page refutes the static notion of culture and the unimodal access to resources that produce meaning through writing. The collected data reveal that the design philosophy of the web pages that the students visited, places a large part of its expectations to the direct connection of the users to other digital sources, thus echoing a dynamic conception of culture, which gives priority to alternative considerations and multimodal constructions of material reality.

Multimodal Factors of Interactive Resources (Hybrid Dialogue) Remarks The measurements only refer to static images, ActiveX and advertisements with a scrolling image. In addition, the system only includes globally recognised and valid archive types of images. The sizes of the images have been recorded, despite the fact that I am not presenting them here. Their size ranges from very small to relatively large (640x480 display data), although large images were very rare. An obvious reason for that is that large images are avoided, because there is a delay in the downloading of the website. The fact that, on average, every web page includes about 12 images shows that the students prefer designers who seek to develop a hybrid dialogue with the user. The interactive relationship between them is interpersonal and mediated by the different Designs represented by each one of them: the criterion of Dual View of Interpersonal Resources may be applied here, given that the designer prompts the user to alternate semiotic modes, having ensured to offer the user alternative modes of dialogue with his/her digital material. Thus, the students searched for their fictitious interlocutors behind the images and, since their choices acquire great

Symeon Degermentzides

217

weight, it is established that the students were open to dialogue, mainly choosing websites with versatile presentation and ideology, usually in the form of multimodal verbal and visual resources.

Multimodal Factors of Synthetic Meanings (Hybrid Metafiction): Background colours per web page Remarks Here, I am measuring the different background colour of the web page. I note that even the slightest change in the colour of the web page (e.g., a pixel), may be considered as a different colour by the system. Some web pages had images on the background; these were not recorded however as different background colours, but rather as images. The formation of semiotic reality as an exchange of informative material between mostly verbal and visual structures on the websites visited by the students is associated with an alternation of background colours and constitutes a multimodal criterion of correlation of hidden analogies in various elements of meaning-making resources: by choosing colour mixing as background of meaning-making resources, the students choose the renegotiation of their abilities (affordances). The different background colours perform multimodal metaphors, compressing different cultural and aesthetic contexts through framing: the connotative interference of similar stylistic or chromatic selections refers to a different cultural or aesthetic pattern, attracting the student/user.

Paragraphs per Web Page Remarks A paragraph is defined as any appearance of the special paragraph mark on the web page, which constitutes an indication of the beginning of a paragraph. Sometimes this criterion may possibly lead to a false measurement; such cases, however, are quite rare. The structure of verbal resources in eight to nine paragraphs per web page reveals that the students prefer a narrator who ‘breaks’ the narrative flow of his/her text to ‘discuss’ with them. This rupture of the narrative/mythical structure runs along with the contribution of different background colours and images. The above-mentioned process of alternation of semiotic modes, which forms the width and the possibilities of reinterpretation of semiotic reality of every web page by the students, is

218

A Multimodal Tool Proposed in the Didactics of Literature

combined with the homological function of the alternation of paragraphs in the narrative resources as possible ‘breaks in the synthetic myth’ that the designer deliberately leaves in order to talk to the student/user, or the latter discovers in order to interpose his/her own interpretation of semiotic reality (the end of each paragraph signals the potential integration of treated meanings and gives the student/user the sensation of temporarily releasing him/her).

Results of correlations in terms of the multimodal factors of Interactive Resources: Diagrams of time spent per web page in relation to the number of its images Time per web page when there is no image Average time spent: 26.4 seconds Time per web page when there are 1–3 images Average time spent: 75.4 seconds Time per Web Page when there are 4–8 images Average time spent: 137.2 seconds Time per web page when there are more than 8 images Average time spent: 56.04 seconds

Summary of Results Relation between Time and Images Remarks With regard to the metadata of web searches in the computer lab of the school, this result admits of a possible interpretation: due to the slow Internet connection of the school network, loading of images – especially of large images – is a time-consuming process. Given the limitations to which the interest of the student, and of the Internet user in general, is subject to, this particular result must not be considered to be cut from the general context where it is placed. On the contrary, the Internet connection speed must also be incorporated into the context of interpretation. Therefore, it is rational to assume that the faster the Internet connection speed is, the greater the number of images that the student may download is, before he/she loses his/her patience.

Symeon Degermentzides

219

However, the greater number of metadata of web searches were saved on the PCs that the students had at home and were transferred with USB flash drives, so that the above explanation is only partially valid. The interpretation of the students’ preference for web pages with four to eight images is explained based on the fact that the fictitious dialogue developed between the two subjects (designer and student) finds a better channel of communication with an average number of images: when they are fewer, the interpersonal interaction is not adequately mediated, while when there are too many images, the interactive relationship between the two persons involved in this metafunction of subjects becomes difficult due to the multitude of interposed images–points acting in a disorienting way.

Results of correlations in terms of the multimodal factors of Synthetic Meanings: Results of Correlations of Time and Background Colours ǹ. One background colour Average time spent: = 93.5 seconds Ǻ. Two background colours Average time spent: = 64.1 seconds C. Three background colours Average time spent: = 77.5 seconds D. Four background colours Average time spent: = 85.3 seconds Ǽ. Five background colours Average time spent: = 74 seconds

220

A Multimodal Tool Proposed in the Didactics of Literature

Correlation Results

Remarks The above diagram shows that the background colour is not directly involved in the determination of the average time the student spends. On the contrary, the only unambiguous element is that the students have spent more time on web pages having a single background colour. In this particular case, the related choice lies in the fact that the students have not paid so much attention to the background colour, but rather to the search for information, focusing their attention on the multimodal factors, e.g., links and images. This means that, while they visit web pages with different background colours and this particular multimodal factor is a pole of attraction, finally, it does not maintain its power for a long time, since the students very quickly focus on verbal resources.

Symeon Degermentzides

221

Results of Correlations of Average Time & Background Colours & Paragraphs per Web Page

Remarks Here, I am making an attempt to correlate the paragraphs and the background colours per web page with the average time of the student/user. We can clearly see that the latter prefers plain background colours, in order to study a text. In addition, the number of paragraphs does not substantially affect the student’s time to a large extent, although – as we can see – the latter mostly prefers one or two paragraphs: certainly, this correlation is also connected with the size of the paragraphs. In broad outline, in the case of number of paragraphs, what was true for the previous case of background colours of the web pages also applies here: while the students are attracted to texts with several alternations of paragraphs – as in the case of different background colours – in essence, these multimodal factors do not play a crucial part and do not keep their interest keen for a long time, since the students practically seek simplicity and resolution in the background, but also seek short texts, of one or two paragraphs, which will give them quick solutions.

222

A Multimodal Tool Proposed in the Didactics of Literature

Conclusions regarding the statistical analysis of multimodal factors It was established from the extensive use of “My First Ex” during the action research that the role of the multimodal factors of every web page in the formation of the student’s thinking is important, since the related process constitutes the starting point in the organisation of digital material with critical thinking and final writing. In this sense, both the factor of the individual absorption rate of interpretable reality and that of each student’s self-activity in original interpretative attempts are associated with special techniques and digital data management skills by the student, i.e., they are related to more specific issues of technological literacy. A large part of the effect of the socio-cultural context on the student’s meaningmaking manipulations with the help of “My First Ex” became detectable and was investigated in the interactive learning environment between the latter and his/her web sources. In particular, “My First Ex” served as a tool for the practical application of multimodal metaphor: the quantitative analysis of digital data and the statistical measurements of multimodal factors based on the students’ choices on the Internet provided typical answers with regard to the mechanisms of multimodal metaphor. The multimodal factors were classified into three categories, evaluating their role and function in the digital texts based on the criteria of hybrid similarities (Representational Structures), hybrid dialogue (Interactive Resources) and hybrid metafiction (Synthetic Meanings) according to the theory of Hybrid Signs. The graded role of multimodal factors as criteria of attractiveness of the websites enabled me to investigate the criteria and the conditions of data selection. Thus, “My First Ex” allowed me to draw useful conclusions about the multimodal choices of the students, the mode of processing of the material under consideration and the factors that influenced the structuring process of new knowledge.

Symeon Degermentzides

223

Bibliography Anthoulias, Tassos. ȆȜȘȡȠijȠȡȚțȒ țĮȚ ǼțʌĮȓįİȣıȘ (Informatics and Education). Athens: Gutenberg, 1989. Degermentzides, Symeon. Unpublished PhD thesis ȂȪșȠȢ țĮȚ ʌȠȜȣIJȡȠʌȚțȩIJȘIJĮ: ıȪȖȤȡȠȞİȢ ʌȡȠıİȖȖȓıİȚȢ ıIJȘ įȚįĮțIJȚțȒ IJȘȢ ȜȠȖȠIJİȤȞȓĮȢ (Myth and multimodality: modern approaches to the didactics of literature). Rethymno: Crete, 2011. Hedegaard, Mariane. "The zone of proximal development as a basis for instruction." In Vygotsky and education: Instructional implications and applications of sociohistorical psychology, edited by Luis C. Moll, 349-371. Cambridge: Cambridge University Press, 1990. Hokanson, Brad and Robert Fraher. "Narrative Structure, Myth, and Cognition for Instructional Design." Educational Technology 48, no. 1 (2008): 27 – 32. Somekh, Bridget. Pedagogy and Learning with ICT – Researching the art of innovation. Oxon: Routledge, 2007. Vygotski, Lev. ȈțȑȥȘ țĮȚ ȖȜȫııĮ (Thought and Language). Rodi, A. (trans.). Athens: Gnosi, 1988.

DEVELOPING STUDENTS’ VISUAL DESIGN COMPETENCE THROUGH SITUATED LITERACY PRACTICES: THE CASE OF THE ERASMUS IP “P.S.BOWMA” CATHERINE DIMITRIADOU AND ANDRONIKI GAKOUDI

Introduction People, places and things included in any learning context are combined in visual ‘statements’, whereby they can articulate a multimodal environment for embedded learning (Dimitriadou 2010). This multimodal environment consists in the material space that includes natural and man-made environment, the structural elements of which reflect social structures and institutions. According to the theory of symbolic interactionism (Dennis and Martin 2005) these elements signify the communities’ potential, reflecting human ideologies and offering meaningful interpretations for culture (Edwards and Usher 2000; Dimitriadou and Kesidou 2008). Thus, the experiential study of an area gives opportunities for the examination of a wide range of literacies concerning the local history, the social structure and the cultural identity of the communities located in the area. The present paper explores the ways situated literacy practices can contribute to the development of student teachers’ semiotic awareness and their competence as educational agents through at least two interrelated and successive procedures: making signs and creating teaching scenarios in the form of multimodal texts. The first part outlines the context of the study, its relation to the structure of the IP and to the space where it was developed. The aims of the study and the method of data analysis are included in the second part. Finally, the outcomes of the analysis are discussed and concluding remarks are made in the third part.

Catherine Dimitriadou and Androniki Gakoudi

225

The context: an Erasmus Intensive Programme (IP) In 2010, a two-week (28 June – 9 July) Erasmus Intensive Programme1 (IP) was developed by three Universities: a) Western Macedonia (WM) (co-ordinating university), Florina, Greece, b) Ljublijana, Slovenia and c) NHL, the Netherlands. The participants of all three universities were given the opportunity to approach the challenges of living in the borderlands and experiencing the landscape as a crossroads formed by three countries: Greece, F.Y.R.O.M., Albania. As its title suggests, “People and Space in the Borderland of Western Macedonia: tracing historical, social and intercultural features” (P.S.BoWMa), the IP was a lifelong learning programme focusing on the exploration of the natural and man-made environment in the borderland of the north-west corner of Greece (www.eled.uowm.gr/ip). During the programme, the borderland of WM was transformed into a vast international classroom where 27 students (21 student teachers and six ethnographers) and 19 teachers were actively involved in situated literacy practices by interacting with the local community through observation and interpretation while being involved in literacy events and practices.

Multiliteracies and Semiotic practices within the IP “P.S.BoWMa” Given that visual literacy could inform the university education being offered across all disciplines – the humanities, the social sciences and even the natural sciences (Elkins 2008; Mitchell 2008, 14) – one of the IP goals was to develop a curriculum and a pedagogic model informed by the theory of Multiliteracies and Semiotic Pedagogy. Additionally, drawing on Barthes (1977), Kress and Van Leeuwen (2001) and Van Leeuwen (2005) the IP curriculum was designed to focus on the visual character of a variety of symbol systems that could function as signifiers for the WM borderland area. These signifiers could also serve as resources for shaping meanings through flexible teaching procedures on the part of the participants. This transformation of signifiers into educational resources highlights the importance of exploiting visual design for the development of the future teachers’ teaching competence. The pedagogic model mentioned above was designed in order to make the multimodal design of texts explicit to students as one way of exploring 1

An IP is a short programme of study which brings together students and staff from Higher Education Institutions of at least three participating countries.

226

Developing Students’ Visual Design Competence

multimodal meaning through the construction of teaching scenarios. To this end, participants x were immersed in an acquisition-rich environment, with a focus on situated practice (experiences and designs were available in the real life worlds); x were given overt instruction through the analysis of how image and word are organised in multimodal texts as well as an analytical vocabulary for understanding the design process in meaningmaking systems; x were asked to connect meanings to their social contexts aiming at the interpretations of the context of designs (critical framing); and x were prompted to recreate and recontextualise meaning across contexts through the compilation of teaching scenarios (transformed practice) (Cope and Kalantzis 2000). The participants’ involvement in the last procedure was based on the idea that a teaching scenario is a highly complex, interrelated sign system with formulations and configurations of symbols, icons and indexes (Bopry 1994, 37).

The concept of design A key concept throughout all the phases of the IP was that of design, which characterised the processes of knowledge co-construction, recontextualisation and situated teaching (Table 4-3). The students were asked to collect material relating to the concept of borderland in the context of WM and transform it into teaching material for a small-scale elementary or secondary school project. Using the affordances of computer technologies (Unsworth 2001, 12-14) they compiled PowerPoint multimodal texts, which were approached as a process of design aiming to induct learners to the idea of the borderland of WM as experienced within the IP. Within this process, the teachers were both sign-makers and designers at the same time. While designing their teaching scenarios, the students would give meaning to things and features around them by interpreting and handling symbolic, i.e. semiotic resources. The students’ conceptualisation of cultural meanings, while being derived from their interaction with, and negotiation of signs, were expected to influence their representations created within the above context. In this sense, the participants were transformed into designers of the context of borderland

Catherine Dimitriadou and Androniki Gakoudi

227

they had perceived, after having recontextualised the content of the subject matters taught in the IP. Table 4-3. The semiotic practices comprising the IP P.S. BoWMa within the concept of “design” The five levels of “design” LEVEL

DESIGN PROCESSES

1st

Constructing the culture of WM in the context of IP

2nd

Recontextualising the culture of WM in the context of knowledge to be taught

3rd

Recontextualising the culture of WM as knowledge to be acquired

4th

Recontextualising the culture of WM in the context of compiling teaching scenarios

5th

Exploring the participants’ creativity and recontextualising the teaching scenarios as research data

DESIGNER The IP organisers as designers of the IP curriculum The teachers as designers of lectures and workshops The students as designers of the knowledge acquired during the IP The students as designers of teaching material The researchers as designers of analytical tools

TOOL

The IP curriculum

Situated literacy practices

Educational content to be accessed

Teaching scenarios

Semiotic analysis of the data

The aim of the study Based on the idea that teachers can transfer representations of the world to the students by using multimodal texts, the aim of the study was to show

228

Developing Students’ Visual Design Competence

that curricula informed by a pedagogy of visual literacy (Simons, 2008) can contribute to developing students’ awareness on the contextual nature of knowledge production and also offer student teachers opportunities to incorporate multimodal representations of the world into flexible teaching practices.

Research questions The research questions posed are as follows: 1. 2.

What symbols and meanings of the region of WM were selected by the IP participants as resources encoding life in the borderland? How did the participants recontextualise these resources as visual representations of the borderland reality in a semiotic pedagogy perspective?

Attempting to answer the aforementioned research questions, emphasis will be placed on the ways in which semiotic resources are deployed in order to develop meaning-making systems in a variety of contexts, thus facilitating teachers’ informed and effective intervention in students’ learning (Unsworth 2001, 2-3).

Study method In order to answer the first research question, the teaching scenarios were analysed as multimodal texts consisting of slides, which in turn were analysed as images following Kress (2003). First, Semiotic Analysis, as a practice of description and an analysis of signification, was employed for the examination of the detailed inventory of the resources used in compiling the participants’ teaching scenarios (Table 4-4). This examination involved the description of ‘objects’ that resulted from the participants’ involvement in a process of social construction through the relationship of the “signifier/signified” signs. Second, Kress and Van Leeuwen’s (2001) grammar of images method was used to analyse the photographs and maps so as to establish how visual representation of WM influenced the participants’ understandings of place. As for the implications deriving from participants’ familiarisation to techniques of creating teaching material incorporating visual representations of reality, assumptions were made as to how they transformed semiotic resources to communicative facts.

Catherine Dimitriadou and Androniki Gakoudi

229

Data analysis Nine teaching scenarios in the form of PowerPoint presentations comprised our data. As the analysis of the PowerPoint slides shows, in Table 4-4, each of the multimodal texts corresponded to a variety of symbols selected for the teaching of the concept of WM borderland. Specifically, the participants chose to refer to representational (denotative) and symbolic (connotative) signifiers (Barthes 1977, Van Leeuwen 2003) in landscapes, pictures, drawings, photographs that correspond to concepts in a narrative or conceptual way: two scenarios selected symbols from the natural environment of the WM borderland (“Arcturos”, “The Dam”), another three made use of historical and religious symbols (“Landmarks in Prespes”, “Tracing Monuments in Western Macedonia”, “Museum and Archaeological sites”), while the remaining four referred to representations that correspond to “doings” and “happenings” as well as to “conceptual patterns” (Kress and Van Leeuwen 2001, 56-61; Kress and Van Leeuwen 2002; Jewitt and Oyama 2003; Bell 2003). In this respect, the nine texts were approached as examples of “ensembles of modes” brought together to realise particular meanings of three kinds: ideational, interpersonal and textual (Unsworth 2001, 10, 7273; Kress 2003, 66). In order to solve the problem assigned to them, the students were prompted to design representations of material, social and semiotic reality of people and space in the borderland and to select the symbols they would use as resources encoding life in the area. As illustrated in ȉable 4-4, the BoWMa borderland was represented by a combination of narrative and conceptual messages presenting unfolding actions (“The Street Market of Florina”) and events (“The Dam”, “Arkturos”, “Museum and Archaeological sites”, “Tracing Monuments in Western Macedonia”, “Landmarks in Prespes”), processes of change (“Borderlines in Fashion”) and transitory spatial arrangements (“Graffiti”, “Florina & Bitola”) (Kress and Van Leeuwen 2001, 59) that illustrate understandings of meanings related to the concept of borderland. In this meaning-making process, the physical resources of the landscape were changed into signs and used to make meaning. Due to space limitations, only four (out of a total of 129) slides have here been analysed in order to illustrate that each of the multimodal texts produced by the students, shapes an argument for a particular understanding of the concept of borderland.

Developing Students’ Visual Design Competence

230

Table 4-4. Inventory of the resources used for the compilation of the teaching scenarios Title of the teaching scenario Arcturos Landmarks in Prespes Graffiti Borderlines in Fashion The Street Market of Florina

8

Photos Photos & without Diagram Map Video print print White/green 6 9

12

Red-purple 2

20

Back

20

Lilac

12

Light blue

Number Background of slides

Florina & Bitola

19

The Dam

13

Tracing Monuments in 12 Western Macedonia Museum and Archaeological 13 sites

White with fine blue lines Brown Mountain Blue sky /water White with fine blue horizontal lines Misty blue

3

9

Wipe effects

9

17 10

1

4 5

5

14

2

2

9

9

9

The teaching scenario, entitled “A comparison between Florina and Bitola”, indicates that the geographical border is not an equivalent to a line between cultures and that we are not able to label cultures as belonging to particular spaces inside boundaries. Instead, it should be understood that cultures often transcend the lines of geographical borders. The group of student teachers who compiled this scenario aspire to highlight the transient nature of borderland culture through a comparative examination of the architectural styles adopted in Florina and Bitola, two cities that sit on the fringes of borderlands the former in Greece and the latter in FYROM. Specifically, the Balkan type of house (Fig. 4-5) is a cultural characteristic reflecting the arbitrariness of dividing cultures through borders in a way that denies the existence of the unique culture that exists between the borderlands. Both Greek and FYROM architecture share common characteristics: stone foundations, lighter upper floors of buildings, small windows on the ground floor while larger windows on the

Catherine Dimitriadou and Androniki Gakoudi

231

first floor. In this sense, the tendency to assume the geographical border as equivalent to a line between cultures is not confirmed by our data analysis.

Fig. 4-5: teaching scenario A comparison between Florina and Bitola

Drawing on Kress and van Leeuwen (2001), the first slide of the teaching scenario, entitled “Arcturos”, is analysed as an inclusive conceptual image that invokes a metaphor (Fig. 4-6). Arcturos, a Greek non-governmental organisation, develops actions for the sustainability and conservation of wildlife in the transborder areas of the south Balkans. One of its first tasks was to provide a hosting facility for confiscated dancing bears. Wild bears are an important feature in the region of Western Macedonia and are invoked as a symbol of the WM borderland through the Arcturos logo. This logo is superimposed on the photo in a background of beech forest, which is associated with the natural habitat of the wild bears living in the region.

Fig. 4-6: The teaching scenario Arcturos

232

Developing Students’ Visual Design Competence

In contrast, the seventh slide of the teaching scenario, entitled “Christian Orthodox Religious symbols in Prespes”, is an example of a conceptual image whose analytical pattern is exhaustive in as much as it shows all there can possibly be to an image of a religious symbol: the Christian cross as background, in the top right- and bottom left-hand corners, two other smaller slides of this landmark appear with a wipe effect, both of which have a heavy black border (Fig. 4-7). The print is in black bold, plus two bullet points. In the top left-hand corner, the name of the landmark appears in a white box with a heavy black border and bold black print. In a metaphorical sense, the Cross in the context of the borderland shows that people living on this side of the border are Christians, they are ‘us’, while people living on the other side of the border are heterodox, and are ‘them’. As a result, the text (two bullet points: “sign of loyalty” and “visual border“) in this slide relates directly to the photos. Another group of students approached the study of borders as imaginary constructs that can be taught by drawing a comparison between the imaginary constructs of borderlands and stereotyping through fashion. “Borderlines in Fashion” compared the countries’ borders with those borders stereotyped through appearances. This teaching scenario focuses on the idea of fashion as a demarcation of border. In semiotics, ideas are signs too: they are indexes. In other words, they are signs that indicate a fact or condition. Fashion is approached as a condition that indicates borders, which are used not only to mark difference, but also to implicate the twin narratives of inclusion and incorporation on the one hand, and of exclusion and dispossession on the other. Those who are dressed following a particular code belong to the same group and are included, are thought of as ‘us’, while those who follow a different dress code are excluded and consequently considered to be ‘them’. Also, focusing on the abstract idea of ‘border’ helps us to identify and analyse the politics of socio-cultural identities which tie individuals in particular groups or border regions. Their lives are part of cultures and forms of meaning which they share only, or, principally with other ‘borderlanders’, on the same or the other side of a constructed abstract demarcation, the borderline.

Catherine Dimitriadou and Androniki Gakoudi

233

Fig. 4-7: The teaching scenario Christian Orthodox Religious symbols in Prespes

Discussion This study employed socio-semiotic analysis in order to consider the way the IP participants transformed semiotic resources so as to represent the concept of BoWMa in their teaching scenarios. It did not provide an indepth semiotic analysis of the semiotic resources referred to by the participants, neither did the analysis focus on all the representations connected to their socio-cultural contexts. Rather, it combined semiotic analysis of multimodal texts as parallel manifestations of how semiotic resources can be recontextualised in educational contexts. Based on the grammar of visual design, as maintained by Kress and Van Leeuwen (2001) in their social semiotic theory of representation, the visual structures of the students’ teaching scenarios pointed to particular interpretations of experience and forms of social interaction that were negotiated by the participants in the co-constructed emergent culture of the 2010 IP P.S.BoWMa. The students, as learners attending the IP, instilled borderland with their meaning-making practices while at the same time, as future teachers they compiled their teaching scenarios through which they recreated the historical and cultural significance of a sign and transformed it into a historical or cultural teaching resource. In other words, the participants, in their capacity as designers of educational material, recontextualised historical or cultural semiotic resources in educational contexts. They were involved in a process of making signs by forming their representations of a region through their immediate contact with it and not through the representations offered in cultural constructs, such as museums, book illustrations, postcards, literature or historic texts (Haskell 1994). During this process, modes and forms were selected in such a way so as to express the meanings that the makers of signs wished to give.

234

Developing Students’ Visual Design Competence

The nine groups of students constructed nine newly made signs shown in table 4-4: The Street Market of Florina The Dam, Arcturos, Museum and Archaeological sites, Tracing Monuments in Western Macedonia, Landmarks in Prespes, Borderlines in Fashion, Graffiti, Florina & Bitola. It can be argued that through the recontextualisation of their multimodal resources during the process of producing teaching scenarios, the participants developed their creativity as a process of meaning-making (Kress 2004, 36-40). Furthermore, in line with Kress (2003, 42) it is argued that because the relation between the signifier and signified is motivated, they did not approach their signs as ready-made entities, consisting of signifiers and signified whose content was determined by social conventions in an absolute sense, as for example the signifier of borderland being signified by the notion of border demarcation. Rather, they were engaged in a process of forming a whole new variety of relations between the signifier and the signified thus constructing newly made signs the meaning of which, as shown, can change according to the local context in which they are encountered. The symbols chosen, as the titles of the scenarios suggest, illustrate that the semiotic resources were used by the participants so as to create images and ideas across geographical and socio-cultural spaces in ways that affect how young people learn and interact. The way the participants devised their teaching scenarios was a semiotic system itself, part of a system of cultural signs determined by the repertoire of images, words, diagrams and other interpretants provided by the students’ knowledge and culture. The conception of the WM borderland was the locus of an encounter between external reality, the recontextualisation processes shown in table 1 and the local dynamics of the semiosis which they created. During these design processes, negotiations and transformations of the conception of the WM borderland brought about semiotic change that reflected the values, structures and meanings of the social and cultural world of the participants as meaning-makers. The students, having been involved in semiotic practices and in the coconstruction of content and procedural knowledge, contributed to their awareness that learning is situated within authentic activities. What is more important, they created their understanding in visual images and learned from their own experiences that they need to engage their learners in a variety of purposeful cross-media activities incorporated in a curriculum and a pedagogic model informed by the theory of Multiliteracies (Cope and Kalantzis 2000). However, it has to be noted that this making of signs is a semiotic reconstruction which constitutes a structured and largely coherent whole

Catherine Dimitriadou and Androniki Gakoudi

235

for the 2010 IP participants only. It is expected that participants attending the same curriculum in future IPs held in the same region could very well approach the concept of borderland in their own uniquely different ways because, as shown above, meanings, as well as resources for meaningmaking, are constantly negotiated and transformed in the process of sign making.

Conclusions This paper aimed to show that IP BoWMA participants developed their semiotic awareness by being transformed into sign-makers and teaching material designers. They decoded the organisational and functional rules of the communities located in the area, recontextualised meanings through a teaching perspective, and produced multimodal texts. What is more important, they demonstrated that learning and teaching can traverse institutional boundaries not only by collapsing disciplinary boundaries, but also the ones between “in-school and out-of-school literacies” (Leander 2001). They constructed contextual narratives (Herman, 2009) that situated concepts in practice after they had approached borderland people and space in situ through situated literacy practices that included activities bound to social, cultural and physical contexts (Anderson et al. 1996). Thus, students made steps towards knowledge construction and development of their visual design competence along the following three axes: (a) identification of signs characteristic of the WM borderland: interstate agreement for the conservation of wild bears, religious identity symbols transformed into borders of exclusion, dressing codes, traditional architecture; (b) recontextualisation of these signs into multimodal texts so as to signify notions, conventions, socio-cultural institutions of the natural and manmade environment of the area; and (c) redesign of the multimodal texts for their use as digital educational material. In other words, the development of the participants’ visual design competence seems to consist in content and procedural knowledge of interdisciplinary topics since their involvement in visual literacy practices contributed to their reading and interpretation of the semiotic universe as well as to a more meaningful understanding of the world. Furthermore, the students, while transforming the WM semiotic resources into meaningful communication events through the manipulation of visual messages and symbols, were actively involved in a process of

236

Developing Students’ Visual Design Competence

symbolic interaction, which, as Dennis and Martin (2005) claim, brings to the surface issues of power and its distribution. Furthermore, the participants made steps to developing their technological literacy, since they used digitally mediated representations of the WM borderland in the context of the IP and from text consumers were changed into mediators and designers of meaning-making systems by developing their computerbased visual literacies. The responses to the research questions highlighted the way in which semiotic resources are deployed to make meaning-making systems in a variety of contexts, thus facilitating teachers’ informed and effective intervention in students’ learning (Unsworth 2001, 2-3). They also led to implications about the way in which the participants’ readiness to design multimodal teaching material could affect both their own and their provisional students’ visual literacy. One way in which Pedagogic Departments can respond to this task is to make student teachers aware that the diversity of signs and cultural meanings that circulate in the classroom need to be broadened (Buckingham 1993). To this end, teachers need to design their teaching on a wide range of modal resources, concentrating, for example, on the semiotic resources of image, as our study showed. Although the conclusions drawn from this analysis are based on a relatively limited amount of data. it is important to note that by transforming their socio-cultural context into semiotic recourses used for the design of multimodal texts, not only do students enhance their creativity through the creation of multimodal ensembles, but they also improve their visual literacy, and most importantly, their teaching competence (Cope and Kalantzis 2000). These conclusions, also, highlight the need for a transformative educational agenda that calls for the redesign of the curricula offered in Education Departments so as to be informed by Semiotic theory and the Theory of Multiliteracies.

Bibliography Anderson, John R., Reder, Lynne M. and Herbert A. Simon. “Situated learning in Education.” Educational Researcher 25, no. 4 (1996):5-11. Barthes, Roland. Image – Music – Text. Translated by Stephen Heath. New York: Hill and Wang, 1977. Bell, Philip. “Content Analysis of Visual Images.” In Handbook of Visual Analysis, edited by Theo van Leeuwen and Carey Jewitt, 10-34. London: SAGE Publications, 2003.

Catherine Dimitriadou and Androniki Gakoudi

237

Bopry, Jeanette. “Visual Literacy in Education: A Semiotic Perspective.” Journal of Visual Literacy 14 (1994): 35-39. Buckingham, David. Changing Literacies: Media Education and Modern Culture. London: Institute of Education, 1993. Cope, Bill and Mary Kalantzis. “A Pedagogy of multiliteracies. Designing social futures.” In Multiliteracies: Literacy Learning and the Design of Social Futures, edited by Bill Cope and Mary Kalantzis, 9-37. New York: Routledge, 2000. Dennis, Alex and Peter J. Martin. “Symbolic interactionism and the concept of power.” The British Journal of Sociology 56, no. 2 (2005): 191-213. Dimitriadou, Catherine. “Embedded learning for teachers’ professional development: Reflection on a Travelling Seminar.” In EUROCLIO Bulletin 29, After The Wall - Teaching History after the Fall of the Soviet Empire (2010): 33-49. http://www.euroclio.eu/site/ Dimitriadou, Katerina and Anastasia Kesidou. «ǼȞĮȜȜĮțIJȚțȠȓ IJȡȩʌȠȚ İʌȚȝȩȡijȦıȘȢ İțʌĮȚįİȣIJȚțȫȞ: Ș ʌİȡȓʌIJȦıȘ İȞȩȢ IJĮȟȚįȚȦIJȚțȠȪ ıİȝȚȞĮȡȓȠȣ» [“Alternative ways of in-service teacher education: the case of a traveling seminar”]. In ǻȚĮʌȠȜȚIJȚıȝȚțȩIJȘIJĮ, ʌĮȖțȠıȝȚȠʌȠȓȘıȘ & IJĮȣIJȩIJȘIJİȢ [Interculturality, globalization and identities], edited by Eleni Hondolidou, Grigoris Pashalidis, Kyriaki Tsoukala and Andreas Lazaris, 279-289. Athens: Gutenberg, 2008 (in Greek). Edwards, Richard and Robin Usher, Globalization and Pedagogy. Space, place and identity (2nd edition). London: Routledge, 2008. Elkins, James. Introduction: The Concept of Visual Literacy, and Its Limitations. In Visual Literacy, edited by Jeremy Elkins, 1-10. New York: Routledge, 2008. Haskell, Francis. History and its Images, Art and the Interpretation of the Past. London: Yale University Press, 1994. Herman, David. Basic elements of narrative. Oxford: Wiley-Blackwell, 2009. Jewitt, Carey and Rumiko, Oyama. “Visual Meaning: a Social Semiotic Approach”. In Handbook of Visual Analysis, edited by Theo van Leeuwen and Carey Jewitt, 135-156. London: SAGE Publications, 2003. Kress, Gunther and Theo van Leeuwen. “Colour as a semiotic mode: notes for a grammar of colour.” Visual Communication 1, no. 3 (2002): 343368. Kress, Gunther and Theo van Leeuwen. Reading Images: The Grammar of Visual Design (5th edition). London: Routledge, 2001. Kress, Gunther. Literacy in the New Media Age. Oxon: Routledge, 2003.

238

Developing Students’ Visual Design Competence

—. “Reading Images: Multimodality, Representation and New Media.” Information Design Journal & Document Design 12, no. 2 (2004): 110-119. Leander, Kevin. “Producing and hybridizing space-time contexts in pedagogical discourse.” Journal of Literacy Research 33, no. 4 (2001): 637-679. Mitchell, William J. Thomas. “Four Fundamental Concepts of Image Science.” In Visual Literacy, edited by James Elkins, 14-30. New York: Routledge, 2008. Simons, Jon. “From Visual Literacy to Image Competence.” In Visual Literacy, edited by James Elkins, 77-90. New York: Routledge, 2008. Unsworth, Len. Teaching multiliteracies across the curriculum. Changing contexts of text and image in classroom practice. Buckingham: Open University Press, 2001. van Leeuwen, Theo. “Semiotics and Iconography.” In Handbook of Visual Analysis, edited by Theo van Leeuwen and Carey Jewitt, 93-118. London: SAGE Publications, 2003. —. Introducing Social Semiotics. London: Routledge, 2005

THE QUEST FOR “VISUAL THINKING” AND THE DOUBLE BIND OF EDUCATION MILTOS FRANGOPOULOS

Introductory Remarks In these hard times we are going through, visual rhetoric comes to the fore. The current conjuncture of global financial crisis offers itself to a proliferation of images based on stereotypes. Greece, being at the epicentre, becomes an easy target of abuse, where reference is made to its widely known ancient past and cultural heritage, through graphic manipulation of landmark monuments such as the Parthenon or works of art such as the Venus de Milo.1 Apocalyptic imagery is in the order of the day: cities as battlegrounds, teargas clouds and impending doom. Certainly, the unending series of local wars keeps feeding us with horrific scenes from the charred remains of Iraqi soldiers to the gory images of Gaddafi’s demise. It is here that some things become unspeakable, and – at least since the footage of the concentration camps at the end of World War II – the presence of the sign in these contexts becomes so pronounced as to be blinding. This, of course, would lead to a different discussion than the one suggested here. However, the ubiquitousness and proximity of such ‘imagery’ does not allow us to forget it – even if ‘safely enclosed’ between brackets. At any rate, in our contemporary context, in our immediate experience of a debate about the current financial crisis, which could easily become political and social, communication is effected through a visual rhetoric laden with ideological messages.

1 Betrüger in der Euro-Familie (2010.02.20). Focus Magazin. Munich: Focus Magazin Verlag GmbH Germany. The 20th February 2010 issue cover, alongside the title of the cover story (“A crook in the Euro-family”), featured a manipulated photograph of the Venus de Milo statue, complete with arms, gesticulating indecently toward the viewer.

240

The Quest for “Visual Thinking” and the Double Bind of Education

But culture throughout has been primarily visual. It is perhaps part of our existence as a species; it is perhaps the way we perceive the world. Exploring origins, one could make forays into the past, or along the meeting of the diachronic and synchronic axes, such as we find in Australian aboriginal art, with human signatures of tens of thousands of years ago, and images that may function as narratives, as descriptions, as initiation rite codes, as maps, as covenants of land custodianship, as ‘dreaming’. One could look toward the Middle Ages, where, as Umberto Eco has suggested, people inhabited a world filled with references, reminders and overtones of Divinity, manifestations of God in things. Nature spoke to them heraldically: lions or nut-trees were more than they seemed; griffins were just as real as lions because, like them, they were signs of a higher truth.2

This would apply to any society at any given moment in history, and a similar inversion is in evidence in a most famous example of visual rhetoric whose analysis ushers in the semiotics of the everyday in the contemporary era: the cover of the 25th June 1955 issue of “Paris Match” magazine.3 The story of Roland Barthes’s encounter with the photograph of a young African saluting in French army uniform is well known. In a way, it replicates the story of Newton and the falling apple, as it makes the ordinary yield something beyond itself, beyond its ‘mere appearance’, and it engenders, if not a law of universal physics, at least a whole new way of seeing things. Surely, ideological critique was not something new by the 1950s, but what Barthes did was to provide, drawing on linguistic theory and social anthropology, a systematic way of discerning and dissecting the ideological trappings of everyday life. What this kind of critique does is to acknowledge the allegorical content – much like in the Middle Ages – but at the same time to undermine it: instead of affirming ideology, it questions it. In tune with modernity, it accepts the basic tenet that everything ideological possesses meaning, but assuming a stance that posits ideology as false consciousness. This introduces an element of tension which accompanies the exploration of signs in the modern social context, in terms of a contradiction 2

Eco, U. (1986). Art and Beauty in the Middle Ages. New Haven: Yale University Press, p. 52. 3 Barthes, R. (1972 [1957]). Mythologies, New York, Noonday Press, p. 59.

Miltos Frangopoulos

241

apparent in all our interpretive activity, while it imbues everything with a degree of complexity.

Modern Examples of Visual Thinking This interpretative process requires a mode of thinking that can map out the elements at play, which can present them visually, so that they could be grasped at once in their totality. This negotiation of complexity and contradiction can be seen in one of the most famous examples of “visual thinking”, Charles Eames’s definition of design (fig. 4-8). It is only by finding the meeting point, the overlapping area of varying interests and concerns that one can begin to design ‘meaningfully’, where design is understood as a socially determined activity.4

Fig. 4-8. Statement of the Eames Design Process by Charles Eames for the exhibition “What Is Design?” at the Musée des Arts Décoratifs, Paris, 1969. © Eames Office LLC (eamesgallery.com)

Negotiating complexity and contradiction is very much in evidence in another example of visual thinking, billed sometimes as a masterpiece of visual literary criticism. It is the graph from Moholy Nagy’s book “Vision in Motion”5 completed in 1946, presenting the structure of James Joyce’s most ambitious literary work, “Finnegans Wake”, published in 1939 (fig. 4-9). 4

See Lupton, E. and Philips, J.C. (2008). Graphic Design; The New Basics, New York and Baltimore: Princeton Architectural Press & Maryland Institute College of Art, p. 11. 5 Moholy-Nagy, L. (1947). Vision in Motion. Chicago: Paul Theobald, p. 349.

242

The Quest for “Visual Thinking” and the Double Bind of Education

The concentric circles (sharing the writer’s initials JJ as the pivotal point) represent the four cycles of Giambattista Vico’s theory of history, while the rectangles on the right list a long series of different levels (familial, historical, mythological, cabalistic, biblical etc.) on which the story can be registered, seen in terms of the roles of the main personages, the family of HCE (H.C. Earwicker), his wife ALP (Anna Livia Plurabelle), and their children (daughter Issy, and sons Shem [Jerry] and Shaun [Kevin]) – and what they stand for. The rays run through the layers of the narrative creating axes around which the story can revolve anew, on different planes of reference – both cosmic and local, such as the four evangelists, the twelve signs of the zodiac, Dublin’s Phoenix Park etc.

Fig. 4-9. Laszlo Moholy-Nagy: Visual Representation of the novel Finnegans Wake by James Joyce, from Moholy-Nagy, L. (1946). Vision in Motion. Chicago: Paul Theobald, p. 347

“Finnegans Wake” aspires to be a repository of all western culture and the chart as a visual presentation of the grand project, rises to the challenge. If we were to indulge in a sort of Panofskean iconological analysis, we would say that this graph could be taking its cue from another visual device used on the cover (dust jacket) of a book of essays dealing with Joyce’s preliminary work on “Finnegans Wake”, published in 1929 by his friends and supporters – among who, were Samuel Beckett and William Carlos Williams – to counter adverse criticism. The book bore the Joycean title “Our Exagmination Round his Factification and Incamination of his

Miltos Frangopoulos

243

Work-In-Progress”.6 This full title can be read round the edge of the circle or wheel whose spokes are made of the names of the contributors to the volume. This geometric pattern, with a reference of motion, carries a hint of the descriptive (progress) as well as of the occult (wheel of fortune), links up with Moholy Nagy’s chart and sends us “by a commodius vicus of recirculation” – to use an original Joycean phrase – to other geometric structures of similar ambitions. The reference here is to that great chapter of “visual thinking”, which is the “Art of Memory” and its “Theatre”.

Ars Memoriae There is a very long story here, taking us back to Greek and Roman antiquity, where we need not go at present, but suffice to say that at the time of the Renaissance an attempt was made to bring together all knowledge under the sun in one place as images within an actually constructed wooden theatre based on Vitruvius – the “Theatre of Memory”, put together by Giulio Camillo (1480-1544) in Venice in the 1530s.7 What this structure was, we learn from two letters written to Erasmus by his friend Viglius, a doctor of jurisprudence, who travelled to Italy in 1532.8 It was a theatre, following the principles of Vitruvius, a large model presumably large enough for two men to walk in and inspect the variously recorded and drawn items of knowledge arranged therein, spreading outward from the orchestra, whose half circle was divided into seven parts, representing the seven pillars of wisdom and the seven celestial bodies. The Art of Memory, based on linking places or events with images was the greatest of aides to rhetoric, and was said to have been invented by the Greek lyric poet Simonides around 500 BCE. (So, we cannot really avoid the journey into the past). Based on a story recorded by Cicero, some 500 years later, Simonides was at a banquet in Thessaly, where he was insulted by his host, who would not pay him the agreed amount for a hymn the poet had composed in his honour. This mean host was to be punished by the gods. For it so happened that shortly after this incident a servant came 6

Beckett, S. et al. (1929). Our Exagmination Round his Factification and Incamination of his Work-In-Progress. Paris: Faber and Faber (see dust jacket). 7 Yates, F.A. (1974 [1966]). The Art of Memory. Chicago: The University of Chicago Press , p. 129. 8 ibid, p.130.

244

The Quest for “Visual Thinking” and the Double Bind of Education

saying that two men requested that Simonides urgently meet them outside the house. No sooner had Simonides exited the house than the roof of the building collapsed. The host and his guests were crushed to death, and they were so completely deformed that their relatives could not tell them apart and could not properly bury them. But Simonides was able to tell who was who by his recollection of the exact place in the hall in which each of them had been sitting.9 This suggested to him the discovery of the truth that the best aid to clearness of memory consists in orderly arrangement. He inferred that persons desiring to train this faculty must select localities and form mental images of the facts they wish to remember and store those images in the localities.10 Giulio Camillo, who put his ideas in writing in a book entitled “L’Idea del Teatro”, was considered somewhat of a quack by Erasmus and at odds with the prevailing climate of the Renaissance, so was soon forgotten after his death. Interest in his work was revived in the 1960s by the research of Frances Yates at the Warburg Institute in London. Yates wrote extensively on the Art of Memory, and by the 1980s the new computer technologies recognised a precursor in Camillo’s systematic storage of memory data in the form of a theatrical space. This was also the time that Bill Viola produced his video installation entitled the “Theatre of Memory” (1985), now at the Orange County Museum in California, while in the early nineties Brenda Laurel published her “Computers as Theatre” (1993).11 The German writer and professor of media aesthetics Peter Matussek has provided a broad picture of the influence of the tradition of the Art of Memory on several major players in the development of personal computer technology, especially what is called Human Interface Guidelines, where he says: […] the Human Interface Guidelines,12 which were developed by Apple in the eighties could well have been borrowed from the traditional teachings of rhetorical ars memoria.13

9

see Cicero, Marcus Tullius (1967). De Oratore, Book II [86.351], with an English Translation by E. W. Sutton, Completed London Heinemann, Loeb Classical Library. pp. 464-5. 10 ibid. 11 Laurel, B. (1993). Computers as Theatre. Boston: Addison-Wesley. 12 Apple Computer Inc. (1987). Human Interface Guidelines: The Apple Desktop Interface, Apple Publications, Reading, Mass. The computer screen icons were developed by Susan Kare (graphic designer) and Bill Adkins (computer

Miltos Frangopoulos

245

Citing the example of the basic "See-and-Point" principle, which recalls the ancient “loci et imagines”, as well as to a series of other concepts. He further adds that “as Nicholas Negroponte has implied, there is an affinity between Simonides of Ceos and Steve Jobs”. 14 Indeed, the first set of Mac desktop icons (1984) bears a striking resemblance to the devices produced by medieval and renaissance practitioners of the “Ars Memoriae”, as in the Romberch visual alphabets, that help conjure up the words and concepts. In one example, drawn from Frances Yates, “Predicatio” is memorised by the bird beginning with a P (Pica or Pie) which she holds. “Applicatio” is remembered by the Aquila (fig. 4-10). “Continentia” is remembered by the inscription on her chest in the “objects” alphabet (the reader can refer to the objects representing C, O, N, T in the “objects” alphabet).15

Fig. 4-10. Grammar as “memory image” (left) and “object” alphabet (right top), from Johannes Romberch, Congestionum Artificiose Memoria, Venice 1533 (after Yates, F.A. (1974), The Art of Memory. Chicago: The University of Chicago Press, p. 114), and 1984 Apple Macintosh 1.0 screen icons (right bottom), designed by Susan Kare (Susan Kare Icons (2011) susankareprints.com)

The Primacy of Words But in all this, the image is still subservient to the word. The iconic device is a mere appendage, an aid to a verbal memory – for since its origin, the programmer). See Meggs, P. B. (1992). A History of Graphic Design. New York: Van Nostrand Reinhold, p. 469. 13 Matussek, P. (2001). “The Renaissance Theatre of Memory”. Janus 8 S. 4-8, electronic version http://www.peter-matussek.de/Pub/A_38 (accessed 29.01.2012). 14 Negroponte, N. (1995). Being Digital. New York: Alfred A. Knopf, p. 109. 15 Yates, F.A., (1974). op.cit. p. 120.

246

The Quest for “Visual Thinking” and the Double Bind of Education

“Ars Memoriae” has one principal goal: to serve the great art of Rhetoric. In the end, in the final analysis (as we would say) it is words we are after. And this prominence of the word can be seen in heraldic art where it is the word and the specific vocabulary of blazonry that are accepted as the true and valid description of the coat of arms. What is perhaps more striking is that in 1593 Cesare Ripa could publish his “Iconologia” without any images whatsoever. All the icons of the “Iconologia”, almost 500 of them, were presented in descriptive texts.16 Even as late as 1604, Karel Van Mander in the Netherlands, as he undertook to present the lives of the great painters of northern Europe,17 in an attempt to emulate Vasari he had to develop a strong argument in defence of his project. Were the painters a worthy subject? Did they warrant such interest as would poets, kings and generals? Indeed, since the mid-fifteenth century, a lively debate had erupted in the Low Countries concerning the status of painting and, indirectly, that of the artist as well. The issue was to define whether it was legitimate to consider painting as a liberal art, comparable to those included in the “Trivium” and the “Quadrivium”. In other terms, one had to prove that the theoretical knowledge of a painter was one of the essential components of their activity, something readily acknowledged for the poet. Van Mander’s book in 1604 pressed for the ‘recognition’ of painting as a ‘liberal’ art, one requiring much more than the menial tasks involved in the actual practice of painting. It was a rather slow process of emancipation, but the artists gradually broke away from the guilds and thanks to the astute Louis XIV found shelter in an institution that recognised their lofty status: “L’académie de peinture et de sculpture” in 1648. This was modelled on the Rome academy that had already been established by Giorgio Vasari and Angelo Montorsoli in 1563, but the royal support and specific direction given by the French king, the most powerful ruler in Europe at the time, provided the prestige needed for painting to become a fine art.18 The subsequent development of the ‘salons’, which became a permanent feature of ‘society’ in the early 18th century, led to exhibitions where art was contemplated on its own merit, where a volte-face occurred as these exhibitions were accompanied by printed leaflets providing 16

Ripa, C. (1593). Iconologia, edited by Maser, E.A., (1971) New York: Dover Publications. 17 Van Mander, K. (1604). Het Schilder-Boek, edited by Miedema, H (1994). In 6 vols. Soest: Davaco, Doornspijk, vol. 1, p. 50. 18 Lemaire, G-G. (2004). Histoire du Salon de Peinture. Paris: Klincksieck, pp. 1518.

Miltos Frangopoulos

247

descriptions of the works displayed: the text serving the picture or a return to ut pictura poesis. Gradually, these descriptive texts developed into essays, such as Diderot’s, which in turn led to theoretical discussions that gave rise to “Aesthetics” and a philosophy of the visual arts.19

The Emancipation of Design In this sense, visual art was emancipated, but at a cost. That is to say it was accepted as a lofty practice only because it proved it could be translated into words. And thus, it was obliged to develop a theory. Still in the beginning not dealing in the highest of knowledge, the noeta, what is perceived by reason, but the experience felt by the senses, the aestheta, hence the discipline of “aesthetics”,20 before the advent of a full philosophy of art. This legacy still permeates the intellectual climate, or has at least left a strong residue. To a large extent the issue is still with us, as when we turn to design, we find that it was only recently acknowledged as a subject worthy of university education (in my country it is still struggling). It could be said, perhaps, that the twentieth century was for design, in terms of education and professional standing, what the seventeenth century was for painting. So, “Art” was finally emancipated, but if, now, “Design” were to become an academic subject proper, it would have to resolve its own status as a “liberal” art, i.e. to prove that it could be translated into words, a logocentric narrative (a blazonry of sorts). It also needed a history, as well as a philosophy. It was in this respect that design education allied itself to semiotics (underpinned by psychology and sociology). And in this sense it is ‘doomed’ to develop a theory. From the point of view of semiotics, of course, all human – indeed all natural – life is ‘semiosis’. From Peirce’s ‘man is a sign’ to Sebeok’s

19

ibid., p. 45. Baumgarten opens his Aesthetica (1750) with this definition: “Aesthetica (theoria liberalum artium, gnoseologia inferior, ars pulcre cogitandi, ars analogi rationis) est scientia cognitionis sensitivae” [Aesthetics (theory of liberal arts, inferior gnoseology, knowledge of beautiful arts, art analogous to reason) is the science of knowledge of the senses]. See: Beardsley, M.C. (1976 [1966]). Aesthetics from Classical Greece to the Present: A Short History. Tuscaloosa: The University of Alabama Press, p. 156. 20

248

The Quest for “Visual Thinking” and the Double Bind of Education

‘global semiotics’ all phenomena are perceived as part of a system of interrelated semata.21 Certainly, this is the era of global communication and of the development of novel media for information and, more generally, sign transmission – a process set in motion with the advent of mass production and mass society. In the new context one could say that the situation is inverted: for in the old “Ars Memoriae” the image was an aid to summon up the word – in the process of oral rhetoric in front of an audience, whereas in the modern era of mass communication the image is all the more often in the driver’s seat, supported by the word (as can be seen in cases ranging from the famous “beat the whites with the red wedge” poster by El Lissitzky in 1919 to the “Silk Cut” cigarette and “Nike” sportswear advertisements in the 1990s). These messages within our lifeworld are filtered through ideological markers and delineate our existential hold on or relation with our surroundings. As we have already seen at the beginning of this presentation, meaning and ideology are inextricably linked. Here, at first instance, the stereotype is at work, which – before attracting our critique – places us firmly in the world. This is perhaps the point made by Robert Venturi’s “Symbolic Communication” drawing, where architectural details function as signs that trigger off interpretations with a profound psychological imprint. For instance, the entrance to the driveway conjures up the image of a grand gateway, the columns at the door to the house the image of a monumental propylaeum, the curving street with the rail fence a sunset in the countryside.22 The process is akin to that observed in the Middle Ages, as presented by Umberto Eco through a reference to the great medievalist Huizinga: The Middle Ages never forgot that all things would be absurd, if their meaning were exhausted in their function and their place in the phenomenal world, if by their essence they did not reach into a world beyond this.23

21

Sebeok, T. A. (2001). Global Semiotics. Bloomington: Indiana University Press, pp. 1-16. 22 Venturi, R., Scott Brown, D., Izenour, S. (1972). Learning from Las Vegas. Cambridge Mass: MIT Press, p. 158. 23 Huizinga, J. (1924 [1919]). The Waning of the Middle Ages. p.194; quoted in Eco (1986), p.52-3.

Miltos Frangopoulos

249

The contemporary world is not radically different. The symbolic and allegorical realms are still with us. As Venturi puts it: We live in a communication era teeming with symbols. Don’t forget that words and letters, which inundate all of our environment, are symbols. We like to say we haven’t an accepted set of symbols the way the Middle Ages had, but we do, via all the advertising media, in great variety and complexity. The valid base for often superficial supergraphics is that it is architecture connecting with the idea of communication in space.24

But the condition may be seen as not determined by historical circumstance, indeed it may be seen as diachronically at work, as Huizinga goes on to suggest: This idea of a deeper significance in ordinary things is familiar to us as well, independently of religious convictions: as an indefinite feeling which may be called up at any moment, by the sound of raindrops on the leaves or by the lamplight on a table. Such sensations may take the form of a morbid oppression, so that all things seem to be charged with a menace or a riddle which we must solve at any cost. Or they may be experienced as a source of tranquillity and assurance, by filling us with the sense that our own life, too, is involved in this hidden meaning of the world.25

Thus, returning to Venturi, we can appreciate his drawing and the deeper significance of those rather trite architectural details. These details, of course, carry another layer of meaning. And here we enter the realm of critique. For this drawing is part of a book that strongly criticises what is known as modern architecture, a book with the tell-tale title of “Learning from Las Vegas”, Venturi’s magnum opus and one of the landmarks of the postmodern movement. This is not the place to investigate the advent of postmodernity, but suffice it to say that the opening-up of the intellectual field into a centreless deconstruction of received ideas that could be debated ad infinitum, coincided with the increasing interest in the theories of the sign, linked to globalisation and the attendant development of mass communication networks.

24

Quoted in Larson, M.S. (1993). Behind the Postmodern Facade: Architectural Change in Late Twentieth-Century America. Berkeley: University of California Press, p.223. 25 quoted in Eco (1986) op.cit. p. 52-3.

250

The Quest for “Visual Thinking” and the Double Bind of Education

Ways of Thinking As these theoretical discourses developing in this domain – bearing upon linguistics, psychology, art history and philosophy – started entering into the field of design education, a feeling of liberation prevailed, as in the well-known case of the Cranbrook Academy in Michigan USA in the early 1980s26. Postmodernity in its ‘fluidity’ was well suited to art and design education, but at the same time its excessively rhetorical discourses placed too heavy a theoretical burden on the average student in the creative disciplines. This, though accentuated by the specifics of postmodern discourse, was a symptom of a more general complication. For once art and design are called to conform to academic standards, two major problems emerge: the first is a contradiction between the intuitive and the rational, and it would seem that those entrusted with the development of curricula for design studies acknowledge this. The Quality Assurance Agency of the Department for Education of the UK puts it as follows in its latest “art and design subject benchmark statement”: The role of the imagination in the creative process is essential in developing capacities to observe and visualise, in the identifying and solving of problems, and in the making of critical and reflective judgements. While convergent forms of thinking, which involve rational and analytical skills are developed in art and design, they are not the only conceptual skills within the repertoire employed by artists and designers. More divergent forms of thinking, which involve generating alternatives, and in which the notion of being “correct” gives way to broader issues of value, are characteristic of the creative process.27 (emphasis added)

This is a concession, albeit not a very generous one, by the university education authorities towards the student in the creative disciplines. A further qualification is perhaps needed to describe more fully the process involved. Delving into these divergent forms of thinking, which could initially be the juxtaposition of the analytical and the synthetic, it is often said that the

26 See Lupton, E. and Abott Miller, J. (1994). “Deconstruction and Graphic Design”. Visible Language, 28(4), pp. 345-365. 27 QAA [Quality Assurance Agency], UK. (2008). Art and Design Subject Benchmark Statement. http://www.qaa.ac.uk/ Publications/InformationAnd Guidance/Documents/ADHA08.pdf (accessed 29.01.2012). p.3.

Miltos Frangopoulos

251

syllogistic approach of deductive or inductive thought is not the one that should be applied in this context. Rather, we should acknowledge an inferential interpretation process, or reasoning, based on conjecture, which does not conform to a rigorous “elenchus”.28 Recently, this approach was glorified in the obituaries of the hero of our times, Steve Jobs. Without the need to drop acid (as Jobs appears to have advised Bill Gates to do) this is a kind of informed guessing, or of lateral thinking, arising on a hunch based on an intuitive grasp of current conditions. If one were to give it a name, one could say that this approach is based on “abductive thinking”. The term may arise from Aristotle’s ĮʌĮȖȦȖȒ, although this has been traditionally translated as “reductive”.29 Abductive appears as a better rendition, and perhaps closer to the kind of unverifiable, unprovable ‘reasoning’ at work here, as we are dealing with the infinitesimal and the almost imperceptible, something extremely complex that may often seem more vague than decisive. This raises the question whether a communicative act, properly speaking, can firmly take place on this basis, and in turn causes a second tension, a further level of complexity and contradiction, in the process, as we try to approach the field of education in the creative disciplines. The tension becomes apparent when we read the list included in the “subject benchmark statement”: 2.2. Learning in art and design develops: ƒ the capacity to be creative ƒ an aesthetic sensibility ƒ intellectual enquiry ƒ skills in team working ƒ an appreciation of diversity ƒ the ability to conduct research in a variety of modes ƒ the quality of reflecting on one's own learning and development ƒ the capacity to work independently, determining one's own future learning needs. 30

The list does not appear very systematic; neither can it ever be exhaustive. And there is a glaring omission: Meaning. 28 The reference here is to Charles S. Peirce’s differentiation of ‘types of reasoning’, as discussed in his ‘Lectures on Pragmatism’ (1903). See: Peirce, C.S. (1960 [1931]). Collected Papers, Vol. V, Pragmatism and Pragmaticism, Cambridge Mass.: Harvard University Press, I, 6, §4, par. 171. 29 Ross, W.D. (1964). Aristotelis Analytica Priora et Posteriora. Analytica Priora 69a20-36. Oxford UP: Oxford Classical Texts. 30 QAA (2008), p. 3.

252

The Quest for “Visual Thinking” and the Double Bind of Education

Design as an Academic Subject Meaning in visual communication is a difficult issue. Perhaps the writers of the document wisely chose not to broach it. The ‘visual’ meaning, harder to produce as it is easier to consume, requires that process of condensation where a number of different stimuli function at once to produce a ‘spark’ – something like Barthes’s punctum31 – that can bring together the manifold in a single image, in a unified sign. But even that, as the decades of the hegemony of deconstruction, evoking the principle of uncertainty, have shown us is not something stable. In such circumstances, it is obviously difficult to explain to students what meaning is in terms of a visual language, as we go backwards and forwards explaining image by text and vice versa, while offering no final verdict. Education is constantly caught between these two sides, made more unstable in the shifting sands of postmodernity, while students find themselves in that situation in the classroom where a move is required from consumption to production, or more precisely, invention. And although an increasing number of people are acquiring new skills, building their own narratives through social networks, facebook, twitter and the like, these have been learned mostly through collateral learning, unwittingly as it were, and hence uncritically. So, here, the crucial question is: what do their stories tell? Invariably, something not very different from what is already served by the mass media. Here is where meaning comes into the picture. For the crucial issue is that which does not appear in the QAA list: The capacity to generate something meaningful, something relevant to our presence in this world. This, of course, can become unduly dense, inordinately theoretical for our students. Perhaps we need to work the other way round and to introduce them to a method that takes us back to a process described by Cassirer: Whatever appears important to our wishing and willing, our hope and anxiety, for acting and doing; that and only that receives the stamp of verbal “meaning”. […] for only what is related somehow to the focus point of willing and doing, only what proves to be essential to the whole scheme 31 Barthes, R. (1986). Camera Lucida. London: Fontana. pp. 26-27. The idea of a revelatory ‘moment’ where things congeal and become comprehensible may be found also in the work of other thinkers such as Walter Benjamin (‘dialectical image’ and ‘monad’, The Arcades Project) and Jacques Lacan (‘creative spark’ in The Agency of the Letter in the Unconscious).

Miltos Frangopoulos

253

of life and activity, is selected from the uniform flux of sense impressions, and is “noticed” in the midst of them–that is to say receives a special linguistic accent, a name.32

It is exactly the point where something becomes important to their willing and doing that meaning may arise. Initially, we may find ourselves, as teachers, in a situation which is the inverse of that described by Cassirer, as the students at first instance cannot find a focus of interest which is properly their own, and they see it as something imposed from without, a focus of interest that is not their own. They feel alienated from the work at hand and it is difficult for them to see the ‘meaning’ of the effort they are asked to put in. The notion that in the end a textual account of what they do is needed, brings on the dreaded ‘critical reflection’ that creative practice was supposed to save them from. This can be seen quite clearly when one approaches the discussion regarding the higher echelons of education, and more specifically on the attempts to build art and design PhD programmes. Indeed, it is very difficult to agree on what exactly such programmes would look like. Victor Burgin, for example, proposes three different types;33 Mick Wilson brings four theses into the debate.34 This is not the place to join this discussion, but the lack of consensus shows that it is not a clear-cut issue – an issue which perhaps boils down to the question: “how much text and how much image?” The second problem is even more serious as it refers to the quality of the current exchange of messages. For as the contemporary setting requires faster and faster deciphering of a rapidly increasing volume of information, a new type of communication is being developed that is addressed to very short attention spans and carrying simplistic content. But, the backdrop of complexity and contradiction I have been suggesting as the main characteristic of our contemporary situation, requires delicate and sensitive communication. It also calls for a deepening cultural understanding. Though a significant amount of players do try to develop exchange on new levels, a rapidly growing number of crude and crass messages are 32

Cassirer, E. (1946). Language and Myth. New York: Dover, p. 38-39. Burgin, V. (2009). “Thoughts on Research Degrees in Visual Arts Departments”, in Elkins J. (ed.). Artists with PhDs; On the new Doctoral Degree in Studio Art. Washington: New Academia Publishing, pp. 72-79. 34 Wilson, M. (2009). “Four Theses Attempting to Revise the Terms of the Debate”, in Elikns J. (ed.). op.cit, pp. 57-70. 33

254

The Quest for “Visual Thinking” and the Double Bind of Education

produced and transmitted. Stereotypes and gross generalisations come out of dusty closets and whole peoples are made fun of, a preamble of demonisation. The “Focus” magazine cover we saw at the beginning of this presentation has had an impact in the blame-game in the current European financial crisis far stronger than any written article. How is communication through abductive thinking to become capable of developing subtle debates? The problem is a wider cultural one. As designer Dietmar R. Winkler has pointed out when communication images, standing alone or in groups, or supporting text messages are constructed on the basis of limited cultural understanding there are great opportunities for breakdowns in the quality of discourse. Worse, insensitivity may create dangerous confrontations.35

Indeed, visual communication requires more sophisticated intellectual baggage than does oral or written rhetoric, precisely because it registers on so many levels at once and deals with the almost imperceptible.

The Double bind But beyond the issue of cultural exchange, which relates to the designer’s sociopolitical and ethical responsibility, there is the issue, of an equally ethical dimension, of the designers’ work, their employment and their daily practice in visual communication. Following Ponzio and Petrilli, who condense quite succinctly a widely accepted description of communication in our contemporary world, it could be posited that communication understood as “communication-production” is communication adherent to the ideo-logic dominating present-day capitalist reproduction. Communication in the globalization era is communication not only in the sense that it extends over the entire but that it accommodates the world as it is.36 35

totally social world planet,

Winkler, D.R. (2009). “Visual Culture and Visual Communications in the Context of Globalization”, Visible Language, 43(1) (electronic version http://visiblelanguagejournal.com/web/abstracts/abstract/visual_culture_and_ visual_ communication_in_the_context_of_globalization accessed 29.01.2012). 36 Petrilli, S. (2007). “Reading Augusto Ponzio, Master of Signs and Languages”, in Petrilli S. (ed). Philosophy of language as the art of listening, pp. 281-327 (electronic version http://www.augustoponzio.com /Critical/19._Petrilli.pdf; accessed 29.01.2012. p. 11).

Miltos Frangopoulos

255

Equally valid, I find, is the caveat for an alternative they propose: However, in the face of a corporate-led globalized world special semioses, different languages and cultures continue to persist and can be interpreted as signs of the potential for critique and resistance. 37

This is the vice in which the teacher is caught, in so far as he or she seeks to motivate students towards a stance that does not merely accommodate the world as it is. Thus the third tension: a predicament constantly restraining originality. How is a student to develop an attitude that challenges perceived ideas, without losing touch with the market which could secure their livelihood? In this series of unresolved contradictions, between word and image, invention and stereotype, rebelliousness and acquiescence, emerges the double bind of education that posits its endless challenge to the teacher of visual communication. But as our brief survey in the first part of this presentation may have shown, the old art of “visual thinking” may yet hold the key for future developments.

37

ibid.



PROJECT MY CITY MY PLACE: A CROSS-CULTURAL COLLABORATION IN GRAPHIC DESIGN MARYAM HOSSEINNIA

Introduction Graphic design students from Minnesota State University Moorhead and The American University of Kuwait participated in a cross-cultural poster exchange. Students were asked to research their environments and produce posters showing their place in this space using artefacts and symbols. The posters were put online for the students to critique. Through the process, students learned to be more aware of their environment and the environment of the other students. In addition, they learned how individuals across cultures use different signs and symbols to represent their personal space. The researcher also analysed the posters in order to see similarities and differences in outcomes. Graphic designers communicate messages through form and content. As these messages or signs are shared across different cultures, there can be quite a contrast in perception. Using the semiotic approach, these messages/signs can be analysed, processed and interpreted to understand their meaning and their influence on people’s interpretation in different parts of the world. The Middle East and the Arabian Gulf are often stereotyped in the media as a place of terrorism. The geographical areas conjure up visions of war, fear and danger. The typical images are of camels walking through dusty deserts on an underdeveloped landscape.1 The West is perceived by Arab youth as the land of opportunities. The images are of pop stars, fast



1 Marvin Wingfield and Bushra Karaman, “Arab Stereotypes and American Educators,” American-Arab Anti-discrimination Committee (March 1995), accessed November 21, 2011. http://www.adc.org/index.php?id=283&no_cache=1&sword_list%5B%5D=stereot ype.



Maryam Hosseinnia

257

food, arts and entertainment. In terms of the latter, America in particular is considered a hotbed of culture, from the film studios in Hollywood to the heart of creativity and the cultural scene in New York City. Kuwait and Minneapolis, Minnesota are two distinctly different places; their geographic location, the extreme contrasts in weather conditions, the contours of the land, and their physical distance from the heart of the art and cultural scene make this collaboration and case study more of an enriching experience. For example, in Minnesota the weather is incredibly cold and snowy six months out of the year, and the land is flat and mainly agricultural. In Kuwait, six months out of the year the weather is unbearably hot, while the land is a flat desert. Minnesota’s economy is based on agriculture, while Kuwait’s is petroleum-based. Because of its prosperous oil-based economy, most Kuwaitis enjoy the same amenities as Americans. The stereotype of underdevelopment is truly a falsehood. Kuwait is a very developed country with access to the same goods and services as in America. Unlike many surrounding Arab nations, Kuwaiti Arabs enjoy a modern and industrialised society. At times, however, the old values and ways of doing things conflict with the changes brought about by modernisation and urban life.2 Kuwait and the United States are allies with close economic ties. Kuwaitis are generally appreciative of the US military’s role during the Gulf War. Most educated Kuwaitis are able to speak English, a fact that contributes to good communication between them and the Americans living there. Furthermore, the younger generation, in particular, increasingly identifies with the Western world, and Western media and the Internet are the two vehicles most responsible for this trend, bringing the country’s youth in contact with the rest of the world.3 College life for students in the United States is the foundation and springboard for learning to live independently. In the US, students experience going to a bar for the first time. They learn to think for themselves and develop their own moral codes. They become more freespirited and develop new social networks. New life experiences, making mistakes, developing careers, falling in and out of relationships and building life-long friendships are part of growing up and experiencing college years in the US. Students at college level are independent, they live either in dorms or share housing with other students. Most students work part-time or even full-time while in school. Some are single parents.

 2 3



Soloman Isiorho, Kuwait (London: Chelsea House Publishing, 2002), 58. Isiorho, Kuwait, 108.

258

Project My City My Place

They have the responsibility of paying bills and some even pay for their own education. America has many sub-cultures, so it is difficult to generalise the average college student experience. Kuwait, on the other hand, has a strong sense of tradition, often based on Islam and their family beliefs. Thus, Kuwaiti university students have very different experiences. They live with their parents, and even married students live in the family home. Most Kuwaiti students do not work nor are they responsible for paying bills. The family is the most important aspect of Kuwaiti society (Kuwait). Parents expect their children to be in contact with them throughout the day. Maintaining traditions is also important. There is a continued segregation between boys and girls. They rarely socialise together, as this is a taboo. Female students hardly ever travel alone. One form of socialisation is the diwaniya (pronounced dee-wahn-eeya). These are usually held weekly. 4 Diwaniyas are formal or informal gatherings in a special room of a house and are traditionally male social events. There are family diwaniyas, but male friends will also get together once a week in the diwaniya to socialise. At the more formal diwaniyas, discussions revolve around politics and general life. While at informal diwaniyas, boys watch movies, eat and play games (cards, video games). Another typical type of social activity is going to malls. Families or friends will gather in malls to have lunch or dinner and go shopping. Another difference between these cultures is the summer experience for college students. It is common for the Gulf State Arab families to travel abroad to places such as Europe or the United States for their summer holidays. In the US, college students may return to their hometowns to work a full-time job. Alternatively, they may travel independently or with friends. Some families continue the tradition of the family summer getaway.

Participating universities Minnesota State University Moorhead (MSUM) is a four-year, public university located in Moorhead, Minnesota. The average number of majors in the Art and Design College emphasizing on graphic design is around 180 students. They offer BAs and BFAs. The American University of Kuwait in Salmiya, Kuwait, is a four-year, private liberal arts institution based on the American model of higher education. The average number of majors in the Art and Design Department is 80. They offer BAs. The programme is six years old, and the language of study is English.

 4



Isiorho, Kuwait, 68.

Maryam Hosseinnia

259

Semiotics in visual communication According to Hall, the genuine task of a visual communication designer is process; it is not product related. The notion of interpreting something unique about the culture or identity using the tools of visual communication, processes and context for creating to understand meaning in a variety of different ways is semiotics. 5 Semiotics in visual communication, in its simplest form, can be described as the study of signs. There are two fundamental aspects to a sign: signifier and signified. Signifier refers to a symbol, sound or image (as a word) that represents an underlying concept or meaning. In the language of visual communication, we work with type, image, colour, composition and symbols to convey a message, and this language in its wide form, is a series of meanings. There is also the language of abstract or ambiguous design: images that leave the perceiver to decide the meaning. The more ambiguous a sign is, the more the viewer has to work through it to identify it. Signified refers to a concept or meaning as distinguished from the sign through which it is communicated. Semiotics, then, is about the tools, processes and contexts we have for creating, interpreting and understanding meaning in a variety of different ways.6 As visual communicators, graphic designers are obliged to design with the audience in mind. In this case study, students began by brainstorming, mind-mapping and collecting artefacts to generate ideas. During each class period, work was pinned on a board for group critique. They were asked to continuously question themselves to further develop their concepts. Questions asked were, “who is the audience?”, “what is the message?”, “how do I communicate my ideas?”, “are my ideas being communicated effectively, and are they clear to the audience on the other side of the globe?”

Project outline Through means of digital technology, students from each of the institutions were required to design a poster about themselves, personally illustrating something unique about the place in which they live. They were to use type to advance their own interests on a subject as well as record observations or comment upon issues. They were able to explore a

 5

Sean Hall, This Means This, This Means That: A User's Guide to Semiotics (London: Laurence King Publishing, 2007), 5. 6 Hall, This Means This, 5.



260

Project My City My Place

broad spectrum of approaches using cultural/historical references to express effective and experimental visual communication. In addition, they were asked to address spatial, hierarchical and typographical concerns that are central to the design process (signifier). The final designs were then exchanged digitally on a blog (mycitymyplace.wordpress.com) to be critiqued by all the students involved (signified). This project was given to the students at the beginning of the spring semester in 2011. Each student had time to investigate their ideas, narrow them down to a single concept and develop them further for final submission. They had two weeks to complete this task. They were free to use any material as long as the final output was on A3 paper and produced digitally. Students were free to draw from found typography and imagery to illustrate the buzz of city life, daily rhythms, cultural background, neighbourhood, transportation, communication, childhood memories and mapping. They were encouraged to use a variety of methods and media to document and to generate designs. This process reflected each student’s unique interpretation of what it is to develop an identity of a place. A list of random questions related to the topic was given to the students to use as a benchmark in their creative process. They were to write and collect data based on the following questions: What is the earliest memory of your chosen place/home? What was the first job in your city/town? What area/street is special to you? One thing you would recommend to a visitor? Favourite shop? Best building? Favourite museum/gallery? Best park/open space? Hobby? Usual way you get about? Best walk/ride? Best view? Things you least like about your home/city/town? The instructor’s objective was to gain insight into the students’ creative processes. In addition, the project introduced collaboration as ideas were exchanged between two cultures. In turn, the project fostered the importance of networking with peers using the Internet.



Maryam Hosseinnia

261

Methodology The project was two-fold: students critiqued one another, while the instructor looked at student outcomes from a design perspective and through semiotics. Four posters were selected for analysis and critique. The result of their work in relation to semiotics and visual communication was broken down into three different ways to decode or to interpret the meaning of the forms within the semiotics context. These were conceptual structure, sign and symbol, and visual structure.

Analyses of Student Posters In the original study, six student posters were randomly selected then critiqued and compared cross-culturally by the author. Here, a few of the results and outcomes are discussed. Figure 4-11 was designed by a male Kuwaiti student, who took a rather humourous approach to the ordinary life of two Kuwaiti men. The subjects in the poster are using colloquial language, mimicking dialects phonetically and linguistically. The student even went so far as to misspell a few words. This shows an honest conversation about politics, culture, entertainment and economy between two Arab men. It also relates the idea of personal through text.

Left: Fig. 4-11. AUK. Male Kuwaiti student, age 29 Right: Fig. 4- 12. MSUM Male American student, age 22.

The use of typographic elements justified on the page in different sizes engages the reader in the rhythmic tone of the their dialogue. The



262

Project My City My Place

symbolic words that carry phonetic and semantic significance can also be carriers of elements such as humour and metaphorical significance, not only through their literal meaning, but also through the relationship between meaning and visual decisions. This student demonstrated the ability to use the bulk of text as visually telling. It is not literally significant in his decisions of size, spelling, colour and harmony with the image and the awareness of space and hierarchy as well as relationships through colour and size. The notion of parcel or recycled paper used on the background and the contrast between digital and illustrative adds a feeling of mystic and comic to the overall composition. Despite the clear content of the text, which a viewer would have to read, the inferences within the text do not work alone; in fact, they are entirely dependent, despite their size and hierarchical dominance on the page, on the two walking figures. Interestingly, the image could represent any of the Gulf countries, which means that it does not necessarily communicate my city/my place. The text gives the aspect of personal. However, for the audience to know this, they would have to be aware of the fact that the dialogue is a very accurate description of a typical conversation between two young Kuwaiti men. One American male student participant chose icons to visually communicate a typical day of his college life (fig. 4-12). A pearl border was combined with a central silhouette of binoculars. The title of the poster “A Field Guide to My Life” is an illustrative play on wildlife manuals. The symbols are carefully illustrated and arranged on the poster in chronological order, mapping his daily life from morning to night. The poster is subtle but effective, impersonal but extremely representative of the student’s life. It is well structured and designed in a straightforward manner. There is no question to this individual’s routine. The cool grey background is neutral and allows the symbols to tell the narrative. A female Kuwait student’s poster focuses on globalisation and the Western influence on the Middle East. It reflects what it is like to be a Muslim woman in Kuwait or the Middle East. The poster consists of a silhouette of a faceless head of a woman wearing a hijab. The colours are washed beige and white. The Arabic word “globalisation” (originally in fuchsia) over the image carries a strong message of how Westernisation has become part of the girl’s identity. It is in coloured font in the foreground, almost on her face. It supersedes her other features and qualities, those of nationally, interest and studies, which are an afterthought in the background. As stated, the most prominent aspect of the poster is the warm fuchsia word “globalisation” written in Arabic. The position of the fuchsia colour



Maryam Hosseinnia

263

on the page, and its contrast to the rest of the form, could be seen as the loss of Arab identity in the sense that the language is now under the pressure and weight of the English language. She seems to convey the message that she is losing her sense of self in the global era.

Left: Fig. 4-13. AUK. Female Kuwaiti student, age 19. Right: Fig. 4-14. MSUM. Female American student, age 20.

A junior-level female student depicts the sequence of her daily activities using digital photography (fig. 4-14). As with other American students in the study, she includes her name on her poster. The majority of the images are of herself, of getting up and coming home. She also depicts her friends, food, computer and workplace. She visually represents her day through common, but personal, glimpses of her life in sequential order. The poster time-lapse photography tells the chronological narrative. This interesting photographic element is almost like stop-motion animation or a flipbook. The first and the last photographs are simply glimpses of the remainder of the student’s day, giving the piece varying speeds and thus hierarchy between the series of photographs. Visually, it is extremely personal; the photographs of her getting out of bed in the morning exemplify this.

Cross-cultural similarities and differences of symbols and visual interpretations Analysing the posters reveals both similarities and differences between the American and Kuwaiti students’ interpretations of the project, their cities and themselves. In terms of the similarities, the sophomore and junior



264

Project My City My Place

students have a good sense of composition and storytelling. Some of the layouts have strong visual structure, while others are vernacular. The use of colour was varied in both groups, and they used a variety of techniques (i.e. drawing, photography and text (handwritten and digital) to express themselves. In terms of differences, the students had very different ways of expressing their personal lives. MSUM student posters are far more personal than those designed by the AUK students, as the former chose images of themselves and their lives as college students. The examples show that the American students are not shy of sharing their private spaces and life; they are more expressive and honest with their daily activities. Figure 4-11 shows two Kuwaiti men walking away from the viewer. Above them is their conversation, which could be a dialogue between any Gulf Arab men. There is no actual identity placed on the people in the poster, except for the clothing, which is traditional for the local geographical area. The Kuwaiti girl’s poster (fig. 4-13) shows a silhouette of a face. It is anonymous. On the other hand, the American girl’s poster (fig. 4-14) is intimate. Kuwaiti students, in general, did not show actual photographs of their faces, remaining anonymous in a project entitled “My City My Place”. As stated, the American student was not shy of sharing her private space and life. This implies the notion of a closed Kuwaiti society and culture. On a daily basis, young Kuwaiti college students expose themselves to their friends and acquaintances at school and around the town. Perhaps revealing themselves beyond the boundaries of their community may be a taboo or at least, uncomfortable. There is also evidence that the students have different interests and priorities. MSUM students showed a variety of objects or artefacts that are part of daily rituals. There was a focus on material items as representations of their likes. On the other hand, there are no direct visual artefacts in the Kuwaiti student posters. Interestingly enough, the Kuwaiti student posters seem to be more introspective, but also allude to the idea of Arabism. The focus is broader, on themselves as members of a society (figures 1 and 3). There is no evidence of them trying to express who they really are to this foreign audience. Perhaps the concept of “Arab Spring” and what has been happening politically in the region have had an impact or an influence on their way of thinking. In their silence, they are artistically expressing an opinion. There is also more of an emphasis on strong traditional values or culture. They had a chance to show what Kuwaitis or Arabs are really like, yet this was not expressed. Instead, the posters show that this generation is cross-cultural. There is a subtle combination of the opposing ideologies. This result can be compared to the opposite outcomes in the American



Maryam Hosseinnia

265

students’ posters (figures 4-12 and 4-14). They show typical American college experiences; they are open, honest and non-judgmental. They share who they are in their place. In terms of content, MSUM students relied more on imagery than text. AUK students’ solutions incorporated an equal amount of text and images, or they relied more on content than visual expression. In addition, only one student used Arabic and English lettering to express a cross-cultural connection (fig. 4-13). Through the silhouette image and Arabic and Latin text, we can differentiate cultures. Even though English is considered the second language in Kuwait, the new generation uses English as its first language. In a way, the Arabic language and culture is in transition as it is being affected by globalisation.

Student and instructor reactions to the project The students, my colleague and myself had many reactions to the project from start to finish. In terms of feasibility, fitting the project into a hectic timetable was not too difficult. The industry requires one to be trained and ready to produce multiple projects at the same time. Today’s technologies made this collaborative project much easier to accomplish in a short amount of time. Because of the short timeline, students had very little oneon-one time with their instructors. We were not too concerned about this because one of our goals was to push the student notion of critical thinking. We wanted them to focus on the delivery of the message and to tap into their personal voice without our interference. The list of questions we provided the students with was meant to activate their creative process. Nevertheless, some students found the list more confusing; they felt that it made it more difficult for them to narrow down their ideas. However, the point was to have students really investigate their total environment. The critical point in this process was at the end when students critiqued each other’s posters. Students quickly became aware that their work was being seen beyond the walls of the studio for the first time. One Kuwaiti student stated that she did not want her family pictures or name to be exposed to the other side; it was all right to share these with students in the class but not with the MSUM students. This unknown or invisible audience made us both, the professors and the students, anxious about the outcome.



266

Project My City My Place

Conclusions This case study discusses a cross-cultural collaboration in design learning between second-year students at the American University of Kuwait (AUK), and second/third year students at Minnesota State University Moorhead (MSUM) in the United States. Fifty students were asked to examine an aspect of their social and/or cultural environments and produce a poster that showed their place in this environment. The posters were uploaded onto a blog for students to critique. This study examined four of the posters. In terms of outcomes, Kuwaiti students used a broader approach to show something about their city and place. American students focused on campus and personal life. In addition, American students used more photography than text. We interpreted the results to show that American students are far more open and willing to share or expose their private lives and space. Yet, the goal of this project was for students to further investigate the creative design process, self-authorship, personal voice, how to think and investigate more about their roots, identity and history. In addition, this project aimed to develop a sense of understanding of the media’s influence on people’s perceptions in different parts of the world. While developing technical skills, students expanded on communication tools to share and exchange ideas, particularly across cultures. The students learned about thinking globally, since their outcomes were viewed outside their immediate environments. In a studio environment, the effective way to encourage content development is to ask students to draw from personal experience and to communicate that to a broader audience. Through critique, we could determine what was important to each design student. Personal mythologies emerged when new forms and meanings were discovered in relation to familiar themes. These cultural myths teach us something about the world in which we live and ourselves. What results is a reflexive communication between the maker and the work, inviting the audience to partake in the conversation. Moreover, they learned how an individual’s visual interpretations of their spaces and senses of identity could be similar and different. The findings attempt to raise awareness about the needs and possibilities for more collaboration with schools from other cultures all over the world, and the development of an interactive learning environment between programmes. Digital technology has opened up the possibilities of collaborative learning on a global level. This promotes



Maryam Hosseinnia

267

their voice and vision and allows them to take risks and experiment with typography and images. Students become more sensitive to content and more aware of different cultures all over the globe. On a deeper level, their understanding of graphic design was amplified as they realised the potential for generating and answering questions on a global level concerning language, identity and perception. As they experienced how graphic design allows them to frame themselves and their milieu, their capacity for discovery and interpretation expanded. Their roles as designers shifted to that of editors, supplementing content rather than generating it. Their works became a compilation of ideas. A designer's involvement with content is defined by his/her ability to assemble visual and verbal materials from diverse sources in an organised and expressive manner, thus creating a coherent narrative or point of view. By encouraging students to develop their own content and messages in their posters, they learned to take a proactive stance, augmenting the standard client-driven model. Thus, they were able to draw upon personal interests to create an original graphic project. In other words, they became self-authoring designers. The collaborative project has allowed for communication and interaction between students from different continents who are uniquely joined through visual communication. Their ideas, thoughts and processes have allowed a greater understanding of each other's lives and abilities, with the results provoking thoughtful contemplation and critical reflection on how they use typography and imagery to communicate meaning. The results and experiences gained from this project will inform the future design strategies and methods by which all involved approach their creative futures.

Bibliography Hall, Sean, This Means This, This Means That: A User's Guide to Semiotics. London: Laurence King Publishing, 2007. Isiorho, Soloman, Kuwait. London: Chelsea House Publishing, 2002. “Kuwait,” Countries and Their Cultures. Access date November 21, 2011. http://www.everyculture.com/Ja-Ma/Kuwait.html. Wingfield, Marvin and Karaman, Bushra, “Arab Stereotypes and American Educators,” American-Arab Anti-discrimination Committee, March 1995. Access date November 20, 2011. http://www.adc.org/ index.php?id=283&no_cache=1&sword_list%5B%5D=stereotype.



268

Project My City My Place

Further Reading Di Piazza, Francesca Davis, Kuwait in Pictures. Minneapolis, USA: Lerner Publishing Company, 2007. Moxey, Keith. “Visual Studies And The Iconic Turn,” Journal of Visual Culture no. 7.2 (2008): 131-146. White, Alex W., The Elements of Graphic Design. New York: Allworth Press Publishing, 2002.



THE RECEIVER IS THE MESSAGE? PETER C. JONES

An investigation into substituting established forms of Market and Target Audience Segmentation with Pedagogic Segmentation This essay outlines the early stages of a practice-based PhD into the effect on communication design methodologies and outputs by substituting established types of market segmentation e.g. A, B, C, D, & E’s, (socioeconomic grading) with theories and categories used by teachers to identify the styles or models by which people learn. One example of these styles is Honey & Mumford’s (1982) “Learning Styles”, which includes sub-types or segments, such as Activist, Reflector, Pragmatist and Theorist. The motivation for this project stems from my own experience as a practicing designer where the prescriptive use of existing research methodologies within advertising, graphic design and marketing can often lead to clichéd and mediocre solutions. Marketing, advertising and design professionals use a range of methodologies to analyse and identify communication problems in order to develop a strategy or project brief. However, core to, or used in tandem with, almost all of these methodologies, is the use of some form of market segmentation or target audience analysis in order to define the intended addressee. Therefore, my approach of radically changing a core component within these established methodologies: i.e. substituting established forms of market segmentation with a range of learning styles or pedagogic segmentation, has not only the potential to act as a catalyst to the formulation and addressing of new communication design criteria, but also to provoke new approaches, possibilities not only within established advertising, graphic design and marketing research processes and outputs but also within pedagogy.

270

The Receiver is the Message?

Proposition and aims This technique of radically changing a core design criteria, in order to evaluate and address a problem from a different perspective, is also an established methodology, for example Altshuller’s “Triz” 1961, Jones’s “Design Methods” 1970 and De Bono’s “Lateral Thinking” 1971. Consequently, I propose to deliver the same core message in several different ways through a variety of media and formats, using extant and emerging communications technologies. Each delivery or prototype will be tailored to a different style of learning or pedagogic segment. In today’s multi-media, multi-platform, multi-channel environment, I would maintain this approach as eminently achievable and practical. My ambition is to create a range of prototype communications using the same core message, each message format will be designed to address and engage a particular learning style or pedagogic segment. Depending upon the physical context of the message (most likely an exhibition), users may then browse and select their preferred message format, indeed they may select more than one and view the same core message through a variety of formats. This approach enables my ambition not to use learning styles or pedagogic segmentation as a diagnostic tool or to categorise the audience directly, but to allow the audience to browse and select their preferred message format. This will not only enable me to evaluate the audience engagement with the prototype messages, but also to critically analyse the impact of the media on the message.

Core message and minimal demographic segmentation The core message is a history of the Western alphabet. This has a broad ranging educational and cultural significance and consequently could, or indeed should be, of interest to anyone that uses an alphabet. However, the latter still requires a form of target audience or demographic segmentation. I have chosen to define this as a Lay Reader. This is to create workable design parameter but still be as inclusive as possible. Within the UK, a Lay Reader is generally described as an individual that has achieved the basic educational attainment expected for a 16-year-old, although their attainment and/or life experience may place them well above this threshold.

Peter C. Jones

271

Problems with Learning Styles & Learner Segmentation A key finding is that the vast majority of learning style models do not appear to be verified by any substantial empirical evidence and/or effectively underpinned by credible theoretical frameworks, as a paper in the American Journal “Psychological Science in the Public Interest” goes on to outline. Although the literature on learning styles is enormous, very few studies have even used an experimental methodology capable of testing the validity of learning styles applied to education. Moreover, of those that did use an appropriate method, several found results that flatly contradict the popular meshing hypothesis. 1

Whilst the Coffield (2004), an earlier more authoritative and comprehensive 200-page report on learning styles, was equally critical. The sheer number of dichotomies in the literature conveys something of the current conceptual confusion. We have, in this review, for instance, referred to: convergers versus divergers • verbalisers versus imagers • holists versus serialists ... The sheer number of dichotomies betokens a serious failure of accumulated theoretical coherence and an absence of well-grounded findings, tested through replication. 2

Pedagogic Segmentation: an alternative to Learning Styles & Learner Segmentation? There is an emerging consensus in pedagogic theory and practice around approaches to learning and teaching based on a conflation and alignment of a variety of more established theories, ostensibly from psychology but also from pedagogy. These emerging approaches or conflations of more established theories take a more pragmatic top down approach and place an emphasis on the teacher evaluating, selecting and implementing an appropriate approach/model or combination of approaches/models depending upon a variety of factors such as: context, curriculum content, the size and nature of the cohort, the mode of delivery and the mode of learner engagement. These changes in pedagogic theory and practice appear to 1

Harold Pashler and others, “Learning Styles: Concepts and Evidence.” Psychological Science in the Public Interest 9, no. 3 (2008): 105. 2 Frank Coffield and others, Learning styles and pedagogy in post-16 learning: A systematic and critical review. (London: Learning and Skills Research Centre, (2004), 136.

272

The Receiver is the Message?

have been largely initiated by, and emanated from, e-learning research projects that are based on more robust interpretations of established pedagogic and psychological theories. The following three have been identified: - “Review of e-learning theories, frameworks and models”, Terry Mayes & Sara de Freitas, JISC e-Learning Models Desk Study (2004). - “JISC Innovative e-Learning with Mobile and Wireless technology”, JISC & The Higher Education Academy (2006) based loosely on Mayes & de Freitas (2004). - “The Missing Link in Promoting Quality Education: Exploring the role of pedagogical design in promoting quality in teaching and learning”, S. Naidu (2006). Whist these focus on approaches to teaching and learning as opposed to learner segmentation, all of the above still use forms of what I have called Pedagogic Segmentation, for example JISC (2004) report uses: Associative, Constructive (individual), Constructive (social) and Situative Learning models, whilst the Naidu (2006) uses models such as Behaviorism, Cognitivism and Constructivism. As can be seen, they are also broadly similar in terms of their segmentation and are based on similar or related theoretical frameworks such as: “Constructive Alignment”, Biggs, J. (1999). “Activity Theory”, Cole, M. & Engestrom, Y. (1993). “Instructivism and Instructional Design”, Gagne, R. M. (1974/1985). “Experiential Learning Cycle”, Kolb, D. (1984). “Conversational Framework/Model”, Laurillard, D. (1993). “Communities of Practice & Situated Learning”, Lave, J. & Wenger, E. (1991). “Stages of Cognitive Development”, Piaget, J. (1936/70). “Operant Conditioning”, Skinner B.F. (1953). “Zone of Proximal Development”, Vygotsky, L. S. (1934/1962). I maintain the pedagogic segmentations as proposed by Mayes & de Freitas, etc. or an adaptation of these could form the basis for credible communication design criteria. As these approaches are based on established theories and frameworks, not only can the latter frameworks be used in the formulation of the design criteria, but they can also be employed in conjunction with other theories and methodologies described later, to critically analyse and evaluate communication design outputs/prototypes and user/ audience experience. To reiterate; the premise of the research is not to diagnose the audience but to observe, document and understand how audiences experience,

Peter C. Jones

273

engage with and assimilate messages. The intention is to present all messages at the same time in the same location (each version of the core message designed to align with a particular pedagogic segment), and the user, then, browses and selects their own preference. Apart from ensuring that any users taking part in the research fit within the broad definition of a Lay Reader, this negates any complex pre- or post-analysis or classification of those involved.

Additional theoretical contexts & frameworks Transdisciplinary research: An appropriate mesh of theoretical contexts The transdisciplinary nature of this research project has made it challenging to identify a ‘mesh’ of appropriate theoretical contexts with which to formulate and address the research question. The core areas are Pedagogy and Communication Design. Pedagogy has its own theoretical canon, it also overlaps significantly with psychology and indeed appears, as outlined previously, to be increasingly using established theories and practices from psychology to form the basis for new and more authoritative pedagogic research. As a comparatively new academic discipline, Communication Design or Graphic Design could be said to be still in the process of establishing its own theoretical frameworks. Current communication or graphic design theory tends to be a mix of art and design history analysed and contexualised with a variety of other theoretical frameworks taken from such diverse areas as Semiotics, Design, Media, Art, Photography, Branding, Advertising and Marketing. This research also requires the addressing of an eclectic range of meanings, discourses and theoretical contexts in order to formulate, contexualise and critically analyse: the design and production process; the delivery of the same core message across a range of media, plus audience experience, engagement with and assimilation of the messages. This eclectic methodology and approach can be said to be typical or indeed core to the theory and practice of communication design that by its nature addresses indeterminate, eclectic and cross-disciplinary problems.

274

The Receiver is the Message?

Design Eclecticism & Indeterminacy The diagram below outlines the areas of research, the principle overlapping between them, with the research question that is a typical example of “Design Eclecticism and Indeterminacy” at the hub.

Fig 4-15, the research proposition as an example of the eclectic and indeterminate nature of the Design process.

This approach is also particularly indicative of the communication/ graphic design discipline. Graphic/communication designers are often required to understand and engage with a wide range of other disciplines, organisations and professions to make their client’s ideas, ambitions, services and products distinctive, engaging, coherent and tangible. In order to achieve the latter successfully, it requires an engagement, understanding and development of creative alignments across a varied combination of disciplines, processes, methodologies, media, ideas and cultural contexts, to address problems that may often contain contradictions that are incomplete, difficult to define or indeed are “fuzzy” or “wicked”. A designer’s strengths lie in the ability to identify, formulate and address problems that can be described as “wicked” (Buchanan 1992). Buchanan elaborates on the indeterminate nature of Design problems.

Peter C. Jones

275

Why are design problems indeterminate and, therefore, wicked? Neither Rittel nor any of those studying wicked problems has attempted to answer this question, so the wicked problems approach as remained only a description of the social reality of designing rather than the beginnings of a well-grounded theory of design. However, the answer to the question lies in something rarely considered: the peculiar nature of the subject matter of design. Design problems are "indeterminate" and "wicked" because design has no special subject matter of its own apart from what a designer conceives it to be. The subject matter of design is potentially universal in scope, because design thinking may be applied to any area of human experience. But in the process of application, the designer must discover or invent a particular subject out of the problems and issues of specific circumstances. 3

The substitution of pedagogic segmentation to create alternative design criteria is an example of the process described above; i.e. the designer discovering or inventing “a particular subject out of the problems and issues of specific circumstances”. Whilst the segmentations proposed by Mayes & De Freitas, (2004) etc. are also examples of “wicked problems”, in that although the definitions or descriptions of these models/segments appear to be as a result of an emerging consensus, these models only offer broad principles and how they may be interpreted and implemented is far from determined or clear-cut.

Design & Graphic Design User Experience & Persuasion Theories and practices relating to user/audience experience and persuasion will also inform this research. Within an academic and professional context there is an established theoretical and practice-based framework of “Design Thinking” and “Design Sensibilities”, particularly in relation to product design, development, user context, experience and interaction, that has been applied, and continues to be applied and developed, to products and interactive communications (Brown, 2008), (Christensen, 2005). Graphic design also addresses a wide range of audience ‘points of action’ (or user experience, interaction and persuasion), whether they are instructional or promotional communications, both are aimed at affecting perceptions and/or behaviour. These techniques have been employed within conventional graphic design for decades, however, as with e3

Richard Buchanan, “Wicked problems in design thinking.” Design Issues 8, no. 2 (1992): 16.

276

The Receiver is the Message?

learning and its influence in generating new pedagogic theory, the developments in communication technology have led to new academic research in communication, which, although focused on digital communications, still have wide ranging relevance to extant or emerging communication technologies and messages. Persuasive rhetoric is as simple as the boldface type highlighting a name when we scan a newsmagazine paragraph. But screen-based electronic media create both the opportunity and imperative for a far deeper application of persuasive rhetoric through interactivity, sound, and motion. These new design dimensions can generate smart persuasive character, attitude, and behaviors to persuade users/readers to make the right moves for effective operation. Persuasion and seductive rhetoric can be developed as theories to explain and evaluate existing communications phenomena, expose and clarify current design strategies, and codify new design strategies for generalized application in communications design. 4

Design Prototyping: Exploratory, Experimental and Operational The design of a poster, a physical product or an interactive digital communication, all tend to be developed through a heuristic methodology of making and reflection. This “Design Sensibility” or “Design Thinking” centres on “Exploratory Prototyping” or through the process of making and reflection, trial and error, developing an understanding not only of the solution but also of the problem. Initially, this may involve creating a range of alternative designs in the form of sketches or “Low Fidelity Prototypes” to explore and clarify the problem and view it from different perspectives, then develop one or more alternatives or “Experimental Prototypes” to a “Higher Fidelity” and develop to a point that strongly suggests the efficacy of the design; only then iteratively refine one of the alternatives to become a finished final proposal or “High Fidelity”, “Operational Prototype” (Manner 1997). Whilst the above describes a range of prototypes and how they may be used, it also suggests that the design process is not simply a sequential mechanistic iterative process. Berente and Lyytinen (2005) describe design as a heuristic approach of not only “mapping the solution space” but also the “problem space”.

4

Katherine McCoy, “Information and Persuasion: Rivals or Partners?” Design Issues 16, no. 3 (2000): 83.

Peter C. Jones

277

The reasoning process of a designer is described as abductive (retroductive) inference (a chain of operations), in contrast to it being inductive or deductive inference, which are well known modes of inference in scientific studies (Peirce 1992). Abduction generates a design hypothesis (a mapping between a problem space and a solutions space), often a good guess by the designer in the face of an uncertain situation, to a given problem and then works with this hypothesis until it is no longer deemed practical – at which time another hypothesis is generated. Simon (1996) describes this form of cognitive activity as nested “generate-test cycles” and argues that they are fundamental to design. He conceives design as problem solving, where designers engage in a “heuristic search” of design alternatives, and then choose (decide) a satisficing design to go forward. When the alternative is shown not to be the proper course, a new cycle of heuristic search begins. During the design process, designers engage in iterative learning about both the problem space and the solution space (Simon 1996, Cross 1989). 5

The design process and outputs can often be said to be based on a heuristic, if not intuitive then, subjective approach that requires an individual and/or group ‘best guess’, based on prior experience, knowledge and expertise. This approach does not always readily align or sit comfortably within established academic research methodologies and practices.

Rationalisation of nomenclature A consequence of the transdisciplinary nature of this research is the range, variety and crossover of the nuanced, similar or contradictory nomenclature, ideas and meanings employed by the various discourses, theories and disciplines. For example, within this paper depending upon the context, meaning and emphasis required: communication, message, artefact, sign, curriculum and user experience can be interchangeable, whilst the same could be said for audience, addressee, user, receiver, learner, student and cohort.

Semiotics An overarching taxonomy Semiotics provides us with a potentially unifying conceptual framework and a set of methods and terms for use across the full range of signifying 5

Nicholas Berente and Kalle Lyytinen. “Iteration in Systems Analysis and Design: Cognitive Processes and Representational Artifacts.” Sprouts: Working Papers on Information Systems 5, issue 4 (2005): 181.

278

The Receiver is the Message? practices, which include gesture, posture, dress, writing, speech, photography, film, television and radio. 6

As Chandler notes above, semiotics has “terms for use across the full range of signifying practices”, therefore I propose to develop a taxonomy or glossary to define the meaning and use of key words or phrases for this research project. I intend to investigate semiotic theory to either: utilise its existing nomenclature or to inform the synthesis of an overarching transdisciplinary nomenclature. Semiotics also provides useful overlaps with other theoretical contexts and frameworks relevant to this research such as narratology as well as the pedagogic/psychological discourses of Vygotsky, Piaget, Lave & Wenger. Equally importantly, semiotics has frameworks for critically analysing and contextualising the relationship between the media and the message from Barthes, to the Technological or Media Determinism of McLuhan to the Linguistic Relativity of SapirWhorf.

Narrative Structure, context and meanings: perception and behaviour Whilst narrative inquiry in terms of observational ethnographic research is not intended as a form of data collection, I do intend to address how users engage and interpret the messages and narratives by using a range of ethnographic and phenomenological data collection methods in relation to user experience i.e. audience engagement with, and assimilation of, a series communication design prototypes. These ‘qualitative’ and ‘exploratory’ data collection methods may range from focus groups, interviews, to real time and/or recorded audio visual observation, to document and analyse the engagement and assimilation that occurs (Dourish, 2001), (Suchman 2007) or use Opportunistic or Judgement sampling (Murphy 1998). The content of the prototype messages will invariably be some sort of narrative that presents the evolution of Western letterforms. Consequently, the structure and presentation of the story can also be formulated, critically analysed and contextualised through narrative theory (Propp, 1928), (Arendt, 1958), (Chapman, 1978), (Stern, 1998). However, there is also a significant overlap between semiotics and narrative theory, for example Narratology (Barthes and Greimas etc.). The above could be said to deal 6

Daniel Chandler. “Strengths of Semiotic Analysis.” Aberystwyth University, http://www.aber.ac.uk/media/Documents/S4B/sem10.html (accessed September 2, 2011).

Peter C. Jones

279

with the structure, context and meanings. Bruner (1986) approaches narrative from a psychological perspective as a way of people understanding experience from “a subjective perspective” Murray (2005). Bruner published Essays for the Left Hand (1962), where he proposed the existence of a ‘library of scripts’ which are available to members of a culture as repertoires of under-standing. In his later contribution to narrative psychology (Bruner, 1986), these two strands of thought are brought together in the opposition between paradigmatic reasoning and narrative thought, Paradigmatic reasoning shares with scientific explanation the mode of inductivism. Through it one sees a world of objects which interact in regular patterns. Narrative thought, by contrast, attempts to maintain a subjective perspective on the world it represents, incorporating aims and fears into the picture. It incorporates both a knowledge of the world and the point of view which beholds it. 7

This cognitive approach to narrative can be said to be the flip side to semiotic theory in that the latter tends to analyse meanings and contexts, whilst narrative psychology tends to focus on how people may use narrative to interpret and understand their experiences. Allied to the latter, there is also an emerging consensus and area of research relating to the power of narrative particularly within the medical/clinical communications targeted at patients and the general public to inform and influence behaviour (Green 2006). Narratives can be an effective means of communicating cancer-related information. Transportation into narrative worlds, or immersion into a story, is a primary mechanism of narrative persuasion (Green & Brock, 2000, 2002). Transportation theory extends the domain of traditional message effects theories, as well as providing mechanisms for behavior change. Trans-porting narratives can both change beliefs and motivate action, and may be particularly useful for conveying cancer information because they reduce counterarguments (and thus help individuals overcome barriers to treatment seeking); facilitate the mental simulation of unknown, difficult, or frightening procedures (e.g., screening); provide role models for behavior change; and create strong attitudes that are based on both cognition and emotion. 8

7

Kevin Murray. “Narrative Partitioning: The ins and outs of identity construction.” http://home.mira.net/~kmurray/psych/in&out.html (accessed September 8, 2011). 8 Melanie Green. “Narratives and Cancer Communication.” Journal of Communication 56, (2006): S163–S183.

280

The Receiver is the Message?

The use of narrative to influence consumer awareness, perception and behaviour has also recently become a recognised practice within graphic design, branding and advertising. How a brand is effectively positioned and differentiated in relation to its competitors then communicated across a diverse range of communications with different objectives and audiences, requires sophisticated levels of co-ordination. Whilst brands may maintain co-ordination and recognition through rigorous control of their visual/audio assets, the developing complexity, audience engagement with and indeed audience authorship of brand related messages, has made the co-ordination of a brand more problematic, and one means of addressing this is by employing a narrative (Garment 2006). The current, highly fragmented media environment calls for an adaptable positioning which is more responsive to a much broader range of audiences, media platforms, media channels and competitive exposures – a cohesive positioning ‘narrative’ which can be extended to communicate the brand to many different audiences that each view the brand from its own perspective. 9

Therefore, a brand narrative or the story of the company/service/product can be employed across a diverse range of communications, enhancing brand co-ordination and recognition.

Technology Determinism & potential media alignments with Pedagogic Segments Whilst technology as such is not a theoretical context or framework, the impact of the media on the message and subsequent user engagement and assimilation of the message are key to this research. Therefore, the choice and application of technology is inextricably linked to the theoretical contexts or frameworks; indeed from a technological determinist perspective, it is technology that imposes meaning whilst driving and shaping these theoretical contexts or frameworks. In conclusion, further research is planned to investigate potential technological alignments with the Pedagogic Segmentation of Mayes & de 9

Garment, John. “Brand Narratives: Positioning in the time of media fragmentation.” Yellow Papers Series, (2008). (no volume, issue or page number stated)

Peter C. Jones

281

Freitas etc., whilst initial investigations suggest that Haptic technology, 360 Degree and Immersive Cinema, Interactive Digital tablets, Augmented Reality and ‘hands-on’ traditional analogue technologies may be aligned and harnessed to create communications that address these various pedagogic segments.

Bibliography Barab, Sasha and others. “Critical Design Ethnography: Designing for Change.” Anthropology and Education Quarterly 35, issue 2 (2004): 254-268. Berente, Nicholas, and Kalle Lyytinen. “Iteration in Systems Analysis and Design: Cognitive Processes and Representational Artifacts.” Sprouts: Working Papers on Information Systems 5, issue 4 (2005): 178-197. Biggs, John. “What do inventories of students’ learning processes really measure? A theoretical view and clarification.” British Journal of Educational Psychology 63, issue 1 (1993): 3-19. Braun, Virginia, and Victoria Clarke. “Using thematic analysing psychology.” Qualitative Research in Psychology 3, issue 2 (2006): 77101. Brown, Tim. “Design Thinking.” Harvard Business Review, June 1, 2008. Buchanan, Richard. “Wicked problems in design thinking.” Design Issues 8, no. 2 (1992): 5-21. Carnegie, Teena. “Interface as Exordium: The Rhetoric of Interactivity.” Computers and Composition 26, (2009): 164-173. Chandler, Daniel. “Strengths of Semiotic Analysis.” Aberystwyth University, http://www.aber.ac.uk/media/Documents/S4B/sem10.html Coffield, Frank and others. Learning styles and pedagogy in post-16 learning: A systematic and critical review. London: Learning and Skills Research Centre, 2004. Cole, Michael, and Yrjo Engeström. “A cultural-historical approach to distributed cognition.” In Distributed cognitions: Psychological and educational considerations, edited by Gavriel Salomon, 1-46. New York: Cambridge University Press, 1993. Christensen, Karen. “Questions for: David Kelley Focusing on the User.” Rotman Magazine, Fall, 2005. Cross, Nigel. Engineering Design Methods. Chichester, UK: John Wiley & Sons, 1989. De Bono, Edward. Lateral thinking for management: a handbook. Maidenhead, UK: McGraw-Hill, 1971.

282

The Receiver is the Message?

Dourish, Paul. “Seeking a foundation for context-aware computing.” Journal Human-Computer Interaction 16, issue 2 (2001): 229-241. Fallman, Daniel. “The Interaction Design Research Triangle of Design Practice, Design Studies, and Design Exploration.” Design Issues 24, no. 3 (2008): 4-18. Fulton, Jane and Michael R. Hendrix. “Developing Design sensibilities.” Rotman Magazine, Spring, 2010. Fogg, BJ. “Creating Persuasive Technologies: An Eight-Step Design Process.” Proceedings of the 4th International Conference on Persuasive Technology, article 44 (2009). —. “A Behavior Model for Persuasive Design.” Proceedings of the 4th International Conference on Persuasive Technology, article 40 (2009). Gagne, Robert and Pamela Driscoll. Essentials of learning for instruction. Englewood Cliffs, NJ: Prentice Hall, 1988. Garment, John. “Brand Narratives: Positioning in the time of media fragmentation.” Yellow Papers Series, (2008). (no volume or issue number stated) Green, Melanie. “Narratives and Cancer Communication.” Journal of Communication 56, (2006): S163–S183. Herman, S. “Creativity and Innovation: Innovation means looking at the world in different ways.” Global Cosmetics Industry, September, 2009. Honey, Peter and Alan Mumford. The manual of learning styles. UK: Peter Honey, 1982. JISC. Innovative e-Learning with Mobile and Wireless technology. London: JISC & The Higher Education Academy, 2006. Jones, John C. Design Methods. New York: John Wiley & Sons, 1992. Kolb, David A. Experiential learning: experience as the source of learning and development. Englewood Cliffs, NJ: Prentice Hall, 1984. Laurillard, Diana. Rethinking university teaching: A framework for the effective use of educational technology. London: Routledge/Falmer, 1993. Lave, Jean and Etienne Wenger. Situated learning: legitimate peripheral participation. Cambridge, UK: Cambridge University Press, 1991. Lockton, Dan and others. “Influencing Interaction: Development of the Design with Intent Method.” Proceedings of the 4th International Conference on Persuasive Technology, article 5 (2009). Maguire, William. “The Yale Model of Persuasive Communication.” Communication Institute for Online Scholarship, http://www.cios.org/encyclopedia/persuasion/Byale_approach1.htm Manner, Walter. “Prototyping.” Bowling Green State University, http://csweb.cs.bgsu.edu/maner/domains/Proto.htm

Peter C. Jones

283

Mayes, Terry and Sara de Freitas. JISC e-Learning Models Desk Study. London: JISC, 2004. McCluhan, Marshall. Understanding Media. London: Routledge & Kegan Paul, 1964. McCoy, Katherine. “Information and Persuasion: Rivals or Partners?” Design Issues 16, no. 3 (2000): 80-83. Murphy, Elizabeth and others. Qualitative research methods in health technology assessment: a review of the literature. Norwich, UK: HMSO, 1998. Murray, Kevin. “Narrative Partitioning: The ins and outs of identity construction.” http://home.mira.net/~kmurray/psych/in&out.html Naidu, Som. “The Missing Link in Promoting Quality Education: Exploring the role of pedagogical design in promoting quality in teaching and learning.” Keynote address given at the 22nd World Conference of the International Council of Distance Education, Rio de Janeiro, Brazil, 2006. Orloff, Michael A. Inventive thinking through TRIZ: a practical guide. Berlin: Springer, 2003. Pashler, Harold and others. “Learning Styles: Concepts and Evidence.” Psychological Science in the Public Interest 9, no. 3 (2008): 103-119. Schultz, Duane P., and Sydney E. Schultz. A History of Modern Psychology. Belmont CA: Thompson Wadsworth, 2007. Slater, Michael D. “Persuasion Processes Across Receiver Goals and Message Genres.” Communication Theory 7, issue 2 (1997): 125-148. Suchman, Lucy A. Human-Machine Reconfigurations: Plans and situated actions. Cambridge: Cambridge University, 2006.

A COURSE IN VISUAL COMMUNICATION TONY PRITCHARD

Design for Visual Communication The Postgraduate Design for Visual Communication course at the London College of Communication sets out to demystify the theories and practices of visual communication. The course is for people converting career or preparing for Master’s study. Tim Molloy, Head of Creative Direction at the Science Museum, has described the initiative as a foundation course for postgraduate study (Molloy, 2009). In fulfilling its remit, the course introduces design principles, related theories and visual research methods to its students. The course attracts non-design students who have studied subjects as diverse as molecular genetics, microbiology, architecture, English and geography. “Eye” magazine said of the course participants: …their strength instead lies in their research capabilities and their capacity to think beyond pure aesthetics. They learn basic compositional and typographic skills within a short period [and] use design as a communicative tool – not as an end in itself (Hobson, 2007).

The Threshold Principles of Design The course introduces design principles such as visual language, typographic hierarchy, colour, structure and information design through practical ‘hands-on’ workshops. We do not use computer technology for these classes as the screen interface can lead to stilted and linear thinking. Instead, we find that by slowing the process down, the experience of learning is intensified. Students learn the significance inherent in the making of signs and symbols through the analysis of design components such as shape and colour, and principles such as scale, contrast and density. By analysing the component parts of visual communication in the abstract and considering them as compositional elements, students become aware of the many layers of potential meaning and interpretation. Colour alone has physiological properties as well as socio-economic, political,

Tony Pritchard

285

cultural and psychological dimensions. It may be used functionally or decoratively, but also conveys subliminal messages through personal association.

Theory with Practice Concepts such as semiotics, denotation, connotation, Gestalt theory, semantic and syntactic typography are introduced whilst exploring practical compositions. These concerns become most apparent when dealing with complexity as found in many information design projects. This is where there is potential for misunderstanding, and miscommunication can occur. The use of colour or shape is commonly used to denote meaning. These elements have their limitations. The use of five colours to differentiate meaning between groups of information is discernable by most people, however the distinctions in meaning become more difficult to decipher once 10 colours are employed for the same purpose. Shapes that have the same colour and characteristics appear to be visually grouped in terms of assumed meaning, however this may not have been the original intention.

Fig. 4-16: Grouping, 30.01.12, created by the author. Based on an exercise by Moritz Zwimpfer. The illustration demonstrates the principle that components that share the same characteristics appear to belong together despite being separated.

286

A Course in Visual Communication

The Workshops There are five key workshops exploring visual communication. Through the “Type Classification” workshop we discuss connotations relating to typeface selection. This can be the association we make to the political leanings of certain newspapers and their choice of typeface for their masthead. Blackletter is commonly employed to denote a sense of tradition and authority in this context. Blackletter might equally be used for the logos of heavy rock bands to denote more sinister and darker overtones. Blackletter can also be seen in the branding of Tequila from Mexico. The type classification may be the same but the semiotic reading is different. The “Colour” workshop engages students with the physiological effects of the interaction of particular colour combinations. Many students use colour and meaning as the focus of major projects, such as mapping colour and symbols around the world. Through the “Visual Language” workshop we consider how abstract shapes can convey concrete ideas and how this is translated into our visual environment. Through a series of set tasks during the “Typographic Hierarchy” session students explore the effects of restrictions on their creativity. They look at micro-typography from the letter to the word, the sentence and paragraph. They consider form and counterform as well as the space in and around the letterform. Subtle manipulations can enhance meaning and the effective transfer of information. The same exercise is explored over a range of formats and the means of gaining emphasis (position, weight and size) to test how the format and type specification alters perception of the information. The “Information Design” project brings together colour, structure, type, charts, tables and images. Having considered the components of design separately, the challenge is to bring each layer into a meaningful relationship to the information and through testing, minimise the potential for misunderstanding. In the following text I shall elaborate on the content and considerations given to the emerging themes within the design principle workshops.

Colour Colour can be described in terms of its physiological, psychological, political and socio-cultural significance. It is associated with various emotional states: green with envy, yellow with cowardice and blue with melancholy. Colour is a means of gaining attention and adding visual dynamism. It can be used to aid navigation through media. It can organise

Tony Pritchard

287

and categorise elements through use of a colour coding system. Colour can help unite nations as in the Olympic symbol and what it represents.

Colour and Culture In Western cultures white is a symbol of purity and associated with weddings. Black is linked to funerals and mourning but is also the colour many of us choose to be formal or stylish. Red represents good fortune in China and is the colour used at weddings. In Western cultures red is associated with danger but also with passion. In India red is a colour of purity. Orange is the colour for Halloween in the US and also the colour associated with the Irish Protestant faith. Yellow is a sacred and imperial colour in many Asian cultures. Green is the colour of Islam but it means a lack of fidelity in China. Blue is a holy colour in the Jewish faith, a sacred colour to Hindus for whom it represents Krishna, a colour of protection in the Middle East and a colour of immortality in China. Purple is a symbol of royalty in European cultures. The course attracts an international cohort. These ideas relating to colour are explored by students from a diverse cultural background, each culture providing a window to their customs and practices. By discussing these issues in an open forum, students become sensitive and informed practitioners.

Colour and Political Significance Political parties adopt colour as part of their identity. In many countries red is associated with left-wing politics and blue with right-wing politics. In 2004, orange took on a more chilling significance. Terrorists in Iraq dressed their victims in orange as a political statement against the Iraqis being held in Guantanamo Bay, who are also dressed in orange. A red cross on a white background is the English flag and the reverse of this, white on red, is the Swiss flag. It is the symbol of the International Red Cross. In Islamic countries the Crusader red cross will provoke other associations. Semiotic analysis enables students to understand how colour and its associated meaning is used in historic context and contemporary media communication and how this can be manipulated to form new meaning.

Shape Shapes have gained social, cultural and political significance. The dynamic primary shapes of circle, square and triangle give form to much

288

A Course in Visual Communication

of the physical world we see. They are the basis of the Roman alphabet. Shape can be a way of grouping and classifying information. Shape and colour are combined in a myriad of permutations to create the world’s flags. They are used to embody visual identities. Often, shapes act as a container for another element. Shapes relate to our human experience of orientation: vertical, horizontal, slanted, rotated and central. Certain shapes have become connected with deeper psychological associations we have as humans. Our planet, the sun and moon are spherical and this shape has a particular resonance with us. Squares are solid and stable. We use these associations when making compositions in the abstract to represent our concrete (real) world. The visual language workshop investigates shape as pure form. These workshops build more personal and conceptual work, such as the “Structure and the City” project. Students in this project establish a focused theme to explore such areas as the significance of colour or shape, in understanding the urban environment.

Line A line has been described as the shortest distance between two points. As simple as a line may seem, it can take many forms: straight, horizontal, vertical, diagonal, curved, continuous, broken, loops and spirals. Lines form shapes when linked to one another and make connections between other items. They can signify a direction, a boundary or a separation. Underlining a word gains emphasis for that word. Lines are used in the construction of diagrams and form the X and Y axes. Lines can be used to separate categories either vertically or horizontally. They are the fundamental components of a timeline; the lines of longitude and latitude on a map; and the construction lines of grids. Lines have various thickness and lengths. They can be dots, dashes and solid lines. Students explore each of these forms and design a book to document and demonstrate their intense investigation of the line as a graphic device and how it may be applied broadly across differing environments.

Squares and Rectangles The square is a fundamental structure implying stability and permanence. Much of our lives are contained within a rectilinear experience, for example buildings, roads and computer screens. The square can be a container of information and can describe a relationship with other elements. Information that is boxed out has been highlighted as having special significance. Rectangles are intrinsic units of construction, they

Tony Pritchard

289

can be placed side by side or one on top of each other forming rudimentary grids. A design begins on a rectangular piece of paper, which is further subdivided into smaller proportionate units of the rectangle. Aerial photographs of cultivated landscapes reveal a patchwork of rectangles. This mapping approach was utilised by Smart Money for their online Map of the Market. The size of the rectangles denotes market share. Colour additionally denotes rises and falls.

Circle The circle is constructed by establishing a centre-point around which another point rotates at a fixed radius. It starts and finishes at the same point. The circle implies movement in either direction. It suggests rotation around a centre. The circle defines the area that it contains and the area that surrounds it by its circumference. It is a symbol of completeness, of wholeness. Transcribed into three dimensions, the circle becomes a globe. Our earth, the moon and the sun are spherical. They all support our life and have significant meaning to us. Wheels rotate around a central axis. The circle is implied in, and transferred to, the many words ending in “centric”, such as egocentric and eccentric. Stone circles have cosmic or religious significance. Domes in religious buildings often represent the heavens. The circle in its smallest form is a dot, which is the basic unit of lithographic printing reproduction; it describes tone and colour. The circle can be a container; it encloses an area, sets a boundary and locates elements as internal or external. Circles can be applied as elements in a diagram, such as pie charts and Venn diagrams.

Triangle A triangle is formed from three sides meeting at three angles; as such the triangle can represent three aspects of something. The triangle is a stable shape and is used as a strengthening device in other constructions, such as girders. Triangular grids can be used for axonometric and isometric projection. Four triangles on a square base is a pyramid. The pyramids of Egypt are a powerful signifier of that country’s ancient history. The triangle is often used as an analogy for a hierarchical organisation.

Shape, Colour and Road signs The triangle, the square and the circle have been used in conjunction with colour to signify specific meanings for the categorisation of road signs. In

290

A Course in Visual Communication

1949, the UN World Conference on Road and Motor Transport in Geneva instigated a protocol to govern the actions of road users. Circular signs give orders. Those with a red outline are mostly prohibitive. Signs with solid blue circles mainly give positive instructions. Triangular signs with a red outline give warnings of potential hazards. Direction and information signs are mainly rectangular. In Britain, blue rectangular signs indicate information on a motorway and green indicates directions on a primary route.

Visual Language and Grammar Visual language is comprised of simple fundamental components such as dots, lines, circles, squares and triangles. Colour, texture and space are also basic elements of visual language. Visual language can be manipulated to express, represent and communicate concepts such as rhythm, speed, distance, movement, density, space, weight, force, impact, proximity and structure. Acquiring knowledge of visual language and grammar will not only be fundamental to a student’s practice, but also enable them to articulate through the spoken and written word the concepts behind their ideas.

Fig. 4-17: Density, 30.01.12, created by the author. This illustration is based on the map by Dr John Snow documenting the deaths from Cholera in Victorian London.

Tony Pritchard

291

Type Classification Typefaces are classified by their form and historical context. Over time, type styles have developed particular associations through usage and are adopted for the atmosphere they can create. Serif typefaces, for example, evoke associations with classicism and tradition. These ideas are discussed with students and they are asked to investigate and express visually the ‘personalities’ of six contrasting typefaces.

Fig. 4-18: The T-ness of T, 30.01.12, created by the author. Students are encouraged to express the quality of the typeface and letterform.

Type Hierarchy The Oxford English Dictionary defines hierarchy as a ranking system, ordered according to status or authority. Hierarchy is a value system and is dependent on who is making the judgment and what the criteria is. The creation of a hierarchy within information is a fundamental method of organising and imparting data. It is the “H” within Richard Saul Wurman’s “LATCH” theory. Typography has a dual purpose, on the one hand it attracts attention through the impact of its dynamic form, on the other it must impart critical information with clarity. Student designers analyse the text and investigate the means of articulating the information, assigning relative importance through typographic techniques. The intention of performing such a task is to encourage people to read information in a preconceived order. Emphasis is given, through contrast, on: size, weight,

292

A Course in Visual Communication

position and colour of type. The use of capitals, small capitals and italics are often used to denote specific meaning. The use of typographic devices such as rules or reversing type white out of a solid colour can also aid the notion of hierarchy. The ability to change the size, weight, colour and percentage tint of a typeface allows designers to create implied depth. The overlapping of type implies that one thing is in front of another and denotes relative importance, i.e. the top layer exerts dominance over the bottom layer. The important concept to be grasped is that a hierarchy is gained through contrast. Slight shifts in contrast are less dramatic than greater shifts in emphasis, for example the difference between 9-point and 10-point type will be less detectable to the human eye than 9-point and 18point. Changes in weight should also be distinct.

Fig. 4-19: Typographic hierarchy exercise, 30.01.12, created by the author. Students are encouraged to make visual and semantic alignments.

Grid Structures The notion of structure implies the ordering of elements into a coordinated whole. Graphic design has adopted “the grid” as a method by which components of a design are brought into a formal relationship to one another. The grid is particularly associated with the work of pioneering modernist designers such as Jan Tschichold, Josef Müller-Brockmann and Wim Crouwel. Grids are apparent in both man-made and natural structures. Georgian architecture exploited a modular approach to the relationship between, and proportion of, windows and doors. The classic

Tony Pritchard

293

Georgian window has a unit structure of three horizontal panes of glass by four vertical panes. A pinecone or sunflower head is comprised of units making up its overall structure. How things are arranged, grouped and ordered can influence the way they are perceived and read, and as a consequence is often an aid to understanding. The student designer needs to evaluate which layout best supports the information to be communicated. The grid is seldom visible; however, the way in which structure is employed can have a dramatic effect on the appearance of the overall design composition. Design academic Ray Roberts has suggested that grids act as “metaphors for the human need to make sense of the world and to position ourselves in control of it” (Roberts, 2000).

Information Design Information design is concerned with explaining complexity through visual means to enable understanding. Information is comprised of components known as data; these components are things like words, numbers, statistics and facts. Data becomes information when it is placed in a context that has meaning to an audience or user. Students investigate the ways in which information is presented, such as maps and timelines. They use the design principles learnt through the workshops to manipulate form and colour to mediate messages.

Semiotics and Visual Communication This conference brought together speakers from across the globe. Despite the subtle nuances inflected from our various backgrounds, the exchange of views seemed more to unite the potential for our broader understanding of a shared visual language in the 21st century.

Bibliography Books Crow, David. Visible Signs. (UK: AVA Academia, 2007). Evamy, Michael. World Without Words. (UK: Laurence King, 2003). Hofmann, Armin. Graphic Design Manual. (Switzerland: Arthur Niggli, 1965). Jacobson, Robert. Information Design. (US: The MIT Press, 1999). Leborg, Christian. Visual Grammar. (Princeton Architectural Press, 2006).

294

A Course in Visual Communication

Noble, Ian and Bestley, Russell. Visual Research. (UK: AVA Academia, 2011). Wurman, Richard. Information Anxiety 2. (QUE, 2000).

Journals Hobson, Jamie. Another way to draw the line. (Eye Magazine. Issue 65, 2007).

Lecture Roberts, Raymond and Roberts, Lucie. Two Generations’ Perspectives. (UK: ISTD/LCP, 2000).

Meeting Molloy, Tim. LCC Academic Consultation Meeting with External Participants. (UK: LCC, 2009).

CHILDREN ARE PAINTING INSCRIPTIONS: PEDAGOGY OF VISUAL COMMUNICATION IN LOCAL HISTORY EVANGELIA SVIROU, IFIGENEIA VAMVAKIDOU AND PARASKEVI GOLIA

This paper is the outcome of a didactic proposal for children (6-10 years old) in order to learn about local history. The historical information concerns life history and the products of the painter Elias Byzantis (19101980), who designed many nameplates in the city of Florina. The researching issue refers a) to the local, financial history as it appears in the specific nameplates and b) to the didactic proposal for teaching local history through visual documents. It concerns a workshop in which we took the role of a curriculum designer1 as we developed a blueprint for revising our existing curriculum along interdisciplinary lines. For the collection of these inscriptions we used the historical methodology, which is the local research in public and private archives. Once the decision is made to conduct historical research, there are steps that should be followed to achieve a reliable result. Charles Busha and Stephen Harter2 detail six steps for conducting historical research: the recognition of a historical problem or the identification of a need for certain historical knowledge, the gathering of as much relevant information about the problem or topic as possible, the forming of hypotheses that tentatively explain relationships between historical factors, the rigorous collection and organisation of evidence, the verification of the authenticity and veracity of information and its sources, the selection, organisation and analysis of the most pertinent collected evidence, and the recording of conclusions in 1

Evaggelia Svirou, Elias Byzantis (1910-1980). Shop signs in Florina, a contribution to the local history (University of Western Macedonia, Faculty of Pedagogic: Florina, 2010), unpublished thesis. 2 Charles H. Busha and Stephen P. Harter, Research Methods in Librarianship: techniques and Interpretations (Academic Press: New York, 1980), 91.

296

Children Are Painting Inscriptions

a meaningful narrative. The lessons took place in February-March of 2011 in Florina at the 5th primary school. The total number of children was 108, from six to 12 years old. All the children that participated came from the 1st class (8 boys, 10 girls), the 2nd class (7 boys, 6 girls), the 3rd class (12 boys, 11 girls), the 4th class (7 boys, 15 girls), the 5th class (8 boys, 5 girls) and from the 6th class (10 boys, 9 girls). We focused on the skills and thinking processes, asking the students to see and analyse the painter’s productions according to the specific genre of the inscription. The students designed their own inscriptions in order to advertise a store. Thus, we were interested in the assessments and products that demonstrate skills and thinking processes, such as essays, productions, recitals, projects, notetaking and in-class participation. Particularly, students were asked to participate in the historical research for the life story of Elias Byzantis using their own ‘gaze’ at the documents we had already collected (family history, maps of the region, newspapers, interviews, the art crafts). They were also asked to “see and read” the sign posts by answering specific questions about the material, the shape, the signs, the typography, the colour, the advertising product. The aim of this presentation was to ‘read’ students’ products using social semiotics in the field of visual communication.

Introduction Before children are able to read and write, they do not know the difference between a line drawing and a letter. When an adult writes an “A”, it is simply another drawing. It is a picture, different than a face or a house, but it is still just another image drawn with a coloured pencil on white paper. Soon, children learn that combinations of these letter-pictures mean more complicated things. When the drawings “A-P-P-L-E” are combined, they form another picture, which we learn stands for the name of the fruit. Now the letter-pictures become word-pictures that can spark other images in our minds of the things they stand for. We further learn that these wordpictures can be combined with other word-pictures to form sentencepictures. To a child, there is no difference between words and pictures – they are one and the same. Thus, visual messages with their own rules of syntax are being read, but this language means nothing to those who can only read words. The wall space and signs in many cities are often coated with multicoloured spray painted messages. Termed vandalism, graffiti or tagging depending on the speaker; these visual messages are actually a complex written form of communication. Graffiti messages may mean the mark of a territorial

Evangelia Svirou, Ifigeneia Vamvakidou and Paraskevi Golia

297

border by a gang, a plea for understanding and hope for the future, grief for a killed loved one, anger towards an enemy, a show of playfulness and humour as part of a national fad, an act of criminal vandalism or simply an individual expression that signals the writer's existence. As with any symbolic communicative system, if you do not know the language, you will have trouble deciphering the message. Educational psychologist Jerome Bruner3 claimed that persons only remember 10% of what they hear, 30% of what they read, but about 80% of what they see and do. Words and pictures can become one, powerful and memorable mode of communication. Interest in local history has boomed since the Second World War. Local history societies have sprung up everywhere like dragon's teeth, Women's Institutes and Mothers’ Union branches contribute to the subject, schoolchildren do projects on it, evening classes and public libraries seethe and bubble with seekers after local truth. In his classic book on local history in England, which came out originally in 1959, W.G. Hoskins attributed the flowering of interest in the subject to the complexity, pace and fundamental incomprehensibility of modern life4. In a world growing daily more impossible to understand and control, people find comfort and a sense of identity in the study of something close at hand, small in scale and of direct personal relevance. The same need, which drives some to strange religions and new cults, sends others to the nearest archivist. Christopher Charlton, who teaches local history among other things in the adult education department of Nottingham University, finds that the factor which brings more people to the classes than any other is today's social mobility. “People move to a new area”, he says, “feel rootless there and want to find out about its history as a way of establishing a place for them”. The recent mushrooming of family history is part of the same phenomenon and many people who start to trace their forebears find their interest extending to the locality as well5. Local history is the subject and the methodology of micro-history and refers to social history, which finds its interest in groups ‘different’ from 3

Jerome Bruner, Actual Minds, Possible Worlds (Cambridge: Harvard University Press, 1986). Jerome Bruner, In Search of Mind (New York: Harper & Row, 1983). Jerome Bruner, ȉhe Process of Education (New York: Vintage Books, 1960). 4 Roger Richardson, The Study of History: A Bibliographical Guide (Manchester University Press, 2000): 140. 5 Department of Adult Education and the ESRC Cambridge Group for the History of Population and Social Structure, Local Population Studies (Nottingham: Nottingham University, 1994).

Children Are Painting Inscriptions

298

the majority, focusing on the experiential aspects of the everyday life. Main axes for the determination of LH are: • • • •

the events and individuals the proximity in space and time (synchronous) the participation of the subjects (indigenous and others) in the historic events the changes, either short-term or long-term, which are happening in the specific host or origin place6.

The process of learning and understanding the background and growth of a chosen field of study or profession can offer insight into organisational culture, current trends and future possibilities. The historical method of research applies to all fields of study because it encompasses their: origins, growth, theories, personalities, crisis, etc. Both quantitative and qualitative variables can be used in the collection of historical information. There is a variety of places to obtain historical information from. Primary sources are the most sought after in historical research. Primary resources are firsthand accounts of information. “Finding and assessing primary historical data is an exercise in detective work. It involves logic, intuition, persistence, and common sense”7. 6

Spyros Asdrachas, Historical Research and historical education Realities and Perspectives Athens: Mnimon, 1982) (in Greek). Spyros Asonitis, “Historical documentation by narrative sources. A slippery route”, Tekmirion, 3, (2001): 1139. Maria Vaina, Theoretical framework of the teaching of local history in the twentieth century (Athens: Gutenberg, 1997). Alain-Jean-Marie Bernard, Geneviève Joutard and Jean-Pierre Rioux, A la recherche du temps présent: Histoire orale et enseignement (Amiens: CRDP, 1987). Fernard Braudel, Mediterranean and the Mediterranean world at the time of Philip II of Spain, Volume A: The role of the surroundings (Athens: M.I.E.T., 1993) (in Greek). Fanourios Voros, “Local History”, Journal Ekpedaiftika, 20, (1990): 33-41. Giannis Giannopoulos, “The teaching of history”, Journal Nea Paideia, 42, (1987): 166-172. Elli Giotopoulou – Sisilianou, The teaching of history in secondary education (Athens: Private Edition, 1965) (in Greek). Alain Croix and Guyvarc'h Didier, Guide de l’histoire locale (Paris: Seuil, 1990). Francois Dosse, History in crumbs. From the Annals to the New History (Heraklion, Crete: University Editions, 1993). Robert Douche, Local History and the Teacher (London: Redwood Press, 1972). David Dymond, Writing Local History. A Practical Guide (Sussex: Phillmore, 1988). 7 Gaye Tuchman, Historical social science: Methodologies, methods, and meanings. Handbook of qualitative research. In Handbook of qualitative research (Thousand Oaks, CA, US: Sage Publications, Inc., 1994), 252.

Evangelia Svirou, Ifigeneia Vamvakidou and Paraskevi Golia

299

Some examples of primary documents are: personal diaries, eyewitness accounts of events and oral histories. Secondary sources of information are records or accounts prepared by someone other than the person, or persons, who participated in or observed an event. Secondary resources can be very useful in giving a researcher a grasp on a subject and may provide extensive bibliographic information for delving further into a research topic. In any type of historical research, there are issues to consider.

The local research This paper is the outcome of a didactic proposal for children (6-12 years old) in order to learn about local history. The historical information concerns life history and the products of the painter Elias Byzantis (19101980), who designed many nameplates in the city of Florina. The researching issue refers: a) To the local, financial history as it appears in the specific nameplates b) To the didactic proposal for teaching local history through visual documents. It concerns a workshop in which we took the role of a curriculum designer as we developed a blueprint for revising our existing curriculum along interdisciplinary lines8. For the collection of these inscriptions we used the historical methodology, which is the local research in public and private archives. Once the decision is made to conduct historical research, there are steps that should be followed to achieve a reliable result. Charles Busha and Stephen Harter9 detail six steps for conducting historical research: • • • • • 8

the recognition of a historical problem the gathering of as much relevant information the forming of hypothesis the rigorous collection and organisation of evidence the verification of the authenticity and veracity of information and its sources

Evaggelia Svirou, Elias Byzantis (1910-1980) Shop signs in Florina, a contribution to the local history (University of Western Macedonia, Faculty of Pedagogic: Florina, 2010), unpublished thesis. 9 Charles H. Busha and Stephen P. Harter, Research Methods in Librarianship: techniques and Interpretations (Academic Press: New York, 1980), 91.

Children Are Painting Inscriptions

300

• •

the analysis of the most pertinent collected evidence the recording of conclusions in a meaningful narrative.

The data Elias Vyzantis, sign-inscription-maker and painter, as a representative of a group of craftsmen and artists who had co-operated with the merchants of the period 1950-1970. He was born in Istanbul in 1910. In 1922, after the events of Asia Minor, the Vyzantis family left Turkey and was installed in Florina. He died in December 1980. His work consists of two parts: sign-making and painting. He used three types of writings, which are still modern in visual, graphic communication: • • •

the upright and italic flat the upright and italic prismatic the unusual with the non-equal width.

Vyzantis’ nameplates were all destroyed during the 70s when the plexiglas and the neon signs with the already-made letters entered the sign-making market of Florina.

Fig. 4-20: A Vyzantis sign10

10

Archive of researchers.

Evangelia Svirou, Ifigeneia Vamvakidou and Paraskevi Golia

301

Elias Vyzantis (1910-1980) The aim of this study is to bring forth the ‘minor’ visual works-signs in the field of the local history ‘from below’ and to emphasise, through Vyzantis’ signs on the everyday economic life, the stores in which they were used. This material is an element of the material and local culture of the people of Florina of the specific period. The identification and the classification of the nameplates constitute the on-site research in the city of Florina. The municipality of Florina is at the northwest part of Greece and borders the municipalities of Prespes, Kastoria, Amyntaio, Edessa and Almopia. It is located at an altitude of 680 metres and has a population of 22.000 (January 2011). Its name, according to some versions, is associated with the rich vegetation of the area, the Flora, the view of vegetation for the ancient Romans and the term which today is translated as flora. Until now, our material consists of 40 signs, which were gathered with research in stores, local archives, newspaper archives and families. After the signs were detected, they were classified and analysed based of Kress’ model. This paper is the result of an educational-didactic proposal for children (6-10 years of age), in order to learn the local history. The historical information concerns the life, the history and the products-works of the painter Elias Vyzantis (1910-1980), who painted many commercial signs in the city of Florina.

Fig. 4-21: A children's sign-The doctors11.

11

Archive of researchers.

302

Children Are Painting Inscriptions

Methodology The first tenet of the syntactic theory of visual communication is “mediated words and pictures have equal importance in the communication process”. Words and pictures are both collections of symbolic images. Words are signs composed of lines, curves and open and closed shapes. Words, as with pictures, can be presented in a variety of colours, forms, depths and movements. Words have their historical roots as images and are still thought to be works of art by typographical designers and calligraphers. The second tenet of the syntactic theory of visual communication is “as symbols with similar historical roots, mediated words and pictures are both symbolic representations”. Before Helen Keller, who was blind and deaf, learned to connect words with the objects she touched in her environment, her mind was filled with nonvisual, non-verbal emotions. The third tenet of the syntactic theory of visual communication is “images are remembered by thinking about them in words”. Whether pictures are not a language because there is no easily definable and reproducible alphabet or because the elements that make up a picture do not follow a discursive, linear flow, most experts agree that images are a collection of signs and as such, become a language when read in the mind. When words and images have equal status within all media of communication, the cultural cues that define a society will not only be more efficiently passed from one generation to the next, but within this generation, here and now, diverse cultures will be able to understand each other a little better12.

The lesson-plan The lesson plans took place during February and March 2011 in the 5th elementary school of Florina. The total number of children was 108 and aged six to 12 years old. All the children that participated were students of the 1st class (8 boys, 10 girls), the 2nd (7 boys, 6 girls), the 3rd (12 boys, 11 girls), the 4th (7 boys, 15 girls), the 5th (8 boys, 5 girls) and the 6th class (10 boys, 9 girls). Didactic interventions were made in each class during ‘flexible zone’, since this was how we could connect in an interdisciplinary way the different fields (history, art lessons, mathematics, environmental lessons). 12 Martin Paul Lester, Visual Communication. Images with Messages. 4th Edition (Belmont, CA: Wadsowrth Publishing Company, 2006).

Evangelia Svirou, Ifigeneia Vamvakidou and Paraskevi Golia

303

We emphasise on: • •

the skills the thought processes.

We asked students to see and analyse the works of the painter according to the particular type of nameplates. Students designed their own nameplates in order to advertise a shop. Thus, we were interested in the assessments and the products that they demonstrated, the skills and processes of thought, such as essays, productions, recital thinking, designs, notes and class participation.

Semiotic analysis For this purpose, following Gunther Kress’ and Theo van Leeuwen’s13 model for the semiotic reading of the children’s visual paintings, we apply and locate the following categories: sequence number, gender, class, thematic, capital and small letters, name, name of owner, type of writing, real name or pseudonym, shapes, symbols, material, text, colour of the painting, type of the sign, (portrait, multifacial, synthesis, landscape, other), description of the sign, telephone number (mobile or landline). We consider these nameplates as syntagmatic forms based on spatial relationships.

Signs’ analysis-results Kress and Van Leeuven14 argue that images are formed in textual synthesis in different ways and thus structure the semiotic reality. According to the grammar of visual design, we will describe the way in which the pictured elements (symbols, shapes etc.) are combined into visual denotations.

Shape All the signs made by the children (1st to 6th grade) are rectangular, because that is exactly the shape that Elias Vyzantis gave to his signs. Rectangular forms shape the elements of the construction, elements that 13

Gunther Kress and Theo Van Leeuwen, Reading Images: Grammar of Visual Design (London: Rutledge, 1996). 14 Gunther Kress and Theo Van Leeuwen, Reading Images: Grammar of Visual Design (London: Rutledge, 1996), 13, 41.

304

Children Are Painting Inscriptions

we shape the world with. Moreover, on a rectangular sign more elements can be portrayed15.

Visual Message The thematic every sign has, varies. Hence, we have flower shops, hairdressers, butcher shops, jewellery stores, bookstores, cafes, taverns, pizzerias, shops-shine, confectionery shops, coffee shops, baking room, café, creperies, police stations (boys whose dad is a policeman), grocery stores, toy stores. We have a few cases of surgery offices and dental clinics while only the children of the 6th grade constructed signs for Internet cafes, since their activity with the computer is more frequent at this age. Finally, we found only one occurrence of a pet shop, a skate shop, a wood shop, a stadium, a car engineer, a glove shop, a fencing club, a truck spare parts store, a sanitary appliances store, a sports journalist office, a pharmacy and a playground.

Typographic signifier Children used capital and small letters, depending on what they wanted to highlight: capital letters if they wanted to point out the name of the store or its thematic or even their name.

Title giving In terms of the name giving, children from the 1st grade did not use original names as other grades (mainly the older children) did. Not all the children gave names to their stores. In the 5th grade, from a total of 14 children, only three gave names to their stores, while in the 6th grade, from a total of 20, only three did not give a name16.

Positioning The position of the owner’s name on the signs varied. The names were positioned: up middle-right-left, in the corner down right-left, down left, 15

Gunther Kress and Theo Van Leeuwen, Reading Images: Grammar of Visual Design (London: Rutledge, 1996), 113. 16 Gunther Kress and Theo Van Leeuwen, Reading Images: Grammar of Visual Design (London: Rutledge, 1996), 186.

Evangelia Svirou, Ifigeneia Vamvakidou and Paraskevi Golia

305

down middle, side right. Therefore, according to how important they considered their name to be, they positioned it in the proper place on the sign. In general, anything considered important is positioned at the top, while what is considered to be less important goes at the bottom17. The horizontal and vertical axes are not neutral dimensions of pictorial representation. Since writing and reading in European cultures proceed primarily along a horizontal axis from left to right (as in English but unlike, for instance, Arabic, Hebrew and Chinese), the 'default' for reading a picture within such reading/writing cultures (unless attention is diverted by some salient features) is likely to be generally in the same direction.

Direction In general, the way children write is horizontal with a few exceptions. A girl from the 2nd grade wrote the name of the store in a semicircular way, while another girl form the 4th grade wrote her name diagonally and in the centre. A variety in directions appeared in the 6th grade, italic, vertical, horizontal, semicircular18.

Human signifiers Forms (men, players, policemen), woman or a woman’s head, a little girl, were used by the children of the 1st, 3rd, 4th grades and one boy form the 6th, for signs made for a tavern, a butcher's shop, a liquor store, a police station, a sports journalist office, a bar, a hairdresser’s salon and in which only female forms are used since they are combined with the female sex.

Symbols Many children from different grades used symbols always related with the product or the services that their store offered, for example shoes and dresses for shoe stores and clothing stores, plates for taverns, pizzas for pizzeria, cakes for patisserie, flowers for a florist and jewels for jewellery store. Nevertheless, there were symbols used only for decorative purposes, as flowers, stars, the sun, a tree etc.

17

Gunther Kress and Theo Van Leeuwen, Reading Images: Grammar of Visual Design (London: Rutledge, 1996), 117. 18 Gunther Kress and Theo Van Leeuwen, Reading Images: Grammar of Visual Design (London: Rutledge, 1996), 105- 109.

306

Children Are Painting Inscriptions

Materials used for Designing The materials used generally by the children were mostly markers, then crayons, one student from the 5th grade used pencil and three students from the 6th grade used pens.

Colour All the signs were multicoloured (very few monochromes). The children used all the colours (red, yellow, green, purple, orange, blue, light blue, pink, brown, black and grey). The colour is used in order to give and to express meanings-situation. The children used the right colours, i.e. when they painted a flower shop they used colours that we see on flowers, lively and bright colours for clothing and shoe shops while red was used for surgeries. The cross of the pharmacy was painted red instead of green by one student, probably combining the medicines with the hospital.

Genre We found landscapes, signs depicting many people at the same time, one portrait by a student in the 4th grade, while most of the signs were syntheses, associating the representational meanings-symbols with what was written on the signs.

Advertising Just one sign had a phone number while many students used a landline number together with a mobile number, or nothing.

Linguistic Material Of particular interest are some signs on which there was a text message. This is something that does not exist on the signs of the artist. For example: “I sell clothes, wood, 1+1 gift, flower shop Labraki 31 we have roses if you buy 50% discount, everything you ask for we have it, flowers are life, read about life”, while some of the signs mentioned the address of the store. It is worth noticing the fact that in the 5th and 6th grades, children used words in English while the younger students did not.

Evangelia Svirou, Ifigeneia Vamvakidou and Paraskevi Golia

307

Fig. 4-22 - A children's sign–VOLT, a sport shop19.

Discussion The semiotic classification and analysis helped us to locate the children’s representations for the specific inscriptions of Vyzantis. Every child represents the knowledge of the world, constructs conceptual representations using signals and symbols. These representations are developed and converted according to cognitive development. According to Bruner20, the school-age child represents the world in pictorial form and much later in symbolic representation. In this research, children read and reproduce the material which concerns the exterior/real representations, but also the conceptual representations with an accent in symbolic. In the sample, the majority of children drew at the level of mental and optical realism21.

Bibliography Asdrachas, Spyros. Historical Research and historical education Realities and Perspectives. Athens: Mnimon, 1982 (in Greek). Asonitis, Spyros. “Historical documentation by narrative sources. A slippery route”. Tekmirion, 3, (2001): 11-39. Bernard, Alain-Jean-Marie, Geneviève Joutard, and Jean-Pierre Rioux. A la recherche du temps présent: Histoire orale et enseignement. Amiens: CRDP, 1987. Braudel, Fernard. Mediterranean and the Mediterranean world at the time of Philip II of Spain, Volume A: The role of the surroundings. Athens: M.I.E.T., 1993. (in Greek). 19

Archive of researchers. Jerome Bruner, On Knowing (Cambridge: Belknap Press, 1962). 21 Paraskevi Golia, Ifigenia Vamvakidou and Evdoxia Traianou, “History and Semiotics: Children Are Drawing “Homeland”, The International Journal of Learning, 16, no 2 (2009): 321-332. 20

308

Children Are Painting Inscriptions

Bruner, Jerome. Actual Minds, Possible Worlds. Cambridge: Harvard University Press, 1986. —. In Search of Mind. New York: Harper & Row, 1983. —. On Knowing. Cambridge: Belknap Press, 1962. —. ȉhe Process of Education. New York: Vintage Books, 1960. Busha, Charles H., and Stephen P. Harter. Research Methods in Librarianship: techniques and Interpretations. Academic Press: New York, 1980. Croix, Alain and Didier, Guyvarc'h. Guide de l’histoire locale. Paris: Seuil, 1990. Department of Adult Education and the ESRC Cambridge Group for the History of Population and Social Structure. Local Population Studies. Nottingham: Nottingham University, 1994. Dosse, Francois. History in crumbs. From the Annals to the New History. Heraklion, Crete: University Editions, 1993. Douche, Robert. Local History and the Teacher. London: Redwood Press, 1972. Dymond, David. Writing Local History. A Practical Guide. Sussex: Phillmore, 1988. Giannopoulos, Giannis. “The teaching of history”. Journal Nea Paideia, 42, (1987): 166-172. Giotopoulou – Sisilianou, Elli. The teaching of history in secondary education. Athens: Private Edition, 1965 (in Greek). Golia, Paraskevi, Ifigenia, Vamvakidou, and Evdoxia, Traianou. “History and Semiotics: Children Are Drawing “Homeland”. The International Journal of Learning, 16, no 2 (2009): 321-332. Kress, Gunther, and Theo Van Leeuwen. Reading Images: Grammar of Visual Design. London: Rutledge, 1996. Richardson, Roger. The Study of History: A Bibliographical Guide. Manchester University Press, 2000. Svirou, Evaggelia. Elias Byzantis (1910-1980) Shop signs in Florina, a contribution to the local history. University of Western Macedonia, Faculty of Pedagogic: Florina, 2010, unpublished thesis. Tuchman, Gaye. Historical social science: Methodologies, methods, and meanings. Handbook of qualitative research. Thousand Oaks, CA, US: Sage Publications, Inc., 1994. Vaina, Maria. Theoretical framework of the teaching of local history in the twentieth century. Athens: Gutenberg, 1997. Voros, Fanourios. “Local History”, Journal Ekpedaiftika, 20, (1990): 33-41.



CHAPTER FIVE: VISUAL ARTS





MARKS, SIGNS AND IMAGES: THE SENSE OF BELONGING AND COMMITMENT WHICH PRE-DATES HISTORY BUT HAS NOW BECOME A POWERFUL GLOBAL LANGUAGE PAUL MIDDLETON

Marks, signs and images that embrace a sense of belonging and commitment predate recorded history. Primitive marks identified groups of people long before a written language emerged, and these marks changed to signs that could communicate with more speed and simplicity through pared-down juxtaposed images. This provided a more efficient form of communication than a complex series of words. In a world where literacy was the preserve of the elite few, visual communication assumed great significance. Images were capable of communicating various messages and concepts through colour, shape and form, enabling access to a wide population regardless of levels of literacy or language origin. These early manifestations of visual language provided a form of communication that was both democratic and accessible. Later, as our preference for the written word became more developed, sophisticated and available to all, such visual language evolved to take a different path. This now forms a uniquely powerful component of modern communication in a globalised society.

The emergence of the primitive mark Primitive peoples used their cave paintings and drawings to communicate and record 35-40,000 years ago. Around 1,000 years before the first emergent writing system, primitive peoples communicated with signs and marks drawn and painted on the walls of their cave dwellings and it is thought this was the birth of a vital system of non-verbal communication. About 1,000 or more years later, the Sumerians (in what is now southern Iraq) began to use symbols constructed from triangular forms stamped into clay as a means of recording transactions, and so began the idea of using simple symbols for the pictorial stage of early written



Paul Middleton

311

communication. This was known as “Cuneiform”. It was another 1,000 years before the famous picture-writing of the ancient Egyptians emerged. The decoding of the Rosetta stone in 1822 aided the understanding that this was not only ‘pictographic’ (i.e. the symbols were literal representations) but also ‘logographic’ – there were signs for sounds made by the words. Importantly, there was also a set of signs for ‘phonetic’ values, in effect – an early alphabet. This was the beginning of various writing systems which are still used in much of the world. In the East, however, things developed differently and the Chinese, Japanese and Vietnamese for example, communicate with ‘ideograms’ or ‘ideographs’ – where each sign communicates a concept. A small proportion is representations of ‘things’, in which case they can be termed ‘pictograms’. As there are many thousands of these, we, in the West, are left wondering how people could possibly learn the meanings of so many symbols. However, the emergence of writing is not the subject of this paper but to look at the other signs and symbols which communicated with such simplicity and economy, and continue to do so today. We might surprise ourselves just how sophisticated a visual language we have developed is, without realising this was happening. We now communicate through signs in ways we take for granted – and normal life would just be unthinkable without them.

A powerful and emotive visual language By showing an audience various symbol images, one can convey concepts, aid recognition and even evoke emotional reactions, making it clear that there now exists a visual language people respond to and comprehend. The images carry a meaning that viewers process based on their previous learning. Here are some examples: x BBC2 number 2 logo – no longer merely a bold red numeral – but instantly a mould-breaking British TV channel. x Perhaps not everyone in the West will be familiar with the Hindu or Buddhist symbol of good luck – a swastika – but we all know what the Nazi swastika means. x We also understand the meaning of the Star of David to Jewish people, but then comprehend a very different meaning when applied to a yellow armband to label a population. x The Red Cross and Red Crescent are symbols of hope all over the world, in conflict and famine.



312

Marks, Signs and Images

x All over the world, the red poppy has been recognised as a symbol of remembrance of those killed in conflict and the more emotive, white poppy, introduced in 1933, is a symbol of peace without violence. Together, they are powerful symbols. x A green rectangle will not be significant to most people – to Libyan people it is a symbol of 42 years of a ruthless regime – and by contrast, the new tricoloured flag is the symbol of victory and freedom for most Libyans. x From a branding expert/designer’s point of view, it is very important to convey the right message with your visual language. The traditional way in England, going back centuries, is the Coat of Arms which incorporates a system of symbols and meaning. However, Catherine Middleton, Duchess of Cambridge might be ‘one of us’ but this (graffiti tag) would be the wrong message.

Fig. 5-1: Tag created in The Graffiti Creator application (www.graffiticreator.net)1

Fig. 5-2 & Fig. 5-3: Catherine Middleton’s (Duchess of Cambridge) coat of arms, 2011. (http://www.bbc.co.uk/news/uk-13127145)2

1

Graffiti creator, http://www.graffiticreator.net BBC NEWS UK. 2011. “Royal wedding: Kate Middleton coat of arms unveiled “ http://www.bbc.co.uk/news/uk-13127145. 2



Paul Middleton

313

The impact of marks, signs and logos within societies and cultures Various alphabets and writing systems evolved throughout the world to give form to thoughts and ideologies, largely to spread political and religious ideas and win support. However, literacy and therefore access to the written word was not widespread until comparatively recently. Images and symbols were used to communicate information to the people at large and sometimes to control them. They had various functions, as can be seen in the previous examples. The concept of branding has been around for about 5,000 years. In the same way that writing began to record financial transactions, the word ‘brand’ came from the practice of indicating ownership of livestock through scorching their hides with a hot iron. Owners had their own design of branding symbol and thus they could differentiate between similarlooking animals. The function here was of Ownership. In Europe, kings, nobles and powerful families were identified by their heraldic marks, evident on their wax seals, embroidered onto pennants, carved versions adorning their homes, an early form of ‘corporate identity’ aiding recognition of the monarch, ruler or perhaps, invading force. In fact, these were often a form of monogram, with symbols to indicate clan, status and achievement – perhaps with words added. Colours were also vitally important. These marks had great impact on societies and cultures, were feared or respected and won loyalty in the way that football insignia do today. The keyword here is Allegiance. However, superstition, religion, politics and magic were the main users of symbols to represent faith, even giving them magical powers. Primitive people believed that symbols could protect them from perceived dark forces and many symbols were believed to have magical powers. Tribal people would tattoo their women with symbols believed to protect them from evil. Even in the 20th century, Berber women were adorned with symbolic facial tattoos. Belief in the evil eye is found in Islamic doctrine, based upon a verse of the Qur'an. The ‘evil eye’ symbol is worn all over the Middle East to protect the wearer, usually incorporated into jewellery. The function here is Protection. Religious symbols such as the Christian cross, Muslim crescent, Jewish Star of David represent the strongly-held beliefs of millions and have developed powers accordingly. Many people wear jewellery and create ornaments depicting their religious creed and attach great value to them. The right to display such symbols has caused much controversy and debate. The keyword here is Belief.



314

Marks, Signs and Images

Symbols have also had more prosaic uses. In the ruins of the ancient city of Ephesus, a graffiti sign in the shape of a handprint resembling a heart survives with a footprint to direct the clients to a nearby brothel, an early commercial use for symbols rather than words. The function here is Information (or perhaps Promotion?). Towards the end of the 18th century, the industrial age was beginning and ‘branding’ in a commercial context really began. Mass-production was made possible by the introduction of steam and oil power and the concept of marketing and advertising goods to consumers was entering western society. By the 19th century, printers were in demand for such ephemera as posters, leaflets and advertisements. Advertisers looked for eye-catching ways to communicate company values to potential customers through visual representations, associating emotional responses and ‘feelings’ with the product, a notion that has remained to this day. The keyword here is Promotion. Today, we are surrounded with powerful images and their hidden messages – advertisers will constantly make efforts to find new ways to gain attention aided by trained graphic designers rather than fine artists such as Millais, whose ‘Bubbles’ painting was used (without his permission) for the famous Pears Soap advertisement in the late 1880s. This could be said to be early ‘commercial art’ and the idea was certainly emulated thereafter3. Nowadays, all of these functions – ownership, allegiance, protection, belief, information, promotion – have become modern ‘brand values’. As the industrial age progressed and global markets opened up, it became even more important to communicate such ‘brand values’ to a wider market – who, though more likely to be literate by now, may not share the same spoken or written language. Effective branding has to encompass most or maybe all of these functions. Branding began as a part of the promotion of products rather than the companies themselves – packaging needed to communicate a clear (if limited) message to attract customers and a logo was a key part of that. Note: this premise is not confined to companies. Not-for-profit organisations, such as charities, also need to follow the same model in order to get their message across for a successful outcome. It is important to get the terminology right, as many words are misused:

3 National Museums Liverpool. “Artwork highlights - Bubbles, by Sir John Everett Millais”. http://www.liverpoolmuseums.org.uk/picture-of-month/display picture.asp?venue=7&id=299.



Paul Middleton

315

x Logo – short for ‘logotype’, which is a custom-lettered word used for a trademark (‘logos’ is Greek for ‘word’ as many people here will know). Examples: Halfords, Boots. x A monogram is letters – perhaps initials – but not a word, for example: IBM or BBC. x A symbol – is a sign without letters, for example Nike’s ‘swoosh’. x A trademark is any symbol, monogram, logo or mark which is legally registered as a representation of a company or product. Trademarks created recognition and confidence, enabling legal protection from competitors’ imitation. x So, all of these represent a brand, but it is important to remember they are not the brand itself. A brand is a rather nebulous concept – hard to pin down and tricky to explain. It exists in the minds of people who recognise the values represented by the devices used in the identity. A form of non-word communication, which will therefore avoid the use of language and can transcend the problems of translation. This is the (usually) commercial application of coded messages. The value of logos became apparent in the early 20th century and some of those proved iconic and are still recognisable to this day. As many packaged goods had imitators, manufacturers added a ‘signature’ to instill confidence and ensure consumers bought the right pack. Kellogs and Boots are familiar examples. Oxo is an example of a ‘created’ brand name – like several others ending in “o”, for example: Rinso, Glaxo and Brasso. The Oxo logo, as with the previous ‘signature’ logos, has barely changed over decades and now relies on the values of nostalgia, rekindling memories of the past, like a good friend we instantly recognise. These brands with their familiar logos dare not tamper too much with the essential forms of the letters without losing the brand itself. Coca-Cola is another obvious example that has changed little in well over a century. Toblerone is another example of an iconic 100-year-old logo, although the lettering has been modified – possibly to its detriment. However, an old symbol has been incorporated in the newer symbol and the image of a bear is hidden in the Matterhorn mountain image.

What does this mean for the 21st century? Gradually, as these organisations became larger, often wielding great power, logos began to represent a vitally important and instantly recognisable face to customers internationally. In today’s global market, brand marks



316

Marks, Signs and Images

are required that convey a message and enable product placement without the need for words. Colours also became key elements in corporate identity and it is even possible to patent a colour (within a context of a brand). For example, although the Kodak logo has been updated over the decades, the Company has largely kept the yellow and red colours as consistent and essential brand elements. (Incidentally, the Kodak trade name came into being entirely on a whim – George Eastman merely liked the letter “K” so invented a name with two!). In this respect, all elements of identities – letterforms, logos, symbols, shapes, colours – are valuable assets and companies will go to great lengths and expense to create their corporate identity, strictly regulate its consistency in use and guard it well from imitators. Many large organisations find it necessary to commission their own corporate fonts to add to the uniqueness in their communications. We have perfected a system to seduce the public who will buy into the consumer utopia and it has become the role of the branding consultant to help companies and organisations to re-imagine their organisations, find a new mission and reposition it through the brand mark. But now, they must contend with a public that takes a more active interest.

The public takes an interest ‘Designer labels’ became a huge influence on the shopping public during the 1990s and a logo-festooned garment, shoe or accessory was essential for ‘street cred’, particularly amongst the young who were targeted by the marketing people, who were purveyors of a desirable lifestyle. Brand logos took on almost magical powers for the young purchasers – Nike was almost a religion. However, when youths were being mugged for their trainers, for some it had gone too far. The phrase “Culture Jamming” was coined as early as 1984 and the movement sparked the formation of groups who abhorred the practices behind the global brands. The term “cultural jamming” was first used by the college band “Negtivland: to describe billboard alteration and other forms of media sabotage. The Canadian founder of “Adbusters”, Kalle Lasn, published “Culture Jam” in the year 2000. Talking about the USA’s corporate giants, he said: “we will jam its image factory until it comes to a sudden shuddering halt”.4 4



Lasn, Kalle. 2000. Culture jam: how to reverse America's suicidal consumer

Paul Middleton

317

Fig. 5-4: This Nike advertisement is a clear reference to the nativity star (http://www.graphicdesigninspiration.net/2012/10/29/creative-graphic-designartworks-by-nike/).5

This new way of thinking had become widespread during the late 1990s, gaining momentum at the start of the new millennium. Naomi Klein’s book “No Logo” was also published in 2000 – and its focus on the negative aspects of branding and the singling out of some large companies, revealed the darker side and misdeeds of their company activities. Nike, The Gap, Microsoft and Shell were targeted. Four years later, Morgan Spurlock’s 2004 film “Supersize me” targeted McDonalds who were forced to address the negative publicity and change their public image – as well as improve their menu. Thus, the logos began to represent other, less positive, values and messages as their unethical practices of exploitation, sweatshop labour and environmentally-damaging methods were revealing their corporate greed. In an attempt to change the way the world does business, Adbusters, the activist organisation, launched ethically-produced ‘Blackspot sneakers’ in 2003 as an alternative to Nike’s Converse trainers with the slogan “Corporate capitalism gets the boot, one pair at a time”. They used a simple ‘spot’ in place of a logo and encouraged its use as a graffiti sign on the premises of the corporate giants.6 These shoes, then, gained a ‘cool’ of binge – and why we must. 5 Graphic Design Inspiration. 2012.“Nike City Motion”. “Great Ad Campaigns: Nike”. http://www.graphicdesigninspiration.net/2012/10/29/creative-graphicdesign-artworks-by-nike/. 6 Harold, Christine. 2004. Pranking Retoric: “Culture Jamming” as media Activism. Critical Studies in Media Communication Vol. 21, No. 3, September



318

Marks, Signs and Images

their own. BP went to a great deal of effort and expense to launch a new ‘green’ identity in 2000, designed by Landor. Their symbol was called the “Helios”, after the sun, god of ancient Greece, and was portrayed as a blossoming flower. It was lauded as an identity that reflected environmental responsibility. However, 10 years later it became a symbol of the worst environmental disaster in decades, following the BP Deepwater Horizon Oil spill in the Gulf of Mexico in 2010 – a disaster for BP’s carefully-crafted brand image. Greenpeace held a ‘crowdsource’ competition for the redesign of BP’s symbol, which attracted much attention – and announced this winner in August 2010. The damage done to the brand was immense and BP lost half its market value – now being seen as an irresponsible organisation.

Fig. 5-5: Greenpeace’s ‘rebranded’ BP symbol, 2010. (http://www.greenpeace.org.uk/files/bp/rebranded/index.html)7

In a clever move, Innocent Drinks moved away from the practices that were being exposed and made social responsibility one of their core brand values when they formed in 1999. Innocent Drinks gives 10% of its profits away to charity via the Innocent Foundation. Excellent Development received a 3-year grant, which enabled them to support the Meka Community Self Help Group to create secure water and food supplies8.

2004, pp. 189–211. 7 Hunziker, Laurent. 2010. Green Peace UK. http://www.greenpeace.org.uk/files/bp/rebranded/index.html. 8 Innocent. 2009. “excellent development scoops another award”. http://www.excellentdevelopment.com/innocent.php



Paul Middleton

319

Currently, they donate generously to Age (Concern) UK.9 Their quirky way of addressing customers as ‘friends we haven’t met yet’ paid off – their business flourishes and they have won many business and marketing awards – and their style of communication has been emulated widely.

A new relationship with consumers A culture of informality has spread to the very language used in modern retail – they want to be our friends and gain our loyalty. Supermarkets have a Facebook page and treating customers as ‘friends’ became a new way of conducting business. A more informal approach has been a successful gambit for many large companies or organisations. Even a person can be a symbol – The Halifax Building Society introduced Howard Brown, one of their employees, in 2000 and he was the face of the Company until 2008, when a harsher economic climate made him ‘too jolly’10. The public is now taking an interest in the designs. On June 4 2007, the London 2012 Olympics logo was revealed by Wolff Olins following a commission the previous year. It was met with a huge outcry of disappointment from the UK public – so much so, that the head of the design team, Patrick Cox, was so hounded by the press at home that he and his family sought refuge in a hotel. The brand mark or logo may become less of a focus, as Wolff Olins’ award-winning 2009 identity for AOL demonstrates. This time the usergenerated backgrounds become the main element of the identity, rather than the logo – a clever idea and an important message about democracy.

Brand personality Successful brands need personality. They also need to build trust – for example, when I buy a mobile phone I probably do not spend time comparing their features – I just know the brand I will buy because I trust the maker’s reputation, reflected in their image. One symbol – the Apple – has entered the public consciousness like few others. It now stands for a ‘lifestyle’ – becoming iconic to the many lovers of Apple products. The original logo from 1976 termed “The 9 innocent. 2013. “so what is the big knit”. http://www.innocentdrinks.co.uk/bigknit 10 Green, Chris. 2008. “Howard's end: he's too happy for a recession”. http://www.independent.co.uk/arts-entertainment/tv/news/howards-end-hes-toohappy-for-a-recession-887114.html.



320

Marks, Signs and Images

Newton Crest” would probably not have worked so well, as Steve Jobs soon realised – it was replaced the same year by the rainbow Apple symbol. Steve Jobs’ death in October obviously received much publicity worldwide, but no image conveyed the message more elegantly – no words needed – than the Apple logo (with Jobs’ profile replacing the ‘bite’) adapted by design student Jonathan Mak, in Hong Kong11.

Communication and responsibility Research has shown that even pre-school children can recognise hundreds of logos and trademarks, before they can recognise their own name. It is now thought that ‘brand loyalty’ has even been established by the age of two – prompting some concern about materialism amongst children. In older consumers, the logo has become the essential tool of contemporary communication, which a new and more ‘savvy’ generation of consumers or viewers is able to quickly and efficiently decode. Successful brands must connect the ‘offer’ to the image as (due to instantaneous communications systems i.e. the Internet etc.) a brand can no longer be used as a ‘cover’ for unsavoury practices. Without realising it was happening, the populations of the civilised world have evolved another language, with just as many symbols and signs as the Chinese writing system we might consider so challenging to memorise. But this visual language begins to transcend the written word and we learn it with surprising speed. So today, logos and brand marks are changing into something even more powerful than in the past, with the ability to impact on societies and economies in ways that go beyond promoting products or services. The increasing trend of corporations to diversify into new areas rests on the logo to articulate brand values alongside ethical commitments and other complex concerns. As we look to the future, the global corporate interests have to embrace the ethical and moral issues which concern their customers, something which can only be a good thing and now gives even more power and significance to logos and symbols – and to the designers who created them.

11 Note: It is not commonly known that Apple’s ‘bite’ was intended as a play on words with ‘byte’. 2009. http://www.edibleapple.com/2009/04/20/the-evolutionand-history-of-the-apple-logo/.



Paul Middleton

321

Bibliography Graffiti creator, http://www.graffiticreator.net BBC NEWS UK. “Royal wedding: Kate Middleton coat of arms unveiled “ (2011). http://www.bbc.co.uk/news/uk-13127145. National Museums Liverpool. “Artwork highlights - Bubbles, by Sir John Everett Millais”. http://www.liverpoolmuseums.org.uk/picture-ofmonth/displaypicture.asp?venue=7&id=299. Graphic Design Inspiration. “Nike City Motion”. “Great Ad Campaigns: Nike”. (2012). http://www.graphicdesigninspiration.net/2012/10/29/creative-graphicdesign-artworks-by-nike/. Lasn, Kalle. Culture jam: how to reverse America's suicidal consumer binge – and why we must. 2000. Harold, Christine. Pranking Retoric: “Culture Jamming” as media Activism. Critical Studies in Media Communication Vol. 21, No. 3, September 2004, 2004. pp. 189–211. Hunziker, Laurent. Green Peace UK. (2010). http://www.greenpeace.org.uk/files/bp/rebranded/index.html. Innocent. “excellent development scoops another award” (2009). http://www.excellentdevelopment.com/innocent.php —. “so what is the big knit”. (2013). http://www.innocentdrinks.co.uk/bigknit. Green, Chris. “Howard's end: he's too happy for a recession”. (2008). http://www.independent.co.uk/arts-entertainment/tv/news/howardsend-hes-too-happy-for-a-recession-887114.html. Note: It is not commonly known that Apple’s ‘bite’ was intended as a play on words with ‘byte’. (2009). http://www.edibleapple.com/2009/04/20/the-evolution-and-history-ofthe-apple-logo/.



SHOWING SAYING: ON SPEECH BALLOONS LIZZIE RIDOUT

Foreword This paper brings together theoretical, historical and practice-based research gathered through the course of 2011. In part, it is a collection of ruminations, preoccupations, truths and tales examining the speech balloon, and its dear, yet distinct relative, the thought balloon. It also forms a brief commentary of the work produced during an artist’s residency, completed by Lizzie Ridout at the Women’s Studio Workshop in Rosendale, NY, USA in 2011/12.

Principles & certitudes The speech balloon, a universally recognised graphic device, is employed as a means to represent both spoken and thought words, most commonly in comics. This is achieved through the use of a form, frequently bubblelike, typically within which is placed typography. On occasion, image may also be used. Traditionally a thought balloon is distinguishable from a speech balloon by a cord of bubbles attaching the principal bubble to the thinker’s head. Speech and thought balloons are simultaneously both a pictorial and a textual device. Initially, we read the words (or images) cradled within the balloon and comprehend their meaning. But our understanding of this meaning is also reinforced by both the choice of typography employed and the structure that the words are contained within. The visual properties of the type, images and balloon may all illuminate further what the character is speaking, thinking or doing.

Lizzie Ridout

323

On physicality Freud was not wrong when he wrote in “The Joke and Its Relation to the Unconscious” that “words are a plastic material with which one can do all kinds of things”. Speech and thought bubbles come in many guises. Some are formal, orthodox in attitude, the suit-wearers of the species. Others are rotund, doltish even, bloated – but confidently so. And still others are susurrate – breathy, ethereal, poised to disintegrate and, in form, rather similar to an exhalation on a cold winter’s day. The visual conventions are manifold. Most commonly, a speech or thought balloon is devised from an outline to define the form, and the surface itself. The boundary line differentiates between what is happening within the bubble form, and what is happening beyond it, in the comic frame. Those boundaries may be angular, beveled, indistinct or blousy as required and desired by their creator. Two things have held my interest in terms of the creative possibilities of these devices: i) The fact that uttered and imagined words and images take up no literal physical space in our reality, and yet in a comic reality they do. If we push this line of enquiry harder, we may question further this physicality. Are both speech and thought balloons essentially flat? May they not also be considered a container? This term suggests volume, mass and solidity. So, if speech/thought balloons are receptacles of information, do they have dimensions? Are they solid or filled with air? Are they transparent or are they white? And what of the contents? Are these also three-dimensional? Do the words float, or are they hung? ii) Although spoken words are on some level understood by all who can make sense of the language, thought is different. It is perceived exclusively by the thinker, and in the case of a comic, also by the audience. The thought balloon makes manifest what would otherwise never be witnessed. David Carrier (2000) describes the interplay of the speech balloon and the comic frame as a: theater with a soundproof glass wall between actors and audience, and with the spectators reading the dialogue from supertitles. Seeing a play in such a theater would be like reading comics.

324

Showing Saying: On Speech Balloons

An intermittent & mercilessly edited account of the history of visualising words Of course there is an extensive and complex history of visualising spoken or thought word in a variety of ways, and the form that we now recognise instantly has experienced various permutations over the centuries. Here, I have extracted some key examples of early speech balloons that have perhaps best examined the nature of the physicality of speech and thought. Within European history, text has been used in painting as a vehicle for adding meaning to the image, often where the gesture, stance and expression of the subject, and object-symbolism may not communicate all. In Renaissance art scrolls billow and unfurl across the canvas. Some quite literally snake out of the subjects’ mouths, clearly representing speech. Others are visual devices that act as an outside narrator to the story, adding important information that cannot be communicated through image alone. They are all as imagined as the speech bubbles in contemporary comics. But words, either religious texts or meaningful maxims, were included in paintings via actual, palpable objects. These items might be engraved (grand carved columns festooned with flowers and ivy or ornate stone urns for example), printed (a book casually left open on a desk or idly poised in the lap of the sitter) or handwritten (a folded note clutched by the subject or a sheath of papers stacked casually in the foreground). Completely independently and far preceding the Europeans, in South America there is compelling evidence to suggest that the Mesoamericans also developed sophisticated written systems to visualise spoken words, songs and music. Images and objects discovered at many archeological ruins depict humans and animals with questionmark-like forms leaping from their mouths. Similar to the ribbon-esque scrolls previously mentioned, the ‘banderole’ or ‘speech scroll’ employed by the Mesoamericans is potentially more abstract in its nature than its European relative. The tongue-like shape darts from the direction of the speaker’s mouth and may link the speaker to a series of other images. If our latterday translations of these devices are correct, these images are glyphs, figures used as symbols to represent words, sounds and ideas. It is suggested that the manner in which these speech scrolls are decorated, may give information about the tone of the words or the identity of the person speaking them. Thomas Rowlandson’s “The Loves of the Fox and the Badger, or the Coalition Wedding” is an adroit example of a strip demonstrating the full gamut of techniques available to the artist in the late 1700s for exploring the interplay between text and image. The piece, a satirical illustration

Lizzie Ridout

325

poking fun at the troubled coalition between Charles James Fox and Lord North after Fox’s Commons victory in 1782, uses a frame format, captions, scrolls, text-inscribed objects and speech and thought bubbles containing both typography and image. In this instance, the contained image, surrounded by radiating lines, suggests a dream. Continuing the practice of using objects to display narrative, in the late 1800s Richard F. Outcault famously created “The Kid”, a yellow nightshirt-cladded street urchin, who speaks to the newspaper-reading audience via his clothing. The speech balloon in this case, is humanoid and certainly in terms of the comic strip, a living and breathing speech/thought balloon. Winsor McCay remains, I suspect, unprecedented as the most sophisticated boundary-pusher of the pliable side to speech and thought balloons. Not only this, several of his works test the concrete possibilities of the comic strip as a whole. “Little Nemo in Slumberland”, published between 1905 and 1914 in, first, the “New York Herald” and then, later, the “New York American”, presents Little Nemo in a series of dream states, in which alternative universes are depicted. An appropriately nightmarish scene shows a series of ever-expanding speech bubbles taking over the entire frame and the characters within it. Most recently Peter Brookes, a political cartoonist, brilliantly transformed words into a physical weapon. His cartoon, published in “The Times” on 29th April during the 2010 UK election campaign, documented what must probably be one of the biggest political gaffes of recent times. Gordon Brown, after discussing Labour’s immigration policies with longstanding Labour voter Gillian Duffy in Rochdale, UK, returns to his car and, whilst still connected to his microphone, is clearly overheard telling an aide that Duffy is a “bigoted woman”. Brookes’ comic strip depicts, in the first frame, the words “bigoted woman” in a traditional speech balloon, poised over the form of Gordon Brown. In frame two, Brown is clasping the speech balloon and pulling it towards himself. In frame three, Brown is forcibly stabbing the speech balloon’s mouth-piece into his own stomach, ultimately impaling himself with the very words “bigoted woman”. It takes just three images for Brookes to mutate a passive, visual convention that we identify with the cartoon and comic world, into a physical form that is capable of inflicting bodily harm – akin to the damage that those two words caused Brown’s political career. The cartoon exposes our own vulnerability in the face of words and their capacity to turn on us when least expected. Whilst most spoken words are uttered then slip away unnoticed, there are always some that will return to plague us.

326

Showing Saying: On Speech Balloons

A residency for creating words, without using words Late in the summer of 2011, I packed my bags and temporarily left my home and my lecturing position at Plymouth University on a brief quasisabbatical. The intention was to spend two months working with the Women’s Studio Workshop on a funded artist’s book residency. I wanted focused time to produce a book – or rather 50 of them – making tangible my varied musings on the speech balloon. Those theories, I had decided, required placement into a practical context. I was curious to see how paper and ink, as image, might alter, validate and enhance what was at that time merely indistinct thoughts and a set of large-scale graphite drawings, cutpaper pieces and sculptures. Furthermore I wanted to see how my one-off pieces might be translated into multiples.

Fig. 5-6: Lizzie Ridout, Soliloquy [After Bakhtin] 2011, Cut paper, 120 x 100cm Image © Lizzie Ridout

“Ways to talk and yet say nothing, or ways to not talk and yet say everything” is the publication that emerged from this collaboration with the Women’s Studio Workshop. I say collaboration: I went armed with theory and proposals; WSW helped me shape this diaphanous pair into something palpable and coherent. Without their commitment to my project and their determination to help me solve the inevitable range of problems that one comes up against in such print-based undertakings, it would not have been resolved into the form it exists in now. The Women’s Studio Workshop was established in 1974 and has since gone on to be the largest publisher of artists’ books in the States. WSW offers artists paid and unpaid studio and book-art residencies, internships

Lizzie Ridout

327

and education programmes. Its publications are held in repositories across America, Canada and the UK including the following, who have committed to acquiring every past and future WSW edition (amongst upward of 200 other libraries and institutions): The Library of Congress, Yale University, Rochester Institute of Technology, Vassar College, Indiana University Bloomington, University of Delaware and Virginia Commonwealth University. It has had the continued support of the National Endowment for the Arts since 2002 and is also a recipient of funding from the New York State Council on the Arts – a State Agency and The Andy Warhol Foundation for the Visual Arts. The publication “Ways to talk and yet say nothing, or ways to not talk and yet say everything” started out as a series of studies examining a collapse in communication between two people through the use of the speech balloon. Later, however, the project became more focused on linguistic theory and semantics. Jackendoff (2001) in his essay “Language in the Ecology of the Mind” reports that language is a pairing of “expressions” and “messages”. “Expressions” being the ‘outer’ or ‘public’ element to language – the utterance or gesture that is tangible to or can be perceived by the person being spoken to. The “message” is the ‘inner’ or ‘private’ aspect of language, therefore the thoughts or concepts that the speaker transmits to the addressee, via the aforementioned expression. In order to convey a message, one needs to do more than just mentally represent it, one needs to be able to express it to the listener, too. Accordingly, a speaker will make a mental representation of what they wish to say, and then this in turn is converted into a series of expressions or movements of the tongue, teeth and lips. From this, a series of noises occur which are then converted by the addressee from expressions back into mental representations, and so finally back to a concept. Many of my early drawings explored this connection between brain and mouth: the interrelation between the message and the expression of the message. And also the intrinsic potential for disparity between the two. After all, there is a difference between knowing what you want to say and then also being able to say it. Further inconsistencies may lie between what is uttered by the speaker versus what is understood by the listener. Philosophers such as Mikhail Bakhtin with theories on spoken and thought words and Jacques Derrida’s deconstructionist principles also particularly informed this early body of drawn, printed and sculptural works.

328

Showing Saying: On Speech Balloons

Fig. 5-7: Lizzie Ridout, Soliloquy [After Bakhtin] 2011, Cut paper, 120 x 100cm Image © Lizzie Ridout

Mikhail Bakhtin (1952) wrote extensively about the “utterance”: its boundaries, its length, its intonation and the speaker’s “speech plan”. His work also concerned the connection between the speaker and the listener, and beyond this, the speaker and the community. Bakhtin proposed that whilst our words belong to us, they also belong to everyone else, including all who came before us. As a result, whilst we may feel that our words are our own, they have been heavily influenced by all with whom we have made contact and those who surround us, and therefore do not truly belong to us, or, indeed to anyone. Jacques Derrida’s methods of critiquing established language theory by turning words, theories and frameworks inside-out and over on their heads have also played some part in how this work and the resulting publication has unfolded. The work attempts to draw attention to linguistic conventions whilst simultaneously deconstructing those same conventions. The speech balloon is one of these conventions. As a visual code that we readily use to represent speech, there is always potential to further explore its usage, meaning and associated past. “Ways to talk and yet say nothing, or ways to not talk and yet say everything” takes a selection of my early drawings and concepts and by utilising a variety of print media (amongst them etching, silkscreen and letterpress) language’s idiosyncrasies are further explored, often via titling that adds a clue to the interpretation of an individual piece’s meaning or justifies the employment of a particular media.

Lizzie Ridout

329

Fig. 5-8: Lizzie Ridout, Compilation of pages from Ways to talk and yet say nothing, or ways to not talk and yet say everything, published by Women’s Studio Workshop, 2012 (Top line from left to right: Surrender / Shadow of a Whisper from Beneath 1000 Pages / Flee / Dialogue of the Deaf / Our Speech is Filled with Others’ Words. Bottom line from left to right: Soliloquy [After Bakhtin] / Excoriate / Deleted Exclamation / Imbroglio) All images © Lizzie Ridout

Thus, whispers are printed with dry, white carbon paper, suggesting a fleeting, chalky shadow of words on a page. Words that are somehow almost hanging in a half-life, or tucked away, best forgotten and slowly dimming over time. A soliloquy is defined in the Oxford English Dictionary as “an act of speaking one’s thoughts aloud when by oneself or regardless of any hearers, especially by a character in a play”. My silkscreened version of the soliloquy meanders across a large sheet, initially slowly, as if in doubt, then finally squeezing itself into every available space. The viewer is also a participant in this private address through their own act of handling the paper and the necessary unfolding of it, in order to reveal this everproliferating piece of diction. The multiple nuances of single words are explored through visual means and paper is treated accordingly. Excoriate means both to censure or criticise severely and also to damage or remove part of the surface of something. Thus, the surface of the page is shot-through with tiny holes, obliterating the flat, calm, white with a flurry of miniature assaults. The final resulting portfolio of prints takes various combinations of paper, ink and press and uses these to add tenor to the monologues,

330

Showing Saying: On Speech Balloons

dialogues and colloquies from both emotional and theoretical perspectives – without ever revealing what has actually been uttered.

On where this leaves me As much of my early investigations into the use of speech balloons attest, these forms continue to hold scope for enquiry into how we speak (and think) and how diverse the solutions are for articulating this in the visual world. There is a multitude of recognised visual conventions that come with established codes of interpretation attached, e.g. the use of uppercase letterforms to suggest shouting, or trembling lines to infer whispers. As previously mentioned, these rules have also been played with, and teased out, by numerous illustrators and artists. And yet, there will always be potential for further consideration into how speech balloons may be used as communicative devices, as a means to infer tone, cadence and the subtle significations inherent in speech itself. Yes, comics are potent forms in which to do this. But equally so is the removal of the speech balloon from this context that we are accustomed to viewing it within. Once isolated, it can be dissected and deconstructed to examine not just the expressive qualities of the utterance itself, but on a more poetic level to be rumination on the philosophical and internal workings of language itself.

 Versions of this article were originally published in Varoom, Issue 16 Autumn 2011, (ISSN 1750-483X) and as part of “Ways to talk and yet say nothing or ways to not talk and yet say everything” (ISBN 1-893125-793), published by the Women’s Studio Workshop, Rosendale NY, USA in 2012.

Bibliography Bakhtin, Mikhail. “The Problem of Speech Genres.” In Speech Genres and Other Late Essays by Mikhail Bakhtin, 60-102. Texas: University of Texas Press, 1987. Carrier, David. The Aesthetics of Comics. Pennsylvania: The Pennsylvania State University, 2000. Freud, Sigmund. The Joke and Its Relation to the Unconscious. London: Penguin Classics: 2002. Jackendoff, Ray. “Language in the Ecology of the Mind.” In The Routledge Companion to Semiotics & Linguistics edited by Paul Cobley, 52-65. London: Routledge, 2001.

Lizzie Ridout

331

Perry, George & Aldridge, Alan. The Penguin Book of Comics. London: Penguin, 1967. Varnedow, Kirk & Gopnik, Adam. High & Low: Popular Culture & Modern Art. New York: Museum of Modern Art, 1990. Varnum, Robin. & Gibbons, Christina. T. (Eds.) The Language of Comics–Word & Image. Jackson, Mississippi: University Press of Mississippi, 2001. Walker, Brian. The Comics Before 1945. New York: Harry N. Abrams Inc., 2004. Walkup, Kathy. (Ed.). Hand, Voice & Vision: Artists’ Books from Women’s Studio Workshop. Rosendale, NY: Women’s Studio Workshop, 2010. Women’s Studio Workshop. “History of Women’s Studio Workshop”, www.wsworkshop.org_about/history.htm Retrieved January 27, 2012

CONTRIBUTORS LIST

A. Editor Evripides Zantides, Assistant Professor of Graphic Communication, Cyprus University of Technology. [email protected] B. Contributors (as appeared in the table of contents) 1. Ralph Ball, Professor, Central Saint Martins, University of the Arts, London, UK. [email protected] 2. Jeff Leak, Senior Lecturer, University of Wolverhampton, UK. [email protected] 3. Theodora Papidou, Ph.D. Candidate, Barcelona School of Architecture (ETSAB - UPC). [email protected] 4. Artemis Alexiou, Associate Lecturer, Manchester Metropolitan University, UK. [email protected] 5. Dora Ivonne Alvarez Tamayo, Dr., UPAEP, Mexico. [email protected] 6. Camelia Cmeciu, Associate Professor, Danubius University of Galati, Romania. [email protected] 7. Doina Cmeciu, Professor, Vasile Alecsandri University of Bacau, Romania. [email protected] 8. Evangelos Kourdis, Assistant Professor of Translation Semiotics, Aristotle University of Thessaloniki, Greece. [email protected] 9. Aspasia Papadima, Assistant Professor of Graphic Communication, Cyprus University of Technology. [email protected] 10. Ioli Ayiomamitou, Research Assistant, Cyprus University of Technology. [email protected] 11. Stelios Kyriakou, Research Assistant, Cyprus University of Technology. [email protected] 12. Nikos Bubaris, Assistant Professor, University of the Aegean, Greece. [email protected] 13. Patrick John Coppock, Adjunct Professor, University of Modena and Reggio Emilia, Italy. [email protected]

Semiotics and Visual Communication: Concepts and Practices

333

14. Jack Post, Senior Lecturer, Maastricht University, the Netherlands. [email protected] 15. Irini Stathi, Associate Professor, University of the Aegean, Greece. [email protected] 16. Theo van Leeuwen, Professor of Multimodal Communication, Institute for Language and Communication, University of Southern Denmark. [email protected] 17. Emilia Djonov, Lecturer, Macquarie University, Sydney, Australia. [email protected] 18. Law Alsobrook, Assistant Professor, Virginia Commonwealth University in Qatar. [email protected] 19. Holger Briel, Associate Professor, Communication, Xi'an Jiaotong Liverpool University. [email protected] 20. Anastasia Christodoulou, Associate Professor, Department of Italian Language and Literature, School of Philosophy, Aristotle University of Thessaloniki, Greece. [email protected] 21. George Damaskinidis, Dr, The Open University, UK. [email protected] 22. Symeon Degermentzides, Lecturer under contract, Greece. [email protected] 23. Catherine Dimitriadou, Assistant Professor of Teaching Methodology, University of Western Macedonia, Greece. [email protected] 24. Androniki Gakoudi, EAP/ESP instructor and teacher trainer, University of Western Macedonia, Greece. [email protected] 25. Miltos Frangopoulos, Deputy Director of Studies, Vakalo Art and Design College, Athens, Greece/ Visiting Research Fellow University of Derby, UK. [email protected] 26. Maryam Hosseinnia, Associate Professor, American University of Kuwait. [email protected] 27. Peter C. Jones, Joint Programme Leader MA Publishing, Plymouth University UK. [email protected] 28. Tony Pritchard, Course Leader, Postgraduate Certificate and Postgraduate Diploma Design for Visual Communication, London College of Communication, UK. [email protected] 29. Evangelia Svirou, Educator specialized in Primary Education, University of Western Macedonia, Florina, Greece. [email protected]

334

Contributors List

30. Ifigeneia Vamvakidou, Associate professor of Modern Greek History and Culture, University of Western Macedonia, Florina, Greece. [email protected] 31. Paraskevi Golia, teacher in primary education and researcher lecturer, University of Western Macedonia, Florina, Greece. [email protected] 32. Paul Middleton, Executive Dean, University of Northampton, UK. [email protected] 33. Lizzie Ridout MA(RCA), Lecturer, Plymouth University, UK. [email protected]