Hearing Eyes, Seeing Ears: Collected Writings on Music in Audiovisual Culture 3031739132, 9783031739132

Table of contents :
Contents
List of Figures
1 Introduction: Conceiving Music’s Relationship to Image
What is Audiovisual Culture?
Sound Plus Image = Double Plus Good
Audiovisual Models of Sound-Image Relations
Book Chapters
Final Statement: Image Redefined by Music
Notes
2 Occult Aesthetics
The Dynamic of Synchronization
Synchronizing Soft Machine(s)
Conclusion
Notes
3 Audiovisual Space
Experiencing Audiovisual Spaces
Rural Sights and Sounds
Nonindifferent Nature
Conclusion
Notes
4 The Audiovisual Elsewhere
Widening Superfield Theory
Audio That Implies Visuals
Technology and Intelligent Sound Design
Conclusion
Notes
5 African American Film Music: Blaxploitation
Genre and Industry
Music
Defining Films
Conclusion
Notes
6 Adapting Stage Musicals in Britain
Traditions
Three individuals: Coward, Newley and Lloyd Webber
Conclusion
Notes
7 New Music for Old Movies
Old Films, New Music
Case Study: Metropolis (1927, Fritz Lang, UFA)
New Synergy, New Psychology
Conclusion: Audiovisual Meditations on History
Notes
8 From Video Game Music to Film Music: Silent Hill
Silent Hill Adaptation
Silent Hill Game Sound and Music
Silent Hill Film Sound and Music
The Integrated and Unified Soundtrack: The Game-Film Continuum
Conclusion
Notes
9 The Triple Lock of Synchronization
Synching Everything Up
Finger on the Trigger
Conclusion
Notes
10 Doctor Who: Britain’s IRCAM and Darmstadt
Relative Histories (1963–89)
BBC Radiophonic Workshop
The Space Between Functionalism and Experimentation
Conclusion
Notes
11 Experimental Music Video: “Why Do I need TV When I’ve got T-Rex?”
Promo History
Promos and Their Makers
Art and the Promo
Conclusion
Notes
Afterword
Index

Citation preview

PALGRAVE STUDIES IN AUDIO-VISUAL CULTURE

Hearing Eyes, Seeing Ears Collected Writings on Music in Audiovisual Culture K. J. Donnelly

Palgrave Studies in Audio-Visual Culture

Series Editor Kevin J. Donnelly, School of Humanities, University of Southampton, Southampton, UK

The aesthetic union of sound and image has become a cultural dominant. A junction for aesthetics, technology and theorisation, film’s relationship with music remains the crucial nexus point of two of the most popular arts and richest cultural industries. Arguably, the most interesting area of culture is the interface of audio and video aspects, and that film is the flagship cultural industry remains the fount and crucible of both industrial developments and critical ideas. Palgrave Studies in Audio-Visual Culture has an agenda-setting aspiration. By acknowledging that radical technological changes allow for rethinking existing relationships, as well as existing histories and the efficacy of conventional theories, it provides a platform for innovative scholarship pertaining to the audio-visual. While film is the keystone of the audio visual continuum, the series aims to address blind spots such as video game sound, soundscapes and sound ecology, sound psychology, art installations, sound art, mobile telephony and stealth remote viewing cultures.

K. J. Donnelly

Hearing Eyes, Seeing Ears Collected Writings on Music in Audiovisual Culture

K. J. Donnelly Faculty of Humanities University of Southampton Southampton, UK

ISSN 2634-6354 ISSN 2634-6362 (electronic) Palgrave Studies in Audio-Visual Culture ISBN 978-3-031-73912-5 ISBN 978-3-031-73913-2 (eBook) https://doi.org/10.1007/978-3-031-73913-2 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2025 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Cover illustration: Oleg Elkov/Alamy Stock Photo This Palgrave Macmillan imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland If disposing of this product, please recycle the paper.

Contents

1

Introduction: Conceiving Music’s Relationship to Image What is Audiovisual Culture? Sound Plus Image = Double Plus Good Audiovisual Models of Sound-Image Relations Book Chapters Final Statement: Image Redefined by Music

1 3 5 8 11 14

2

Occult Aesthetics The Dynamic of Synchronization Synchronizing Soft Machine(s) Conclusion

21 24 28 39

3

Audiovisual Space Experiencing Audiovisual Spaces Rural Sights and Sounds Nonindifferent Nature Conclusion

45 48 51 56 62

4

The Audiovisual Elsewhere Widening Superfield Theory Audio That Implies Visuals Technology and Intelligent Sound Design Conclusion

67 69 71 76 82

v

vi

CONTENTS

5

African American Film Music: Blaxploitation Genre and Industry Music Defining Films Conclusion

87 90 92 96 101

6

Adapting Stage Musicals in Britain Traditions Three individuals: Coward, Newley and Lloyd Webber Conclusion

105 106 114 121

7

New Music for Old Movies Old Films, New Music Case Study: Metropolis (1927, Fritz Lang, UFA) New Synergy, New Psychology Conclusion: Audiovisual Meditations on History

125 126 130 134 137

8

From Video Game Music to Film Music: Silent Hill Silent Hill Adaptation Silent Hill Game Sound and Music Silent Hill Film Sound and Music The Integrated and Unified Soundtrack: The Game-Film Continuum Conclusion

143 143 146 150

9

The Triple Lock of Synchronization Synching Everything Up Finger on the Trigger Conclusion

165 166 168 179

10

Doctor Who: Britain’s IRCAM and Darmstadt Relative Histories (1963–89) BBC Radiophonic Workshop The Space Between Functionalism and Experimentation Conclusion

185 186 192 194 199

154 160

CONTENTS

11

Experimental Music Video: “Why Do I need TV When I’ve got T-Rex?” Promo History Promos and Their Makers Art and the Promo Conclusion

vii

203 206 209 213 216

Afterword

221

Index

227

List of Figures

Fig. Fig. Fig. Fig. Fig. Fig. Fig. Fig. Fig. Fig. Fig. Fig. Fig. Fig. Fig. Fig. Fig. Fig.

2.1 3.1 3.2 4.1 4.2 4.3 5.1 6.1 6.2 7.1 8.1 8.2 8.3 9.1 9.2 10.1 11.1 11.2

The Passenger Midsomer Murders Stalker: Shadow of Chernobyl Mount Vernon Arts Lab Quatermass and the Pit Lustmord’s The Monstrous Soul Superfly Noel Coward in The Italian Job Anthony Newley in The Strange World of Gurney Slade Metropolis Silent Hill 2 Silent Hill 3 Silent Hill: Revelation Dead Space Plants vs Zombies ‘The Sea Devils’ Nine Inch Nails’ Closer U2’s (Even Better Than) The Real Thing

35 53 61 73 74 79 90 114 118 132 145 147 151 172 176 196 212 216

ix

CHAPTER 1

Introduction: Conceiving Music’s Relationship to Image

“Hearing Eyes, Seeing Ears” refers to the insight of the McGurk Effect, that our hearing and seeing senses are not separate but merge, are not just cross-referencing but provide a unified signal.1 This is particularly pertinent for audiovisual culture and is at the heart of its potency as a medium. Indeed, the audiovisual is central to modern culture, with screens and speakers (including headphones) dominating communication, leisure and drama. While this book mostly addresses film, it also deals with sister media television and video games, registering that there is a ‘common core’ of synchronized image and sound at the heart of these different but related media. The traditions of sound allied to the moving image, what Michel Chion calls ‘audiovision’,2 include principles of accompaniment and industrial processes from film that have been retained and developed in other media. Each chapter marks a case study making for a varied collection that embraces rich history and different traditions, as well as the distinct aesthetic boldness of different genres and formats. This book engages with the rich history, varied genres, different traditions and variant strategies of audiovisual culture. However, it also points to and emphasizes the ‘common core’ of flat moving images and synchronized sound and music which marks a dominant in electronic media culture (what might be called ‘screen and speaker/diaphragm culture’).

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025 K.J. Donnelly, Hearing Eyes, Seeing Ears, Palgrave Studies in Audio-Visual Culture, https://doi.org/10.1007/978-3-031-73913-2_1

1

2

K. J. DONNELLY

This book addresses ‘audiovisual culture’ as a unity of music, sound and image across a broad field. Perhaps surprisingly, audiovisual culture’s ‘common core’ of electronic sound and flat images is rarely registered, with most studies emphasizing specific and defining aspects of different media. A significant aspect of the book is the positing of a common core to audiovisual culture despite it usually being approached as different media. Consequently, the book not only includes writing about film but also television, music video and video games. As well as diversity, audiovisual culture has a strong degree of unity, and the book addresses theoretical aspects and sound and image aesthetics, rather than simply musicology or image-based narrative analysis. Addressing music as both diegetic and non-diegetic, as both songs and score, the book has a wide variation of material united under its principal concerns. Concentrating on context as the defining aspect of music, analyses aim to attend the precise interaction between music and other elements of audiovisual culture as defining overall configurations. It emphasizes the uncertain nature of music’s effect in audiovisual culture, which can be crucial in articulating the whole and gaining a semi-conscious effect, particularly in emotional terms. This book approaches music in audiovisual culture as a complex merged signal rather than as idealized separate ‘music’ and ‘the film’/ ‘TV show’ or ‘video game’. This unified signal is arguably the most defining characteristic of audiovisual culture, although this unification is far from a simple operation or effect as it is premised upon the perceptual phenomenon of humans construing images and sounds as being the same coherent object.3 The book focuses on aesthetics, addressing both music and image aesthetics, strategies and traditions, as well as connecting with the psychological engagement engendered by the audiovisual. It also illustrates how music in audiovisual culture is able to make complex, subtle and sometimes contradictory meanings, through working with, or seemingly sometimes against meaning made elsewhere, or making ‘footnotes’ to ideas and meanings not immediately present. This is illuminated by a sense of historical and other contexts, traditions and technological horizons.

1

INTRODUCTION: CONCEIVING MUSIC’S RELATIONSHIP …

3

What is Audiovisual Culture? There is a tendency not to recognize or register audiovisual culture as a specific object, perhaps inspired by its multi-faceted aspect. What is incontrovertible is that audiovisual objects share a common core of aesthetics and are built around the perception of sounds and images, often held together in a tight synchronization. So, audiovisual objects might be understood as something specific: synchronizing sound and (usually) moving flat images. These are structured in a manner derived from film, which was the first object to bring together moving screen images and synchronized recorded sound. Michel Chion discussed the process of ‘synchresis’,4 a spontaneous weld that appears to bind together sound and image in audiovisual culture. They seem to ‘fit’ to us, even when married at random. This probably tells us more about our perception, and its desire to make a coherent signal than it does about how images and sounds fit together. Yet following the principles of isomorphism, there is a sense not only of equivalence between certain sounds and images but also of ‘what image fits well with what image’ (and vice versa). Characteristics that define the audiovisual focus on the relation of sound and image. These relate to synchronization and the perceived bond between sound and the diegetic world depicted on screen. Indeed, these are highly specific to audiovisual culture, making for a definition through specificities while it also shares many crucial elements with other sound and image culture. Sound and image synchronization is the first line of definition, aligning recorded images to recorded sound.5 This can take the form of anything from tight synchrony to total disconnection between the two. This is arguably the central defining characteristic of audiovisual culture and often moves between synchrony and lesser or greater asynchrony in a dynamic relationship.6 From tight synchronization to the ‘acousmatic’, which Michel Chion (after Pierre Schaeffer) addressed as a total disconnection between sound and image, and furthermore the distinction between on screen (showing the source of the music) and off screen (where the audience assumes it is part of the diegetic world although its source is not visible).7 Such theories engage with the earliest theories of recorded sound’s relation to the film image, such as Eisenstein’s notion of ‘parallel’ or ‘counterpoint’ relationships and co-authorship of the ‘Statement on Sound’ in 1929.8

4

K. J. DONNELLY

Secondly, the distinction between diegetic and non-diegetic sound. This relates to whether music and sounds are part of the diegesis, the world depicted on screen, or from outside that. Most sounds in film and other dramatic audiovisual forms are diegetic, part of the illusion of the world on screen. Rick Altman wrote about the differences between diegetic and non-diegetic music: “By convention, these two tracks have taken on a quite specific sense: the diegetic reflects reality (or at least supports cinema’s referential nature), while the [non-diegetic] music track lifts the image into a romantic realm far above the world of flesh and blood”.9 While conventionally these two relationships have had particular functions and psychological implications, they have not been unambiguous and music and other sound can have an equivocal or changing relationship to the diegesis, and the distinction is conventional and certainly not always prevailing.10 Indeed, many theorists have had issues with the diegetic/non-diegetic distinction as a viable theoretical division,11 which derived from traditional film-making conventions that classified music as ‘source music’ (part of the world on screen; diegetic) or ‘score’ (music written to help film narration; non-diegetic). These two aspects, synchronization and diegetic status, regularly relate to a third: music and sound providing a clear sense of spaces. Technology has been important here, with stereo sound allowing for the aural illusion of ‘real’ space. Contemporary spatialized stereo sound using multiple speakers and multiple variants of sound signal to each, furnishes an unparalleled sense of space. It remains an audio illusion, however, yet retains a strong feeling of immediacy and ‘reality’, yet even mono sound (a single channel and sometimes a single speaker) provides a sense of space with spatial cues (relative volumes of recorded sources, reverb) within the mono sound ‘image’. Synergetic construction and implication of spaces can be understood as a primary aspect of audiovisual culture, although it is far from straightforward, as Jeff Smith notes in his compelling discussion of music’s relation to narrative space.12 Film and subsequent audiovisual culture play to the affordances of audio and visual, and perhaps at best plays to the mixing of the two. Initially at least, this process is perceptual rather than cognitive (a higher order of activity). Thus, perhaps it is more fitting to approach the audiovisual as fundamentally physiological rather than simply a mental process. Sound and image fit together in a way that can be highly effective, making for a clear synergy between the two channels and human senses. The

1

INTRODUCTION: CONCEIVING MUSIC’S RELATIONSHIP …

5

‘wiring’ of image and sound in unity is matched by, and matches, the wiring of our brains in relation to perceptual processes. The McGurk Effect is a startling illustration of the unified essence of audiovisual culture, indicating how the sounds and images of audiovisual culture are merged into a single, unified signal. In an article in the journal Nature in 1976, psychologists Harry McGurk and John MacDonald outlined a perceptual phenomenon whereby filmed images of a person’s mouth enunciating a particular vocal sound, matched to a sound recording of a different vocal sound, add up to the perception of a new sound that objectively is not present in audio.13 This is often illustrated with the repeated sound of a person saying ‘ba’ being accompanied successively by images of the same person saying ‘ga’ and ‘da’, and almost universally human beings perceive that the vocal sound is changing, even though in reality the sound is exactly the same.14 This shows that we hear with our eyes as well as our ears. There are corresponding audiovisual perceptive phenomena that prove we also see with our ears. One of these is the ‘Bounce Inducing Effect’, where our perception of screen movement can be changed by the addition of sound at certain moments.15 Sound appears to drastically affect visual perception. Both of these perceptual phenomena illustrate clearly how adding together recorded sound and moving image produces a highly particular effect of their direct relationship.16 This cross-sensory effect underlines how images and sounds may seem totally different together from apart, as the McGurk Effect proves the active nature of vision and hearing, as well as their transformed relation when added together, making for a ‘synaesthetic’ outcome. Indeed, it indicates that fully isolating sound or image may be invalid. It certainly suggests that blanket assertions of ‘film being a visual medium’ (or similar assertions) should be long banished. The implications for sound may in fact be more profound. Indeed, any appeal to the ‘purity’ of musical experience is of course, also thrown into question.17 Through synergetic effect, the two tracks add up to something more than separately.

Sound Plus Image = Double Plus Good Arguably, we have been living in an age dominated by audiovisual culture since the early twentieth century. The proliferation of audiovisual media since the middle of that century accelerated significantly since the widespread impact of digital media, platforms and distribution beginning

6

K. J. DONNELLY

just before the turn to the twenty-first century. This is a relatively young culture, since the large expansion of Internet capabilities and reach in the mid-1990s in Europe and North America, and later elsewhere. In recent years, music has come to play an increasingly important and evident role in audiovisual culture. Alongside this, scholarly writing about the theory and history of ‘audiovisual’ music and culture now firmly has arrived. The best of this writing acknowledges that it is far more than a simple ‘addition’ to the images of film. In fact, audiovisual music might be understood as not music at all, or at least as music with profound differences from other music—particularly ‘absolute music’ that stands on its own. Indeed, in musical terms it might be better understood as a context, for any music imported to film or other audiovisual culture transforms into something else through merging with its new surroundings. A good example of this is Gioachino Rossini’s William Tell Overture, the opening of his opera that premiered in 1829. This exciting music for the opera about a Swiss national hero bowman became more solidly associated with the cowboy television show The Lone Ranger (1949–57, ABC), and the music had already been used in an early radio version. The final section of the overture, ‘Finale: The March of the Swiss Soldiers’ appeared eminently suited to galloping horses and other things. In 1971, Stanley Kubrick used the overture in A Clockwork Orange as an accompaniment to a comically sped-up sex scene. This illustrates just how malleable musical recordings are, and how important images are for changing musical meanings. Reciprocally, becoming an integral element of a combination with images might also mean the music composed for a film is only a partial object on its own without its visual counterpart, while the images without their music can be lifeless and emotion-free. This often calls for a form of music that is significantly different from other music, setting up a condition of music that has the ability to incorporate images into itself, as much as images can achieve their plenitude through incorporating music. Similarly, pre-existing music, sometimes of well-known origin, must transform to successfully become film music. The process is not the simple addition of music to images, but a complex process of merger and transference between the two media. Indeed, the qualities of mediocre music can seem entirely transformed through the addition of images, and reciprocally prosaic images and narrative situations can be made compelling through adding engaging music. There has always been an assumption (rightly or wrongly) that music can ‘save’

1

INTRODUCTION: CONCEIVING MUSIC’S RELATIONSHIP …

7

a poor film, while music videos often illustrate how dull songs can be totally transformed by added images. There is a clear phenomenon where a song that you think is awful can appear in a film or television show and remarkably it seems far better, sometimes to the point of seeming to be excellent. Of course, this tends to be only a temporary effect, and upon again hearing the song outside of the audiovisual context it can revert to its original unpleasing effect. Music video as a medium is premised upon this phenomenon, with visuals regularly being used to enhance the music. With the onset of music television in the 1980s, on a couple of occasions I was inspired to rush out and buy a record after experiencing a music video, only to find that the song recording alone was mediocre, the disc never to touch my turntable again. Of course, it is not just the visuals are ‘good’ but that the sound and images have cohered into something more effective. This is a clear indication of the McGurk Effect and the mutual implication and transformation that takes place between sound and image, which is registered in Michel Chion’s definition of ‘added value’, where an image or narrative situation can be given a seemingly intrinsic stronger sense, emotionally or impressively, through sound or music.18 Music certainly ‘aestheticizes’ the image, making it into something less of a recording of a pro-filmic event— what was in front of the camera—and more of an emotional experience. Similarly, the addition of images appears to ‘spatialize’ music, expanding its sense of movement and sense of its particular space. There is an old film industry saying that incidental music can either be ‘camera’ or ‘set’. While unfortunate in its ocularcentrism, this filmmaking metaphor for music tells us something significant: film music often primarily will function more as one or the other, perhaps working more as a narration or as a part of location (diegetic setting). It is able to mould the audience’s momentary responses and match the dynamic undulations of film mood, character and narrative. All too regularly, musical scores in films and other audiovisual dramas are described in functional terms, in relation to narration and providing support and information for the narrative. This fits a conception of film and audiovisual drama as simply a ‘story’ and with the assumption that when music isn’t supplying information and guidance, it is there to furnish an emotional impact. Although schematic, the ‘camera or set’ dichotomy remains useful: it might better be rendered as music being ‘narration’ or ‘atmosphere’. Music as either: narrative, providing and clarifying information and aiming towards closure, or: in the moment, sensual and substantial.

8

K. J. DONNELLY

While it can clearly be one or the other, it can, of course, alternate or be both at the same time in certain instances. However, understanding that music can be ‘set’ (furnishing a sense of place and feeling of ambience) rather than simply ‘camera’ alone allows for a sense of it being understood as having some solidity, a sense of space and atmosphere, rather than conceiving it purely as a dramatic ‘storytelling’ device. It is clear that most film and other audiovisual culture regularly is conceived and often has been constructed following a nominal structure of background and foreground, with incidental music regularly conceived as background.19 While this comes from both perceptual requirements (for example, prioritizing audible dialogue) and the characteristics of the medium itself (such as the use of shallow focus), it is not only a wellentrenched convention but also a given for analysis. The implication is that the foreground is important while the background is unimportant. However, there ought to be more acknowledgement of how the two are in an interdependent relationship. The sense of audiovisual culture’s habitual foregrounding of dialogue makes the rest of the sounds seem receded background or sometimes simply ambience, yet the background supplies crucial context and dynamic definition for the foreground. This process is a remarkably precise description of the foundational Gestalt Psychology principal of ‘figure and ground’.20 This is a proven reality of human perception—which is not a bad foundation for audiovisual theory. Indeed, better than most. People regularly apprehend visual fields in terms of these two intimately interconnected elements, one in the foreground and one behind it, but with the crucial understanding that the background is absolutely defining of the foreground rather than simply being an unimportant backdrop. The set, both in the sense of the space constructed for pro-filmic activity and through a succession of shots, and as the metaphor for an aspect of music function in audiovisual culture, ought then to be understood as an essential and critical component of the audiovisual, both in visual and sonic terms.

Audiovisual Models of Sound-Image Relations As already noted, this book embraces incidental music in film (including musicals), TV, music video and video game music but also wants to register that while sharing a common core, these individual aesthetic models of production and consumption have significant differences.

1

INTRODUCTION: CONCEIVING MUSIC’S RELATIONSHIP …

9

Indeed, they might well constitute different emotional and psychological models. Broad models of relationship between music and image cohered around particular audiovisual forms. They also relate to antecedent forms to some degree. So-called ‘silent cinema’ with its live music drew upon earlier conventions of live music for live events. The development of written music libraries for silent films, like Giuseppe Becce’s Kinothek (published in 1919) and Ernö Rapée’s Encyclopedia of Music for Pictures (published in 1925), allowed some degree of standardization, as cinema musicians would build accompaniments from a limited repertoire of excerpts that were concordant with the moods on screen. The coming of mechanically synchronized recorded sound in the late 1920s and the crystallization of what Bordwell, Staiger and Thompson call the ‘Classical Hollywood Cinema’21 served to standardize musical accompaniment for films further.22 The mature studio system in Hollywood followed a production line arrangement which led to a distinct blueprint of incidental music in terms of its production, its sonic character and its function. This included an aspiration to guide the audience’s emotions and not to be consciously registered in relation to the illusory world on screen.23 In the early 1930s, film music solidified into a solid plan that defined the music that would appear in the background in mainstream films. This model is what Claudia Gorbman called ‘Classical film scoring’ and Kathryn Kalinak called the ‘Classical Film Score’,24 a blueprint for music in film drama that has developed since the 1930s although in terms of principles and function remains largely in place. Significant developments include novel modes of scoring in mainstream cinema matching more general changes in the film,25 and what has been called ‘musical sound design’,26 which mixes music with other sonic elements to create an extended and unified palette as the ‘integrated soundtrack’. The film musical is a highly distinctive audiovisual format, as a hybrid of the stage musical and the affordances of film as a medium. Song sequences have visuals that overwhelmingly are determined by the requirements of the music, as Rick Altman notes,27 while visual style often includes frontal staging in a tableau-like frame (that looks like a stage), allied to the direct look of the performer into the camera, and lyrics that use personal pronouns, or even reverse shots of an audience, all of which implicate the film viewer/auditor in a direct relationship with the musical performance.28 This format has proven to be malleable while retaining its basic characteristics surrounding song sequences.

10

K. J. DONNELLY

Music for television drama has increasingly developed a similarity with film scores, while music video clearly owes something to film, particularly musicals. Television may look and sound as if it developed in audiovisual terms from film, but in many ways, it owed more to radio formats.29 For television drama, conventions from the stage and film were imported, while live music shows would often simply ‘broadcast’ music as if from a concert. The later development of music video, which blossomed as a distinct form by the 1980s, led to dedicated channels of music television. The proliferation of music television has established a new audiovisual form, the music video or pop promo,30 that has furnished a new logic to television, seeping out of specialist channels and into commercials and action films, or perhaps rather channelled back into some of their originator formats. Geoff King and Tanya Krzywinska noted that in March of 2000, Sony had more revenue from video/computer games than from film for the first time,31 marking something of an eclipse of film by the gaming industry. Video game music can sometimes owe a lot to film scoring, despite its wholly different interactive nature.32 Up until the 1990s, there was a tradition of distinctive ‘game music’ (sometimes also known as 8-bit music or ‘chiptune’ music), which emerged from the sonic limitations of the hardware and limited amount of dedicated software in the game package. Starting with the Sony Play Station (PS1), released internationally in 1995, video game consoles were able to have CD-standard sound.33 Since then, many video games increasingly have tried to sound like dramatic films, even though their procedures are by necessity radically different. ‘Interactive’ or ‘dynamic’ music, which reacts to a range of possible developments in gameplay, still follows a broad concept of non-diegetic music or accompanying score. Some of the incidental music in video games sounds remarkably similar to film scores, such as, for example, Nobuo Uematsu’s music for the Final Fantasy series of games or Jeremy Soule’s striking music for The Elder Scrolls series of games.34 Beyond this, it is possible for Internet websites to have dedicated music that loads with the rest of the HTML or uses other dedicated software. All these examples add up to a world of constant and anticipated musical accompaniment to the image. Indeed, small screens have come to dominate the large screen in terms of audiovisual culture, and perhaps headphones of some sort have eclipsed speakers sending sound out into the air.

1

INTRODUCTION: CONCEIVING MUSIC’S RELATIONSHIP …

11

Book Chapters This book addresses a wide range of material across the breadth of audiovisual culture. It ranges from theoretical approaches to highly specific studies of concrete instances of audiovisual culture. The opening chapters have a theoretical character. “Occult Aesthetics” argues that film and other audiovisual media are based on a hidden (‘occult’) process of synchronizing image and sound and a play of dynamics in this process. While audiences engage with the ‘surface’ of film, being enchanted by its illusions of people, places and activities, the crucial underlying level is one of abstract connections, in emphatic moments of synchronization as well as the more mundane moments that hold sound and image together. The interplay between synchrony and asynchrony is the central dynamic of audiovisual culture more generally. “Audiovisual Space” explores musicdominated space in films, with sound providing different, sometimes contradictory spaces, for films at the same time, and not simply tied to the ‘realistic’ requirements of the diegesis. The McGurk Effect established that not only are vision and hearing active but also that they merge in a ‘synaesthetic’ manner in human perception. Electronic audiovisual culture can tell us much about human physicality, and artificial space in films, not only the illusory diegesis but also the emotional and cognitive space of music, which are fundamental aspects of audiovisual culture. Music in audiovisual culture has developed into an established and dominant blueprint that its aesthetic can spread beyond audiovisual culture. This extension beyond the boundaries of its object is detailed in the chapter “The Audiovisual Elsewhere”. Film and other audiovisual culture have exerted a significant influence on music beyond film, as well as having been an agent for musical recordings and having been influenced by music culture. This chapter also notes the increased ‘musicalization’ of the film soundtrack, which has paralleled the expansion of traditional notions of music in the wake of musique concrete, later experimental music and peripheral developments in so-called popular music. The next few chapters investigate different formats of music-image relations, including dramatic scores, musicals and silent films. “African American Film Music: Blaxploitation” addresses so-called ‘Blaxploitation’ films, which were a popular and vigorous American film genre in the 1970s, that depicted the African American urban experience. Music was often one of the key defining aspects of Blaxploitation films, drawing

12

K. J. DONNELLY

upon contemporary African American popular music and working in an unconventional manner in comparison with mainstream films of the time. Although fluctuating in popularity over the years, film musicals have remained a vibrant film genre. The chapter “Adapting Stage Musicals in Britain” illustrates that film musicals always had a close relationship with their stage counterpart, and Britain has been no exception. Indeed, in the 1930s, home-made musicals were among the most popular films in the country. In later years British musicals became less prominent, although stage adaptations have exploited the existing profile and readymade audience created by the stage original. Indeed, the packed theatres of London’s West End have never seen the lean years suffered by film musicals. This chapter investigates films that have drawn heavily upon stage originals, sometimes changing them notably, as well as looking at some key figures in developments. A concern with silent cinema is the focus of “New Music for Old Movies”. Yet this is not an account of film history but rather an account of how, particularly since the Millennium, silent films have returned from near oblivion. Indeed, silent films have become more popular than they have been since the 1920s when they were seemingly made obsolete by films with synchronized recorded sound. Their revival has come partly through a rebranding of past culture as heritage often restaged as live event, but perhaps more pertinently through the addition of new musical accompaniments. This novel culture can allow new opportunities for music and a rethink of the place of old silent films in contemporary culture, and this chapter reviews the possibilities and looks at some of the boldest and most controversial ‘modernizations’ of old silent films with new music. The following two chapters address music in video games. “From Video Game Music to Film Music: Silent Hill ” addresses the first two Silent Hill films (2006 and 2012), which were constructed directly around the existing soundtracks of the source video games. Exploring the adaptation of the Konami ‘survival horror’ video game series (the first one in 1999) illustrates the similarities and differences between the two audiovisual formats. Although adopting some cinematic conventions, the films duplicate the game soundtracks and reverse some film norms, most clearly in having sound effects with an emotional cast and music that is unemotional and mechanical, like an inanimate object. The concern with video game music carries on into “The Triple Lock of Synchronization”, which directly addresses the particularities of synchronization in digital

1

INTRODUCTION: CONCEIVING MUSIC’S RELATIONSHIP …

13

games. Contemporary audiovisual objects unify sound and moving image in our heads via the screen and speakers/headphones, yet video games not only synchronize these but also player input to form a close unity of the three. This synchronization unifies the illusion of movement in time and space and cements it to the decisive interactive dimension of gaming. In most cases, the game software’s ‘music engine’ assembles the whole, fastening sound to the rest of the game, allowing skilled players to synchronize themselves and become ‘in tune’ with the game’s merged audio and video. This constitutes the critical ‘triple lock’ of player input with audio and video that defines much gameplay in digital games. Moving the focus of the next two chapters to television and music video, the following chapter, “Doctor Who: Britain’s IRCAM and Darmstadt”, addresses a fine example of cutting-edge avant-garde music finding a place inside widely distributed popular culture. The initial run of the British television show Doctor Who (1963–1987) was remarkable for its use of music, sometimes produced by the BBC Radiophonic Workshop. In what was nominally a children’s programme, it used avant-garde and cutting-edge electronic music. At least partly, this was functional, with the low production value of the series in visual and dramatic terms being bolstered significantly by its strange and exotic music. Like much science fiction, arguably, the music in Doctor Who’s first three decades did a lot of the ‘imagining’ for a show plagued by cheap set and atmospherefree lighting. The BBC Radiophonic Workshop, which was showcased by the programme, increasingly appears to be the contemporaneous British counterpart to European centres for electronic music experimentation, the UK equivalent of such celebrated places as Darmstadt in (West) Germany or IRCAM in Paris. Another form of small screen experimentation is addressed in the book’s final chapter, “Experimental Music Video: “Why Do I Need TV When I’ve Got T-Rex?”” In the 1980s and 1990s, avant-garde audiovisual techniques appeared in mainstream media, most clearly where music video crossed over with more popular television. While in many cases, young filmmakers schooled in film history as well as film-making experimented with new audiovisual technology, in some cases, seasoned avant-garde artists moved into a more lucrative area with potential for larger audiences, although retaining their marginal aesthetic sense. Indeed, a collapse of the distinction between ‘art’ and ‘popular’ culture was most clearly evident in events related to music and television.

14

K. J. DONNELLY

Final Statement: Image Redefined by Music In Music in Cinema, Michael Chion begins with a subtitle: “music redefined by cinema”— which registers how far the music is qualitatively changed by audiovisual context.35 Indeed, sometimes music in film and other audiovisual culture seems to make more sense as not being understood as music at all. At the very least its logic is changed by its multimodal context. Of course, this process also has its reciprocal: images are changed by the music, which is why Hollywood has spent and continues to spend a significant amount of film budgets on music. Since the advent of synchronized sound, the film industry has had remarkably close ties with the music industry. Jeff Smith noted that the emergence of commercial discs as an important ancillary market encouraged films to use popular music.36 Commercial tie-ins have been important since the 1920s but the audio and (audio)visual industries have also had close technological ties. Music in films and beyond has followed the dominant processes in the music industry, with recording developing from a simple impression of a live event to recent multi-track recording and careful mixing that includes all sorts of compensations and adjustments. Indeed, not only is technology a, perhaps the, defining aspect of audiovisual culture, of most music, of all moving image cultures, but it also provides the hinge, the crucial point of unity between sound and image. As such, it needs to be taken far more seriously. The ‘forgetting’ of mediation is pervasive in the modern world as a parallel to the increasing penetration of media into all aspects of life. It is easy to forget that the impressive and sometimes overwhelming effect of film and television is the product of a mixture of vibrating diaphragms in speakers (or headphones) and light projected onto a screen or dots of light on an electronic screen. Music regularly works to enhance the effect of the image and vice versa. This mutually reinforcing effect is a form of aesthetic synergy, where the two add up to more than the sum of their parts. Sensuous, rhythmic and qualitative aspects of each generate a complex interaction, before aspects such as narrative context and cultural associations of the music and images, respectively, are taken into account. It is, therefore, not surprising that it is a thanklessly difficult task to attempt to account for the effectiveness of music in combination with the moving image. This, I would venture, is one of the main reasons why there are few sustained attempts to provide convincing and authoritative accounts of the powerful

1

INTRODUCTION: CONCEIVING MUSIC’S RELATIONSHIP …

15

aesthetics and emotional strength in the unifying ‘lock’ between music and the moving image. It remains quite common to hear repeated assertions that human beings are ‘essentially visual’ animals.37 I’ve never been fully convinced of this. Indeed, like many people I have spoken to, if given a choice I would rather lose the faculty of sight than hearing. Perhaps physiologically we have evolved to respond to vision in the first instance (and maybe with hearing close behind, and in tandem), but perhaps culture has reoriented this to some degree. It is hardly an outlandish suggestion to posit that audiovisual culture has elevated sound to an equal (or near equal) in the human perception mechanism. Indeed, interactions between sound and image, as well as the more obvious strategy of giving plenitude in one and scarcity in the other, illustrate how audiovisual culture has taken the basic evolved interaction of hearing and vision and made them into something potentially far more elaborate and complex. Thanks to those who provided some comments on earlier versions of the book’s chapters, including Claudia Gorbman, Carol Vernallis, John Richardson, Ann-Kristin Wallengren, Danijela Kulezic-Wilson, Liz Greene, Tim Summers, Melanie Fritsch, David Butler, Laura Mulvey and Jamie Sexton. Thanks to Erin Wiegand at Palgrave, and thanks to anyone else who helped in some way, no matter how little. The book’s chapters include some already published material, although to a lesser or greater degree amended: “Occult Aesthetics” in K.J.Donnelly, Occult Aesthetics: Synchronization in Sound Film (Oxford University Press, 2013), pp.70–93; “Audiovisual Space” in K.J.Donnelly, The McGurk Universe (Palgrave, 2022), pp.177– 214; “The Audiovisual Elsewhere” in “Extending Film Aesthetics: Audio Beyond Visuals” in Claudia Gorbman, Carol Vernallis and John Richardson, eds., The Oxford Handbook of New Audiovisual Aesthetics (Oxford University Press, 2014), pp.357–371; “African American Film Music: Blaxploitation” in “Singing Across 110th Street: Music in Blaxploitation Films” in Magical Musical Tour: Pop and Rock in Film Soundtracks (Bloomsbury, 2015), pp.79–92; “Adapting Stage Musicals in Britain” in “Stage to Screen: the British Stage Musical Adaptation” in British Film Music and Film Musicals (Palgrave, 2007), pp.122– 137; “New Music for Old Movies” in “How Far Can Too Far Go?: Radical Approaches to Silent Film Music” in K.J.Donnelly and AnnKristin Wallengren, eds., Today’s Sounds for Yesterday’s Films: Making Music for Silent Cinema (Palgrave, 2016), pp.10–25; “From Video Game

16

K. J. DONNELLY

Music to Film Music: the Silent Hill Adaptations” in “Emotional Sound Effects and Metal Machine Music: Soundworlds in Silent Hill Games and Films” in Danijela Kulezic-Wilson and Liz Greene, eds., The Palgrave Handbook of Sound Design and Music in Screen Media: Integrated Soundtracks (Palgrave, 2017), pp.73–88; “Video Game Music and the Triple Lock of Synch” in “The Triple Lock of Synchronization” in Tim Summers and Melanie Fritsch, eds., The Cambridge Handbook of Video Game Audio (Cambridge University Press, 2020), pp.94–109; “Doctor Who: The UK’s Darmstadt and IRCAM” in “Between Sublime Experimentation and Prosaic Functionalism: Music and Sound in Doctor Who” in David Butler, ed., Time and Relative Dissertations in Space: Critical Perspectives on Doctor Who (Manchester University Press, 2007), pp.190–203; “Experimental Music Television in the UK: Why Do I Need TV When I’ve Got T.Rex?” in “Experimental Music Video and Television” in Laura Mulvey and Jamie Sexton, eds., Experimental British Television (Manchester University Press, 2007), pp.166–179.

Notes 1. See sustained discussion in K.J.Donnelly, ed., The McGurk Universe: The Physiological and the Psychological in Audiovisual Culture (New York: Palgrave, 2022). 2. Michel Chion, Audio-Vision: Sound on Screen (New York: Columbia University Press, 1994). 3. David Alais and David Burr, “The Ventriloquist Effect Results from Near-Optimal Bimodal Integration” in Current Biology, vol.14, no.3, 2004, pp.257-262. 4. Chion, op.cit, 1994, pp.221,224. 5. In some cases, live sound and music persist from silent cinema, such as in concerts of film or video game music. Synchronization remains the key. 6. K.J.Donnelly, Occult Aesthetics: Synchronization in Sound Film (New York: Oxford University Press, 2013), pp.9–12. 7. Chion, op.cit., 1994, p.73. 8. Sergei Eisenstein, Vsevolod Pudovkin and Grigori Alexandrov, “Statement on Sound” in Richard Taylor, ed., and transl., S.M.Eisenstein: Selected Works, volume 1, Writings 1922–1934 (London: BFI, 1988); Sergei M.Eisenstein, ‘Synchronization of

1

9. 10.

11.

12. 13. 14.

15.

INTRODUCTION: CONCEIVING MUSIC’S RELATIONSHIP …

17

Senses’ from The Film Sense (New York: Faber and Faber, 1943), p.69. Rick Altman, The American Film Musical (London: BFI, 1987), p.11. Jeff Smith, “Bridging the Gap: Reconsidering the Border between Diegetic and Nondiegetic Music” in Music and the Moving Image, vol.2, no.1, 2009, p.1. Indeed, there has been something of an ongoing debate about issues to do with the distinction, evident in: Caryl Flinn, Strains of Utopia: Gender, Nostalgia, and Hollywood Film Music (Princeton, NJ: Princeton University Press, 1992); Guido Heldt, Music and Levels of Narration in Film: Steps across the Border (Bristol: Intellect, 2013); Anahid Kassabian, Hearing Film: Tracking Identifications in Contemporary Hollywood Film Music (London: Routledge, 2001); David Neumeyer, “Diegetic/Nondiegetic: A Theoretical Model” in Music and the Moving Image, vol.2, no.1, Spring 2009, pp.26–39; Nina Penner, “Rethinking the Diegetic/Nondiegetic Distinction in the Film Musical” in Music and the Moving Image, vol.10, no.3, Fall 2017, pp. 3–20; Robynn J. Stilwell, “The Fantastical Gap between Diegetic And Nondiegetic” in Daniel Goldmark, Lawrence Kramer, and Richard Leppert, eds., Beyond the Soundtrack: Representing Music in Cinema (Berkeley, CA.: University of California Press, 2007), pp.187–202; Ben Winters, “The Nondiegetic Fallacy: Film, Music and Narrative Space” in Music & Letters, vol.91, no.2, May 2010, pp. 224–244. Smith, op.cit, 2009, p.22. Harry McGurk and John MacDonald, “Hearing Lips and Seeing Voices” in Nature, no.264, 1976, pp.746–748. People with hearing difficulties who tend to partially lipread will also confirm that this phenomenon is not limited to audiovisual culture and yet it is particularly clear when rendered as sound and screen. Allison B.Sekuler, Robert W.Sekuler and Renee Lau, “Sound Alters Visual Motion Perception” in Nature, no.385, 1997, p.308. Allison B.Sekuler and Robert W.Sekuler, “Collision Between Moving Visual Targets: What Controls Alternative Ways of Seeing an Ambiguous Display? in Perception, no.28, 1999, pp.415–432.

18

K. J. DONNELLY

16. Other notable research into this includes: Katsumi Watanabe and Shinsuke Shimojo, “When Sound Affects Vision: Effects of Auditory Grouping on Visual Motion Perception” in Psychological Science, vol.12, no.2, March 2001, pp. 109–116; Massimo Grassi and Clara Casco, “Audiovisual Bounce-Inducing Effect: Attention Alone Does Not Explain Why the Discs are Bouncing” in Journal of the Experimental Psychology of Human Perception Performance, vol.35, no.1, February 2009, pp.235–43; Massimo Grassi and Clara Casco, “Audiovisual Bounce-Inducing Effect: When Sound Congruence Affects Grouping in Vision” in Attention Perception Psychophysics, vol.72, no.2, February 2010, pp.378–86; Marcello Maniglia, Massimo Grassi, Clara Casco and Gianluca Campana, “The Origin of the Audiovisual Bounce Inducing Effect: a TMS Study” in Neuropsychologia, vol.50, no.7, June 2012, pp.1478–82. 17. It worth emphasizing that while the McGurk Effect is based on speech, there is plenty of experimental evidence for audio changing visual perception, and vice versa. Another point worth noting is that while the McGurk Effect appears to be universal, I would be interested to understand more about whether it has a weaker effect in countries where dubbing foreign language films is common. While it would be reasonable to suggest it might be, the effect is not limited to audiovisual culture, although it is extremely clear here. 18. Chion, op.cit., 2021, p.220. 19. Theo van Leeuwen, Speech, Music, Sound (Basingstoke: Macmillan, 1999), p.15. 20. K.J.Donnelly, The Synergy of Music and Image in Audiovisual Culture: Half-Heard Sounds and Peripheral Visions (New York: Routledge, 2023), p.7. 21. David Bordwell, Janet Staiger and Kristin Thompson, The Classical Hollywood Cinema: Film Style and Mode of Production to 1960 (London: Routledge, 1985). 22. Although the regime of ‘silent cinema’ and its sound-image relations remained inside the Classical model, for instance in action sequences which were shot without sound and given loud musical accompaniment in post-production. 23. Claudia Gorbman, Unheard Melodies: Narrative Film Music (London: BFI, 1987), p.73.

1

INTRODUCTION: CONCEIVING MUSIC’S RELATIONSHIP …

19

24. Kathryn Kalinak, Settling the Score: Music and the Classical Hollywood Film (Madison, WI.: University of Wisconsin Press, 1992), pp.xv–xvi; Gorbman, op.cit., 1987, p.70. 25. David Bordwell, “Intensified Continuity: Visual Style in Contemporary American Film” in Film Quarterly, vol.55, no.3, Spring 2002, pp.16–28; Jeff Smith, “The Sound of Intensified Continuity” in John Richardson, Claudia Gorbman, and Carol Vernallis, eds., The Oxford Handbook of New Audiovisual Aesthetics (New York: Oxford University Press, 2013), pp.331–357. 26. Many scholars have discussed the notion of the ‘integrated soundtrack’, unifying music, sound effects and voices in an overall aesthetic (and potentially ‘musical’) ‘sound design’: Noel Burch, “On the Structural Use of Sound” in Elisabeth Weis and John Belton, Film Sound: Theory and Practice (New York: Columbia University Press, 1985), p.205; K.J.Donnelly, “Saw Heard: Musical Sound Design in Contemporary Cinema” in Warren Buckland, ed., Film Theory and Contemporary Hollywood Movies (London: Routledge, 2009), pp.104–107; David Neumeyer, Meaning and Interpretation of Music in Cinema (Bloomington: Indiana University Press, 2015), p.6; Liz Greene and Danijela Kulezic-Wilson, eds., The Palgrave Handbook of Sound Design and Music in Screen Media: Integrated Soundtracks (London: Palgrave, 2016); James Buhler, Theories of the Soundtrack (New York: Oxford University Press, 2019), p.260; Danijela Kulezic-Wilson, Sound Design is the New Score: Theory, Aesthetics, and Erotics of the Integrated Soundtrack (New York: Oxford University Press, 2020); James Buhler and Hannah Lewis, eds., Voicing the Cinema: Film Music and the Integrated Soundtrack (Urbana: University of Illinois Press, 2020). 27. Altman, op.cit., p.81. 28. Jim Collins, “Toward Defining a Matrix of the Muscial Comedy: The Place of the Spectator Within Textual Mechanisms” in Rick Altman, ed., Genre: The Musical (London: Routledge and Kegan Paul, 1981), p.138. 29. Michel Chion characteries television essentially as “illustrated radio” or “image Radio”. Op.cit., 1994, pp.157, 165.

20

K. J. DONNELLY

30. As Carol Vernallis Argues, Pop Video Should be Seen as a Distinct form Rather than a Derivative of Other Audiovisual Genres. Experiencing Music Video: Aesthetic and Cultural Context (New York: Columbia University Press, 2004), p.x. 31. Geoff King and Tanya Krzywinska, “Introduction: Cinema/ Videogames/Interface” in Geoff King and Tanya Krzywinska, eds., ScreenPlay: Cinema/Videogames/Interfaces (London: Wallflower, 2002), p.7. 32. Karen Collins, Game Sound: An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design (Cambridge, MA: MIT Press, 2008), pp.183–87. Tim Summers, Understanding Video Game Music (Cambridge: Cambridge University Press, 2018), pp.22–25. 33. This Technological Sea Change Arguably Meant an End to the Specificity of (and ‘Genre’ of) ‘video game music’. Rod Munday, “Music in Video Games,” in Jamie Sexton, ed., Music, Sound and Multimedia: From the Live to the Virtual (Edinburgh: Edinburgh University Press, 2007), p.51. 34. Summers, op.cit., 2018, p.143. See further discussion in Michiel Kamp and Mark Sweeney eds., “Musical Landscapes in Skyrim” in William Gibbons and Steven Reale, eds., Music in the RolePlaying Game: Heroes & Harmonies (New York: Routledge, 2019), pp.179–196; and Paul Martin, “The Pastoral and the Sublime in Elder Scrolls IV: Oblivion” in Game Studies, vol.11, no.3, December 2011. http://gamestudies.org/1103/articles/ martin [accessed 2/5/2022]. 35. Chion, op.cit., 2021, p.1. 36. Jeff Smith, op.cit., 2009, p.2. 37. Apparently, When Asked People Overwhelmingly Prioritize Seeing Over Hearing. Fabian Hutmacher, “Why is There so Much More Research on Vision Than on any Other Sensory Modality?” in Frontiers in Psychology, vol.10, 2019, p.2246.

CHAPTER 2

Occult Aesthetics

‘Occult’ does not necessarily mean ‘satanic’, pagan or in any way evil or negative; it merely means unapparent. The term can be used to describe any hidden workings or processes that are unable to be observed. For example, ‘occult bleeding’ is a medical term for bleeding that takes place out of sight. Often, the term ‘occult’ tends to have connotations of magic and the mystical, invoking the notion of mysterious ritualistic underpinnings of the everyday world. All these aspects are relevant for the discussion of the ‘occult aesthetics’ behind audiovisual culture. Aesthetics can work in mystical, magical, and unapparent manners, and the synchronization of sound and image in the cinema is an exemplary case in point. It is far from straightforward. Despite the impression of an unproblematic composite of sound and image, the relationship of the two is always more complex than it initially might seem. Horror films appear to know more about the occult of aesthetics than other films: They play around with this lynchpin of cinema far more than other genres. There is something potentially disturbing about the cinema’s illusion that has a determinedly supernatural character. Film could be understood as a conjuration: as a magical act of illusionism that dazzles the audience into believing it.1 This was a persistent notion in the early days of film. One of early cinema’s alleged effects was known as the ‘train effect’, where audiences feared physical impact upon seeing the Lumière brothers film © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025 K.J. Donnelly, Hearing Eyes, Seeing Ears, Palgrave Studies in Audio-Visual Culture, https://doi.org/10.1007/978-3-031-73913-2_2

21

22

K. J. DONNELLY

L’arrivéNewe d’un train en gare de la Ciotat (1895), specifically when the train on screen travels towards the camera.2 The illusionism of this phenomenon was burlesqued in Uncle Josh at the Moving Picture Show (1902), when a country rube visiting town gets up next to the screen in a cinema, thinking that he can interact with the uncanny shadows of people depicted. Although it may have been a cause for mirth, the illusion of film has an impact: We believe it on some level. In fact, to a degree, the film keeps us as children, constantly amazed and engrossed in its magic. Perhaps it is no surprise then that there has been a glut of ‘magic’ films recently that are almost as popular with adults as they are with children, such as the Harry Potter films, the Narnia films, The Golden Compass (2007) and The Lord of the Rings films. The success of the literary originals aside, some might argue that the degree of magic portrayed in films has risen with the capacity for computer animation, but this has always been an insistent strand of cinema, certainly since Georges MéNewlièNews’s startling conjuring tricks and wild special effects in films at the turn of the twentieth century.3 The ‘hidden’ aspect of all such magic is crucial: In his discussion about melodrama, Peter Brooks invokes what he calls the ‘moral occult,’ which . . . is not a metaphysical system, it is rather the repository of the fragmentary and desacralized remnants of sacred myth. It bears comparison to unconscious mind, for it is a sphere of being where our most basic desires and inner dictions lie, a realm which in quotidian existence may appear closed off from us, but which we must accede to since it is the realm of meaning and value. The melodramatic mode in large measure exists to locate and to articulate the moral occult.4

The weakening of religious belief in the wake of the Enlightenment and industrial revolution led to the seeming disappearance of religious representations of good and evil, which became hidden or partially masked by the surface of reality. Brooks argues that melodrama as a form engages this repository of ideas, and similarly, I argue here that there is a half-hidden ‘occult’ in the fundamental aesthetics of sound cinema: one that plays across representational and emotional concerns with basic perceptual movements between synchronization and asynchrony. The marriage of sound and image is a form of hidden rite, matching the occultist belief that clandestine rituals control and underpin the world while remaining hidden from the naïNewve and unsuspecting population.

2

OCCULT AESTHETICS

23

Furthermore, the secret marriage of sound and vision includes the fragmentary residuals of silent cinema, as well as the promise of wholeness. Tom Gunning suggests that recorded and synchronized sound cinema was likely a product of the desire to reunite hearing and vision, which had been divided by technology a few decades earlier, “... a desire to heal the breach”.5 Theory has something of the occult itself. When dealing with culture as intangible as the shadows of film and the fleeting squeaks of musical sounds from a speaker, theory can take on something of the form of divination, trying to find something not immediately apparent or obvious. Indeed, theory has a generative function in that, in its best form, it grapples with unknowables, making them into something more tangible—although never quite making them unproblematically understandable. In other words, theory can never make them into a simplistic reduction that passes itself off as guaranteed knowledge. Retaining a sense of the occult means realizing that hidden aspects might be the most important ones, and by their obscured nature, the most difficult to pin down. Despite its concealed formation, there was a determinedly ritualistic element to the synching up of sound and image in the early years of cinema—before it was achieved mechanically—that remains at the heart of cinema. To achieve convincing and immersive illusion, the ritual of synchronization must be completed to satisfaction. Yet similarly, theoretical investigation, as well as scientific experimentalism, involves clear, ritual aspects, including following certain strict and quasi-religious procedures to open the portal from the known to the unknown. As noted previously, there were some highly specific meanings ascribed to the term ‘synchronization’ at the time of the arrival of sound cinema, derived specifically from fitting music to a film. These had implications beyond simply making a unity of sound and image. Robert Spadoni notes: Synchronized voices are understood to accompany moving lips rather than to issue from them. This connotation helps to explain instances in which the word is applied, during the early sound period, to describe sloppy post-dubbing and even foreign speech that has been dubbed over English-speaking mouths. The sense of synchronization as something more provisional than essential accorded well with a viewer’s impression of synchronized speech as a marvelous mechanical gimmick. The novelty

24

K. J. DONNELLY

of the technique guaranteed that even if there had been no discernable problems with the synchronization, the quality of the sound, or the disparity between the locations of the loudspeakers and the speaking mouths, viewers still would have experienced a heightened awareness of the artificial nature of cinema as a direct result of sound.6

He points to the audience’s sensitivity to material aspects of the film medium at this historical juncture and suggests that this was exploited by films at the time, particularly horror films. According to Spadoni, the uncanny nature of film sound in transitional period cinema is translated into supernatural, seemingly monstrous bodies on screen, such as that of Frankenstein’s monster, Dracula, or the characters in Tod Browning’s Freaks (1932). Thus technology offered a potential for complex representation through the uncanny assemblage of sound and image. Rather than improving the illusion, Spadoni suggests that it initiated a form of reception interference or static, with an effect more intense than mere illusion, expressing an embodiment of the aesthetic disturbance afforded by the cultural possibilities of technological developments.7 The illusionism of cinema is doubtless a profound effect, but the potential disturbance invoked by asynchrony can also be an extreme effect, although different in nature. I would argue that there is a residual psychological disturbance to the divorce of sound and image in cinema, despite the conventional use of asynchrony for passages of films. There appears to be some evidence that early sound films unsettled audiences—who had a heightened awareness of the new formulation of cinema—as images appeared less real than silent, but more uncanny because sound was a strange new addition to the familiar images on the screen. In addition, David Toop points to the sonic at the core of the notion of the uncanny more generally.8 The heart of cinema then contains a potential disturbance at the centre of one of its most enduring and engaging aspects: the illusion of sound and image unity.

The Dynamic of Synchronization Sound film tends to routinely travel in the midst of moments of synchrony between sound and image, and points when there is no apparent synchronization. Approaching audiovisual culture from this more abstract perspective illuminates it in a form that removes the overly familiar aspects

2

OCCULT AESTHETICS

25

that have militated against sustained and detailed theorization of sound in films, and the notion of sound cinema more generally. Points of synchronization constitute something of a repose, a default position of normality that furnishes moments of comfort in a potentially threatening environment that can be overwrought with sound and image stimuli. Correspondingly, the lack of synchrony between sound and images has to be characterized as potentially disturbing for the audience, perhaps even as moments of aesthetic and representational danger. Indeed, McGurk and McDonald noted that asynchrony tends to prove upsetting.9 Thus, from this perspective, the interplay between the two becomes the central dynamic of audiovisual culture, and its objects can be reconceived and newly understood along these lines. Indeed, contemporary mainstream film is sometimes conceived as a movement from set piece to set piece, with filler material in between. (Indeed, the exigencies of film finance and production dictate that certain ‘featured’ sequences are nodes where the budget is concentrated.) We can rethink film, though, as a different form of temporal movement, between moments of synchronized repose and unsynchronized chaos. Films contain a large amount of asynchronous sound that we tend not to consciously notice or register: They aim to ensure that we do not linger on these moments. However, every film that has a synchronized soundtrack will evince this sort of forward development or movement. Rather than merely conceiving of this as an industrial process and a byproduct of the conventions of framing, recording and post-production, I mean to engage this as something potentially more profound. It can be approached as an abstract, unconscious and aesthetic drama, where film might play out momentary and instinctual understandings of and responses to the world. Within this system, precise synchronization and complete asynchrony represent different extremes of film and extremes of experience for the viewer/ auditor. Asynchrony (non-synchronization), or at least an uncertain relationship of synchronization between images and sounds, renders the audience uncertain, making them uneasy or anxious. At the opposing pole, (absolute) synchronization suggests to us, or dramatizes for us, a situation where all is well with the world: Everything is in its rightful place. Ambiguity about synchronization (or a total lack of it) is potentially unsettling. At the very least, it is a different ‘mode’ from synchronized ‘normality’ on screen. Scientific insights and biological determinism might help to understand how synchronization appears to serve films. Human beings likely react to the discontinuity between what is seen and what

26

K. J. DONNELLY

is heard on an unconscious or preconscious level. According to developments in neuroscience and sociobiology, we possess physical, hardwired processes whereby we are informed about the space we occupy through a combination of the senses. A disparity between visual perception of a space and its apparently attached sound (or vice versa) will have an instant effect. Such unprocessed reactions likely set in process an unconscious unease or dissatisfaction. This mechanism may have evolved for human defence and might inform us, for example, that a dark ravine with no echoes might contain a large, unseen predator. Such biological concerns about sound perception and its place in our survival were doubtless transposed into cinema and exploited by that most sensual of audiovisual media. Since the advent of 5.1 Surround Sound cinema, soundtracks have spatialized their elements as never before. Features such as the in-the-wings sound effect still can, if used in an unsubtle and crass manner, make us partially turn our heads, forcing an involuntary physical reaction to sound. It is worth remembering that in psychobiological terms, sounds that emanate from anywhere except directly in front of us, initially at least, are perceived as a potential threat. Indeed, as these points attest, there is something absolutely primal about the synchronization of sound and image. Clearly, the senses of hearing and seeing are not totally separated. The cross-referencing of the two, making for a seamless continuum of perception, would have to be taken as the normality of human physicality. The exigencies of the human body clearly are partially activated and altered in significant ways by the cinema. I suspect that moments of synchronization between sound and image provide feelings of coalescence, joining up and ultimately integration. Integration on an aesthetic level homologizes feelings of integration on a level of well-being and ultimately social integration. Following Adorno’s approach to culture as ‘concentrated social substance’,10 I would suggest that the abstract play of synchronization in films (indirectly at least) mirrors the social and psychological processes of understanding our place in the world and our perception of risk in modern life. Film manipulates audience emotions and desires. Gestalt psychology concerns similar ‘vectors’ of energy, and film appears to marshal audience energy in its movement from tension to repose, sometimes very crudely.11 Similarly, the movement between asynchrony and synchronization forms vectors of emotional and psychological dynamics. These ideas are evident in some musical theory. For example, music theorist Leonard B. Meyer notes that

2

OCCULT AESTHETICS

27

Both music and life are experienced as dynamic processes of growth and decay, activity and rest, tension and release… Emotional behavior is a kind of composite gesture, a motion whose peculiar qualities are largely defined in terms of energy, direction, tension, continuity, and so forth. Since music also involves motions differentiated by the same qualities, ‘musical mood gestures’ may be similar to behavioral mood gestures. In fact, because moods and sentiments attain their most precise articulation through vocal inflection, it is possible for music to imitate the sounds of emotional behavior with some precision. Finally, since motor behavior plays a considerable role in both designative emotional behavior and in musical experience, a similarity between the motor behavior of designative gestures and that of musical gestures will inforce [sic] the feeling of similarity between the two types of experience.12

The analytical dominance of a film model as being narrative or representational in essence has marginalized approaches that attempt to think of film as dynamic movement or emotional cathexis, which usually describes an investment of emotional or mental energy in something (or a form of channelling psychological and emotional energy). Of course, certain types of film have been founded very explicitly upon the direction of emotional energy and manipulation of audience anxieties and desires. Synchronization is only one of the ways that energy is cathected, while there are a plethora of dynamic devices that regulate tension and release in arousal and emotional terms, alongside narrative revelations and cathartic outlets of extreme excitement, etc. In an isolated article that discusses synchronization, John Belton noted: This perceptual process of testing or attempting to identify sound can, through a system of delays that postpone the synchronism of sound and source, be manipulated to create suspense, both in the area of voice/ dialog and in that of sound effects, calling attention to sound as a device by playing with our perception of it. The identification of a voice with a body can be delayed, as in the case . . . of The Wizard of Oz (1939), in which the Wizard’s unmasking occurs at the precise moment that synchronization is established; the achievement of synchronization creates a unity whose completeness spells the end of a hermeneutic chain within which an enigma is introduced, developed, prolonged and resolved. Or in the more complex case of Psycho (1960), in which off-screen sound is employed to create a nonexistent character (Mrs. Bates), the particular revelation of the sound’s source carefully avoids synchronism: we never see Bates speak in his mother’s voice; even at the end, his/her request for a blanket comes

28

K. J. DONNELLY

from off-screen and his/her final monologue is interiorized. Image and sound here produce a tenuous, almost schizophrenic ‘synchronization’ of character and voice, which precisely articulates the fragmented nature of the enigma’s ‘resolution’ and completes an ‘incompletable’ narrative.13

Belton reaffirms the importance of the moment of synchronization as a point of stability in the film’s development but significantly also in the film’s dynamic landscape. This ‘pull’ is so strong that we not only search for that unity but also desire it deeply in its absence. Consequently, we often imagine that simultaneity equates with causality; in other words, sound appearing at the same time as image is often understood as a single event. This is more the case in cinema than in the real world, where visual and sonic aspects have limited characteristics of quality, space, movement and conventional relationships. An example that occurs on television sometimes is when a news report with a voiceover is accompanied by an image of a person on-screen speaking. We have a desire to match the two into a unity. This is further illustrated by the confusion caused by the difference in speed between sound and light: a gunshot that is heard after someone is hit by a bullet; jet planes that fly by dragging their roars behind them. The drifting apart of the components of single events is an ontological disruption and a cognitive disturbance. This is why a confusing confluence of sound and image at the same time can short-circuit our perception momentarily. This principle of simultaneity often suggests that a causal connection might easily be related to the Gestalt principles of proximity, whereby objects that are close together are construed as belonging together; continuity, whereby elements that appear to create a united form are perceived as intimately connected or common fate, whereby objects seemingly moving in the same pace and direction are perceived as unified.

Synchronizing Soft Machine(s) The physical hardware of human beings is of paramount importance for understanding how sounds and images are perceived, despite being largely ignored by film theory. After all, concepts do not merely materialize in consciousness from nowhere. The grounding for all understandings is the physical limitations set by the human body, and these are exploited explicitly by sound and image culture. The mechanism of the human ear involves a succession of physical processes initiated by sound striking the

2

OCCULT AESTHETICS

29

eardrum before a neural signal is sent to the brain. Sound waves vibrate the eardrum, which articulates a mechanism of small bones that impart movement impulses to the cochlea, the spiral-shaped canal of the inner ear. The eighth cranial nerve runs from the cochlea directly to the cerebral cortex.14 Human hearing conventionally has a range from 20 Hz to 20 kHz (this is ten octaves, although it decreases with age), and frequencies below 20 Hz are not processed by the ear but can be felt by the body. We would do well to remember the physical basis of our dealings with the outside world, particularly in the case of sound, which actually imparts a physical impact upon us: We are touched and vibrated by sound waves. Vision, of course, is no less mechanical in its operation. Yet despite the regular assertion of film as a visual medium, it seems conventional to approach it as a cerebral medium, one that appears almost ready-formed in our consciousness. Neuroscience supports the notion that the brain functions as a parallel processor, with different brain locations specializing but often dealing with matters simultaneously rather than in a simple modular manner where certain parts of the brain deal exclusively with specialized functions.15 Although scientists increasingly know more about the brain, its processes are still beyond detailed and definitive account. Sometimes understanding may be forced into a déNewjàNew vu, splitting the signal in the brain with a delay in reception and processing, particularly as a discrepancy exists between the speeds of aural and visual perception and processing. This lapse could be compounded by the physical makeup of the brain as a parallel processing device that channels impulses to different regions and works on them simultaneously. Asynchrony in sound and image might lead to a divided and fragmented state approaching schizophrenia. This schizoid state is embodied by the aesthetic fact of a division between the unsynched sound (i.e., non-diegetic music) and the co-temporal image (the illusion of people talking on screen in two-dimensional space). Brains appear to be able to go ‘out of synch’. Normally, the left temporal lobe receives messages from the right, but if the parietal regions of the brain (which have a spatial role) go quiet, the frontal lobe (often thought of as the ‘executive’) can think that the body has disappeared.16 Asynchrony between the two hemispheres of the brain can impart the impression that something strange is happening, and bizarre occurrences such as out-of-body experiences can take place.17 Such hemispheric asynchrony can divide and confuse broad cerebral functions. Many functions

30

K. J. DONNELLY

and activities in the brain involve more than one location. Broadly speaking, however, emotion and music processing take place predominantly in the right hemisphere of the brain (the ‘creative’ side that is associated with the left ear). Indeed, there is a surprisingly strong division of brain activities in different hemispheres of the brain. While the left hemisphere is predominantly ‘verbal’ and concerned with language and logical thought, the right hemisphere is predominantly ‘nonverbal’ and deals with more emotional matters.18 Music is mostly (but not exclusively) processed by the left hemisphere, unless it is essentially a vehicle for words.19 The human brain seeks and finds patterns, and this has clearly had a central determining role on structures and formations in art and wider culture. A similar perceptual process to the ‘cocktail party effect’ (named after the fact that people can hear their own name mentioned across a noisy room at a party) called psychoacoustic masking is utilized regularly and unconsciously by people to focus on certain sonic elements in a complex soundscape, while managing to ignore other sounds to the point of their not registering. This ability also enables us to phase out crackly noise on an old record or poorly tuned radio and listen to the music as if it is not obscured, or for people who live next to a noisy road to not register the background drone of traffic. This is part of essential pattern-seeking mechanisms in the human brain that are easily best illustrated with reference to sonic phenomena. The brain appears to use a correlative process for recognizing sonic patterns in a similar manner to how electrical circuits look for signal patterns.20 Schemata are held up to the complex of sounds looking for a fit. This can lead to a socalled psychoacoustic phantom effect, where we hear what is not there through over-concentration on expectations of hearing certain sounds. A low threshold of acceptance will lead to a finding of the patterns too easily and in places where they might not exist. The brain can fill in the missing parts of the pattern, although white noise allows for elements to be singled out into a distinct pattern where one might not exist independently. There is a similar process with vision, which can take place in situations of very low light. Heightened states of perception (as a result of anxiety, for example) can lead to this effect, such as in situations of potential danger or in houses that are supposedly haunted.21 The term ‘pareidolia’ describes the phenomenon of the brain discovering patterns where none exists, or the perception of significant meaning or a pattern in something ambiguous: in other words, the act of finding sense in the

2

OCCULT AESTHETICS

31

senseless.22 A more extreme form is ‘apophenia’, originally a psychotic condition of finding unapparent connections and significance in things where meaningful patterns actually do not exist. This is particular to images and sounds.23 In perceptual terms, this involves a vague stimulus being perceived as distinct, often seeing animate objects in the inanimate. Examples could be a cloud being construed as a human form and the famous Rorschach blot test, which is projective of the patient’s personality and state of mind as the blots are ambiguous but standardized.24 Yet such pattern recognition is of course also responsible for extreme detective work and scientific breakthroughs as much as it is for illusions and delusions. In some cases, sound can be interpreted more readily for patterns than visual aspects. For example, ‘phantom fundamentals’ are low-pitched sounds heard where none is present. In this situation, the brain interprets patterns from differences in present sounds’ audible harmonics and ‘creates’ the impression of a lower note.25 If pareidolia suggests a misinterpretation of stimuli, it goes some way towards explaining the illusion of sound and image as unified and living on screen.26 The McGurk Effect is concerned with the matching of phonemes (basic units of speech sound) and visemes (basic units of speech in visual terms). Perception of speaking involves both sound and visual cues, rather than simply the former as many might expect. This phenomenon demonstrates that there is not a solid perceptual-cognitive division of sound and image. Chion describes the process of synchresis as a phenomenon of almost gravitational ‘pulling’ of sound and image together.27 An associated form or mode of psychology is attached to the synchronization of sound and image. The illusion of unity is enough to engender a sense of reality on some level. This is not quite the same as outside the cinema, yet it has its own situation, rules and beliefs. Such organization, according to Wolfgang KöNewhler, “... refers to the fact that sensory fields have in a way their own social psychology”.28 Such Gestalt psychology confirms (rather than is inspired by Kantian idealism), which suggests that we do not perceive the world as it is but impose a sense of cause-effect relationships on it that are derived from something approaching ideal forms inside our head. Aesthetics have solid psychological implications. Indeed, one might imagine that it is their heart. Moreover, such forms might have materiality as proformas or templates, and these can sometimes become confused and misapplied. Being ‘out of synch’ is arguably a mental state. In essence, a similar malady relates to

32

K. J. DONNELLY

sound. ‘Schizophonia’ describes the splitting of sounds from their original contexts. The founder of acoustic ecology, R. Murray Schafer, is the originator of the term: I coined the term schizophonia in The New Soundscape intending it to be a nervous word. Related to schizophrenia, I wanted it to convey the same sense of aberration and drama. Indeed, the overkill of hi-fi gadgetry not only contributes generously to the low-fi problem, but it creates a synthetic soundscape in which natural sounds are becoming increasingly unnatural while machine-made substitutes are providing the operative signals directing modern life.29

Schafer makes many relevant points. His notion relates directly to technology and its ability to foster a disjunction of spatial perception. This state is a little like permanent existence in an uncertain echoing environment, where the precise origins of sounds are not immediately apparent and are difficult to pinpoint.30 Such a sense of uncertainty about the source of a sound can be a cause for anxiety. Acoustic ecology’s answer to current stressful environments is to limit sounds without a clear source sound or to emphasize sound that is continuous and fairly predictable, so that we know where we are with it. The same applies to other areas of culture: Music that aims to promote relaxation habitually lacks any startling sound events or surprising elements. There should be no sounds that demand us to enquire of their origins. As we might expect, music that targets the opposite effect will often contain precisely what is lacking in such relaxing music. Cinema also exploits these characteristics. A startling sequence in David Lynch’s Lost Highway (1997) involves the spectral ‘mystery man’ (as the film’s credits call him) using a mobile telephone. Here, despite standing next to him, this obscure character, played by Robert Blake with a ghostly whitened face, informs the film’s protagonist Fred (Bill Pullman) that he is at Fred’s home ‘right now’. He entreats Fred to call and speak to him on a mobile phone and verify this statement. Fred calls and then speaks to the very man who nevertheless appears to be standing opposite him. The film might have suggested to us that it was some sort of ventriloquist trick but instead plays it seriously. The audience is left in little doubt that there are some bizarre, supernatural machinations afoot. The mystery man’s entrance and exit are marked by a sound dissolve, whereby the diegetic music at the party is removed to leave silence as the backdrop to his conversation with Fred. On one level,

2

OCCULT AESTHETICS

33

this is telegraphing the extraordinary narrative situation, but on another level, this sequence is emphasizing the artificiality of the illusion created by sound and image synchronization. Fred’s reaction to the mystery man’s division of sound and image is one of shock. Such a rupture of asynchronous division within a film premised upon conventional sound and image relations inaugurates something of a trauma. Indeed, it is through the undermining of the established audiovisual normality in a film that such a contrast proves traumatic. At least partly, this extreme example under discussion works precisely as a rupture to audiovisual naturalism, the unremarkable sound and image relations that the audience grasps as normal and does not register as an organized discourse. Asynchrony more generally can mark a degree of psychological trauma in films through its negation of the contract of illusory unity in sound and image relations. The clinical state of psychological trauma is damage to the psyche, often occurring when an event overwhelms an individual’s ability to deal with the emotions, ideas and memories of that experience. It might be a horrifying experience or a violation of familiar ideas about the way the world works that leads to confusion and anxiety. Indeed, dangers may always be perceived as potentially present even when they are not. So, trauma can range from the extreme to the mild, with causes that also vary correspondingly.31 Trauma can invoke feelings of low self-esteem, depression and possibly even a sense of loss of identity. People suffering from trauma need to avoid triggers or stressors. If we transpose this psychological condition to film perception, we might imagine that audiences could crave the normality of mainstream cinema while finding overtly challenging aesthetics or difficult avant-garde films as something to avoid as stress producers. Indeed, it is reasonable to suggest that there is much about mainstream cinema that provides a sense of reassurance for audiences, who know where they are with the conventional regime of mainstream representational films.32 Perception and its limitations can inspire anxiety and agitated mental states. For instance, there are highly disturbing states such as the neuropsychological disorder akinesthesia (sometimes called akinetopsia), which is the inability to perceive movement, in which the sufferer perceives instead a series of still images. This motion blindness is a rare condition, often arising from brain damage. Another perceptual-cognitive state of anxiety appears far more commonly: Cognitive dissonance involves holding two seemingly contradictory ideas simultaneously. This leads to anxiety and stress (or other negative emotional states). Normally, ideas held simultaneously by

34

K. J. DONNELLY

an individual are considered consistent and thus ‘consonant’ with one another, and moments of dissonance short-circuit the functioning system to a greater or lesser degree. Lionel Festinger’s original theory from the late 1950s stated that cognitive dissonance occurs when two cognitions that are absolutely inconsistent with one another are held by an individual.33 This position is a highly anxious mental state to occupy, and the mind’s desire is to urgently attempt to negate one of the notions as soon as possible. We have a drive to ameliorate such cognitive dissonance and thus limit the amount of mental stress to which we are subject. Processes to reduce such cognitive dissonance include ego defence mechanisms such as rationalization. Indeed, self-justification procedures become central in allowing a cohabitation of such incommensurable concepts.34 The socalled double bind is a situation in which a subject receives different and contradictory messages or impulses, and this untenable situation of cognitive dissonance often produces a state of physical arousal, a tangible state of anxiety, where the individual is visibly upset or disconcerted. Similarly, motion sickness can occur when the brain receives conflicting messages from the eyes and ears about acceleration. Ears detect only changes in acceleration, rather than movement itself—totally failing to notice constant movement. If a train journey is not smooth, focusing on the page of a book during acceleration leads to the brain deciding that the disjunction between sonic and visual information is illness of some sort, triggering a vomiting reflex. States of arousal, as psychologists term heightened awareness and physical symptoms of anxiety or excitement, have become an essential component of certain leisure activities. The proximity of disturbed and aroused states can also be exploited. For instance, amusement park rides are founded upon the concept of anxiety as stimulating and enjoyable. Despite taking place in a controlled situation and environment, there is the threat of physical danger, which is at the heart of the extremity of experience and enjoyment derived from that. Films also offer this experience on a number of levels, so it would be no surprise if anxiety derived from cinematic aesthetics was not also, in its own way, enjoyable and exciting while also being physically and emotionally anxious. The denial of sound and image synchronization at crucial points in films manifests something similar to a momentary episode of cognitive dissonance. The lack of synchronization potentially offers a disunified communication to the film audience. Though in some cases, this is not consciously exploited by filmmakers, in others, it clearly inaugurates a

2

OCCULT AESTHETICS

35

short-term policy for the purposes of cinematic impact. An illustration of sound and image disjunction leading to cognitive dissonance is evident in Michelangelo Antonioni’s The Passenger (Professione Reporter, 1975). There is an astonishing sequence in which the reporter (Jack Nicholson) is assuming the identity of the man he met the previous day who has now suddenly died. As he pastes his own picture into the dead man’s passport, the soundtrack consists of a conversation between the reporter and the dead man that is temporally unconnected to what we see. After some minutes, the camera focuses on a reel-to-reel tape, revealing that we are listening to a conversation that was recorded the previous night. If we think of this sequence in terms of a classical sound counterpoint, the key is in the space between the meaning of the soundtrack and image track and their seeming temporal dislocation. However, thinking of it in more abstract terms, the key moment becomes the double-take of the sound and image at the point when we realize that we are listening to a tape. Indeed, this is a very dramatic moment, but it is also an imperative instant in structural and perceptual terms (Fig. 2.1). The initial audience reaction is likely to imagine that the sequence is taking place in the character’s head. We might think this for a while, but the sequence sustains and proceeds beyond the conventional for subjective sequences, and the expected return to normality fails to appear. Of

Fig. 2.1 The Passenger

36

K. J. DONNELLY

course, the return to film style normality does appear after a long period, but with the retrospective motivation of the uncoupled soundtrack as an audiotape running. The shot of the reel-to-reel tape machine is the visual answer to the question that has been posed by the film’s asynchrony. The ambiguity of sound and image relation asks to be addressed by the audience and retains their mental state in a split frame of mind, holding what we expect to be a unity apart and short-circuiting cognition until this dissonance is resolved. An ostensibly mainstream narrative film, Point Blank (1967), directed by British director John Schlesinger, effects a chaotic sound and image style. The film has an elusive narrative about the seeming return from the dead of Walker (Lee Marvin) to avenge himself on his former partner and wife, both of whom double-crossed him. As we might expect from a film premised upon disconnection, embracing narrative ambiguity, dream sequences, flashbacks and subjective images, there is an intermittent but insistent appearance of asynchrony. At the film’s beginning, Mal Reese is talking to the film’s protagonist Walker, telling him about the situation at a prison. As his voice continues, the image track transforms to Reese literally showing Walker around the prison itself. During this shot, Reese is gesticulating and clearly addressing Walker, but this is not synchronized as a voiceover continues. This opening section contains a number of incoherencies if the film is approached in traditional terms, including temporal confusion and repetition. At the point when Walker is approaching his wife’s (and Reese’s) house, the sequence begins with Walker in a medium shot walking purposefully towards the camera with loud and echoed footsteps making an insistent beat. This sound continues as the image track moves though a succession of shots lacking any synchronized sound, with the footstep beat persisting relentlessly, until the point when Walker smashes through the front door of the house and grabs his wife in a simultaneous blast of sound and movement. This is the moment when synchronization returns, as the lead-up to this has contained images and spaces that clearly were not related to the sounds of the footsteps, apart from in a symbolic sense of determined advance upon his wife. Point Blank begins in a most confusing manner, following a poetic path, but one that is intent on setting up a confused state in the audience. A less immediately challenging beginning to a film (but one with a similarly problematized synchrony) is Sunset Boulevard (1950). It begins and concludes with a voiceover from a dead man, whom we see floating inertly in a swimming pool. This is a curious and rare effect that appears to

2

OCCULT AESTHETICS

37

confound or challenge cinema’s sense of credible illusion. Sunset Boulevard does not emphasize the impression or indeed ask the audience to work it out, but rather just uses it as a mildly troubling frame for the rest of the film’s action. Such asynchrony can sometimes act like Brechtian alienation (Verfremdungseffekt ). It has the ability to alienate the audience from the comfort of the illusion of a habitually coherent world of sound and image on screen. This strategy usually aims to make audiences momentarily ‘stand outside’ the film and take a critical distance from the film as an illusion. It is premised upon assumptions about audiences and their expectations of cinematic convention, meaning that it is largely only effective as an occasional device in the context of illusionistic and immersive film style. If this strategy is used in avant-garde films, it is often immaterial as audiences suspend the expectations of mainstream film conventions and often replace them with others more particular to avant-garde cinema. Some of the French New Wave films exploited both the engaging illusion afforded by film and the potential for undermining it for aesthetic and political reasons. In Jean-Luc Godard’s Weekend (1967), for example, there is a battle sequence that consists of a diorama of model soldiers and planes that is accompanied by real war sounds. This self-conscious disjunction of sound and image wields the realistic sounds of war to alienate any emotional immersion and destroy any illusion of the models as representative of war. This approach appears less poetic and more parodic. Perhaps on one level, it is even poking fun at the film convention of using model shots in large-scale battle scenes. Whatever its actual aim, it alienates the audience from the illusion of the world on screen as a reality to which they are unseen perceivers, causing a disconcerting effect.35 In recent years, such stylistic approaches have become more widely utilized as a specific artistic technique in films. The British biographical film of the celebrated eponymous violent prison inmate, Bronson (2009), uses a variety of sound and image dislocation techniques to tell a complex and static story in an engaging but semiotically rich manner. The film opens with a stylized shot showing Bronson (Tom Hardy) facing the camera in a medium shot. He addresses the film audience: “I’m Charlie Bronson”. The shot cuts to a reverse shot that is immediately behind him, then to a different medium shot (with a different camera angle and different lighting). His voice retains the same acoustic quality, but now it becomes a voiceover, no longer synchronized as his mouth remains closed. The shot returns to the original frontal medium shot with the

38

K. J. DONNELLY

voice synchronized again. Accompanying this shot is some minimal nondiegetic music consisting of a single tone of deep bass to which is added a high string tone. As this proceeds, the image cuts to a shot of Bronson naked behind the bars of a cell doing push-ups, lit with an unlikely and stylized red light. Singing begins on the soundtrack at the precise moment when an image appears of a warden opening a door in long shot. An immediately ensuing shot of fighting in a cell is accompanied by a lack of any diegetic sound but the song persists. The image track then cuts to the original medium shot of Bronson as he laughs (with synchronized diegetic sound having returned), and the film’s title is superimposed beneath his face. This sequence revels in its visual discontinuity, but in terms of sound, the song continues throughout much of it while synchronized sound alternates between matching unity and dislocation. Sonic continuity is often the foundation for visual discontinuity, emphasizing the semiotic inter-reliance but aesthetic divergence of the two tracks. Extreme forms of disjunction are highly disorienting, particularly when there is sustained sound and image independence. Here, synch points become much more important, as something to hold onto in a wash of perceptual ambiguity. A more extreme example is evident in Gus Van Sant’s controversial film Elephant (2003), which was based on the events of the Columbine High School massacre. The film uses some sound art/soundscape recordings to startling effect. Hildegard Westerkamp’s recordings TüNewren der Wahrnehmung (‘Doors of Perception’) (1989) and Beneath the Forest Floor (1992)36 and Frances White’s Walk Through Resonant Landscape #2 (1992) appear in the film without adaptation, although at times with some diegetic sound appearing simultaneously. These are not music in the traditional sense, but more assemblages of sound field recordings in a structured whole. At one point in Elephant, student Nathan (Nathan Tyson) walks to meet his girlfriend, traversing a sports field, into school corridors, out into a yard in between and then back inside the school to meet her. The sequence begins with diegetic sounds from the sports field, as well as what appears to be the distant non-diegetic sound of Beethoven’s Moonlight Sonata played on a piano. At the point of the cut when Nathan crosses the threshold to enter the building, there is a radical shift in the film soundtrack. While the nondiegetic Beethoven remains, instead of different ambient diegetic sounds for the school interior, the Westerkamp piece ensues. It begins with the sound of a choir that could almost be diegetic sounds but then progresses into the body of the piece, which incorporates most notably many sounds

2

OCCULT AESTHETICS

39

of doors opening and closing along with other field recordings that clearly have no connection with what the audience is seeing on screen. The expectations of diegetic sounds are confounded as events happen silently until a degree of diegetic sound reappears later in the sequence. Yet the sounds on the soundtrack are clearly not music in the conventional sense, nor are they existentially connected to any origins on screen. This sets up a sense of dislocation through the dichotomy of representational images and unrelated sounds as a replacement for those that we might imagine as ‘belonging’ to the images. When Nathan is in the courtyard, Westerkamp’s piece presents the expansive soundscape of a train station, which, although its appearance matches the change in space back to exterior, the expected change in diegetic sound is instead replaced with a radical disconnection. Some of the sounds are almost fitting, but never quite, while much of the sound confounds expectations about sound film synchronization conventions. As Nathan opens the door back into the school’s interior, the sound of the door is synchronized with the sound of a door in Westerkamp’s piece, returning a sense of synchronized unity of sound and vision and restoring the illusory effect of narrative film.37 This is an example of ‘musical’ sound in film, with what might be construed as film sound effects ceasing to be mere sound emanations from the world on screen and becoming a musical aspect of the whole. It makes plain the artificiality of the welding of sound to image as an illusionary unity but also emphasizes the aesthetic interplay of elements evident in most films.

Conclusion In their initial article about the McGurk Effect, McGurk and McDonald note that asynchrony tends to be upsetting.38 Asynchrony has the potential to disturb, whether it is through sustained loss of the synchronized norm or through momentary anomalies. For instance, in a lengthy sequence from the Australian thriller Black Water (2008), a woman walks through trees in a swamp accompanied by echoed and rather tuneless music and well-defined sounds of her footfalls and grasping of trees. As she reaches the end of the trees, we hear isolated sounds of water gurgling—suggesting that there might be something there—but the water in the shot appears calm. This minor disjunction causes some disquiet, but it is not blatantly ‘telegraphed’ so it is not immediately evident to the audience, perhaps merely setting up an unconscious anxiety.

40

K. J. DONNELLY

Alternation between asynchrony and synchrony is a common strategy, although it rarely registers as a specific device of audiovisual culture. At times, it marks a fundamental dynamic movement that functions largely by its processes remaining unconscious rather than on the surface of the film. There is a significant difference between not having precise synch points and clear asynchrony as a policy. Asynchrony as a policy is evident in many avant-garde films and in a fair number of music videos. However, in narrative films, there is a strong tradition of movement between clear synchronization of sound and image and situations when the relationship between the two is more equivocal. While Eisenstein stressed the montage potential of asynchrony, where it can open up the possibility of different meanings, I am more concerned with a lack of synchronization as a potential breach in the fabric of illusion, where ambiguity might disrupt the immersion of cinema’s unity of sound and image. We cannot assume that a lack of synchronized sound and image is ‘radical aesthetics’ as the alternation between precise moments of synchronized sound and image, and degrees of movement away from this, is endemic in audiovisual culture. This alternation augurs normality, with some films and video games playing around with latent psychological disturbance. So, while conventional, this is a step beyond the illusionistic processes of audiovisual culture and constitutes a dramatic movement by stylistic aspects only, through the crucial but unacknowledged ‘occult’ lynchpin upon which it is reliant.

Notes 1. See, for Example, Tom Ruffles, Ghost Images: Cinema of the Afterlife (Jefferson, NC: McFarland, 2004). 2. Stephen Bottomore, “The Panicking Audience? Early Cinema and the ‘Train Effect’” in Historical Journal of Film, Radio and Television, vol. 19, no.2, 1999, p.177. 3. AndréNew Gaudreault, “Theatricality, Narrativity and ‘Trickality’: Re-Evaluating the Cinema of Georges MéNewlièNews” in Journal of Popular Film and Television, vol. 15, no.3, 1987, pp.110–119. 4. Peter Brooks, The Melodramatic Imagination: Balzac, Henry James, Melodrama, and the Mode of Excess (New Haven, CT: Yale University Press, 1995), p.5. 5. Tom Gunning, “Doing for the Eye What the Phonograph Does for the Ear” in Richard Abel and Rick Altman, eds., The Sounds of

2

6.

7. 8. 9. 10.

11. 12. 13.

14.

15.

16.

OCCULT AESTHETICS

41

Early Cinema (Bloomington, IN: Indiana University Press, 2001), p.16. Robert Spadoni, Uncanny Bodies: The Coming of Sound Film and the Origins of the Horror Genre (Berkeley, CA: University of California Press, 2007), p.17. Ibid., p.30. David Toop, Sinister Resonance: The Mediumship of the Listener (London: Continuum, 2010), pp.126–127. Harry McGurk and John W. Macdonald, “Hearing Lips and Seeing Voices” in Nature, no. 264, 1976, p.746. Andy Hamilton, “Adorno” in Theodore Gracyk and Andrew Kania, eds., The Routledge Companion to Philosophy and Music (New York: Routledge, 2011), p.393. Rudolf Arnheim, Visual Thinking (Berkeley, CA: University of California Press, 2004), p.139. Leonard B. Meyer, Emotion and Meaning in Music (Chicago: University of Chicago Press, 1961), pp.261, 268. John Belton, “Technology and Aesthetics of Film Sound” in Elisabeth Weis and John Belton, eds., Film Sound: Theory and Practice (New York: Columbia University Press, 1985), p.65. The human ear consists of three chambers. Sound enters from the ear flap (the pinna or auricle), follows into the outer ear, a cavity that leads to the tympanic membrane or eardrum, which vibrates as the sound wave hits it. Beyond the ear drum, the middle ear contains air (and thus becomes pressurized during flight). This contains three bones: the stapes (stirrup), the incus (anvil) and the malleus (hammer). The vibrating eardrum causes movement in the malleus, which in turn moves the other two bones, with the stapes pushing the oval window, which causes movement of the liquid in the inner ear. The liquid-filled cochlea is the central organ of hearing in the inner ear and is sensitive to the effects of motion and gravity, affecting the sense of balance. Here, movements in the liquid are converted by sensitive hairs on the surface into an electrical impulse, which is then fed into the neurological system. Mariano Sigman and Stanlislas Dehaene, “Brain Mechanisms of Serial and Parallel Processing During Dual-Task Performance” in Journal of Neuroscience, vol. 28, no. 30, July 2008, p.7585. The primary sensory areas of the brain include areas of the upper brain, primarily the auditory section of the temporal lobe and

42

K. J. DONNELLY

17.

18. 19. 20.

21.

22. 23. 24.

25.

26. 27. 28. 29. 30.

insular cortex, the somatosensory section of the parietal lobe and the visual section of the occipital lobe. The association areas of the cortex are involved in primary perception and also more complex thought. Indeed, activities in the parietal lobe are often associated with meditative states. Cinema arguably might take on some of the characteristics of meditation and offers something perhaps related to an out-of-body experience. Vernon B. Mountcastle, Medical Physiology (St. Louis, MO: C. V. Mosby, 1974), p.579. Anthony Storr, Music and the Mind (London: Harper Collins, 1997), p.35. Jennifer A. McMahon, “Perceptual Constraints and Perceptual Schemata: The Possibility of Perceptual Style” in Journal of Aesthetics and Art Criticism, vol. 61, no.3, Summer 2003. Michael Shermer, The Believing Brain: From Ghosts and Gods to Politics and Conspiracies—How We Construct Beliefs and Reinforce Them as Truths (London: Times Books, 2011). Also see Daniel Dennett, Consciousness Explained (Boston: Back Bay Books, 1992). Graham Reed, The Psychology of Anomalous Experience: A Cognitive Approach (Buffalo, NY: Prometheus, 1988). This human tendency towards ‘patternicity’ is discussed in detail by Shermer, op. cit. The shapes are without structure, which is provided by the patient’s mind, illustrating unconscious concerns. The test’s scientific status is not agreed upon. This phenomenon can be taken advantage of by audio equipment that wishes to provide lower notes than the system appears capable of delivering. Particularly so, given that auditorium sound is not necessarily even originating from the same direction in which the audience is facing. The notion of synchresis is an attempt to make a theoretical replacement for the illusory sense of ‘reality’ found in cinema. Wolfgang KöNewhler, Gestalt Psychology. An Introduction to New Concepts in Modern Psychology (New York: Liveright, 1947), p.20. R. Murray Schafer, Our Sonic Environment and the Soundscape: The Tuning of the World (Rochester, NY: Destiny, 1994), p.91. The Institute of Sound and Vibration Research at the University of Southampton has a particularly impressive ‘live’ echo chamber,

2

31.

32.

33. 34.

35.

36.

37.

OCCULT AESTHETICS

43

where even short spoken sentences are unintelligible through sound reflection. There is a constant danger of re-experience through a trigger. After a while, emotional exhaustion might take over. This can lead to a state of complete lack of emotions, communicational distance and even problems with memory. Freud’s work centred on psychological trauma. This is not the same as Jacques Lacan’s work, which is more concerned with understanding through language. For instance, Lacan’s “the Real” has a traumatic capability outside symbolization, where categories fail. It is thus an object of anxiety par excellence. Jacques Lacan and Wladimir Lanoff, “Fetishism: The Symbolic, the Imaginary and the Real” in Journal for Lacanian Studies, vol. 1, no.2, 2003, pp.299–308. Lionel Festinger, A Theory of Cognitive Dissonance (Stanford, CA: Stanford University Press, 1957). Some have suggested that Festinger’s original formulation needs rethinking. For example, Elliot Aronson, “The Theory of Cognitive Dissonance: A Current Perspective” in Leonard Berkowitz, ed., Advances in Experimental Social Psychology, vol. 4 (New York: Academic Press, 1969), pp.1–34; Joel M.Cooper, Cognitive Dissonance: 50 Years of a Classic Theory (London: Sage, 2007). In the television series Space 1999 (1975) “End of Eternity” episode, alien Balor fights the inhabitants of Moonbase Alpha to a fully silent soundtrack. Later, when lead characters Konig, Bergman and Helena wonder about the art in Balor’s asteroid prison, we are shown shots of paintings of chaos with tortured and aggressive faces, accompanied by extra-diegetic sounds of screaming and terror. These pieces also appear in Van Sant’s following film Last Days (2005), which is based on the days leading up to the suicide of a rock star clearly based on Nirvana’s Kurt Cobain. According to Randolph Jordan’s extensive analysis, Westerkamp’s pieces, rather than simply expressing alienation, connect interior subjectivity in the film to a plane outside the diegesis, indeed outside the film. “The Schizophonic Imagination: Audiovisual

44

K. J. DONNELLY

Ecology in the Cinema” (Ph.D. diss., Montreal: Concordia University, 2010), p.263. See further discussion in Danijela KulezicWilson, “Sound Design is the New Score” in Music, Sound and the Moving Image, vol. 2, no.2, Autumn 2008, p.129. 38. McGurk and Macdonald, op. cit., p.746.

CHAPTER 3

Audiovisual Space

The play of dynamics is one of the defining logics of audiovisual culture, with constant change to perceptual and cognitive stimulation at its heart. Consequently, the sense of visual and sonic space and movement within that is of paramount importance. Audiovisual space is fabricated through dynamics in camera movement, editing, deep space sets, as well as depth cues in sound, sound reverb and spatial separation. This chapter addresses audiovisual landscapes in film, television and video games. Contemporary audiovisual culture is based not simply on the illusion of movement but more crucially on the illusion and effect of sound and image being merged into a coherent whole. These mark important moments exercising not only the medium but also the physiological requirements of its human audience. Rudolf Arnheim noted: The organism … is by no means a closed system. Physically, it counteracts the running-down of usable energy within itself by constantly drawing resources of heat, oxygen, water, sugar and salt, and other nutrients from its environment. Psychologically, too, the living creature replenishes its fuel for action by absorbing information through the senses and processing and transforming it internally. Brain and mind envisage change and crave it; they strive for growth, invite challenge and adventure. Man prefers life to death, activity to inactivity.1

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025 K.J. Donnelly, Hearing Eyes, Seeing Ears, Palgrave Studies in Audio-Visual Culture, https://doi.org/10.1007/978-3-031-73913-2_3

45

46

K. J. DONNELLY

So, human beings have a mental, perceptual and cultural appetite and requirement, as much as they have physical needs. Our senses crave stimulation. Indeed, denying them stimulation can cause anxiety and psychological issues. On the other hand, an overload of stimulation can prove exciting but ultimately emotionally overwhelming and perhaps mentally disconnecting. Indeed, ‘sensory overload’ is considered a distinct condition with general symptoms including difficulty in focusing, irritability, restlessness, stress, anxiety, a higher level of sensitivity to stimuli and feeling overly excited.2 Indeed, most of the time our physiological need is to expend as little energy as possible on perception and cognition. So, recognition of repeated images, ideas and situations is a positive thing, both for ‘bottom-up’ perception which locks on to repeats and also for ‘top-down’ cognition, which needs to expend less energy understanding. It is no accident, then, that the repetition of elements and stimuli configured in conventional relationships is at the heart of audiovisual culture. We think of audiovisual culture as being ‘dynamic’, yet these dynamics are held within quite strict boundaries and often are repeated in a stereotypical manner. Dynamics, like all other parameters of audiovisual culture are set and guided by conventions. Sometimes these can be strict and heavily circumscribed. As I stated, easy recognition of objects in terms of sound and image is a positive occurrence. Conventions and familiarity set up expectation, which can aid perception (keeping us relaxed rather than anxious) and ‘top-down’ cognition (allowing rapid processing and lessening cognitive load). Sound and image dynamics define space in film and subsequence audiovisual culture and are at the heart of it. Predominantly, in terms of image this is a two-dimensional flat space with the illusion of depth. Action and framing furnish a further sense of dimension. Depending on the number of speakers and the number of different musical signals (often called channels), the ‘sound stage’ of audio will be more or less wide and differentiated. Of course, the two merge together in perception into a seamless space that is viable as an illusion. Even in situations where the sense of space in the sound does not fit the sense of space in the image, we will not perceive it as a mismatch unless the difference in radical. An instance of this might be a large room with little contents and hard wall and ceiling surfaces, accompanied by a ‘dead’ sound with no reflective sounds, yielding a very close sound as if in an enclosed box. Such as instance would signal a psychological dimension, we might

3

AUDIOVISUAL SPACE

47

be experiencing the character’s point of view, or there may be something ‘unreal’ about this space. Audiovisual space is all about dynamics as without them there is no sense of space. The separation of sounds on the soundtrack is also crucial, as it can supply a space that gives something to allow movement across it, or perhaps even to deliver the possibility of movement. Vistas, panoramas and landscapes more generally tend to be characterized by dynamics. A sense of distance is often offset by something closer in the foreground, for example, and it is indeed rare for pleasing landscapes to lack a dynamic sense. With large amounts of the world’s population living in cities where viewing distance is limited, the attraction of looking long distances has become a premium for sea view hotel rooms or chalets up mountains. Landscapes in audiovisual culture are similarly attractive and can help give us a sense of a short ‘holiday’ in the location on screen. Thinking along these lines is reminiscent of the apartment-bound cats whose owners supply them with a video depicting the outdoors and featuring small animals for their excitement. Landscapes in films, television and video games are audiovisual illusions and yet on occasions they appear extremely tangible and enterable. They are habitually a combination of images and sounds recorded at different places. In the case of music accompanying landscape shots, it is clear that the music was recorded in a studio most likely in an urban environment a long way from the images. Yet even in other situations it may not be what we are expecting. In natural history documentaries, for example, sound is almost never recorded at the same time as images and very rarely in the same location. Indeed, library sounds can be used, too, to fabricate environmental sound that appears fairly faithful to (or at least not incongruous to) the location and recorded images. Perhaps, in some oblique way, experiencing landscapes in audiovisual culture is related to experiencing them as a healthy dynamic experience, and as I noted, perhaps a little like a holiday. There is often an interest in showing us remarkable views and places as spectacles perhaps make us into armchair tourists.3 The backdrop for an interest in ‘healthy’ vistas is that many of us spend far too much time looking at an electronic screen, and often with only a keyboard separating it from our bodies. Approaching the audiovisual as dominated by concerns of perception and thus characterizing it as essentially physiological in nature, can lead to a reconceptualization of audiovisual culture as an area for exercising perception, something like a gymnasium. ‘Healthy’ use of the eye varies focal

48

K. J. DONNELLY

length and mixes distant and close vision, while a similar procedure might be argued for hearing. Keeping perceptual faculties in good order involves an attention to dynamic signals, and much audiovisual culture endeavours to provide this. Indeed, it is more than likely that the dynamics of artificial deep space are beneficial, perhaps even like a perceptual ‘workout’. This is not essentially to do with ‘survival fitness’. Although a case certainly can be made, it is a requirement of human hardware (eye lens, etc.), which only indirectly might have evolved for fitness. So, the process might perhaps be incidental. Perhaps a translation across two ‘borders’, which makes the process less of a simple adaptation and more complex negotiation of determinants. This chapter will explain the concept, while attending to important aspects such as the artificiality of electronic audio and visual space, the alternation of proximity and distance effects and the exploration of perceptual overload as an occasional but dramatic effect.

Experiencing Audiovisual Spaces Landscape painting and later still photography have made us think that ‘landscape’ is essentially visual. All actual landscapes include sounds an integrated element. Since the advent of film and later integrated audiovisual culture, the rendering of landscapes habitually involves a prominent sonic, and often musical, component. Landscapes can appear real enough to give an impression of being almost enterable. This has been emphasized and concretized by film and television. For instance, in the British film Three Cases of Murder (1955) segment called ‘The Picture’, where characters can enter a house in a painting. Or the BBC television adaptation of M.R.James’ The Mezzotint (2021) where something appears imperceptibly to be moving across the picture towards its new owner. These play upon the notion that we believe in the image. The painting and etching, respectively, exceed the medium’s flatness and texture, becoming real. Audiovisual culture allows for an increased sense of ‘enterability’, particularly through the deployment of sound. Sound can envelop us, removing the perpetual distance of the image.4 Indeed, there is a sense that sound can embody its own landscape and this has been exploited by corresponding visuals. Although not quite the same, the moving image can give a sense of the audience entering through the camera moving forward, and perhaps between a movement between long shot and close up, although this is not as literal as sound and music.

3

AUDIOVISUAL SPACE

49

Harper and Rayner contend that in cinematic landscape, music retains an equal place with other elements, but remains defined through the image.5 This is a traditional view of the image as prime and sound as secondary, and traditionally Film Theory has suffered from a failure to adequately address sound. I would suggest that music can redefine the image rather than simply being defined by it. This would assume a fundamentally transformative relationship with the images of landscape shots. One that appears more complicated and more far-reaching in its effect and not simply defined by the image. There has been a widespread assumption that music ‘adds to’ the image in audiovisual culture. A more sophisticated understanding would be that it rather converts it, perhaps through a process of ‘mutual transference’. Music accompanying landscape shots might mean that music is ‘landscaped’ and landscape is ‘musicalized’, in a reciprocal relationship that embodies and homologizes the perceptual-cognitive process taking place in our heads. The distinctive formation makes a particularly potent combination of the indexical realism of the image with the emotional immediacy of music. These ‘musicalized screen landscapes’ potentially can become an emotional representation, which might appear less of a representation of a place than a representation of an emotion. Yet rather than simply a case of music furnishing the emotion of the image, the image reciprocally provides something important for the music. As a part of this, it is possible that the image provides an (imaginary) visual spatialization of musical structure—one that will not correspond with actual musical structure—but adumbrates an ‘emotional sense of structure’, or what might better be thought of as a seemingly logical sense of emotional structure. These elements have a defining relationship in the overall effect of the whole. A good example is Maurice Jarre’s music added to the initial landscape shots of the desert in David Lean’s Lawrence of Arabia (1962). The sequence inaugurates with Lawrence extinguishing a match with his fingers cueing the heroic orchestral music bursting forth alongside a succession of desert shots joined by crossfades. This combination of sound and image is so iconic it has been reused and parodied, in the James Bond film Moonraker (1979), for example. The music here is the counterpart to the images as drama rather than an atmospheric equivalent to the desert, though. While the sustained notes are perhaps isomorphs of the sweeping sand dunes, the music supplies a sense of movement lacking

50

K. J. DONNELLY

in the static desert shots. On other occasions, music appears to embody the emotional tone of the landscape more, such as Ry Cooder’s slide guitar music accompanying barren American deserts in Wim Wenders’ Paris Texas (1984).6 In other cases, music can sometimes subtly integrate with the images to form an audiovisual environment as a whole. Music on its own can also carry a charge of landscape implication. Socalled ‘ambient Music’ contains a distinct implication of images, or at least something pertaining to a landscape or location as a complement to the music. In The Ambient Century: From Mahler to Moby, Mark Prendergast points to Brian Eno’s central importance to ambient electronica.7 Eno’s seemingly programmatic album Ambient 4: On Land (1982) evinced a lack of purposeful progression or development, alongside an evacuation of melody/harmonic movement. The music thus formed its own sense of environment, rather than utilizing traditional aspects of dynamic movement (a concern with melody, harmony and rhythm). This form of static musical structure tends to foreground texture and sonority. It also mixes what is traditionally understood as music with what appears more like sound effects or ambient sounds. As Brian Eno himself notes on the cover of Ambient 4: On Land: “… cluster all disparate sounds into one aural frame: they become music”.8 It is difficult not to be aware that there is a similarity to film scores that have had an important role in furnishing aural impressions of locations and associated images. Ambient 4: On Land was an influential signpost to the future genre of ‘ambient music’. Most of the music lacks notable melody or harmonic movement and instead is premised upon slow musical events and a focus on textures. Timbres are defined by electronic treatments and do not sound like distinguishable instruments. Indeed, many of the sounds are from previous recordings or library sound effects. For instance, The Lost Day involves a periodic keyboard that sounds more like the slow clanking of metal ropes on spinnakers in a yacht marina. This punctuation provides an important temporal structure but also provides a sense of spatial distance, along with much in the way of deep indistinct sounds. Certainly, compared to other music of the time, this must have sounded strange. Its character is closer to environmental sound than traditional music. It lacks dynamics or development and seems less a piece of coherent music than a span of ambient sound that easily could be employed as backdrop in a film. All the sounds were subjected to lengthy signal processing and development in the studio—what in the film industry they would refer to as ‘post-production’. Continuity and slow unfolding of sound make

3

AUDIOVISUAL SPACE

51

this music a far cry from the traditional standard song format of ‘popular music’. Instead, the recordings present a strong impressionist sense of ‘painting pictures’ though sound. Music in its wake has had a significant impact on audiovisual culture and indeed, Ambient 4: On Land sounds relatively straightforward today due to the degree of influence in later music, and in scores for film and television.

Rural Sights and Sounds Images of the English countryside have been common on film and television programmes. It has often rarely been acknowledged that this image includes an associated sound/music. This has cohered into welded together countryside panoramic long shots and ‘pastoral’ music. A good example of this is the Wessex described in detail in Thomas Hardy’s novels. This was translated into music by Gustav Holst in his tone poem Egdon Heath (1927); and screen adaptations of Far from the Madding Crowd in 1967 and 2015, with music by Richard Rodney Bennett and Craig Armstrong, respectively, and in both cases suitably rustic to fit the films’ country landscapes. Similarly, American Westerns also are associated with a particular sound, in terms of music this involves a repertoire of particular instruments and combinations of timbre. These might be understood as a clear equivalent of sound and image, which merges into a strong whole through structural isomorphism and homology. This process of equivalence between a certain type of music and certain types of images might be understood with reference to notions of ‘congruence’ as well as with the structural idea of isomorphism. While isomorphism posits that there is a formal mirroring or structural equivalence between sound and image, the notion of congruence investigates the psychological and cultural sense that certain images seem to fit well with certain sounds (‘congruence’). Foundational work on this was completed by Annabel Cohen with some more recent work by David Ireland.9 The sense of whether we think certain music and certain images as particularly ‘congruent’ has clearly had an important role in setting traditions, for both production and expectation. However, some films and television dramas aim for a sharp contrast between music and images through the use of anachronistic music, such as Plunkett and Maclaine (1999), Sofia Coppola’s Marie Antionette (2006) and the BBC television series Peaky Blinders (2013–22). These challenge expectation, although the more expectation is challenged, of course, the less it remains

52

K. J. DONNELLY

a solid expectation. However, conventional usage tends to dominate and there can be little to gain from radical divergence of sound and image confounding expectation. The British television series Midsomer Murders (sometimes known as Barnaby or Inspector Barnaby overseas) has been one of the most successful British television exports.10 The series began in 1997 and is still in production. It is a rurally set police procedural drama, where an inspector and his assistant investigate what habitually is a succession of gruesome murders in an atmosphere mixing pretty and polite locations often with an air of irony and camp. The ‘whodunit’ prides itself on finding novel ways for murders to be achieved in this country location that is peopled with eccentric and exaggerated characters, while also allowing for regular cameos from ageing celebrity actors (Fig. 3.1). The location shots are regular and one of the programme’s attractions. Indeed, the ‘Home Counties’ locations, set across Buckinghamshire, Berkshire, Hertfordshire and Oxfordshire, are listed as points for tourists to visit in the book Midsomer Murders on Location, which is available in the Oxford Tourist Information office. The back cover notes: “This book is a must for all Midsomer Murders enthusiasts as well as those interested in visiting some of England’s finest countryside”.11 The distinctive visuals have an equally distinctive sonic counterpart. The music for Midsomer Murders plays an important part in the programme. It is produced by experienced television music composer Jim Parker and has a ‘mock-classical’ character, not only punctuating the action and providing structure for the narrative but also providing a playful sense to the arch murders and investigation.12 In Midsomer Murders, a small repertoire of reused recorded cues appeared in each episode, alongside new ones that might be specific to the episode. The programme’s main theme, as is traditionally the case, establishes a sense of emotional tone as well as cutting out an individual sense of the show’s character. A waltz with a swinging tonic-dominant bass, it has an arch and drily comic feel. Its edge of irony is reminiscent of composers such as Michael Nyman or Danny Elfman, whose work can sound like it is not straightforward and is asking you to join in with a parody. Perhaps the clearest antecedent might be the Stranglers’ Waltz in Black, which had become a staple as television library music.13 The Midsomer Murders theme uses a series of parallel minor chords, which sounds unusual in the light of waltzes tendency to use strong tonal structures of harmony.14 Its melody is also distinctive and exotic, being performed on the theremin,

3

AUDIOVISUAL SPACE

53

Fig. 3.1 Midsomer Murders

one of the earliest electronic musical instruments and used often for its eerie sound in music for science fiction films such as The Day the Earth Stood Still (1951) or for disturbed psychology as in Spellbound (1945). These musical aspects provide a solid sense of the programme’s distinctive character and mark it out as unusual for a detective show. On the opening titles, the theme accompanies images of the locations and actors—in other words, the diegetic action has already started. This already provides a significant ‘cast’ on the images and provides a certain levity to images which can often include murder and its aftermath, not what might be considered lightly in many television dramas. The theremin reappears, too, in the incidental music. This happens at moments of revelation, or dramatic points that lead into advert breaks, where the instrument makes a startling rise in pitch climax. This resembles a scream and is similar to the sort of portamento that can be achieved on a slide trombone. While this bears similarities to horror film music, it retains its arch, ironic connotation. These moments are a good example of the unity of narrative development and sonic dynamics, both peaking at the same time and underlining the significance of events on screen. A prominent repeated theme is called ‘The Village’ and is clearly based

54

K. J. DONNELLY

on the classical music tradition of music inspired by the rural fox hunt and idealist notions surrounding it. As such, the melody is played by the French horn, a common ‘pastoral’ instrument, and the melody again appears perhaps slightly comic, although not overtly so. It appears to parody the sense of urbane behaviour—which oddly is associated with the blood sport—and this polite melody is given an even more formal sense by the addition of urbane obligatos and the light triple-time pulse. Sometimes this piece is rearranged and has its melody performed by the clarinet, another instrument regularly associated with the simplicity of the rural and highly evident as a solo instrument in English pastoral classical music.15 So, such music not only furnishes a sense of the rural to reinforce the images on screen of English villages and countryside, but also suggests a hidden underneath of blood sport and irony, where things are not what they appear to be. This overall unity of sound and image has provided the show with a unique atmosphere that is the ‘static’ sold element with a surface of different narrative intricacies on top of it. It is not only the incidental music that is fully integrated and a clear part of the Midsomer Murders ’ character. Sounds are used in a way that extends beyond simple guarantors of diegetic realism. The programme uses sounds of nature very prominently. These are stereotypical sounds of rural England, which reinforce a sense of nostalgia (for lost rurality) and a shared English culture based on land and tradition. Garden shots regularly include a singing blackbird, while sometimes the undercurrent of negativity is indicated by a hoarse call from a magpie. Night sequences almost always include the sound of a lone fox bark. Ironically, all these sounds, and particularly the latter, are far less rural these days in Britain. They are certainly less evident in the intensive farming-based countryside and far more evident in ‘ruburban’ (rural suburban) gardens, or perhaps in these garden outposts in rural villages. Midsomer Murders thus moves aspects that traditionally are on the edges of television dramas into being closer to a featured aspect. These nature sounds, which are all clearly added in post-production, are unsynched and off screen and comprise an important psychological aspect of the landscape that is the heart of the programme. For instance, in the episode ‘A Tale of Two Hamlets’ (from 2003) as a family sits on their lawn talking to Inspector Barnaby, we hear the cooing of a collared dove, then a tolling church bell, along with a thrush and some crows. These are as much, perhaps even more, of the ‘landscape’ of the programme than the images or the narrative

3

AUDIOVISUAL SPACE

55

and characters. They are also not present for the purposes of realism, and while they are an ‘emotional landscape’ they also function structurally. One potential reason for the importance of the soundscape and its function as landscape is that in some cases, when rural village scenes in England are being shot, they avoid showing the reverse shot, which is a large road full of cars or some form of recent building development. Of course, this is not always the case but Britain has experienced an unprecedented expansion of cities such as London into the nearby countryside in recent decades. Indeed, the visual construction of Midsomer country is highly selective and takes images from diverse sources to collage an imaginary rural location. Overall, Midsomer Murders ’ sound and music provide the dimension of ‘landscape’ that images cannot (unifying fragmented illusions of the rural) as well as providing a wry commentary on the action. Novelist and filmmaker Iain Sinclair stated that the version shown in France (Inspecteur Barnaby) with its translated French subtitles altered the programme, making it appear less comic and more of an ‘existential drama’.16 It is not necessarily that the translation loses and gains something but that the programme has certain ambiguities that can be emphasized or not and that sonic irony can easily lose any impact due to wider context, as potentially can comedy, too. While it might easily be argued that the landscape images owe something to the English tradition of landscape painting, the music clearly has a debt to the tradition of ‘English Pastoral’ art music.17 This is most commonly associated with British composers such as Ralph Vaughan Williams, George Butterworth, Frederick Delius, Gustav Holst, Frank Bridge and others and was prominent in the first half of the twentieth century. Some prime examples of this particular sound include Butterworth’s A Shropshire Lad Rhapsody and The Banks of Green Willow; Delius’ Brigg Fair, Holst’s Egdon Heath and Vaughan Williams’ The Lark Ascending and Symphony no.3 ‘Pastoral’. Indeed, the piece ‘Driving Home’ by Parker is reminiscent of Butterworth’s A Shropshire Lad Rhapsody. While this ‘English Pastoral’ was not a movement as such, it appears to have a broad stylistic similarity, often being based on folk songs and with prominent woodwind. Its sense of the rustic music of England’s past homologized with a Romantic notion of Britain’s countryside and the (largely lost) simple rural life. It also coincided with a period of strong nationalism in Europe that was evident in the Arts. In his book Music in England, Eric Blom declared that English Pastoral music was

56

K. J. DONNELLY

the “counterpart of English lyric poetry …”.18 It is perhaps unsurprising that this tradition is also clearly evident in music for British film and television costume drama and films set in Britain’s past. While Richard Rodney Bennett’s gentle rustic flute theme for Far from the Madding Crowd is a good example, so are the BBC Great Expectations (1999) with music by Peter Salem, or the film Akenfield (1974) that featured prominently a piece of Michael Tippett’s music (Fantasia Concertante on a Theme of Corelli). Geoffrey Burgon’s music for Brideshead Revisited (1981, Granada) was perhaps less pastoral and more ‘drawing room classical’ which is the norm for recent British television costume drama. This seems more about class than landscape, although it might be argued that the rural landscape includes (and sometimes overwhelms) the peasant/ working class whereas the ruling class is shown in expensive clothes and buildings, which demand a different form of music.

Nonindifferent Nature Audiovisual culture increasingly has developed the notion of the ‘emotional sound effect’, which has become particularly prominent since the increasingly collapsed distinction between music and sound effects.19 A clear outcome of losing the distinction between music and other sounds is that sound effects become more emotional—like music, while music has the option to become less emotional but more integrated with the action. Sound effects can grow to be part of ‘music’ and take on some of its traditional functions. Also, at least partly, this is due to digital and ‘musical’ software being used for both sound effects and music, and their being mixed also through the same programmes. The merging of these two traditionally separate functions is matched by a similar loss of distinction between sound and images. Here, and in other rural set films and dramas, there can be a sense where the particular images need a particular image at the same time, and indeed, vice versa. The sound of English Pastoral music is ‘completed’ by images of the English countryside, as well as the other way around. From the late 1950s to the early 1980s, British regional television company Southern Television began its daily broadcast with the specially written piece Southern Rhapsody by Richard Addinsell. This was accompanied by a slow cross-faded montage of still photographs and film of predominantly city landmarks, country and seaside locations across the region, the south coast of England. It is clear not only that the

3

AUDIOVISUAL SPACE

57

producers thought the images and music ‘fitted with each other’ but also that they were the equivalent on some level. This sense of integration between landscape and music (and other sound) is hardly new. Sergei Eisenstein was interested in their expressive possibilities, discussing what he called ‘nonindifferent nature’ where landscape shots had an emotional function similar to music and lacked the burden of providing narrative information. Rather than being neutral and representational, such images carry a suitable emotional charge which is in tune with the rest of the film. He calls this ‘the musicality of landscape’.20 Eisenstein’s notion of ‘nonindifferent nature’ posits an equivalence between landscape shots and music.21 Both are more than simple backdrop and supply an emotional charge and valence for the film. Both are able to move beyond simple communication and narrative function to elicit a sense of thick atmosphere and emotion. Eisenstein was interested in emotional landscape shots and their ability to step outside of narrative and diegesis. Later he was concerned with sound film beyond basic ‘talking films’ and often thought and theorized in musical metaphors. Landscape shots matched with non-diegetic music would potentially redouble this effect. Rather than simply a representation of location, film landscapes with music regularly become transformed into something extraordinary and, by implication, with their own integrity. Eisenstein appears to pull back from this, though, and focuses on landscape shots’ place within the film system, in relation to other shots and elements, where they offer emotion to their surrounding pieces of celluloid. Yet, they can also be understood as something in themselves, as semiautonomous objects within film. So, this is not about ‘representation’, and the music does not ‘add’ to the image, but rather converts it, perhaps into an ‘emotional representation’, which is less of a representation of a place than it is a representation of an emotion. Even the most specific footage of a place becomes something notably different as film and music combined. ‘Double nonindifferent nature’ is not a straightforward process, however. The mixture of the two channels and corresponding senses can produce an ‘artefacting’ or ‘aliasing’ effect, yielding certain ideas, emotions and implications as ‘spectral’. Despite the genetic fusion of landscape image and music, there can remain something of a ‘gap’ between sound and image, which emanates from technological realization and perception. The ‘feeling’ of landscapes resembles the supernatural.

58

K. J. DONNELLY

The images become ‘more’. They are complex, not straightforward and appear to have subtle connotations and implications. Potentially, illogical emotions might appear. While pretty music any moving image could fit to any music, some, of course, fits better than others. But sometimes the effect can be strange and uncanny. This is due to the process being more important than the content (as classical Eisenstein sound montage, or ‘vertical montage’). This is not simply about feeling the landscape, but the conversion of the landscape to something else: music providing a sense of depth (and a sense of different time scales) to the landscape image. Landscapes welded to music are significant for video games, too. Tadhg Kelly, video game designer, producer and creative director, argues, ‘forget the person. The art of game design is all about the place’.22 and additionally, Geoff King and Tanya Krzywinska claim that ‘more than simply a background setting, the world of the game is often as much of a protagonist, or even antagonist, as its inhabitants’.23 This is certainly true for so-called ‘Open World’ games, where the player can traverse what appears to be a real space, choosing a path and movement within a highly distinctive location. Good examples of this include the Elder Scrolls games and Silent Hill games. In Atlas of Emotion, Giuliana Bruno points to the walk in a city as a ‘haptic geography’ that links film and architectural concerns and then she proceeds to suggest that most Italian Neorealist films might be understood as city walks.24 A relatively recent video game genre appears quite literally to be a walk: so-called ‘Walking Simulators’, sometimes known as ‘exploration games’. In these games, the player has little or nothing to do in gameplay and simply traverses the world on screen at will. Examples include Dear Esther (2012, The Chinese Room), The Vanishing of Ethan Carter (2014, The Astronauts) and Gone Home (2018, Fulbright/ Blitworks). In the same mould, independent video game The Old City: Leviathan (2015, Postmod) is not based on skilful action fighting or thoughtful puzzling. This ‘walking simulator’ involves a detailed visual environment that is open to the player’s aimless exploration and contemplation. As a counterpart to the sumptuous and mobile visuals, there is an extensive and atmospheric music soundtrack by Swedish dark ambient industrial musician Atrium Carceri. Rather than music derived from an interactive video game tradition,25 the music firmly retains a distinctive aesthetic from ambient music and retains a character of sombre uneasiness. Less

3

AUDIOVISUAL SPACE

59

concerned with gameplay and more directly concerned with embodying environment, it lacks notable dynamic shifts and development. This decentres the player and makes them a visitor in a large dominant and independent sound and image-scape. Before creating the music for The Old City: Leviathan, Atrium Carceri’s music sometimes appeared to be descriptive of cities, such as Inside the City (on Phrenitis from 2009) and A Stroll Through the Abandoned City (on Kapnobatai from 2005). Indeed, it is almost as if the game has actualized the potential of the music. The Old City: Leviathan’s music adopts a different function from most video game music, potentially bringing with it aspects of electro-acoustic environmental music for a game that espouses the tenets of psychogeography, through imbuing a landscape with emotional characteristics and mysterious implications. The Postmod website states: “Players have the option to simply walk from start to finish, but the real meat of the game lies in the hidden nooks and crannies of the world; in secret areas, behind closed doors …”26 The music is not only an integrated part of the experience, but also follows a similar process of being open to exploration and contemplation. In fact, in a number of ways The Old City: Leviathan problematizes the notion of the ‘video game’ and indeed some game players do not consider such ‘walking simulators’ as games at all. Prominent games theorist Espen Aarseth stated: “Games are both object and process; they can’t be read as text or listened to as music, they must be played”.27 Yet, the contemplative nature of games such as Leviathan allows a player to listen to the music while wandering or remaining inert. It appears to be a remarkably pure instance of focus on audio and visuals, as there is little narrative that the player doesn’t have to follow anyway and little that resembles the traditions of gameplay in electronic games. The opportunity for immersion is high, though, as there is little to distract the player from wandering and appreciating. As I noted already, Atrium Carceri’s music is not ‘functional’ in the regular video game sense and sounds very similar to his other music. The aesthetics could almost derive from this ambient music that dominates the game’s logic. The music changes for different locations in the game, which is common in video games where the player or avatar moves through a space resembling an actual location. Philip Kirby notes that ‘geographical approaches’ to video games can often exploit or rely on music.28 In The Old City: Leviathan impressive images and atmospheric

60

K. J. DONNELLY

music cohere into a sombre but engaging view of the empty city, devoid of a population. This is a compulsive ‘vision of the end’ of humanity, embodying the Freudian death drive as a video game, if you like. The lack of gameplay simulates an afterlife scenario, being a tourist after the demise of humanity, while the music suggests the ghost of human agency and the implication of absent humanity. Some video games exploit a similar interest in exploring location, while also including goal-oriented activities for the player. S.T.A.L.K.E.R.: Shadow of Chernobyl (2007) is a first-person shooter survival horror video game developed by Ukrainian game developer GSC Game World and published by THQ in 2007 following a long development. The game is set in an alternative reality, where a second nuclear disaster occurred at the Chernobyl Exclusion Zone in Ukraine, causing strange changes in the area around it. The game features a fragmented line of progression and includes role-playing game (RPG) elements such as dealing with non-player characters. Inspired by Andrei Tarkovsky’s Stalker (1979), an adaptation of the Strugatski brothers’ novel Roadside Picnic, and the game’s extremely large traversable area is based at least partly on ‘the zone’ in the film. Similarly, it is a place with strange mutations and developments, but also with changing weather. While the desolate zone appears to be a continuity, it is in fact a large number of different successive 3-D locations that are accessed via distinct portals. Like, The Old City: Leviathan, the game’s music (credited to MoozE) is more like ambient music used for atmosphere and as a merged part of the location rather than being dynamic game music that develops and changes with specific gameplay. The third game in the series, S.T.A.L.K.E.R.: Call of Pripyat (2010) includes playable locations modelled directly on actual places in the Chernobyl exclusion zone, such as the town Pripyat, the village of Kopachi and Yaniv Railway Station (Fig. 3.2). The use of meticulously copied actual locations can add a significant dimension to the sense of actuality in playing video games. Good examples include Assassins Creed (2007, Ubisoft) and its ancient Rome setting and Assassins Creed Unity (2014, Ubisoft) in Paris, InFAMOUS Second Son (2014, Sucker Punch) in Seattle and Tom Clancy’s The Division 2 (2016, Massive Entertainment) in Washington, D.C. This ‘realist’ tendency is very different from the dreamlike use of landscape in many games. Indeed, in recent years, the approach of ‘psychogeography’ has become popular, even though it seems to lack a solid definition. One

3

AUDIOVISUAL SPACE

61

Fig. 3.2 Stalker: Shadow of Chernobyl

facet is the notion that we have an emotional understanding of landscape, as well as landscapes appearing to be partly ‘inside the head’ of the observer rather than fully separate external objects. The Silent Hill games are premised upon a landscape that appears to take place inside the protagonist’s head, while the independent game Barrow Hill: The Dark Path (2016, Shadow Tor studios) has the player move through different locations around an English village in a hypnogogic situation. Indeed, landscapes can be ‘read’ by those with skills understanding the symptoms of the past (old mines, fields, old rail lines or roads, villages destroyed, etc.), while there are also other means of dealing more ‘magically’ with landscape, such as ‘divining’ and ‘dowsing’. So, the landscape and our understanding of it is not a simple thing but one premised upon taking basic perceptions and being able to accept that the ‘face value’ of our initial signal should not be taken as the full truth but that we need to ‘think behind’ and imagine, in order to understand more fully. This

62

K. J. DONNELLY

is a fundamental questioning of perception that has been and remains a persistent strand in audiovisual culture. It is a common strategy for culture to set up a state of ambiguity and some confusion between ‘real’ and interior states of mind, and indeed certain genres arguably rely upon this. There might be some evidence that sound is particularly susceptible to this confusion. Flinker et al. suggest that judging from audio cortex activity, human brains cannot easily differentiate between our own voice and that of others, suggesting that a solid demarcation between inside and outside our heads may be fragile.29 On one level, we approach diegetic worlds as on some level being ‘real’, through the so-called ‘suspension of disbelief’. Indeed, as already noted much culture, such as literature and film, are premised upon making fantasy seem real. On the reverse side of this, we can become confused between manifestations of psychological ‘inner states’ in a seemingly ‘realistic’ format. In some diegeses, two different worlds can be present and set apart, which illustrates precisely the process of formulating constructions of ‘fantasy’ and ‘reality’ within the same space. These can play to or across the traditional tendency for audiovisual culture to use sound for interior states of mind and image for more objective situations. Yet this can often be less schematic, and rather than posing a clear sense of ‘a real world’ and an interior world, stylistic aspects can augur a sense of momentary change or amendment to subjectivity. Distortions of perception as a way of expanding human perceptual experience and reach, and of questioning perception itself, or perhaps of perception questioning cognition.

Conclusion Physiological baselines provide psychological horizons. Dynamic variation is the lifeblood of audiovisual culture, and this is most clearly evident in the depiction of landscapes where sound and image are able to forge a highly dramatic sense of space. This audiovisual space is created through a repertoire of image and sound techniques that yield a desirable illusory space. Mimicking distant vistas with extremely long shots and large spaces through sound and music with electronic reverb, audiovisual landscapes have a haptic as well as emotional sense. This chapter addressed the central play of dynamics evident in audiovisual culture, which might be conceived as one of its central and defining logics. This will include discussions of culture that limit dynamics and points where sensual ‘overload’ and tiredness are an essential part of cultural objects.

3

AUDIOVISUAL SPACE

63

Dynamics are crucial for perception and their configuration into a consumable format in audiovisual culture arguably has a function in helping regulate the brain. They certainly exploit its central tendencies. Rudolf Arnheim noted, “We envisage the human mind as an interplay of tension-heightening and tension-reduction strivings. The tendency towards tension-reduction cannot run its course unopposed, except in the final disintegration of death”.30 While video games clearly exploit this, it is also a clear strategy in thrillers and horror films, and sometimes this process is mirrored extremely clearly in the incidental music’s regular tension-release structures. These are premised upon repose and anxiety, usually built by narrative development, although in perceptual terms they can follow the music’s building of tension and ultimate release, and so engage both top-down and bottom-up processes rather than simply being based on the illusory situation on-screen and narrative context. They remain importantly physiological in their engagement of the body: not only the perceptual organs but also connections to rates of breathing and heartbeat. The sense of space and aesthetic dynamics are mutually implicating and productive. Landscapes in the real world are characterized by distance and dynamics, and audiovisual culture negotiates these attributes through audiovisual dynamics, long shots supplying visual space, while music and sound cues also have their own sense of dynamics. Both sound and image have their own intrinsic dynamics but, crucially, merge into a new configuration of audiovisual dynamics. Although the metaphors are not good as it is ocularcentric, there is a longstanding saying that the music can either be ‘camera’ (part of narration) or ‘set’ (part of atmosphere, emotional tone). Some television programmes, for instance, provide an instant emotional repertoire and sense of imagination for the show as part of its audiovisual landscape. We can feel like we make a ‘visit’ to some drama programmes, like a short holiday, perhaps. Midsomer Murders is a fine example of this, concentrating on a coherent audiovisual landscape for the audience to ‘enter’. In a similar manner, Star Trek (retrospectively relabelled ‘The Original Series’, 1966–69) has a strong feel for its spaceship location through not just coherent visual structure of the sets but crucially also through the selection of ambient spaceship sounds that belong with it. Indeed, the visuals without those sounds would not be convincing, and for those of us who have watched the show regularly repeated since childhood, there

64

K. J. DONNELLY

is the reassuring feeling of visiting the ‘place’ of the bridge of the starship USS Enterprise. This sense of coherent sonic and visual landscape is evident in many dramas, and in particular in science fiction series, even though they aim for an exoticism of their visual and sonic landscapes that they might visit as ‘alien planets’. While we might imagine this comes from the tradition of ‘sound effects’ rather than music, I would suggest not. Sound becomes wielded in the manner of music, as a modal and emotional element rather than necessarily an object to furnish a sense of reality to the programme world. For instance, the strangely foregrounded animal sounds in Midsomer Murders do little if anything for narrative development but instead give a sense of texture to the show that marks it out as distinctive. Indeed, in the case of the latter, it would be a jarring surprise to actually be shown the night time off-screen barking fox. The fox’s sound has become decoupled from the animal and reintegrated into the electronic landscape, integrated with dark images of the characteristic location.

Notes 1. Rudolf Arnheim, Art and Visual Perception: A Psychology of the Creative Eye (Berkeley: University of California Press, 1974), p.411. 2. Anon, “7 Sensory Overload Symptoms to Look Out For” at Experia.USA, 15 November 2020. https://www.experiausa.com/blog/7-sensory-overload-symptoms-to-look-out-for/ [accessed 27/6/2022]. 3. John Urry implied this in his influential book The Tourist Gaze (London: Sage, 1990). 4. Virtual reality (VR) is ‘enterable’ as a 3D space, particularly with immersive headphones. In fact, headphones are particularly significant, allowing us to carry a particular soundscape around with us like our own mind. 5. Graeme Harper and Jonathan Rayner, “Introduction—Cinema and Landscape” in Graeme Harper and Jonathan Rayner, eds., Cinema and Landscape: Film, Nation and Cultural Geography (Bristol: Intellect, 2009), p.19. 6. Cooder’s music is based primarily on the blues piece Dark Was the Night Cold was the Ground by Blind Willie Johnson.

3

AUDIOVISUAL SPACE

65

7. Mark Prendergast, The Ambient Century: From Mahler to Moby (London: Bloomsbury, 2001), pp.131, 52–53. 8. Brian Eno, cover notes of Ambient 4: On Land (1982 EG Records EG 2311107). 9. Annabel J.Cohen, “Congruence-Association Model of Music and Multimedia: Origin and Evolution” in Sui-Lan Tan, Annabel J.Cohen, Scott D.Lipscomb and Roger A.Kendall, eds., The Psychology of Music in Multimedia (Oxford: Oxford University Press, 2013), pp.17–37; David Ireland, Identifying and Interpreting Incongruent Film Music (New York: Palgrave Macmillan, 2018). 10. BFI Television Yearbook (London: BFI, 2003), p.39. 11. Sabine Schreiner and Joan Street, edited by Antony J.Richards, Midsomer Murders on Location (Cambridge: Irregular Special Press, 2010). 12. However, the strategy changed in 2019 to have new individual scores by different composers. 13. Which appeared on the album The Gospel According to the Meninblack (1981). Although perhaps not as arch, a similarly ironic sounding waltz, Shostakovich’s Waltz No. 2 from Suite for Variety Stage Orchestra, which was used effectively in Kubrick’s Eyes Wide Shut (1999). 14. The chord progression starts on a minor chord and then moves down two semitones to make a parallel minor chord, followed by a 7th chord two semitones lower and then a seventh chord a semitone lower (Im-VIIm-VI7-V7). This is a variation on what is sometimes called ‘the Aeolian Progression’ a relatively common descending chord progression in minor keys (or the Aeolian mode). 15. The track is called ‘Driving Home’ on the CD release. 16. He states this in Grant Gee’s film Patience (After Sebald) (2012). 17. Which English modernist composer Elisabeth Lutyens allegedly called ‘Cowpat’, a name that stuck. 18. Eric Blom, Music in England (London: Pelican, 1943), p.200. 19. K.J.Donnelly, “Emotional Sound Effects and Metal Machine Music: Soundworlds in Silent Hill Games and Films” in Liz Greene and Danijela Kulezic-Wilson, eds., The Palgrave Handbook of Sound Design and Music in Screen Media: Integrated Soundtracks (New York: Palgrave, 2016), pp.73–88.

66

K. J. DONNELLY

20. Sergei M. Eisenstein, Nonindifferent Nature: Film and the Structure of Things, Herbert Marshall, trans. (Cambridge: Cambridge University Press, 1987), p.389. 21. Ibid., p.389. 22. Tadhg Kelly, ‘Worldmakers [Game Design]’ at Whatgamesare. www.whatgamesare.com/2010/12/worldmakers-game-design. html [accessed 17/06/2015]. 23. Geoff King and Tanya Krzywinska, Tomb Raiders & Space Invaders: Videogame Forms and Contexts (London: I.B. Tauris, 2006), p.76. 24. Guiliana Bruno, Atlas of Emotion: Journeys in Art, Architecture, and Film (London: Verso, 2002), pp.64, 30. 25. Video game music traditions are discussed in detail by Karen Collins, Game Sound (Cambridge, MASS.: MIT Press, 2008); and Richard Stevens and Dave Raybould, The Game Audio Tutorial: A Practical Guide to Sound and Music for Interactive Games (London: Focal Press, 2011), p.169. 26. Visiting http://postmodsoftworks.com/ will show you that the company no longer exists and developed only this game. 27. Espen Aarseth, “Computer Game Studies, Year One” in Game Studies, vol.1, issue 1, July 2001. http://gamestudies.org/0101/ editorial.html [accessed 12/5/2009]. 28. Philip Kirby, “Musical orientation in virtual space: videogame score and the spatiality of musical style and topic” in Social and Cultural Geography, vol.23, issue 6, 2020. https://www.tandfonline. com/doi/full/10.1080/14649365.2020.1821392 [accessed 20/ 2/2021]. 29. Aideen Flinker, Edward F.Chang, Heidi E.Kirsch, Nicholas M.Barbero, Nathan E.Crone and Robert T.Night, “Single Trial Speech Suppression of Auditory Cortex Activity in Humans” in Journal of Neuroscience, vol.30, no.49, 2010, pp.16643–16,650. 30. Arnheim, op.cit., p.411.

CHAPTER 4

The Audiovisual Elsewhere

Film music and other sounds from the soundtrack have extended film aesthetics beyond the bounds of film into other media and cultures. Sound design now can use musical software to enhance sound effects in films and music composers to incorporate sound effect recordings. Soundtrack elements now appear to have an ‘aesthetic’ character. Technology has engendered a spatial sonic arena wherein sonic elements have mixed into a sensual and psychological field. Modern film soundtracks often evince a conceptual or aesthetic unity strikingly similar to musical unity, evident in disc releases unconnected to the cinema. In films sounds on their own work in a different way, implying visuals that we then expect to see or imagine. That sound implies visuals are crucial to “paratextual” soundtracks outside of film. Thirty years ago, Noel Burch noted that classical cinema occasionally treated its sound components as musical material. He pointed to Force of Evil ’s (1948) use of dialogue as both dramatic action and “musically organized sound”.1 In recent years such a “musical” approach to the film’s music and sound as a “whole entity” is less remarkable. Film soundtracks now often evince a conceptual or aesthetic unity along the lines of musical principles, both in structure and suppositions about individual sounds. The increasing involvement within a larger sonic scheme of elements of the film soundtrack partners with the enhanced importance © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025 K.J. Donnelly, Hearing Eyes, Seeing Ears, Palgrave Studies in Audio-Visual Culture, https://doi.org/10.1007/978-3-031-73913-2_4

67

68

K. J. DONNELLY

of off-screen sound (it is no longer necessary to painstakingly show everything on screen) in modern audiovisual aesthetics.2 The crucial point is that sound implies visuals. This has fundamental implications for “soundtracks” outside the cinema, as well as larger ramifications for discussions of sonic culture generally. Technological developments have ensured a shared fate for film and music. Digital recording and manipulation have enabled more complexity in sound design in mainstream films as a matter of course. In the music industry, it has also afforded the easy construction of complex music through sampling sounds from diverse sources and precise, almost molecular manipulation. It should be no surprise that film and music have converged, and in the light of expansive notions of what constitutes music, have seemingly fused on music’s terms. Film remains at the apex of audiovisual culture, providing inspiration and aspiration for other media. Digital technologies and small electronic screens, rather than hastening the death of film, have transfigured it into a new format that retains fundamental continuities with its traditions.3 One vital aspect of this, the aesthetic of “musical accompaniment” or “dramatic musical scoring”, has proliferated, becoming increasingly evident in media and culture beyond cinema. Indeed, little more than a permeable membrane separates “film music” (or music made for film) and music that is “absolute” or self-contained, but also combines well with images. Although film has always enjoyed a central role in “the audiovisual”— defining the term probably more than any other agent—its aesthetic traditions of assumptions, techniques and effects have also been exported to other media. We should bear in mind that film soundtracks can constitute an aesthetic unity, with music incorporating sound effects and the whole having something of a ‘musical’ sense to it. Rather than simply being a cog in a mechanism, soundtracks are able, to a degree, to become autonomous without explicit moving images but through the implication of accompanying visuals. “Soundtrack albums” had already extended the reach of individual films beyond their physical bounds and screen time. By the 1980s and the saturation of industrial “synergy”,4 soundtrack albums were growing in parallel with their associated films, culminating in albums declaring, “Music Inspired by the Film”, though in some cases having no discernible connection to their nominal parent.5 Over the years a small but insistent area of musical culture has emerged, either structured or inspired by the recordings and culture of

4

THE AUDIOVISUAL ELSEWHERE

69

film soundtracks, with practitioners exploiting aesthetics and compositional procedures derived from film, integrating traditional musical forces with sounds not previously considered musical.6

Widening Superfield Theory Both technology and aesthetics have dictated a headlong rush into establishing a unified aural field of music and other sounds in many films. Partially imposed by the overwhelming adoption of digital technologies, with their new filtering and enhancing procedures, sounds and music have increasingly become conceptualized in electronic terms (envelopes, bandpass filters, frequency spikes, pitch shifting, etc.). Although it could be argued that avant-garde films had already trodden this path before the coming of digital sound, it can be more convincingly claimed that digital technologies, adopted wholesale, have determined the “unified” approach to film sound and music. I realize that technological determinism is hardly the critical flavour of the month, but we must accept that technological capabilities set the horizons of aesthetic capabilities, and in the vast majority of cases, users adopt the technology’s path of least resistance. Michel Chion referred to the new sonic space offered by highdefinition sounds and directional multispeaker surround sound as the ‘superfield’, which alters the perception of space and thereby the more general rules of audiovisual scene construction.7 This superfield is not only qualitatively different from previous sonic space, but also marks an extension of off-screen space and possesses, “a kind of quasi-autonomous existence with relation to the visual field”.8 This “quasi-autonomy” frees film sound from slavishly backing up what is seen on screen. Developments in sound have been matched by developments in image. Spatialized, multitracked sound favours passive off-screen sound (which for Chion is off-screen sound that is not narratively foregrounded). Such passive off-screen sound works to establish more close-ups and wide spaces, as well as freer movement among shots. Sound also helps prevent the audience from experiencing spatial disorientation, while contributing to a new group of aesthetics that rework the basic conceptions and practices of film. Of course, incidental music in films and music in films more generally has attained a degree of independence, and other sounds have not only drawn upon this aspect, but also followed its path to an extent. In recent years an increasingly aesthetic rather than representational conception of sound in the cinema has emerged. Traditionally cinema

70

K. J. DONNELLY

sound was conceived and constructed in a basic compositional template. At its heart were clarity and intelligibility of dialogue, alongside uncluttered but functional composition of diegetic sound elements. Nondiegetic music occupied an unobtrusive position in volume and pitch, except at privileged moments. James Lastra notes that new sound pioneers such as Walter Murch, “discovered that the founding gesture of sound design, as it were, is both the complete severing of sensory experience from representation, and a compulsive linking of the two in an indissoluble unity that appeared to efface representation”.9 Sounds may almost lose their representational attributes as they serve emotional and psychological functions, the traditional domain of non-diegetic music in films. Some art films followed a long-standing tradition of imbuing sound effects with weighty emotional and symbolic significance, endowing sound with a primarily aesthetic role. Tarkovsky’s Stalker (1979) is a case in point, perhaps most notably during the sequence of the lengthy and memorable train journey to “the zone” where the soundtrack is dominated by the not-quite-regular diegetic clacking of the train on the track. The metallic sounds remain in the foreground, undergirded by some subtle chords, mixed from a small orchestra and synthesized non-diegetic music by Eduard Artemyev. As the journey progresses, the track noise becomes increasingly treated with electronics, so that it not so much represents the illusory world on screen as bestows an interior sense of compulsive mental pattern and transition, an equivalent to the visuals of the film’s emotional landscape. Discussing Apocalypse Now (1979), Jay Beck notes that “Coppola wanted a sound mix that emulated the quadraphonic musical recordings of the early 1970s, both in their multichannel location of the speakers as well as in their psychedelic sounds and style”.10 With music as inspiration, it is hardly surprising that on occasion diegetic sound effects adopt a function closer to that traditionally assigned to incidental music. For example, Ju-On: The Grudge (2003) includes extraordinary sequences in which sound could be interpreted as either music or sound effects. In both cases the ambiguity is doubtless part of the general effect of the film. Ju-On contains deep, ambiguous rumbles that might be construed as diegetic supernatural sounds. The status of many of the sounds remains unclear: they might be diegetic, or they might be part of the scary musical furnishings of the film. Indeed, deep sub-bass rumbles and metallic sounds with a wide frequency range have exploited the dramatic and psychological possibilities of an extended

4

THE AUDIOVISUAL ELSEWHERE

71

range of bass and treble tones available to 5.1 Dolby sound and Chion’s multispeaker superfield.11 The merging of music as an art of time with sound effects and their way of delineating space produces a physical sense of time–space continuum, remarkably similar to what Albert Einstein described in “Theory of Special Relativity” (1905). Through the addition of (musical) time as the fourth dimension, non-diegetic music plus diegetic sounds establish a fully relative conception of the superfield. The superfield is no mere add-on; rather, it involves fusion at the genetic level, rendering the soundtrack as a distinct object. A point of view from Gestalt psychology would not only focus on the field as a whole, but also register that individual aspects are perhaps less important than the dynamic relationship between the elements gathered together in that field. This might be a more fruitful approach to film soundtracks than the cognitive approach dominant in film studies, which focuses on isolated cues and atomized and hierarchized elements. A Gestaltist position would engage meaning as being produced through addition rather than copresence and interrelationship of the part to the entire entity of the soundtrack (and film).12 Furthermore, sound implies images, often suggesting a particular origin. In much the same way that Gestaltist perception diagrams prove the process of “reification,” wherein we mentally “complete” objects that are only partially present, so a Gestaltist approach might suggest that we “imagine” the completion of the soundtrack’s audio into a full “quasi” film, finishing the whole that is implied by significant component parts.

Audio That Implies Visuals “Hear the pictures! See the music!” was Disney’s synesthetic slogan for Fantasia in 1940. Filmmaker Robert Bresson claimed that sound always evokes an image, whereas an image never invokes a sound.13 Indeed, the propensity for sound to suggest images has been exploited by cinema and other media.14 The Gestalt of audio might include an absent visual component; the “audiovisual” ought also to embrace audio that implies visuals (and equally visuals that imply audio). Film soundtracks have influenced audiovisual culture outside the cinema—most obviously other screen culture, but also cultural forms with little or no relation to images. Examples range from Pink Floyd’s Dark Side of the Moon (1973) and Jeff Wayne’s War of the Worlds (1978) in the 1970s to more recent ambient music recordings.

72

K. J. DONNELLY

Ambient music owes much to film music, or at least to a concept of the “cinematic”; it is premised on “atmosphere” and the construction of soundscapes, much as film composers and sound designers aim to construct.15 Ambient music is often “image-friendly music”, which can thus be used in television programmes or films very easily and so has the enviable capability to be sold twice over. Brian Eno’s album Ambient 4: On Land (released in 1982) was a harbinger of the genre of ambient music.16 Fundamentally different from “popular music”, most of this recording lacks notable melody or harmonic movement, instead focusing on slow sonic events and textures; it aims to serve as a sonic backdrop rather than as a traditionally “musical” event. As Eno himself notes on the album’s cover, “cluster all disparate sounds into one aural frame: they become music”.17 Like film scores, such music aims to furnish aural impressions of locations and associated images (Fig. 4.1). ‘Visual music’ has remained an insistent margin of music, sometimes eschewing melody and concentrating on atmospheric soundscapes and ambiences, while aiming at an extension beyond simple music. Embracing drama, film and television culture, as well as non-musical inspiration, certain albums appear to derive more from films than from anything else. Mount Vernon Arts Lab’s album The Séance at Hobs Lane (2001; rereleased in 2007) consists predominantly of electronics, but sometimes mixes them with conventional instruments and features guest musicians such as Coil and Portishead’s Adrian Utley. An extraordinary recording, The Séance at Hobs Lane appears almost to be a psychogeographical investigation of subterranean London. As such it could not be farther from ‘popular music’, yet when approached as a concept album, its concept remains obscure and occult. Tracks include titles such as The Vauxhall Labyrinth and The Submariner’s Song (whose title is ironic—nothing remotely resembles a traditional ‘song’ on the album). The sleeve notes state (Figure 4.2): Imagine a landscape of sets built for a never-quite-produced British science fiction film of the 1950s. Hobs End was the invented tube [London underground] station in Quatermass and the Pit (an MVAL [Mount Vernon Arts Lab] touchstone). The Séance at Hobs Lane is a late-night gathering, convened to reconnoitre this subterranean world.18

Quatermass and the Pit , a celebrated BBC television science fiction serial broadcast in 1958–1959, was remade into a film by Hammer

4

THE AUDIOVISUAL ELSEWHERE

73

Fig. 4.1 Mount Vernon Arts Lab

Studios in 1967 (US title: Five Million Years to Earth). In the film excavations at the Hobs End underground station in London uncover what is assumed to be a bomb, later to be revealed as a buried spaceship. Sightings of a mysterious, short, goblinlike figure occur in the vicinity. Professor Bernard Quatermass works out that the spaceship brought aliens to Earth, who then modified some humans with their genetic material. This discovery leads to an atavistic battle between those humans who are possessed by their alien DNA and those who are not. The Séance at Hobs Lane is shaped by isolated elements from the television serial and the film. Its second track relates directly to the shady, diminutive figures that have been sighted at Hobs Lane at various times. Hobgoblins sounds reminiscent of The Stranglers’ Waltz in Black, an electronic waltz with an

74

K. J. DONNELLY

Fig. 4.2 Quatermass and the Pit

angular ‘absurd’ melody, whose archly comic tone has made it a popular sample for continuity music and library music on television. As it turns out, this is the nearest to a conventional piece of music on the album. Sir Keith at Lambeth consists of austere, continuous electronic tones (a deeper mechanical drone and more high-pitched, white-noise-based sounds). Indeed, this sounds more like a soundscape field recording from a zone of heavy industry than a piece of music. The higher tone fluctuates in intensity, at times sounding almost like a high-pressure water hose (or a distant boring machine), at others like a continuous alarm. Near the start of the track the deeper-pitched tone sounds at times close to a powerline or mains electric hum. The concluding solo note, with its slow vibrato, is less like anything heard regularly in electronic music than the sort of electronic sound effects used in science fiction films and television, most particularly for ray guns or auguring devices (such as the ‘tricorder’ in the 1960s series Star Trek). The principal member of Mount Vernon Arts Lab, Drew Mulholland,19 plays vintage synthesizers, including an EMS VCS3 (Putney).20 These are now much sought-after early synthesizers that sell for high prices. Made by Peter Zinovieff”s EMS company in the early 1970s, they featured on Pink Floyd’s Dark Side of the Moon and were used by the BBC Radiophonic Workshop to provide sound effects and music for television

4

THE AUDIOVISUAL ELSEWHERE

75

science fiction programmes such as Doctor Who (1963–1984), Blake’s Seven (1978–1981) and Hitchhiker’s Guide to the Galaxy (1981).21 One of EMS’s directors was the composer and electronics pioneer Tristram Cary, who provided the music, both orchestral and electronic, for the film version of Quatermass and the Pit. Some of the cues bear similarity to pieces on The Séance at Hobs Lane. For example, “Vessel” has a structure of repeated (looped) figures, one rising in pitch gradually like an engine gaining speed, one dropping lower, echo-repeated and more percussive in character. Similarly, “Maelstrom” is based on electronic echo and sounds like a spaceship taking off or the sound effect used in films to signify extrasensory perception (consisting of a very gradual rise in pitch). Carey also provided musique concrète for Doctor Who in the early and mid-1960s, extraordinary soundscapes that are somewhere between nondiegetic music and diegetic ambience for the Doctor Who stories “The Daleks” (1963) and “The Daleks’ Master Plan” (1965).22 Some of this avant-garde-inspired music is remarkably similar to The Séance at Hobs Lane, which overall is austere and often based on repetitive drones and basic analogue synthesizer sounds. Its tracks develop little, and there are no vocals for the listener to hang onto. An extension of film, The Séance at Hobs Lane provides a springboard for the imagination, rather than functioning as music per se.23 For example, the opening track, The Fog Detonator, consists of a succession of sounds of fireworks exploding (with a slight reverb, suggesting hard reflective surfaces). Though the title may or may not refer to some obscure, imaginary fog-dispersing device particular to London, sonically the track suggests a ‘pre-CD event’, or a particular location captured through field recording. It is doubtful that the recording could be listened to without imagery arising in the listener’s head. This CD remains a long way from conventional “music”. In essence, it implies an elsewhere: stories, ideas and locations with a strong visual element. None of these is summoned through sung words, but rather, more often through sounds with an almost identifiable origin. The sonic textures remain sparse, consisting throughout of few layers of sounds and with a sense of organizational focus evident in film soundtracks. Most of this album comprises acousmatic sounds, divorced from their unapparent (visual) origins. It implies vague visuals rather than presenting ‘face value’ sounds. The Séance at Hobs Lane derives less from a conventional avant-garde background (like Stockhausen) than from the edges of so-called popular

76

K. J. DONNELLY

music (such as Coil, who have produced film music as well as music on disc). Films and their soundtracks can function as a cultural repository,24 and not only in aesthetic terms. Film also functions as a repository of key ideas and abstract notions, seemingly having displaced literature as the custodian of the majority of the cultural imaginary.25 Equally, however, it is film’s place at the apex of audiovisual culture that is the aesthetic inspiration. Yet film also functions as a repository of sounds and sonic memory, alluding to an elsewhere and evoking visuals through esoteric and often elusive processes.

Technology and Intelligent Sound Design Miguel Mera and David Burnand suggest that an aesthetic modernism in film soundtracks aspires to unify music and other sounds.26 Although this may not be the norm across all mainstream cinema, one sees significant evidence of it taking place. As Mera and Burnand also suggest, there is not only a technological basis in unified film music and sound effects, but also an impulse from the fringes of cinematic and aural culture. The sea change from analogue to digital media occurred shortly after the turn of the millennium, completing a process that had started a couple of decades earlier in the audiovisual arts. The radical shift from analogue to digital sound had a notable impact on many aspects of cinema,27 enabling ever more precise manipulation of all its aural elements: music, sound effects and dialogue. Although big-budget films still wanted to have the sound of real orchestras, by the late 1990s musical scores could be constructed and edited on a computer screen at home, using nonlinear editing software and musical “soft studio” technology. Digital signal processing (DSP) and integrated soft studio packages were widely available by the late 1990s for home use and have followed a trajectory of increased capabilities and decreasing prices. The adoption of the resulting “digital audio workstation” (DAW) has revolutionized music production, allowing easy construction of relatively high-quality music on a home computer. Versions of this product include, among others, Steinberg Cubase, Apple Logic, Ableton Live and Propellerheads Reason. Each has its own strengths; Logic for Apple Macs and Cubase for PCs are the most popular among professional musicians. They are all based on integrated music recording and processing with dedicated software and a powerful sound card in the computer, converting sound impulses or computer

4

THE AUDIOVISUAL ELSEWHERE

77

instructions into a digital code that is sampled upon entry and synthesized upon exit from the terminal. One dividend of these modes of digital sound is that through their massive capabilities for signal processing, users have gained an increasingly acute awareness of aspects of sound and more developed sensitivity to precise sonic qualities. This new consciousness of sound is patent for those engaged in digital sound and music production, but also likely evident to wider populations who are consumers of more sonically sensitive material than previous generations. This situation promotes a ‘sound for sound’s sake’ approach, wherein primary aspects of sound and its microlevel manipulation become the most attractive part of the process for practitioners. This clearly can lead to an ambiguity about what might constitute ‘music’ and what might constitute ‘non-musical sound’, which at the very least has challenged the limited concepts of music and sound effects in wide circulation. Digital sound processing technology is used not only by ‘musical’ people, but also by sound designers and editors, to manipulate sound effects in precisely the same way musicians do when they compose music. Indeed, sound designers use “musical software” and digital products designed primarily for the production of music (such as Pro Tools) for processing both dialogue and sound effects.28 Whereas on the one hand, composers are perhaps more aware of sound as a compositional element in its own right than ever before, on the other hand, film sound people approach soundtracks in a manner that might be termed ‘musical’, or at least betrays a musical awareness of the interaction of elements and their individual sonic qualities. Consequently, music for films, and indeed film sound effects, can be produced by people with no musical or technical training. Equally, though, those with musical knowledge or sensibility can be effective sound designers for films. The increased ‘musical’ character of soundtracks is evident not only in the ways that sound is used and conceived, but also in its organizational imperatives. These include spatializing aspects that allow the copresence of sounds that might mask one another when mixed for mono or twotrack stereo. Also, pitches can be varied and sounds rendered through musical instruments. For example, the sound effects for Aliens (1986) were processed through a Fairlight CMI digital synthesizer.29 An example of someone with a musical background being involved in film sound effects is Alan Howarth. He is probably best known as a composer and synthesizer programmer who has worked with John Carpenter on films such as Halloween II (1981), Escape from New York (1981) and Prince of

78

K. J. DONNELLY

Darkness (1987). Yet he also created the rumbling sound for the starship at the beginning of Star Wars (1977) by layering recordings of white noise, air conditioning and sounds he recorded at jet engine laboratories. 30 Howarth also produced sounds for Star Trek: The Motion Picture (1979) and Poltergeist (1982) using keyboard synthesizers as well as field recordings. For Star Wars Episode 3: Attack of the Clones (2002), Ben Burtt also used electronic musical instruments for sound effects.31 George Pal’s War of the Worlds (1953) used electric guitars for the sound of the Martian death ray, and one of the more recent remakes (2005) used pitched, musical sounds for the Martian war machines. Along similar lines, Terminator Salvation (2009) used pitched musical sounds for the large machines that attacked the humans, providing a degree of novelty to the cinematic convention of predictable, mechanical sounds. The increasingly evident ‘musicalization’ of the soundtrack might owe something to practices developed in avant-garde film-making and sound art. The breakdown of the partition between diegetic sound effects and non-diegetic music began once the uniqueness of their functions was questioned. The avant-garde has worked with an expanded idea of music since the early part of the twentieth century, blurring the boundary between music and not music.32 This novel reconceptualization of music embraced noise and environmental sound; its inclusive notion of music and composition has exerted an influence on the holistic concept of current film sound design.33 In fact, avant-garde ideas of music are so expansive that they might conceivably include even the visuals of film itself.34 The avant-garde impulse to expand musical vocabulary to include what habitually were classified as ‘non-musical’ sounds has likely had a strong but unregistered impact on film soundtracks. A notable example is musique concrète, which developed around composer Pierre Schaeffer in Paris after World War II and processed library and field recordings into a form of abstract music. The practice was premised on manipulation of sound recordings on magnetic tape (changing speeds, running backward and using electronic filters and effects). The music created in this manner owed less to traditions of art music than to sound assemblages that were taking place in film-making (Fig. 4.3). Brian Williams, a musician and sonic artist whose career has involved working across audiovisual art and mainstream media, released discs under the name Lustmord. One of his early releases, Paradise Disowned (1984), included field recordings and musical performances in strange spaces, such as a nuclear bunker, a slaughterhouse and Chartres Cathedral. He

4

THE AUDIOVISUAL ELSEWHERE

79

Fig. 4.3 Lustmord’s The Monstrous Soul

worked with the experimental rock group SPK but continued releasing his own work as well. The Monstrous Soul (1992) includes very prominent samples from the British horror film The Night of the Demon (1957) (known as Curse of the Demon in the USA), whereas The Place Where the Black Stars Hang (1994) has a first track built on the almost continuous synthetic sounds of a massive propeller swooping around, towards and then away from the listener. The implication of not traditionally ‘musical’ and more clearly visual aspects is unmistakable. He has tended to use first analogue treatments, such as echo and reverb, and later digital sound treatment of field recordings, allied to often minimal atmospheric electronics and threatening sub-bass rumbles. Though some people have classified these recordings as ‘dark ambient’ music, their sound worlds

80

K. J. DONNELLY

owe a great deal to the kinds of ambient atmosphere evident in some horror films. Indeed, the product of the collaboration between Williams and electronic musician Robert Rich, Stalker (1995), was inspired directly by Tarkovsky’s atmospheric and enigmatic film of the same name. Like Eno’s On Land, this music aims to build a sense of a sonic landscape or ‘soundscape’. More recent albums, such as [O T H E R] (2008), continue to rely on large-scale electronic echo and reverb and sounds such as clanking metal bars, as well as more conventional pitched instruments. It was no surprise when Williams transported his penchant for such “atmospheres” into films, although instead of providing scores for films in the traditional sense, he has collaborated with his former colleague from SPK, Graeme Revell and later with Paul Haslinger. Williams has been billed as a ‘musical sound designer’ on over forty films, including The Crow (1994), Streetfighter (1994), Strange Days (1995), Mighty Morphin Power Rangers (1995), From Dusk Till Dawn (1996), Spawn (1997), The Saint (1997), Bride of Chucky (1998), Pitch Black (2000), Underworld (2003), Touristas (2006) and Underworld: Rise of the Lycans (2008).35 He has been responsible for providing certain sounds (field recordings, treatments, samples) and ambiences that can be inserted directly into the film or manipulated by a collaborating composer as part of the score. Williams’s position, despite the word ‘music’ in his job title, is good evidence of the degree of intermixing of musical and field recording components that now prevails in many films, giving a strong sense of unified sound design. In addition to providing sometimes quite conventional incidental music for television drama (Fox’s The Chamber [2001] and SciFi Channel’s Proof Positive [2004]), Williams has composed music for the video games Master of Orion 3 (2001), Cashflow 101 (2001) and Cashflow 202 (2002). His career arc has stretched from sound art into films (and also into television and video games), although his early (non-audiovisual) output already resembled the music heard in horror film soundtracks. His musical career illustrates the crossover, while also indicating how far various media exploit music that incorporates a high degree of “nonmusical” sounds. Indeed, one might argue that Williams’s case is far from exceptional: that horror film music and sound design have influenced the sound of video games, which in turn has fed back into horror films. Christophe Gans’ film version of the video game franchise Silent Hill, released in 2006, appears to be a unique case, however. The film’s sound is based on the existing sound world of the game: Akira Yamaoka’s

4

THE AUDIOVISUAL ELSEWHERE

81

highly effective and memorable video game music was used as a skeleton around which to build the film. The game’s sound world did not have to undergo many changes to make it to the film soundtrack; indeed, the lack of development between the media illustrates the extent of the sonic overlap. Video games like World of Warcraft (with music by Jason Hayes) and Nobuo Uematsu’s music for the Final Fantasy series of games, for example, owe much to other music, particularly film music.36 The dynamic sense of emotional content in orchestral film music has carried over as a desirable aspect in games, which share with film as many aural aspects as they do visual ones. As might be expected, video games that emanate from film franchises by and large contain scores that are derived from film score models.37 Good examples are Clint Bakajanian’s score for the Star Wars Jedi Knight series, Matt Griskey’s for Star Wars Episode 3: Revenge of the Sith and Harry Potter and the Goblet of Fire, and Jamie Christopher’s for Lord of the Rings: The Battle for Middle Earth. Established film music composers also have written for video games, such as Harry Gregson-Williams, who composed the music for the Metal Gear Solid series; Michael Giacchino wrote music for the Medal of Honor video game series before going on to write incidental music for television, such as the serial Lost (2004–2010, Bad Robot-ABC), and then for film, including Star Trek (2009) and Let Me In (2010). These examples demonstrate how easily music emanating from the film score aesthetic can adapt to other media, emphasizing the common core of much audiovisual culture rather than obvious differences and pointing to film aesthetics as the main unifying aspect. Clearly, apart from technology, a central determinant of developments has been the personnel involved and their ability to work across media. This encourages markedly similar approaches to different activities, illustrating the strength of certain types of music to determine something essential about character regardless of the media format.38 In recent years the sounds and emotional functions of film have proven a most significant inspiration, underlining its position as the epicentre of audiovisual culture. Music for films and instrumental music not intended for films are becoming increasingly similar. Music that bears the influence of films and their soundtracks has been used in films and television programmes. This marks a return in aesthetic terms and guarantees the directness of influence and connection. It also reveals the proximity of audiovisual media and how far the blueprint of film soundtracks has had a tangible effect on their near neighbours. Although the film and music industries have always

82

K. J. DONNELLY

been closely related, the proximity of film and musical aesthetics discussed here is part of a larger process leading to fading boundaries between film, music and other culture.

Conclusion Film soundtracks’ influence outside the cinema vividly illustrates the importance of film and other audiovisual culture as a paratext, with its sound and music processes gaining purchase in other areas of culture. In these instances, the audiovisual can be purely sonic, through its implication and evocation of associated visuals. Technology has engendered an expanded spatial sonic arena, wherein sonic elements (sound effects, music) have mixed into a sensual and psychological field. This development has led to varying degrees of autonomy for the soundtrack; rather than simply supporting the image track of film, it is able to imply and invoke images itself. This, in turn, has had a marked influence on—or engaged with—an already existing area of music that manifests audiovisual landscapes through sound only. In recent years the development of converging digital sound technology has allowed sound designers to use musical software to enhance sound effects in films and enabled music composers to produce their own music through the incorporation of sound effects. Such developments, in line with wider technological and aesthetic convergence and harmonizing platforms and industries, mean that music no longer is simply a “bolton” to films, but rather something more akin to a genetic integration. Technology and personnel, as well as aesthetic traditions, have instigated unified sound design, while the use of digital electronics for both music and other film sounds results in music increasingly being conceptualized in sonic terms. There is a reciprocal “musical” character to sound design in many films. Here, sound effects, ambience, and even dialogue have more of an “aesthetic” character rather than only a representational value. Does this evolution signal the end of traditional film aesthetics or a new dawn? Whatever it may be, it heralds the migration of traditional film dynamics and conception into other areas of culture. This augurs a new psychological involvement, an embodied aesthetic psychological state in films and elsewhere, enabled by the sensual aspects of sound and the emotional elevation and directness of music.

4

THE AUDIOVISUAL ELSEWHERE

83

Notes 1. Noel Burch, Theory of Film Practice (Princeton, N.J.: University of Princeton Press, 1981), p.96. 2. Michel Chion, Audio-Vision: Sound on Screen, ed. and trans. Claudia Gorbman (New York: Columbia University Press, 1994), p.85. 3. Lev Manovich notes that the now dominant principles of new digital media existed in earlier film format, and film has remained an interface and locus point for surrounding developments. The Language of New Media (Cambridge, Mass.: MIT Press, 2001), p.50. 4. R. Serge Denisoff and George Plasketes, “Synergy in 1980s Film and Music: Formula for Success or Industry Mythology?” in Film History, vol. 4, no. 3, 1990, p.257. 5. A prominent but critically overlooked example is musical recordings made by actors. There are many examples of these, including David Hemmings’s Happens (1967), Richard Harris’s A Tramp Shining (1968), William Shatner’s The Transformed Man (1968), Bruce Willis’s Return of Bruno (1987) and Scarlett Johansson’s Anywhere I Lay My Head (2008), to name but a few. 6. Although there is an undoubted similarity to radio plays, the aesthetic is arguably derived more from film, and indeed, film is an unregistered influence on the trend for organic ‘concept albums’ prevalent from the late 1960s. Dark Side of the Moon integrates voices and sound effects with the music, whereas War of the Worlds sounds like a film musical. 7. Chion, op.cit., pp.149–150. 8. Ibid., p.150. 9. James Lastra, “Film and the Wagnerian Aspiration: Thoughts on Sound Design and History of the Senses” in Jay Beck and Tony Grajeda, eds., Lowering the Boom: Critical Studies in Film Sound (Chicago: University of Illinois Press, 2008), p.135. 10. Jay Beck, “The Sounds of Silence,” in in Jay Beck and Tony Grajeda, eds., Lowering the Boom: Critical Studies in Film Sound (Chicago: University of Illinois Press, 2008), p.71.

84

K. J. DONNELLY

11. Deep, almost infrasonic sounds have been exploited by horror films, such as Lost Highway (1997), Dark Water (2002) and Paranormal Activity (2007), which used such sounds to indicate the onset of the titular activities. 12. See, for example, Kurt Lewin’s field theory of behaviour, which addresses a totality of events in a specific environment when dealing with group behaviour. Resolving Social Conflicts and Field Theory in Social Science (Washington, D.C.: American Psychological Association, 1997). 13. Burch, op.cit., p.90. 14. Of course, radio drama was an influence on film soundtracks, but also has arguably become influenced by film soundtracks. 15. There is also often subaudible presentation. Nicholas Cook and Anthony Pople, eds., The Cambridge History of Twentieth-Century Music (Cambridge: Cambridge University Press, 2004), p.502. However, there are different kinds of ‘ambient music’. Perhaps the most prominent use is the branch of electronic dance music that is not aimed at dancing (like Aphex Twin or Boards of Canada); there is also the ‘New Age ambient music’, which often mixes relaxing but conventional music with environmental sounds. These are both very different from the music I am discussing here. 16. Prendergast points to Eno’s centrality, but also to later developments from electronic dance music. The Ambient Century: From Mahler to Moby (London: Bloomsbury, 2001), p.131, pp.52–53. 17. Brian Eno, cover notes to Ambient 4: On Land, EG 2,311,107 (EG Records, 1982). 18. Lawrence Norfolk, sleeve notes to Mount Vernon Arts Lab, The Séance at Hobs Lane, GBX009 (Ghost Box, 2007). 19. Remarkably, Mulholland became “composer in residence” (a Leverhulme fellow) in the Geographical and Earth Sciences Department at the University of Glasgow. Drew Mulholland, Hayden Lorimer and Chris Philo, “Resounding: An Interview with Drew Mulholland,” Scottish Geographical Journal, vol. 125, no. 3–4, (September–December 2009), pp.379–380. 20. VCS3 synthesizers are less contemporary electronic keyboard instruments than they are a piece of audio hardware dominated by dials and a pinhole matrix for patching between modular sections of the instrument.

4

THE AUDIOVISUAL ELSEWHERE

85

21. Louis Niebur discusses how the BBC Radiophonic Workshop “complicated the boundary between ‘sound effect’ and ‘music’.” “The Music of Machines: ‘Special Sound’ as the Music of Doctor Who” in David Butler, ed., Time and Relative Dissertations in Space: Critical Perspectives on Doctor Who (Manchester, UK: Manchester University Press, 2007), 205. 22. Cf. Quatermass and the Pit: The Film Music of Tristram Cary, Volume One, CNS 5009 (Cloud Nine Records, 1996). Among other things, Carey provided the electronic effects for Casino Royale (1966) and musique concrète for Doctor Who in the early and mid-1960s. His album Doctor Who: Devil’s Planets, The Music of Tristram Cary, WMSF 6072–2 (BBC, 2003) showcases extraordinary soundscapes that are somewhere between non-diegetic music and diegetic ambience, from the Doctor Who stories “The Daleks” (1963) and “The Daleks’ Master Plan” (1965). 23. In an interview, Mulholland discusses the film only, suggesting its sound world is the inspiration for the album. Despite its significant reputation, the television serial was virtually impossible to see until the advent of home video. Mark Pilkington, “Mount Vernon Arts Lab: True Sound of the Underground,” Fortean Times, 2001, repr. June 2007, at http://www.forteantimes.com/features/interv iews/483/mount_vernon_arts_lab.html [accessed 13/12/2010]. 24. Digital popular music has not only raided old musical recordings, but also exploited sound samples from films, as much as it has had recourse to images and narratives inspired by films. 25. There are many examples of music videos inspired directly by film, such as Madonna’s Material Girl (1985), which was based on Marilyn Monroe singing Diamonds Are a Girl’s Best Friend in Gentlemen Prefer Blondes (1953), and Rob Zombie’s Living Dead Girl (1998), which is based on The Cabinet of Doctor Caligari (1920). 26. Miguel Mera and David Burnand, eds., European Film Music (Aldershot, UK: Ashgate, 2006), p.5. 27. Gianluca Sergi, The Dolby Era: Film Sound in Contemporary Hollywood (Manchester, UK: Manchester University Press, 2004), p.30. 28. Even in the 1980s, Alan Howarth, working on James Cameron’s Aliens (1986), processed sounds through a Fairlight CMI (computer musical instrument), one of the first sampling musical

86

K. J. DONNELLY

29. 30. 31. 32. 33.

34.

35.

36.

37.

38.

instruments. Anonymous, “Sounds of Sci-Fi II,” Computer Music, no.151, May 2010, p.64. Ibid., p.64. Ibid., p.62. Ibid., p.65. David Toop, Ocean of Sound: Aether Talk, Ambient Sound and Imaginary Worlds (London: Serpent’s Tail, 2001), p.78. As Randolph Jordan notes of Hildegard Westerkamp’s avantgarde musical pieces that were used in Elephant (2003), “One could easily go through the entirety of the film and believe that the sounds of Westerkamp’s work were actually elements created by sound designer Leslie Shatz, or perhaps even recorded on location.” “The Work of Hildegard Westerkamp in the Films of Gus Van Sant” in Offscreen, vol. 11, no. 8–9, August/ September 2007, p.4, atwww.offscreen.com/Sound_Issue/jordan_ westerkamp.pdf [accessed 22/06/2010]. An example of the dissolution of the border between film and music is musician DJ Spooky’s 2004 Re-Birth of a Nation, a ‘remix’ of D. W. Griffith’s Birth of a Nation (1915), which not only adds voice-over commentary and music but also adds graphics to the screen and recuts the film. Originally a live multi-screen event, it was later made available as a DVD. Indeed, it is not uncommon to have dedicated sonic atmosphere providers, such as Mel Wesson’s credit for “ambient music design” for The Da Vinci Code (2006). Rod Munday argues that since the advent of CD-quality sound for video games there is no such thing as specific “video game music,” as it is derived from other existing genres of music. “Music in Video Games,” in Jamie Sexton, ed., Music, Sound and Multimedia: From the Live to the Virtual (Edinburgh: Edinburgh University Press, 2007), p.51. Karen Collins notes that it is wrong to simply evaluate video game music by film music standards and concepts, although there are points of significant connection. Game Sound: An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design (Cambridge, Mass: MIT Press, 2008), p.5. In concert hall music, “program music” usually aspires to depict or alludes to something outside the music and arguably affects a synesthetic character.

CHAPTER 5

African American Film Music: Blaxploitation

African Americans had a significant, arguably defining, impact on popular music in the twentieth century. Their impact on Hollywood film music (musicals apart) was less apparent until the 1970s. The cycle of ‘Blaxploitation’ films marks a point where filmmakers, instead of looking to film convention and traditions of film scoring, aimed at procuring popular musicians to produce highly dramatic music which could then be used as a frame onto which to place the film images. In the majority of cases, rather than follow the Hollywood convention, where music would be written to accompany the precise requirements of a ‘nearly finished’ film, here the music was a semi-independent item, also released on disc, which would often have the images of the film to some degree cut to its requirements. As Lawrence Novotny notes, music is one of the key elements of Blaxploitation films: “many black exploitation films include contemporary rhythm-and-blues soundtracks that match the filmic images in theme and content. Significantly, both well-known and up-and-coming African American artists such as Curtis Mayfield, Earth Wind and Fire, and Isaac Hayes composed the soundtracks. The films often feature extended montages that are enhanced by the funky, upbeat songs”.1 In some cases, the soundtrack records for these films eclipsed the popularity or longevity of the films themselves.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025 K.J. Donnelly, Hearing Eyes, Seeing Ears, Palgrave Studies in Audio-Visual Culture, https://doi.org/10.1007/978-3-031-73913-2_5

87

88

K. J. DONNELLY

The 1970s phenomenon of ‘Blaxploitation’ films, premised upon depictions of African American urban culture, habitually included a strong input from popular music. For instance, Isaac Hayes produced the iconic title music for Shaft (1971), Earth, Wind and Fire supplied the whole score for Sweet Sweetback’s Baadasssss Song (1971), Curtis Mayfield produced both songs and incidental music for Superfly (1972) and Bobby Womack produced music for Across 110th Street (1972). Many of these films exploited black popular music, sometimes to the point where the character of the films almost verges on being film musicals in their wielding of songs and grooves. Indeed, the music as much as the distinctive imagery forms an essential ‘landscape’ for Blaxploitation films. This aesthetic landscape not only involves primarily urban settings but distinct stereotypes connected to African American communities and narratives involving street crime and violence. In terms of music, the sound embraces American soul songs and instrumental music based on sounds derived from soul, funk and jazz. Blaxploitation films were strident in terms of representation and featured sex and action as well as violence. Although often quite crude in conception and execution, and cheaply made, these films have boldness and energy, have retained something of a cult status and are considered remarkable in American popular culture. The genre’s significance is summed up well by Walker, Rausch and Watson: Blaxploitation films changed how black men and women were portrayed in films. Gone were the old negative stereotypes that had dominated films for over 70 years. Replacing them were new archetypes, including drug dealers, pimps, and hardened criminals. The new images were initially a welcome change for audiences, and the studios were more than happy to be making money, but the constant barrage of morally ambiguous anti-heroes soon led to a backlash.”2

The cinema screen appeared to offer freedoms that were not yet widely available to African Americans in US society. In contrast with many mainstream Hollywood films of this period, these films are not overshadowed by anxiety and fear but exude a sense of positive outlook and confidence.3 Concurrently, African American musicians were producing confident and novel popular music, running from developments in mixing jazz and rock led by musicians such as Miles Davis and Herbie Hancock, while soul and funk music were developing different forms that would go on to

5

AFRICAN AMERICAN FILM MUSIC: BLAXPLOITATION

89

have a dominating influence on dance music in the following decades. Black musicians were also producing strident music that mixed soul traditions with rock and extremely pronounced beats to produce ‘funk’. James Brown, Sly and the Family Stone and Funkadelic were particularly eclectic. James Brown excelled live in concert but also produced imaginative and complex studio recordings. Their musical style at least fed into Blaxploitation film soundtracks. It was striking that among the artists producing music for films, Sly and the Family Stone and Funkadelic did not provide soundtracks to any Blaxploitation films, although the leader of the latter, George Clinton, later wrote some successful film music. Some of this soul and funk music that fed into Blaxploitation films conveyed a serious political message, which was evident in some of the films’ music, while there was also a connection to proto-rap culture.4 Blaxploitation films evince a high degree of what might be called ‘rhythmic scoring’, where the beat of the music dominates and the film follows. The most common strategy in these films is the use of a ‘groove’, a repetitive rhythmic continuity which is particularly effective in accompanying action sequences or scenes without dialogue. For instance, in Super Fly (1972), foregrounded music usually of the character of a ‘groove’ accompanies the many scenes of walking or driving through the streets of New York. Such music has the character of being soul or funk music often without any singing to remain as a so-called ‘backing track’, dominated by a bass guitar riff and drum beat, which also is similar to the basic rhythmic foundation for improvisation evident in some fusion or jazz rock. A groove might be defined as arguably the successor to jazz’s sense of ‘swing’. A repetitive rhythm comprising a meshing of instruments, often cohered around the drum kit and bass guitar, which has particular emphases and nuances that give it a particular ‘feel’ or sense of movement (Fig. 5.1).5 The overwhelming majority of the musicians producing scores for Blaxploitation films had no experience of scoring films in the traditional sense. The assumption from the point of view of filmmakers was that musicians did not need to know the ‘craft’ of writing music for the particular requirements of film, and that assuming they were good at making music, and that such music could then be edited into the film and would be effective in itself. In many cases, the musicians involved would supply a handful of songs and instrumental versions of those songs, sometimes as extended instrumental workouts. These were usually written to fit the general character of the film in question and not written to fit a rough cut

90

K. J. DONNELLY

Fig. 5.1 Superfly

of the film, as had been the convention for film music production during the Hollywood studio era.

Genre and Industry The genre began in 1970 with Cotton Comes to Harlem and then Sweet Sweetback’s Baadasssss Song (1971), followed quickly by Shaft (1971).6 The first was a studio production, while the second was an independent production with Melvin Van Peebles starring, directing and producing. It cost only $450,000 but had made $12 million after a year.7 Many Blaxploitation films were made outside the mainstream, with independent productions, such as AIP’s Black Caesar (1973). Hollywood was quick to take advantage (certainly in distribution). In 1974 and 1975 Blaxploitation films buoyed the US film industry to its highest domestic box office in 28 years according to Variety.8 The films were almost all commercially aimed and aimed primarily at African American audiences through representing the black urban experience. However, there was a certain ambiguity about whether these films offered a positive and culturally specific version of African American life, or whether they continued a tradition of African American representation in pejorative terms. The films tended certainly to be filled with stereotypes, often celebrating crime

5

AFRICAN AMERICAN FILM MUSIC: BLAXPLOITATION

91

and misogyny. However, they usually depicted extremely strong African American characters, often striving against negative white characters. Josiah Howard offers a tentative definition of Blaxploitation films as “… 1970s black-cast or black-themed films … created, developed, and most importantly heavily promoted to young, inner-city, black audiences”.9 Lawrence Novotny’s study defines Blaxploitation films as “movies made between 1970 and 1975, by both black and white film directors alike, to exploit the black film audience”.10 According to Mikel Koven, the term ‘Blaxploitation’ was coined by the Variety reviewer of Cotton Comes to Harlem (1970).11 Combing ‘black’ and ‘exploitation’, the term was clearly intended to be pejorative, suggesting the low-budget sensationalism of exploitation cinema. However, others suggest it came as a critical term from black culture. The term was possibly coined Julian Griffin, the head of the Los Angeles branch of the NAACP (the National Association for the Advancement of Colored People), who was campaigning against films which they thought demeaned black people.12 Blaxploitation films initially were aimed at and consumed by black urban audiences in the United States. They were part of the ‘Coalition Against Blaxploitation’ (along with other groups of moral guardians) and this pressure among other things led to the petering out of this film cycle by the end of the 1970s. While on the one hand, these films catered specifically for black audiences, on the other they offered an exotic vision of urban African American culture for white audiences. Part of their impetus came from exploiting a specific, even ‘ghetto-ized’ audience, while some filmmakers approached such film-making as a ‘separatist’ enterprise, refusing ‘integration’ into the mainstream cinema on the terms of the dominant Hollywood studios. However, the Hollywood studios had been through something of a crisis in the 1960s. Overseas investment, particularly in British film production, had impoverished American film-making, but buoyed independent production companies such as AIP (American International Pictures). The return of investment to American films focused on relatively cheaply made films such as Bonnie and Clyde (1967) and Easy Rider (1968). Blaxploitation films similarly offered rapid and cheap production with rough and ready production values for an industry that was looking for narrower margins and more fragmented audiences. Although Blaxploitation was seemingly a coherent film genre, it hybridized easily with other genres, and this worked in a manner as if films could be ‘blackened’ through the substitution of black characters

92

K. J. DONNELLY

and concerns. Examples include distinctive runs of westerns and horror films (such as Blacula [1972]), as well as comedies and kung fu films (such as Black Belt Jones [1974] starring Gloria Hendry). The exceptions were derivations of the Hollywood gangster and crime genre, which cut out a character all of its own with particular urban settings and flamboyant characters.13 While women were presented as objects of desire, they also appeared as confident action stars, in films including Cleopatra Jones (1973) with Tamara Dobson, T.N.T. Jackson (1974) with Jean Bell, Black Belt Jones (1974) with Gloria Hendry and Coffy (1973), Foxy Brown (1974), Friday Foster (1975) and Sheba Baby (1975) with Pam Grier. Despite the diversity and variations across the Blaxploitation genre, music tended to be fairly constant in style. In terms of setting, these were overwhelmingly urban films and many were set in New York’s Harlem (literally ‘across 110th Street’ as the film proclaims), although not exclusively, as for example, The Mack (1973) was set in Oakland, Detroit 9000 (1973) in Detroit, Together Brothers (1974) in Galveston and Sugar Hill (1974) in Houston’.

Music Many of these films were notable for their music, and indeed, Blaxploitation films often took advantage of African American popular music to give a contemporary urban flavour to the films. This was a bold move, and well beyond the possibilities for music in film evident for popular musicians desiring to work in mainstream film production. A principal attraction of these films was the depiction of colourful characters from contemporary American urban life, while another clear attraction was the constant appearance of the contemporary sounds of black popular music. Indeed, African American popular music had become one of the main areas of pride and signifiers of black identity so it was no surprise that Blaxploitation films relied upon it so heavily. Music was a central defining characteristic of these films, using a sound palette for non-diegetic music that later filtered into mainstream use. This embraced drum kit and bass guitar grooves, choppy rhythmic wah wah guitar and bass guitar lines as ostinato melodies, among other aspects. In the overwhelming majority of cases, these films were not ‘scored’ as such in the traditional sense, but rather pieces of music were written to fit general situations in the film. The popular musicians involved simply imported the modes of soul and funk music to film. Indeed, the ‘scores’

5

AFRICAN AMERICAN FILM MUSIC: BLAXPLOITATION

93

produced by many soul and funk artists bear direct resemblance to their regular musical output. Rather than attempting to produce something approaching a conventional film score, they clearly produced music the way they knew how. This, of course, would mean that the ‘soundtrack album’ would not be a radical departure from the rest of their recordings and thus had a readymade clientele in the artist’s existing fans and record buyers. Much of the music in Blaxploitation films is song-based, either being songs or instrumental music that derives broadly from song structures. Sometimes the incidental music consists simply of ‘grooves’, which could easily be made into songs. Indeed, rather than follow the traditions of film scoring Blaxploitation films readily imported musical fabric and techniques from African American popular music, with a strong emphasis on the rhythmic impetus of the music and its structure based on 4-bar units. In relation to the traditions of film scoring, the process was closer to existing library music being ‘tracked’, simply cut-in over a piece of film footage, and controlled by means of either being cut (stopped suddenly) or being faded with a volume control knob. Certain timbres tend to dominate and provide a characteristic ‘sound’ of Blaxploitation scores. The music regularly is based on a muscular ‘rhythm section’ of electric guitar, bass guitar and drum kit, often playing a repetitive riff-based groove. The electric guitar is played almost as a percussion instrument, played as chopped chords or heavily palm-muted notes, this has a particularly distinctive sound which appears in film scores since at time but often with a desire to connote something approaching a black urban 1970s milieu. Perhaps the most iconic form of this is the choppy guitar played with a wah wah effects foot pedal that is perhaps the defining aspect of Isaac Hayes’ opening theme to Shaft. Similarly, Gene Page’s instrumental main theme for Blacula (1972) is based on a wah wah guitar groove alongside some less common wah wah bass guitar. Another featured instrument, or rather series of instruments, would be percussion. Most commonly bongos and congas, which supply a repetitive pattern of alternating drum pitches higher than the drum kit. A good example is the chase music from Willie Dynamite (1974) by J.J. Johnson. Hand-played drums, most obviously the congas played rapidly for action, such as in the pool hall sequence in Black Belt Jones (1974), which adds then big band brass to the fast-beat kinesis. Latin percussion was also evident, such as the wooden stroking sound of the guiro in the running music from Sweet Sweetback’s Baadasssss Song.

94

K. J. DONNELLY

Perhaps one of the most characteristic musical aspects of many Blaxploitation film scores is the use of bass guitar to produce a repetitive ostinato upon which the rest of the music is built. These can be part of a full song, rhythm section activity as a ‘groove’ or carrying on their own as part of a so-called ‘drop out’ where instruments stop playing for a period, leaving one to carry the music during a more quiet passage. This is evident, among other places, in Curtis Mayfield’s music for Super Fly (1972). Other common musical elements included closely co-ordinate horn sections, playing supporting melodies, and often punctuating rhythmically off the beat. An occasional but notable timbre was the use of breathy solo flute, which was something of a fashion in the late 1960s and early 1970s in film scores more generally. This appears in the quiet opening to the song Brother’s Gonna Work It Out by Willie Hutch, from The Mack (1973) and at the start of the main theme by Isaac Hayes in Truck Turner (1973), in which he also starred. Flute (by Jimmy Vinson) is also prominent in guitarist Don Julian’s score for Savage! (1973), which consists of a number of ensemble pieces broadly characterized as fusion but with a strong funk beat. One effect of popular music appearing in films was that certain aspects of popular music sounds were more prominent and noticeable, such as the emphasis of an open high hat cymbal beat, which often on records simply became a norm, in films could become a defining part of a groove to accompany action. Indeed, aspects such as this, which were merely a conventional passing element in a pop song could become more significant for music in films, defining the sort of minimal texture that was a rarity in popular music but could unify a long-scale piece of music. The use of the drum kit was almost universal in the non-diegetic music of Blaxploitation films. For African American audiences, Blaxploitation films must have sounded extremely familiar. The distinctive rhythmic character and timbres were derived directly from developments in wider African American popular music. Although instruments such as electric pianos (primarily Fender Rhodes or Wurlitzer) had become a more widely available instrument from the late 1960s onwards, they were a staple of soul and funk music and its cousin jazz rock. The importing of these musical aspects created a distinguishing ‘sound’ for Blaxploitation films, which almost never contained mainstream-sounding orchestral scores.14 Indeed, evident in many of these scores is an influence from fusion or jazz rock, which emerged from jazz musicians embracing rock rhythms and electric

5

AFRICAN AMERICAN FILM MUSIC: BLAXPLOITATION

95

instruments. However, this is not to suggest that there is a direct confluence into these scores from fusion music, it is more that the black popular musicians who were making this music adopted techniques to sustain music without singing, such as extended solos or changes in texture that were the sort of musical strategy common in jazz and related music. Influential figures in fusion included keyboard players such as Herbie Hancock and Chick Corea, guitarists John McLaughlin and Larry Coryell, bass player Ron Carter and drummer Billy Cobham. Miles Davis was a catalyst in developments, assembling crossover musicians for albums such as In a Silent Way (1969) and Bitches Brew (1970), while one of his collaborators, Herbie Hancock, later produced a landmark funk-influenced album, Head Hunters (1973). Elsewhere, urban-set films followed suit with action underscored by drum kit beats and electric bass guitar ostinato forming flowing ‘grooves’. In 1971, Lalo Schifrin’s score for Dirty Harry (1971) exploited the sort of jazz rock sound evident in Miles Davis’ recent Bitches Brew double album. Having a similar musical palette to many of the later Blaxploitation films, the score incorporates drum kit, bass guitar and a wordless female vocal, with energetic exotic percussion keeping the score and the film’s action moving. However, it also used orchestral strings, sometimes inspired by modernist art music with resonating cluster chords. The sequel, Magnum Force (1973), was also scored by Schifrin and although in many ways it sounds like a continuity, it in fact sounds notably similar to Gordon Parks’ score for Shaft’s Big Score, which was released a year earlier. It is in effect energetic funk-inspired jazz rock. I am not suggesting a direct influence, merely noting that there is a similarity. Schifrin also used a similar musical language in Enter the Dragon (1973), which although it was a Bruce Lee vehicle also had an African American character played by Jim Kelly who went on to star in a handful of Blaxploitation films.15 Musicians of distinction were attracted to these films. Roy Ayers had been in demand as a specialist jazz vibraphone player. Starting as a jazz musician in the US West Coast jazz scene of the early 1960s, he later began to mix jazz with other forms, notably funk in the 1970s and hiphop in the 1990s. As a highly influential musician and composer, Ayers’ career has involved a massive variety of recordings with a great many different musicians. Indeed, to the point where his scores for Blaxploitation films only get the barest mention in summaries of his career. In 1973, Ayers recorded the memorable and effective score for Coffy, which starred Pam Grier and was about a nurse who becomes a vigilante to fight drug

96

K. J. DONNELLY

dealers. While jazz musicians such as Ayers and J.J. Johnson produced Blaxploitation film music, there were many more soul or rhythm-andblues musicians who made music for these films. Examples include Willie Hutch (The Mack [1973]), Osibisa (Superfly T.N.T. [1973]) Foxy Brown [1974]), Marvin Gaye (Trouble Man [1972]), Bobby Womack (Across 110 th Street [197] with J.J.Johnson) and Barry White (Together Brothers [1974]).

Defining Films Sweet Sweetback’s Baadasssss Song remains a startlingly original film, following …. It starred, was written and directed by Melvin Van Peebles. Sweet Sweetback’s Baadasssss Song also has a score credited as “written and composed by Melvin Van Peebles” and “featuring Earth, Wind and Fire”. Van Peebles had a background which had included recording music. He had recorded an album for A&M, called Ain’t Supposed to Die a Natural Death which included a version of the song Come On Feet Do Your Thing which appears in the film. Van Peebles provided musical ideas and some melodies, which were then developed by the musicians.16 Earth, Wind and Fire were not famous at this point before they had achieved a number of big hit records in the mid and late 1970s, although they had already released their self-titled debut album immediately before working on the film. A wholly independent production, the film initially had trouble getting distribution,17 although ended up making a lot of money for Van Peebles. The film’s music involves the constant use of musical ‘grooves’, repetitive rhythmic backing tracks, and a few songs. The memorable repeated organ-based theme accompanies a running montage as Sweetback escapes from the white police. Beginning with some startling colour effects on the image, he runs through hills in a semi-urban location as the continuous musical piece develops from an extended saxophone solo to an electric piano solo. The same music appears for shots of cars driving in the street. The music is built around a brisk beat and a repeated two-bar organ motif of three major chords (I-mIII-mVII). This is supported by a leading bass riff and forms the foundation for sustained improvisation by saxophone and horns, and later electric piano. Although the music initially sounds like a soul backing track, its lack of singing and instrumental solos bears a stronger resemblance to jazz. The music’s continuous structure and dynamics make an effective and energetic accompaniment to the chase on

5

AFRICAN AMERICAN FILM MUSIC: BLAXPLOITATION

97

screen. Indeed, variations on this appear for other dialogue-free moments in the film particularly if they include movement. One arrangement has some prominent electric piano, one of the key instrumental sounds in later Blaxploitation films and in 1970s soul and funk music. The melody is related to the chant which appears at the conclusion of the film (“they bled your brother”), which is sung in a call and response pattern reminiscent of a religious song. There is a looseness to the relation of sound and image in Sweet Sweetback’s Baadasssss Song, which comes from much of the film having been shot silent, meaning that the soundtrack was mostly built in post-production, which allowed for more prominent music.18 The concluding section of the film underlines the film’s singular aesthetic strategy. The music involves a strange vocal chorus which addresses the character Sweetback, alongside ‘vox pop’-style images where people (who seemingly are not actors) declare that they have not seen him. This is a means of expressing the police hunt for him and includes some startling repeated footage and dialogue (such as where a woman asks, “Did I have a LeRoy?”).19 This extensive flight sequence has run most of the way through with the three-chord riff appearing on and off, concluding with Sweetback arriving in Mexico. It contained no sustained dialogue and almost constant music, giving it a perpetual pace and purposefulness. Shaft (1971) was produced by Hollywood studio MGM, who had bought the rights cheaply and expended a low production budget on the film. Isaac Hayes supplied the iconic title song and some instrumental pieces for the music, although the music was a collaboration with Johnny Allen, who was a highly successful arranger for records, specializing in string and brass arrangements. Hayes’s theme has a long instrumental introduction, premised upon a choppy guitar rhythm played through an opening and closing wah wah pedal. The key was the combination of the defining guitar with orchestral forces, which worked a rhythmic arrangement around the guitar’s central and continuous position in the arrangement. Amid muscular orchestral stabs, Hayes’ gravelly baritone voice half sings and half speaks the words with interjections from a chorus of female backing vocalists. The lengthy title track accompanies an extended opening/title sequence, which depicts John Shaft (Richard Rowntree) walking through the streets of New York. Shaft won an Oscar for Hayes’s theme song and an Oscar nomination for the score. The sequel was Shaft’s Big Score (1972), which was made with a larger budget. The score was by the director, Gordon Parkes, although it included songs

98

K. J. DONNELLY

sung by O.C. Smith and one by Isaac Hayes. The score embodies Blaxploitation film music’s mixture of jazzy orchestra and muscular rhythmic grooves dominated by drum kit and electric bass guitar. The music draws on eclectic sources. For instance, there is a swinging jazz piece for the funeral (replacing diegetic sound), while the sex scene which is obscured by special lens effects, is accompanied by a smooth song sung by O.C.Smith. This slow love ballad (Don’t Misunderstand) obliterates diegetic sound and works anempathetically with the images, first showing an exotic dancer in a club where this appears to be diegetic music but then introducing parallel editing cuts away to a violent attack.20 The third film in the series, Shaft in Africa (1973) had an even larger budget and was directed by experienced English film director John Guillermin, who went on to direct The Towering Inferno (1974). On this occasion, the score was by Johnny Pate and it featured three songs by the Four Tops, who were a perennially successful musical act by this point. The score exhibited key characteristics of most Blaxploitation scores, matching dense drumming with wah wah guitar semiquaver pulse and close-playing ‘big band’-style horns. The director of Shaft, Gordon Parks snr, was a veteran filmmaker, photographer and musician, and one of his sons, Gordon Parks jnr, went on to direct another of the canonical Blaxploitation films, Super Fly (1972), less than a year after his father had directed Shaft. Super Fly encountered many difficulties during the shoot; the studio stopped the money supply and producer Sig Shore managed to raise finance to finish the film from African American businesspeople.21 While Shaft made the genre popular Super Fly developed and set the style, iconography and terms of representation.22 The film also showcased an outstanding Blaxploitation score. The music by Curtis Mayfield is highly impressive including a number of songs which comment on the film’s action, such as Pusher Man and Freddy’s Dead. Although the film appears to glorify drug dealer Priest (Ron O’Neal), Mayfield’s songs provide a more critical commentary which does not directly tally with the film’s tone. This is not to suggest a disjunction. The music fits the film well but has a far more critical tone than the rest of the film. Pusherman is sung from the point of view of the drug dealer, although it mostly appears in the film without singing. It includes exotic percussion, tuned drums (possibly rototoms), alongside Mayfield’s high-pitched and intimate voice, the use of the male falsetto voice of which is not uncommon in soul music. The music is riff-based—on bass guitar with

5

AFRICAN AMERICAN FILM MUSIC: BLAXPLOITATION

99

electric guitar support—augmented by busy varied percussion played across a steady drum kit beat. The beat being slightly ‘pushed’—the first beat of the bar arriving slightly early—helps generate the ‘groove’ and sense of rhythmic energy. As a song, Pusherman appears as nondiegetic music when the car drives through the street at night after the heist. When protagonist Priest enters a club and meets people, Curtis Mayfield is performing on stage, singing Pusherman, with a predominance of medium close up shots of him. The song also appears for the remarkable sequence which represents a montage of making drug deals, making drugs and taking drugs but through a succession of still images. This surprising aesthetic move includes split screens but significantly also includes some cutting to beat, which retains continuity with the Super Fly’s use of rhythmic music as a central structural tenet of the film. Indeed, Super Fly regularly exploits the rhythmic impetus of the music, with Priest’s arrest near the end of the film involving a groove of percussion, featuring congas heavily, alongside percussively clicking wah wah guitar, leading to a few horn punctuations. Earlier in the film, Priest and his woman walk on a bridge and by the river in the snow accompanied by a bossa nova rhythm-based cue which fades out quite crudely when they talk and then then back in again once they finish. This illustrates how the music was not written to fit the action but has been edited and faded to fit the final cut of the film. Super Fly and Freddy’s Dead both made an impact on the singles charts at the time of release. The title song appears a few times as instrumental versions in the film, including opening the film, but the existence of an instrumental version (as ‘score’) and a sung version illustrates the dual function of much Blaxploitation music: as a score providing an essential character for the film, but also as output from musicians whose main area of production was as music artists rather than in the film industry. This marks a form of synergy where the music’s primary function is as music in itself rather than tied directly to the film. These two songs, along with Pusherman, dominate the film’s soundtrack. The pre-title sequence follows two junkies walking through the streets accompanied by Super Fly as non-diegetic music with diegetic sound pushed to the margins. The onset of the titles is accompanied by an instrumental arrangement of the song Freddie’s Dead. Later sequences, including one of the streets beginning with a close up of the front of a car use this instrumental version of Freddy’s Dead. The sung version of Super Fly accompanies Priest dealing with white men in a restaurant, replacing diegetic sound fully to occupy

100

K. J. DONNELLY

the foreground of the film. Super Fly was Mayfield’s first score. He went on to score Claudine (1974, with Gladys Knight and the Pips), Let’s Do It Again (1975, with the Staple Singers), Sparkle (1976, with Aretha Franklin), A Piece of the Action (1977, with Mavis Staples), and prison film Short Eyes (1977), where he appeared as a prisoner singing the song Doo Wop is Strong in Here. Black Caesar (1973) had a certain moral ambiguity, following the rise of a black criminal in Harlem. It starred Fred Williamson (who had originally been a football player) and Gloria Hendry and had music by James Brown. Written, produced and directed by Larry Cohen for AIP (American Independent Pictures), Black Caesar was an iconic and successful gangster film that owes something to Warner Brothers gangster films of the 1930s Little Caesar (1931) and Angels with Dirty Faces (1938). The score and diegetic music for the film was written and performed by James Brown, often dubbed ‘the Godfather of Soul’. This is proclaimed at the start of the film and one of the final credits lists the availability of the soundtrack album on Polydor Records. On the end titles, the film credits the song Big Daddy to the writing team of Lenny Stule and Joelle Cohen. Although he had a rough cut of the film to work from, Brown and his band did not produce music that followed traditional patterns of musical underscores. Instead, they produced pieces of music that they thought fit the mood of the scenes that were deemed to need music. Director Larry Cohen noted: James did a fantastic job but unfortunately the music he submitted wasn’t timed out to fit the actual scenes in the film. I called up his manager and told him that James had not done what he had been contracted to do and his manager just said, ‘So, he gave you more than you paid for!’ I said, ‘Unfortunately, that’s not the way it works. We need the music to fit the scenes exactly – that’s why we gave you a copy of the film.’ In the end, I made it work but AIP was furious. James had done the same thing with another of their films, Slaughter’s Big Rip-Off . Consequently, when it came time to do a sequel, James had to do his music on spec. When AIP refused it, he released it as a solo effort called The Payback and it became the biggest album of his career.”23

Cohen’s description of Brown supplying ‘too much music’ which did not ‘fit’ illustrates how many inexperienced film composers worked: through simply providing music that fitted the character of the film rather than to fit specific dramatic moments or precise dynamic developments. In

5

AFRICAN AMERICAN FILM MUSIC: BLAXPLOITATION

101

some cases, such musicians simply supplied songs. There are a few songs in Black Caesar. The film opens with Brown singing Down and Out in New York City, which was written by Bidie Chandler and Barry DeVorzon. The soul funk character forms a suitable homology with the images of urban America, despite the action on screen having been set in the early 1950s and the music sounding distinctly something from two decades later. Inaugurated by a brutal shooting, the song is crudely silenced by a rapid fade rather than dovetailing with the film’s action. The incidental music in the film consists of ‘grooves’, repetitive rhythmic pieces that have little dynamic development and merely provide a background for the images. These are simply cut-in and provide a strong atmosphere and energy to the sequences where they are used. Songs are used in a similar manner, with one appearing for a traditional montage sequence. In each case, music tends to appear when there is little or no dialogue. Another song, Mom’s Dead, appears at his mother’s funeral and reappears slightly later, on both occasions continuing under dialogue.

Conclusion Although in recent years Blaxploitation films have been generally celebrated it has been to a degree at the expense of their controversial status in the 1970s. Shaft ’s title song tells of priorities of the film (and the genre following in its wake). The opening line is: “Who’s the black dick who’s a sex machine to all the chicks”. Strong gender division is reinforced by the song’s use of popular music, especially soul, tradition of a male lead singer and female backing singers. (As if the singer is the protagonist and the backing female singers are ‘all the chicks’.) While this might, perhaps unconvincingly, be dismissed these days as steeped in irony, or the whole might be received by current audiences in a sophisticated ‘ironic’ mode, criticisms cannot easily be dissipated. The NAACP (National Association for the Advancement of Colored People) and Rev.Jesse Jackson denounced Superfly, stating that the films made the audience feel momentarily powerful but at the expense of not only failing to help the cause of African American empowerment but also denigrating black culture. Indeed, Blaxploitation films almost uniformly showed African Americans in relation to crime, drugs, pimps and prostitution. Picketers outside cinemas held placards reading: ‘Black Shame, White Profits’.24 Indeed, they were uniformly cheaply made films usually made by white producers and investors but significantly they primarily

102

K. J. DONNELLY

targeted young black audiences.25 African American TV producer Tony Brown stated: The ‘blaxploitation’ films are a phenomenon of self-hate. Just look at the image of Super Fly. Going to see yourself as a drug dealer when you’re oppressed is sick. Not only are blacks identifying with him, they’re paying for the identification. It’s sort of like a Jew paying to get into Auschwitz. Blacks who contribute to the making of these films are guilty of nothing less than treason.26

These sentiments indicate the vehemence of opposition that Blaxploitation films received. Yet they were popular with urban black audiences because they represented the community’s locations and represented blacks as heroic and powerful. Despite the well-founded criticisms, these films gave opportunities to African American filmmakers and musicians. For instance, a number of black film directors, such as Gordon Parks Snr., Melvin Van Peebles, Oscar Williams, Gordon Parkes jnr., Bill Crain, Michael Schultz and Ivan Dixon came to the fore working on Blaxploitation films. Furthermore, Blaxploitation films moved music to the centre of dramatic films. Not only giving scores for African American musicians to work in film, the type of music that graced Blaxploitation films became evident elsewhere in other films and television programmes, and the use of repetitive rhythmic ‘grooves’ as a backing of musical continuity for action sequences became prevalent in mainstream films.

Notes 1. Lawrence Novotny, Blaxpoloitation Films of the 1970s: Blackness and Genre (London: Routledge, 2008), p.20. 2. Andrew Walker, Andrew J.Rausch and Chris Watson, Reflections on Blaxploitation: Actors and Directors Speak (Lanham MD: Scarecrow, 2009), p.ix. 3. Another definition comes from Canadian film fanzine Trash Compactor suggested that “… true Blaxploitation … occurred when wagon jumping white boy producers hired primarily black casts to provide chiefly black audiences with ‘what they wanted’. Namely this was to witness ‘The Man’ held up for ridicule by the jive-strutting soul brothers. Of course most of the green from these ‘black’ only films found its way deep into white lined pockets

5

4. 5.

6. 7. 8. 9. 10. 11. 12. 13.

14.

15.

16.

17. 18.

AFRICAN AMERICAN FILM MUSIC: BLAXPLOITATION

103

that were basically just mining the then current off Hollywood ‘in thing’ of Black Awareness.” Anon, “Back in Black (Again)” in Trash Compactor, vol.2, no.6, Summer 1992, p.15. Darius James, That’s Blaxploitation (New York: St.Martin’s Griffin Press, 1995), p.90. For a more technical analysis of the notion of groove, see G.Siros, M.Miron, M.Davies, F.Gouyon and G.Madison, “Syncopation Creates a Sense of Groove in Synthesized Musical Examples” in Cognitive Neuroscience, vol. 5, no. 1036, 2014. Josiah Howard, Blaxploitation Cinema: the Essential Reference Guide (Godalming, Surrey: FAB Press, 2008), p.10. Ibid., p.10. Ibid., p.15. Ibid., p.7. Novotny, op.cit., p.18. Mikel Koven, Blaxploitation Films (London: Kamera, 2010), p.11. Howard, op.cit., p.12. Some of the actors involved came from positions of celebrity elsewhere. For instance, athletes featured in many films, with Fred Williamson (football player), Jim Brown, Rosey Grier and singers, such as Isaac Hayes, Curtis Mayfield, Gladys Knight, comedians like Rudy Ray Moore, Richard Pryor and Redd Foxx and glamour models such as Gloria Hendry, Jean Bell and Marki Bey. This particular musical style has a rapid influence on US TV shows, particularly cop shows with urban settings, perhaps most notably Starsky and Hutch (1957–79, Spelling-Goldberg/ABC). At the same time, Roger Moore’s debut film as James Bond, Live and Let Die (1973), took direct inspiration from Blaxploitation films particularly in its New York scenes. However, the score, written by ex-Beatles producer George Martin owes little to these films as does Paul McCartney and Wings’ title song. Perhaps this was an assertion of the Britishness of the series through music in the face of a film which is wholly set in the USA and focuses particularly on the African American community. Van Peebles’ basic notation is reproduced in Melvin Van Peebles, The Making of Sweet Sweetback’s Baadasssss Song (Edinburgh: Payback Press, 1996), p.158. Walker, Rausch and Watson, op.cit, p.171. Van Peebles, op.cit., p.91.

104

K. J. DONNELLY

19. While Sweet Sweetback’s Baadasssss Song uses some radical techniques, perhaps this is the most startling due it still being a rarity in cinema. An argument might be made for this homologizing later hip hop record ‘scratching’ where the same piece of music was moved back and forth, to repeat the same excerpt. 20. The large ensemble that recorded Parks’ music included wellknown jazz musicians Joe Pass on guitar and Freddie Hubbard on trumpet. 21. James, op.cit., p.80. 22. Howard, op.cit., p.11. 23. Larry Cohen interviewed in Ibid., p.27. 24. Howard, Ibid., p.12. 25. Ed Guerrero, Framing Blackness: The African-American Image in Film (Philadelphia: Temple University Press, 1993), pp.68–69. 26. Howard, op.cit., p.12.

CHAPTER 6

Adapting Stage Musicals in Britain

There has been a familiar relationship between the British stage and the British cinema, which has been manifested most clearly in the relationship between stage musicals and film musicals. This is what might be expected, in that the overwhelming majority of stars, musicians, songs and shows in musical films have originated on the stage. However, there have been some distinct surprises in what has failed to be translated from stage to screen. Some very successful British stage musicals have not managed to make the transition to the screen, while others, conversely, have been made into film musicals in Hollywood rather than at home. London’s West End theatres consistently have been full of highly successful musicals, many of them home-grown. The abiding question is to wonder why these have only rarely been translated into musical films. Over the past few decades, there have been massive stage successes for Andrew Lloyd Webber’s stage musicals, not only in London’s West End but also on Broadway and elsewhere abroad, and yet these have not yet been exploited by British cinema to the degree that might have been expected, although Hollywood has cashed in on a few of them. A number of notable individuals have had a great impact on the British musical stage, with varying degrees of success in films too. It is something of an enigma as to why Noel Coward never made a British film musical. Even his successful stage musicals were only rarely made into © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025 K.J. Donnelly, Hearing Eyes, Seeing Ears, Palgrave Studies in Audio-Visual Culture, https://doi.org/10.1007/978-3-031-73913-2_6

105

106

K. J. DONNELLY

films. It is interesting how maverick talent on the British musical stage (and in the popular music industry as well) has failed to be exploited by the British film industry to the degree that it might have been. Apart from Coward, this chapter will look at the career of another renaissance man, Anthony Newley, who made a British film musical but found more success in Hollywood, and Andrew Lloyd Webber. This has been against a backdrop of very successful runs in London’s West End, where the musical has been a perennial while on screen it gradually has become marginalized in film production, evolving into a more occasional production rather than a staple of cinema. In terms of scholarly attention, there has been a surprising lack of interest in the transposition of musical successes from the stage to the screen. For Instance, Geoff Brown’s useful article on the relationship of British film to theatre concentrates overwhelmingly on non-musical stage productions and their adaptation, making hardly a mention of musicals.1 This chapter will chart a perspective on the relationship between the British musical stage and the screen, concluding with a more detailed consideration of three notable figures of the British musical stage and their relationship to the screen.

Traditions What became known as the ‘English musical comedy tradition’ can be traced largely to plays with music in the eighteenth century, most notably John Gay’s The Beggar’s Opera, which debuted in 1728 and used familiar song tunes. This tradition was crystallized by the later series of successful musical comedies of Gilbert and Sullivan. This pairing produced 14 comic operas between 1871 and 1896, including Ruddigore, H.M.S. Pinafore, Iolanthe, The Mikado and The Gondoliers. These were massively popular and have remained something of a perennial ever since. They certainly cast a shadow over stage musicals in the early part of the twentieth century. In his review of the state of English music in 1945, critic Eric Blom looked negatively at developments in popular music over the first four decades of the century: the popular aspects of music in our days. They are not pleasing. Operetta has steadily degenerated since Sullivan’s time, and although Lionel Monckton, Howard Talbot and others – we must disregard the vogue of Viennese purveyors like Léhar, Leo Fall and Oscar Straus – still

6

ADAPTING STAGE MUSICALS IN BRITAIN

107

managed to whip up some pleasant froth, the English modern musical comedy falls quite flat.2

Blom noted that the American musical stage of the time was superior to the British. His review is a strange and highly opinionated picture that (certainly in terms of its discussion of “serious” music) now seems highly eccentric, but most likely embodied a certain conservative and elitist view of musical activity. Indeed, it is striking just how far modernism and innovation in British music are ignored by Blom’s account. The European (largely Austrian) tradition of operetta had a significant impact in Britain as elsewhere. Its most notable indigenous manifestations were the comic operas of Gilbert and Sullivan, which were comedies with a sprinkling of rousing, comic songs. The earliest adaptation of Gilbert and Sullivan to the film was The Mikado (1938) directed by Victor Schertzinger. It was produced by the D’Oyly Carte company, which had been formed by theatre manager Richard D’Oyly Carte, who had done much to keep Gilbert and Sullivan’s collaboration together at the time and had developed theatres on the back of the success of these musicals. The biographical aspect of the pairing was emphasized in Launder and Gilliat’s The Story of Gilbert and Sullivan (1953) and Mike Leigh’s Topsy-Turvy (1999), which also included extensive onstage performance sequences, concerning the production of The Mikado. However, it is worth noting that the perennial favourites of Gilbert and Sullivan translated rather less to the screen that might have been expected. British cinema had a good hunting ground for its material on the British stage, where successful stage musicals were often relatively quickly adapted for the screen. Ivor Novello was not only a successful actor and writer but a tremendously successful songwriter. Among other songs which he wrote, Keep the Home Fires Burning became massively popular during the First World War. Novello wrote exotic and escapist musicals during the 1930s, such as Glamorous Nights (1935), Careless Rapture (193) and The Dancing Years (1939), the latter of which had an extended run throughout the Second World War.3 The Dancing Years had something of the operetta about it in that it started in early 1900s Vienna, but was still essentially a play with songs. Chu Chin Chow debuted on stage in London in 1916 and ran for a number of years. It was a fine example of the popular desire for exotic Orientalism, being inspired by the story of Ali Baba and the Forty Thieves from the Arabian Nights and set in Baghdad. As might be expected, it

108

K. J. DONNELLY

included Westerners under the spell of inscrutable and treacherous Easterners and included the song Any Time’s Kissing Time. It had some large-scale musical numbers with dances and flamboyant costumes, which were translated into the film version of Chu Chin Chow (1934), directed by Walter Forde and featuring George Robey, Fritz Kortner and SinoAmerican actress Anna May Wong. It had derived its production pretty much straight from the stage version of the time, a clear option for film production the world over. Probably, the most successful British stage was Me and My Girl , which debuted in 1937. Its success inspired a rapid adaptation to celluloid, and it was made into the film The Lambeth Walk (1939). Me and My Girl had been one of the most successful West End musicals of all time. It was written by L. Arthur Rose and Douglas Furber and had music by Noel Gay (who had been born Reginald Armitage but took a nom de plume that conflated Noel Coward and singer Maisie Gay). Lupino Lane produced and starred in the play, about a Cockney who learns that he is really a lord and goes to live at a country hall. The Lambeth Walk took its name from the most famous song in the play and reduced the number of songs in the film to only two (The Lambeth Walk and Me and My Girl ).4 British stage musicals had been popular at home, but in the late 1940s and early 1950s they were increasingly marginalized by imports from Broadway. Rodgers and Hammerstein’s earliest collaboration Oklahoma! came onto the British stage in 1947. According to Dominic Shellard, it “recast the template of the musical”.5 It broke the form’s convention of starting with a big number, included a murder and was extremely simple in terms of set and song melody. This led to a welter of American musicals hitting London, although there were still home-grown musicals, such as Bless the Bride (1947, at the Adelphi) written by Vivian Ellis, King’s Rhapsody (1949, at the Palace) and Call Me Madam (1952, at the Coliseum). Discussing the period from the end of the Second World War to the mid-1950s, Shellard notes that The appeal of the musicals for British audiences was matched only by the appeal of their profitability for theatre producers. Whether a license to print money or not (adumbrating the West End in the 1980s and 1990s), the sheer proliferation of the genre on the London stage at this time does refute the notion that this was a glamour starved period for the London stage.6

6

ADAPTING STAGE MUSICALS IN BRITAIN

109

Some significant British productions of the time included The Boy Friend, first staged in 1953 and written by Sandy Wilson. It was a pastiche of musical comedy from the 1920s and made its lead Julie Andrews into a star. This was made into a film by Ken Russell in 1971, starring the model Twiggy. Another notable production was Salad Days, debuting in 1954 and written by Julian Slade and Dorothy Reynolds, which was about a magic piano that caused people to start dancing. Other British stage musicals included Grab Me a Gondola (1956, including the songs I Want a Man not a Mouse and That’s My Biography), Sandy Wilson’s Divorce Me, Darling (1964, including the songs No Harm Done and Out of Step), Lionel Bart’s Twang! (1965, including the song Dreamchild), and Phil the Fluter (1969, about Irish entertainer Percy French and including songs he made famous such as That’s Why the Poor Man’s Dead and Abdul Abulbul Ameer). None were made into films. On the other hand, The Good Companions (1956) was remade with little added to the version 20 years earlier (1937), which also stood close to the stage adaptation from 1929 of the original novel by J.B. Priestley. This backstage musical had all songs performed on stage (using the performance mode) and set up a solid divide between backstage drama with the music hall company and their onstage musical activities.7 Another stage production to find rapid adaptation to the screen was Expresso Bongo in 1958, released as a film in 1959, starring dramatic actor Laurence Harvey and singer Cliff Richard, one of Britain’s earliest rock’n’roll singers. Lionel Bart’s successful career as a stage songwriter began with Lock Up Your Daughters and Fings Ain’t Wot They Used t’Be (both in 1959), the latter of which was one of the most successful musicals of the turn of the 1960s. Its depictions of the working class showed something of what was to come from that decade, where polite middle-class plays (musicals included) receded dramatically. Bart started out writing songs for rock’n’roll singer Tommy Steele, and later for Shane Fenton and Anthony Newley. He wrote the massive hit Little White Bull for Steele, which had appeared in Tommy the Toreador (1959). In the same year, Fings Ain’t Wot They Used t’Be was staged by Joan Littlewood at the Theatre Workshop in Stratford, east London. It was full of lowlifes and criminals and looked back decidedly to John Gay’s The Beggar’s Opera two centuries earlier. In 1960, Bart’s Oliver!, derived from Charles Dickens’s Oliver Twist, opened and was a significant success. This was made into a film in 1968, directed by Carol Reed. It retained some of the rough and

110

K. J. DONNELLY

ready aspects of Bart’s stage original. Reed’s hellraising nephew Oliver Reed played Bill Sykes while the lead role was played by newcomer Mark Lester, a charming child who was unable to pitch anywhere near what the songs required. The crucial role of Fagin was taken by Ron Moody on stage and screen, rewriting and the role from Dickens’s original and Alec Guinness’s film version (1948), with the Jewish writer and actor arguably removing some of the clear earlier anti-Semitism. Bart went on to write Blitz! (1962), about a family during the Second World War, and the Liverpool-set Maggie May (1964), although his career went into hiatus later in the decade. The 1960s saw regular adaptations of stage musicals as films. Anthony Newley’s Stop the World I Want to Get Off opened on stage in 1961 and was made into a film in 1966. Oh What a Lovely War! also opened on stage in 1961, a politicized play developed at the Theatre Workshop and was made into a film in 1969. It was an extraordinary re-enactment of events of the First World War accompanied by popular songs of the time. The film concludes with a startling crane shot that moves outwards and upwards to reveal a gigantic field war graves. There was a British–American co-production of Half a Sixpence (1963), which had been adapted for the stage by Beverley Cross and composer David Heneker from H.G. Wells’s Kipps. Rock’n’roll singer-turned all-round entertainer Tommy Steele starred both on stage and on screen. A rapidly-adapted successful stage play, A Funny Thing Happened on the Way to the Forum, was made into a film and released in 1966 as a British–American co-production. The songs by Stephen Sondheim took a back seat to the comedy, with the film featuring American comedians Phil Silvers and Zero Mostel. Other notable stage musicals from the middle of the decade included The Canterbury Tales (1969), which attempted to update Chaucer with pop music and Charlie Girl (1965), which was about a class divide and starred Joe Brown and Anna Neagle. However, these were not adapted for the screen, as film musicals became something of a rarity, not only in British film production but also in the output of Hollywood. The 1970s began with a financial “crisis” for British film production. The loss of American investment forced the industry to tighten its belt and make films less frequently and with substantially lower budgets. The British stage was not notably affected, producing hit musicals including Anthony Newley and Lesley Bricusse’s The Good Old Bad Old Days (1972), Andrew Lloyd Webber and Tim Rice’s Jesus Christ Superstar (1972), which was made into an American film the following year, while

6

ADAPTING STAGE MUSICALS IN BRITAIN

111

there was the less-mainstream success of The Rocky Horror Show (1973) and children’s musical Smike (1980), an adaptation of Nicholas Nickleby written by Simon May, one of the most successful writers of television theme music in Britain. There were film adaptations of more upmarket musical product. For example, The Tales of Beatrix Potter (1971) was an adaptation of a Royal Ballet stage show that took the stage ballet and set it among Christine Edzard’s sumptuous film sets. With similar upmarket credentials, Peter Schaffer’s Amadeus was first staged in 1979. It was directed by Peter Hall, who had worked on opera at Glyndebourne, and had Mozart’s music arranged by respected composer Harrison Birtwhistle.8 It started at the National Theatre and then moved to the West End in the early 1980s. It used plenty of loud recordings of Mozart’s music, often in a highly dramatic manner, most notably perhaps his Requiem. Amadeus was made into an American film in 1984, directed by Milos Forman. Ken Russell became something of a specialist director of musical films, often following art music subjects. Working for the BBC, he had made dramatized documentaries about the lives of composers including Claude Debussy, Bela Bartok and Edward Elgar. These were highly acclaimed for their imagination and illustration of the composer’s music. After Billion Dollar Brain (1968) and Women in Love (1969), Russell directed The Music Lovers (1970), starring Richard Chamberlain as Tchaikovsky, which again exploited the potential dramatization of existing classical music with visual bravura and spectacle. He then went on to make more in the way of imaginative composer biopics, Mahler (1974) and Lisztomania (1975). Russell’s Tommy (1975) had originally been developed as a stage show in 1969 emanating from The Who’s concept album of the same name. It was the apogee of a cycle of “rock operas” that were definitely ambitious, some might say pretentious. Denisoff and Romanowski discuss the overblown artistic status accorded to Tommy and how it was performed in opera houses and legit theatres: “A Columbia Pictures advertisement was pure hype: ‘Tommy is greater than any painting, opera, piece of music, ballet or dramatic work that this century has produced’”.9 Certainly such terms were never used for the discussion of The Rocky Horror Picture Show, which started as a stage production at the Royal Court Upstairs (a more modest setting than their main auditorium) in 1973 and was made into a film in 1975, changing little. It pastiched elements of science fiction films, Hammer horror films and 1950s rock’n’roll, making a potent and

112

K. J. DONNELLY

remarkably individual mix that has retained and built on its cult status ever since. Towards the end of the twentieth century, British musicals were not only a success on West End stages but also exported well, most notably to Broadway, at least partially reversing the trend since the 1940s of the British musical stage being dominated by American products. These productions were often characterized by a barrage of special effects, including amplified music, hydraulic set movement, pyrotechnics and special lighting effects. These musicals also tended towards the sentimental—not a trait normally associated with British culture—and contained very accessible music, written in a highly populist and quite conservative style. In many ways, they looked back at the earlier tradition of operetta. An important figure in these developments was theatre producer and impresario Cameron Mackintosh, who rose to a pre-eminent position in London’s West End, producing a startling succession of moneyspinning musicals. He famously and arrogantly called Broadway “just another stop on the American tour”, and has since been honoured in Britain with a knighthood. His collaboration with Andrew Lloyd Webber was an essential part of the buoyancy of London’s theatreland at this time. Lloyd Webber’s musicals will be discussed in more detail later. Apart from Lloyd Webber’s work, Mackintosh brought the French stage musical Les Misérables to London in 1985, engaging the Royal Shakespeare Company and respected director Trevor Nunn. Many other theatrical successes of the last decades of the twentieth century included Willie Russell’s plays that regularly incorporated his songs, such as the stage musical Blood Brothers (1983), about two brothers separated at birth and adopted, who later meet and clash.10 Other successful musicals included Chess (1986), written by Bjorn Ulvaeus and Benny Andersson from Abba, Miss Saigon (1989) and Schönberg and Bubil’s Martin Guerre (1995). One highly notable aspect of the success of stage musicals at this point was the importance of merchandising, with all manner of readily identifiable logos adorning products. Les Misérables had a female child with hair blowing in the wind, while Cats had a pair of cat’s eyes with the irises dancing. Marketing included soundtrack albums and singles, much as did the tradition for film musicals, glossy souvenir programmes full of photographs, T-shirts, badges and posters—in fact, these were all the trappings that had become a staple of rock concerts in the 1970s, along with a few other items, such as coffee mugs, stuffed toys and key rings.

6

ADAPTING STAGE MUSICALS IN BRITAIN

113

Cats was the likely starting point for this saturation marketing. Musicals on stage have always retained popularity in London’s West End “theatre land”. However, in the 1980s and 1990s, musicals became endemic on London’s principal stages, driven by the increasing appeal of such theatre to a middle-class “middle brow” audience. According to Dominic Shellard, The re-emergence of a genre popular in the 1950s was the single most noticeable feature of the West End in the 1980s, but instead of relying on American imports, London became famous for its efficient and profitable staging of home-grown musicals. Their dominance has drawn much adverse comment and they are regularly accused of variously blocking the entrance of writing into the West End, restricting choice, driving up prices and being intellectually light-weight, but their enduring popularity testifies to their ability to entertain on a grand scale (often marrying impressive technical feats with soaring melodies), their capacity to reach beyond the normal pool of theatregoers, and the ingenuity with which they are marketed.11

A catalyst in this development was the musicals of (later Sir) Andrew Lloyd Webber and later international dancing show successes such as Riverdance and Lord of the Dance, both of which were derived at least partially from Irish folk music and dance. There were some isolated successes that translated well to screen. For example, the stage success of Little Voice led to a film adaptation in 1998. This was in some ways much more traditional than many other stage musicals of the time, utilizing established show tunes. These are performed in a club by reluctant singer “Little Voice” (Jane Horrocks) at the behest of a hustling manager (Michael Caine), who memorably sings Roy Orbison’s It’s Over on a karaoke machine at the film’s conclusion. The British stage has always had a healthy appetite for musicals, which have been perennial successes. Screen musicals have not always been so successful and since their heyday in the 1930s have become increasingly only an occasional occurrence in British cinema. This is despite the fact that adaptation is often easy and that a massive stage success is almost a guarantee to cinema audiences. Furthermore, the film musical is a cinematic form that has a closer relationship to its stage parent than perhaps any other. However, some notable stage phenomena have failed to make the transition to film.12

114

K. J. DONNELLY

Three individuals: Coward, Newley and Lloyd Webber Across this historical backdrop of broad developments, painted in bold brushstrokes, the careers of a handful of noteworthy individuals play in an uneven fashion. These personages I will discuss are Noel Coward (1899– 1973), Anthony Newley (1931–1999) and Andrew Lloyd Webber (1948– ). Noel Coward’s career started from being a child prodigy for the stage to being a morale-boosting cheerleader during the Second World War. After the war, his position was more ambiguous, with his plays appearing quickly outmoded and he reinvented himself as a cabaret turn in places like Las Vegas and on American television, repackaging himself as the consummate Englishman for the American market. Alongside and as a part of this, he increasingly made cameos in films, perhaps the most notable being Our Man in Havana (1959) and The Italian Job (1969), the latter of which was directed by Peter Collinson, a protégé of Coward’s (Fig. 6.1). Noel Coward had a mixed career as a playwright, actor and songwriter (and even wrote the musical score for a few films). His stage career was at its peak in the 1920s and 1930s, with him reaching celebrity status with The Vortex in 1924, which he wrote and starred in, and where no punches were pulled in a drama of drugs and sex. Coward had and projected a highly distinct persona, which made him a more public face

Fig. 6.1 Noel Coward in The Italian Job

6

ADAPTING STAGE MUSICALS IN BRITAIN

115

than many others in the world of the London stage. This included a very self-conscious sense of sophistication and only a thin veil over his homosexuality. Rather than writing musicals, Coward created plays with songs, bringing energy, speed and wit and sophistication to London’s West End theatres. In 1916, Coward appeared in a stage musical with Cicely Courtneidge and written by her husband Jack Hulbert, called The Light Blues. Later, he wrote songs for stage reviews such as London Calling in 1923 (where Gertrude Lawrence sang Parisian Pierrot ), On With the Dance in 1925 (which included Poor Little Rich Girl ) and This Year of Grace in 1928 (which starred Jessie Matthews and Sonny Hale and included the song A Room With a View). In 1929, Coward’s Bitter Sweet made its debut. It was a “nostalgia musical” that looked back to Viennese operettas of the turn of the century and was about an heiress who falls in love with her music teacher and included the song I’ll See You Again. In 1930, Private Lives was a massive success, and Coward starred in it opposite Gertrude Lawrence, with whom he had a very close personal and official relationship. The play concerned a divorced couple who meet up again on holiday in Switzerland with new spouses and was made into a Hollywood film in 1931, starring Robert Montgomery and Norma Shearer. He went on to write Cavalcade, which opened in 1931 and celebrated moments in British history using an ordinary family to unify events. It was a large-scale extravaganza, which included a massive set with hydraulic movements. Coward then went back to reviews with Words and Music in 1932 (which included Mad About the Boy and Mad Dogs and Englishmen). Bitter Sweet is concerned a woman marrying a musician and then being wooed by a rich man. It was made into a film directed by Herbert Wilcox in 1933 and made a star of Anna Neagle. It contained some memorable song sequences, including a notably static camera for I’ll See You Again and 360 degree pans for another song. Bitter Sweet was also adapted into a film by Hollywood in 1940, starring the highly successful musical pairing of Jeanette MacDonald and Nelson Eddy, who were famed for a series of lucrative film operettas. In 1934, he wrote, directed and acted in Conversation Piece, which was more of a turn back towards the Victorian operetta. In 1936, tonight at 8.30 had an innovative format of being nine short plays that would be performed on successive nights and in random order. Four of these were musicals. One of the short plays, Still Life, was extended to make the film Brief Encounter (1946), directed by David Lean.

116

K. J. DONNELLY

During the Second World War, Coward worked tirelessly for the war effort, writing, producing, directing and starring in In Which We Serve (1942), about the crew of a sunken British ship. He also wrote and acted in This Happy Breed on stage (1942), and wrote the adaptations of his play Blithe Spirit (1941) as well as Brief Encounter. For In Which We Serve, Coward even wrote the incidental music, in collaboration with Ray Douglas and Clifton Parker, underlining his dominance and authorship of the project. Coward wrote a succession of stage plays with music, including Operette (1938), an attempt at a Viennese-style operetta, Sigh No More in 1945, starring Cyril Ritchard and Joyce Grenfell, Pacific 1860 (1946), Ace of Clubs (1950), After the Ball (1954) (an adaptation of Oscar Wilde’s Lady Windermere’s Fan) and Sail Away in 1961 (starring Elaine Stritch and set on a luxurious ocean liner). These were not filmed and, in fact, only a few of his stage plays were adapted for the screen. Noel Coward’s film career began as an extra pushing a wheelbarrow in a street scene in D.W. Griffith’s British-made film Hearts of the World (1918). He acted in the American film The Scoundrel (1935), but in David Lean’s This Happy Breed (1944) which was adapted from Coward’s play of 1939, his acting role was replaced by Robert Newton. Blithe Spirit (1945) was also adapted by David Lean from Coward’s stage play of 1941, and included Margaret Rutherford repeating her stage success as the clairvoyant Madame Arcati. Coward starred in The Astonished Heart (1950), directed by Anthony Darnborough and Terence Fisher, which was derived from a short play of Coward’s. This Gainsborough production also starred Celia Johnson and Margaret Leighton, while Coward also wrote the film’s musical score. For CBS Television, he wrote and appeared in Together With Music (1955), a revue with a cast of two (Coward and Mary Martin). His filmed plays included The Vortex (1928), Easy Virtue (1928), Cavalcade (1933 US) and Design for Living (1933).13 Coward’s extraordinary Cavalcade, which followed an English family from the turn of century to the 1930s, included the song Twentieth Century Blues, and the Hollywood production (by Fox), starring Clive Brook and Diane Wynard, won an Oscar for best picture. It is something of an enigma as to why Coward’s plays and songs did not make a string of successful films. His reputation was established and retained largely by the stage rather than film, or television, which he once suggested was something “one appeared on rather than watched”. Yet his

6

ADAPTING STAGE MUSICALS IN BRITAIN

117

foray into films was also limited. He wrote to his mother, “I’m not very keen on Hollywood I’d rather have a nice cup of cocoa”.14 Coward was a key figure of the British stage and popular music in the twentieth century, although he often did not tie the two in traditional stage musical formats, the plays being more like plays with occasional song interludes. According to Cole Lesley, Coward’s assistant, he did not learn from the innovations brought about by Oklahoma! and its integration of songs to further plot development.15 While there were some attempts to use Coward as a power for the British cinema in the late 1920s. Indeed, “Gainsborough production chief Michael Balcon pointed to Coward’s material as a means of countering undue American influence on the content of British films” but this strategy failed.16 Coward was at his peak just around the time that sound cinema was beginning and later he certainly failed to endeavour with a musical contribution to British cinema in the 1930s, as his life took on more of the character of a celebrity that he retained until his final years. Anthony Newley was another remarkable figure in British music, stage and film. He wrote musicals and songs, sang and recorded songs, directed for stage, film and television and acted. His career started at a tender age, as one of the principal British child leads, and most famously as ‘the Artful Dodger’ in David Lean’s adaptation of Charles Dickens’s Oliver Twist (1948). By the mid-1950s, he was appearing in West End plays such as Cranks and NY , while also appearing in small parts in films. By the end of the decade, he was prominent in films such as Idle on Parade (1959) and Jazz Boat (1960). Newley starred in ATV’s The Strange World of Gurney Slade in 1960, a remarkable television serial that prefigured the postmodern techniques of over 20 years later. He also had a short life as a chart-topping pop singer with songs like Strawberry Fayre and Pop Goes the Weasel in the early 1960s. He utilized a very distinctive cockney-style singing voice, which later was influential on British pop singers who wanted to adopt an indigenous English-inflected singing voice, most clearly David Bowie. Newley’s first play was very successful. Stop the World I Want to Get Off (opened in 1961), which was about ruthless ambition, starred Newley as ‘littlechap’ and featured the later hit song What Kind of Fool Am I ? This became a theme song for Newley. The show also included the songs Once in a Lifetime and Gonna Build a Mountain, and the show was a notable success in London and then on Broadway. This success led to a

118

K. J. DONNELLY

film adaptation released in 1966, directed by Philip Saville and starring Tony Tanner in the part originally played by Newley (Fig. 6.2).

Fig. 6.2 Anthony Newley in The Strange World of Gurney Slade

Newley wrote songs and appeared in American film Doctor Dolittle (1967) with Rex Harrison in the title role. He wrote, directed, wrote music for and acted in Can Hieronymous Merkin Ever Forget Mercy Humppe and Find True Happiness? (1969), a highly singular project that demonstrated the degree of risk that was being taken with films at the time. While the film was certainly highly personal to Newley, this white elephant film made certain that a career as film director did not beckon. Instead, he followed the lead of Noel Coward and went into cabaret in Las Vegas. An impressively varied career finished with a short run in the BBC’s flagship soap opera EastEnders in the late 1990s. Overall, Newley completed a remarkable career spanning the stage, films and records and involving writing, song-writing and directing. Indeed, if any single person can claim to have crossed all the boundaries of popular culture, it is surely Newley. No other individual can claim

6

ADAPTING STAGE MUSICALS IN BRITAIN

119

the range of successes that he managed, and this includes the adoption of experimental procedures in The Strange World of Gurney Slade and Can Hieronymous Merkin Ever Forget Mercy Humppe and Find True Happiness? He was an absolute rarity in his ability to excel in multiple fields, although he was erratic in veering from massive popular success to experiment and self-indulgent obscurity. Probably the best example of this was his trajectory from number one hit Strawberry Fayre to Moogies Bloogies, his collaboration with electronic musician and experimentalist Delia Derbyshire, who arranged and realized the Doctor Who theme for BBC television. Less of a renaissance person but massively prolific, Andrew Lloyd Webber has been one of the most successful composers of stage musicals and undoubtedly the most successful British composer of musicals since Arthur Sullivan. Testament to his prodigious success is that he has been recognized by the British establishment, being knighted in 1992 and then made a Lord in 1997. Most of his productions, particularly the earlier ones, are collaborations with lyricist Tim Rice. Lloyd Webber made his debut with Joseph and his Amazing Technicolor Dreamcoat in 1968, which was written for school production, with lyrics by Tim Rice. This has been a perennial favourite, with a television version of the successful British stage production made in 1991, starring Australian pop singer and actor Jason Donovan. There was another version on screen in 1999, starring 1970s teenybopper Donny Osmond in the lead role. In 1971, again in collaboration with Rice, his Jesus Christ Superstar opened. It began as a studio recording before it was a stage show. This was probably the first rock opera, with all the dialogue sung rather than a distinct divide between songs and acted scenes. Jesus Christ Superstar’s startling success and long run in London started off a halcyon period for London’s West End and home-made productions. It was made into a film in 1973, an American production directed by Canadian director Norman Jewison, who had directed the highly successful American musical Fiddler on the Roof (1971). Lloyd Webber has made a remarkable series of successful stage plays. Tell Me on a Sunday (1979) was also staged for television in 1979. Evita (1976), co-written with Tim Rice, was about the wife of Argentina’s leader Juan Peron. It spawned three hit singles (Julie Covington’s Don’t Cry For Me, Argentina, Barbara Dickson’s Another Suitcase in Another Hall and David Essex’s Oh What a Circus ). Cats (1981) was based on T.S. Eliot’s Old Possum’s Book of Practical Cats, included a hit for Elaine

120

K. J. DONNELLY

Page (Memory) and was staged by Trevor Nunn and thus had the patina of legit culture to it. There was a television programme that filmed the stage production in 1998. Starlight Express (1984) was a spectacle involving a roller-skating cast and hydraulic stage (lyrics by Richard Stilgoe), and Phantom of the Opera (1986) (with Stilgoe and Charles Hart, after Gaston Leroux’s novel) had a hit single for Michael Crawford with The Music of the Night. Aspects of Love (1989) had lyrics by Don Black and Charles Hart and was developed from a novel by David Garnett. It was adapted for television in 1993 and directed by Lloyd Webber himself. It starred Michael Ball as the soldier who falls in love with an actress. Lloyd Webber wrote Sunset Boulevard (1993), Whistle Down the Wind (1997) in collaboration with rock singersongwriter Jim Steinman, The Beautiful Game (2000) in collaboration with comedian Ben Elton, Bombay Dreams (2002) and The Woman in White (2004) after Wilkie Collins’s novel. Lloyd Webber cemented his critical position in relation to the success of London’s West End theatreland. After setting up the Really Useful Theatre Company, at the turn of the Millennium, he bought ten London theatres to add to those he already owned. Lloyd Webber has also worked with Alan Parker, who is something of a specialist in film musicals. Evita (1996) was a British–American co-production, starring Madonna, Antonio Banderas, Jimmy Nail and even featured Gary Brooker, singer in the 1960s group Procol Harum, in a small role. Phantom of the Opera, which opened in 1986, starring Michael Crawford and Sarah Brightman, Lloyd Webber’s wife at the time, was made into an American film in 2004, directed by Joel Schumacher and starring Gerard Butler.17 Films Aspects of Love and Sunset Boulevard have been long planned to be films yet have yet to materialize. There is certainly a cinematic possibility for his plays. Indeed, Sunset Boulevard and Whistle Down the Wind, both of which were based on existing films, while Bombay Dreams was premised upon the production of a Bollywood film.18 One of Andrew Lloyd Webber’s most successful shows, Cats, was adapted in 2019 as a British–American co-production directed by Tom Hooper. It was ambitious but became a massive box office and critical failure,19 and had CGI issues that led to an enhanced version of the film being released shortly after the initial release, which had sorted out a plethora of visual errors in the digital graphic skin on top of the filmed images. Despite including a new song co-written by Lloyd Webber with

6

ADAPTING STAGE MUSICALS IN BRITAIN

121

pop star Taylor Swift (who appeared in the film as Bombalurina), Cats was perhaps not helped by the policy of replacing the stage cast with higher-profile actors and comedians (such as Judy Dench, James Corden and Idris Elba). Although aesthetically interesting, the film’s reception has given it the status of a landmark failure of iconic stature. Some have declared Lloyd Webber’s work to be well-crafted but not very creative. Indeed, there is a tendency to rely on only one or two very memorable melodies per show. In Cats, for example, the melody for Memory keeps reappearing as a reassuring refrain that holds the rest of the musical enterprise together. Some of his compositions have been dismissed as pastiche by music specialists, although this is irrelevant to the hordes of paying fans who have made his musicals among the most popular ever. He has also been criticized for lacking the craft of American musical composers such as Rodgers and Hammerstein or Stephen Sondheim, in that his shows rely on spectacle and pyrotechnics rather than dynamics and formal development and integration. However, it cannot be denied that he produces directly engaging, highly successful stage shows that consistently have wrested the initiative away from the USA to Britain, in terms of dominance of the world musical stage.

Conclusion In Britain, the stage has been particularly virile and successful but the overwhelming majority of British stage musical successes have not translated to the screen. However, by the mid-2000s, successful stage musicals such as Summer Holiday, Hairspray and Footloose were derived from film originals. This appeared to be a turnaround in the dominant way of conceiving the relationship between stage and screen, which traditionally saw the stage as the originator and the screen as the adaptor. The successful film Billy Elliot (2000), about a young boy from a mining community who wants to be a ballet dancer, was transported to the London stage as Billy Elliot: The Musical. The film was a notable success both at home and abroad, receiving three Academy Award nominations. The stage version was created by many of the same team who had made the film, including director Stephen Daldry, choreographer Peter Darling and even film producers Working Title. One significant addition was a full score of songs and music by (Sir) Elton John. Similarly, Chitty Chitty Bang Bang was produced very successfully on stage in 2002, and derived

122

K. J. DONNELLY

directly from the film of 1968 (and the original book by Ian Fleming, published in 1964). At times, there has been an extremely close relationship between the British musical stage and the musical films, most notably in the 1930s. Since that time, the two have drifted apart to a situation in recent years where London’s West End theatres are full of successful musicals, many of which have a British origin, while the British film industry has made only a handful of musicals in the last decade. Stephen Guy, discussing the 1930s, has pointed out: The notion of cinema as a branch of musical theatre was also signalled by the friendly attitude of the variety profession towards it (unlike radio, which it saw as a menace killing off theatre audiences and offering a paucity of poorly paid work by compensation). For variety artists the cinema was seen as a good potential source of employment, as well as being genuinely beneficial for the profession.20

In more recent years, the buoyant state of the musical stage in Britain has begun to engulf the tradition of film musicals and even led to respected film actors onto the stage. Significant individuals such as those I have discussed—Coward, Newley, Lloyd Webber and others (Novello and Bart)— have had differing and often indifferent relationships with the screen and often more consistent ones with the British stage. Indeed, their careers are testaments to the strength of the British musical stage. Coward’s key work was for the stage and his career peaked too early for synchronized sound. Newley was eclectic and iconoclastic, but managed a remarkable career on stage, recording, film and television; indeed, one that has not been duplicated. Lloyd Webber, on the other hand, found his principal successes with the stage and has remained loyal to it despite the more recent adaptations of his massively successful musicals. His plays seem suited to big-budget adaptations, and thus have not only been slow to be adapted but also have tended to be American or American–British co-productions. The success of a handful of American-made musicals in the late 1990s (such as Baz Luhrmann’s films and Chicago [2002]) has inspired further investment in screen versions of Lloyd Webber’s musicals.

6

ADAPTING STAGE MUSICALS IN BRITAIN

123

Notes 1. Geoff Brown, “Sister of the Stage: British Films and British Theatre” in Charles Barr, ed., All Our Yesterdays: 90 Years of British Cinema (London: BFI, 1986). It also focuses on the adaptation of more highbrow or ‘legit’ plays. 2. Eric Blom, Music in England (Harmondsworth: Penguin, 1945 [f.p. 1942]), p.199. 3. See further discussion in: Michael Williams, Ivor Novello: Screen Idol (London: BFI, 2003). 4. Gay wrote few stage plays, but instead concentrated on songs, such as Leaning on a Lamp Post, written for George Formby. 5. Dominic Shellard, British Theatre Since the War (London: Yale University Press, 2000), p.22. 6. Ibid., pp.23–24. 7. It includes comedienne Joyce Grenfell, Celia Johnson and even Anthony Newley in a small part. 8. It initially starred Simon Callow and Paul Scofield. 9. R.Serge Denisoff and William Romanowski, Risky Business: Rock on Film (London: Transaction, 1991), pp.216–217. 10. Russell’s stage play Shirley Valentine was also made into a successful film. 11. Shellard, op.cit., p.190. 12. Although some have made a less than direct transition. For example, see my discussion of the influence of Riverdance in James Cameron’s Titanic (1997). “Riverdancing as the Ship Goes Down” in Sarah Street and Tim Bergfelder, eds, The Titanic in Myth and Memory: Representations in Visual and Literary Culture (London: I.B. Tauris, 2004), pp.205–214. 13. Coward wrote the songs for The Grass is Greener (1960), directed by Stanley Donen. This British film had a Hollywood cast, including Cary Grant, Deborah Kerr, Jean Simmons and Robert Mitchum. 14. Cole Lesley, The Life of Noel Coward (Harmondsworth: Penguin, 1976), p.159. 15. Ibid., p.220. 16. Ibid., p.150. 17. The film was co-produced by Lloyd Webber, while in 2006 the stage musical became the longest running show on

124

K. J. DONNELLY

Broadway, http://news.bbc.co.uk/1/hi/entertainment/459408 4.stm [accessed 10/1/2006]. 18. Lloyd Webber even wrote some film incidental music, for Gumshoe (1971) and The Odessa File (1974). 19. The “Lights, Camera, Traction” guide looked into numerous indicators of failure and found that Cats was the worst film in the 2010s and the second worst since the Millennium. Michele Debczak, “It’s Official: Cats Is the Worst Movie of the 2010s” at Mental Floss, 21 March 2021. https://www.mentalfloss.com/art icle/643930/cats-worst-movie-of-decade-2010s [accessed 4/3/ 2024]. 20. Stephen Guy, “Calling All Stars: Musical Films in a Musical Decade” in Jeffrey Richards, ed., The Unknown 1930s: An Alternative History of the British Cinema, 1929–1939 (London: I.B. Tauris, 1998), p.114.

CHAPTER 7

New Music for Old Movies

Currently, silent cinema has become more prominent than at any time since the end of the 1920s, when it was ousted by the cinema of recorded sound. Apart from festivals and special screenings, silent films are a part of education and art.1 Where there are silent films, there are almost always musical accompaniments. Many DVD and BluRay releases of silent films have new specially written music for the films while others go out of their way to have ‘authentic’ scores constructed to be as similar as possible to what accompanied the film at the time of its initial release. There is a profound difference between scores that are founded on such historical principles and those that aim at furnishing something new for the film. While the former offers the museum and history as guarantor of pedigree, the latter offers a potentially new culture although this can mean breathtaking novelty or a crude hybrid. In recent years, a whole new breed of musician has become involved, not only pop musicians using electronics but also experimental musicians and so-called ‘turntablist’ DJs, who can spin discs to the action. There are a number of possible reasons for the increased interest of popular musicians, not least the seemingly perpetual ‘crisis’ in the music industry and demographics, particularly the proliferation of ageing musicians wishing to branch out. Perhaps silent films have become like Shakespeare plays, which are—restaged in versions ranging from ‘Elizabethan original’-style © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025 K.J. Donnelly, Hearing Eyes, Seeing Ears, Palgrave Studies in Audio-Visual Culture, https://doi.org/10.1007/978-3-031-73913-2_7

125

126

K. J. DONNELLY

to radically ‘modern’ versions. Or perhaps like some art music, which has garnered painstaking ‘historical performance’ (such as Christopher Hogwood’s work with the Academy of Ancient Music) or ‘updates’ (such as those by jazz pianist Jacques Loussier, or electronic musicians Walter/ Wendy Carlos or Isao Tomita). While opinions vary on these ‘originals’ and ‘updates’, there was more controversy tied to the electronic ‘re-imaginings’, suggesting that they might be ‘going too far’ from any original intention. Indeed, some of these new musical scores seem to move a significant distance from the film’s original musical horizons, and some would say straying too far. There are some highly esoteric, crass, outré musical accompaniments to silent films, and some of these seemingly have little to do with events on screen. There has been a simplistic assumption that silent films always had ‘appropriate’ musical accompaniment. In Silent Film Sound, Rick Altman has written about how in the first years of the twentieth century, music for silent films was sometimes played by blind musicians.2 This is less surprising from a musical point of view as blind musicians were not uncommon in this period, and in the first decade of cinema there was often little concept of unified harmony between film and musical accompaniment. This historical fact might serve as an emblem for similar processes now: where some musicians have a certain indifference to film.

Old Films, New Music Scholars have taken little notice of the current vigorous silent film culture. It has avoided theory and commentary but has thrived in the hothouse of film festivals. In 2010 for instance, the 13th British Silent Film Festival, at the Phoenix Arts Centre, Leicester, included The Bridal Party (Rasmus Breinstein, Norway, 1926) with a new score by Halidor Krogh, Tol’able David (Henry King, 1921) with a new bluegrass music score by Damien Coldwell (appearing with Nick Pynn on fiddle and Appalachian dulcimer, and Lee Westwood on guitar) and the Louise Brooks film Beggars of Life (William Wellman, 1928) with live music by the Dodge Brothers. This was billed as ‘The Dodge Brothers performing to Beggars of Life’, indicating that the film is not the primary object but more of an accompaniment to a concert of sorts. Indeed, the Dodge Brothers only rarely perform concerts with films and the event questions the basic tenets of accepted relationships between silent film and musical ‘accompaniment’.3

7

NEW MUSIC FOR OLD MOVIES

127

Silent films have spawned a wide-ranging and successful industry in recent years, with tours of screenings with live music and international festivals, such as Pordenone in Italy, which embrace a sizable number of screenings with live music. There is a degree of vertical integration, with films being toured with musicians and then released on DVD. One flourishing ensemble, the Alloy Orchestra, even has their own ‘sister company’ called Box 5 which restored film prints to be used by them and others. DVD releases have allowed for multiple soundtracks for the same film, sometime, but not always, stemming from live performances of music to a film screening. Some films seem to attract a large amount of interest from musicians. For instance, Häxan (1922) was released with a choice of three scores: the late sixties recut with W.S.Burroughs with Daniel Humair and Jean-Luc Ponty (usually called Witchcraft Through the Ages ), Geoff Smith’s hammer dulcimer music or Bronnt Industries Kapital’s electronic music. There are also DVDs available with music by Matti Bye, Gillian B.Anderson and Art Zoyd. Some of these cases illustrate a rethinking of film scores as more malleable things, which sometimes can retain only a loose relationship with the screen. Indeed, isolated scores on DVD can make a sound film into a new experience through promoting the background score at the expense of dialog and sound effects. Similarly popular is The Cabinet of Dr.Caligari (Das Kabinett des Doktor Caligari, [1919]), which has had accompaniment from In the Nursery, Timothy Brock, Rainer Viertlböck, dulcimer music by Geoff Smith, a jazz score by Mark Dresser and Lynne Plowman with the London Mozart Players [2009]. Many of these releases were on the Tartan label, which since has been taken over. The culture is a live as well as a recorded phenomenon, with many make successful careers from writing, performing and conducting music for silent film, including Ben Model (resident piano accompanist at the Museum of Modern Art in New York), the Mont Alto Picture Orchestra, Donald Sosin, Gillian B.Anderson (conductor and composer), Dennis James (Wurlitzer organist), Neil Brand (pianist), Stephen Horne, Philip Carli, Günter Buchwald (composer and conductor), Carl Davis (conductor and composer), Timothy Brock (composer), Robert Israel, Steven Ball, Steven Severin, Jon Mirsalis, the Silent Orchestra, the Alloy Orchestra and the Paragon Ragtime Orchestra. This list is by no means exhaustive and attests to the extensive and vigorous nature of this ‘new’ region of old culture. These silent films on DVD with their alternative musical scores not only expand film into a larger object through the array of accompanying

128

K. J. DONNELLY

extras, but also align the ‘rebirth’ of silent film with culture that retains a loose relationship with images, or even almost no direct relationship at all. Silent films might be approached as having a polar approach from their music: Historically accurate versions vs ‘Novel’, radical versions

These are not absolute poles but the diagram represents more of the possibilities of a sliding scale between the two. New accompaniment for old silent films will appear somewhere on the vector between being historically accurate and an attempt at something new and radical. The ‘historically accurate’ version retains a traditional character to the score, often guided by knowledge of an ‘original version’. The approach is scholarly and the processes often follow those of historical research, with the music as an outcome of archival work. These films are often presented at dedicated silent film festivals attended by aficionados and might be seen as a film-musical manifestation of museum culture. Often they follow the principal sources of music from the period, the film music compendiums of the era, such as Giuseppe Becce and Hans Erdmann’s Kinothek (a declension of Kinobibliothek) (1919) and Ernö Rapée’s Motion Picture Moods for Pianists and Organists (1924). The ‘novel’ versions tend to use a rhetoric of cultural renewal or ‘updating’ in their aspiration to make the silent film a valued object to a wider contemporary audience. There is a strong tendency for certain types of silent film to be favoured, particularly German Expressionist horror films or other ‘popular’ silent classics. The films in question are rarely anything too obscure, or anything far beyond the choice of a general audience. Nosferatu or Man with a Movie Camera, for example, are constant sources exploited by musicians. This augurs a courting of a mainstream audience rather than the aficionados who go to silent film festivals. (One might even suggest these are ‘silent films for people who don’t like silent films’.) Rhetoric about ‘updating’ the films abounds. For example, the publicity for the ‘Breakin’ Boundaries’ event at the Roundhouse in London on 28 May 2008, stated: “New electronic artists produce their own soundtracks to and reworkings of classic silent movies including Micropolis (a short film constructed from Metropolis ), Häxan, an abstracted version of Seven Seals and more”.4 I can find absolutely

7

NEW MUSIC FOR OLD MOVIES

129

nothing about a film called Seven Seals and wonder if their lack of attentiveness to the films might have led them to mistake Ingmar Bergman’s The Seventh Seal (1957), a sound film but with large quiet sections. On the one hand, there is the aim for ‘original intentions’ or ‘the author’s wishes’, approaching film as a historical entity, bound by and emanating from its ‘own time’, and on the other a notion that silent film exists ‘to be updated’ (perhaps to be “improved”). According to silent film music conductor and composer Philip Carli: “An exciting score in an idiom that would be familiar to the original audience for the film leads the modern viewer to accept dramatic conventions that might otherwise seem stilted or even unintentionally comical, rather than highlighting these as more ‘modern’ scores can do”.5 The contradictory point of view is that modern music allows a point of entry for audiences unfamiliar with the conventions of silent cinema. Whichever approach is taken, music is inclined to function as a frame for the film, either enabling us to see as audiences in the past saw the film, or effecting to update the experience and offer us a contemporary point of entry to a difficult and antiquated film. A further dichotomy that does not quite map directly on to the polar divide under discussion, pertains to the aesthetic assumptions of the music supplementing a silent film. This does not involve a simple divide between ‘historical’ and ‘novel’ approaches. Generally speaking, images inspire particular sonic strategies and aesthetic approaches. The dominant in cinema has been the notion of a ‘homology’ of film and music, where aspects such as dynamics, ‘mood’ and kinetics are treated as if they should ‘match’. On the other hand, there is the possibility of merely imposing an existing musical approach or tradition on the film, perhaps even in an indifferent and indiscriminate manner. As a homology music must, to a greater or lesser degree, be subordinated to the requirements of film as image. It is, therefore, essentially image-based (and also dialogue based in sound cinema). Consequently, since the 1920s the dominant tradition in mainstream cinema has been that after shooting and editing, a film will then have a musical score written to fit its particular requirements. This tradition has established a certain format and style for film scores that marks them out as a distinctive musical form. However, the imposition of an existing musical approach or style yields a different outcome. This approach is arguably closer to the tradition of film musicals, where pre-existing songs or stage shows lead to a certain format, where images become an accompaniment to the primacy of the music. In terms of

130

K. J. DONNELLY

silent films, this can lead to a situation of a ‘gig with images’—and where some scores for silent films are available on CD only (such as accompaniments to Metropolis by Art Zoyd and the Club Foot Orchestra), confirming the primacy of the music rather than the film images (or indeed the composite). On occasions, film can appear incidental to the music, following a tradition outside the mainstream cinema. For instance, since the 1960s, some clubs have projected almost random film images which are incidental to the club space and the dominance of music in that space, while avant-garde cinema can involve procedures radically different from the cinematic norm. At best, music might be conceived as an equal partner with the silent film, but in some cases it manifests more than a partner.6 Music as an addition is able to remove or add to the original intentions of the film. There are some cases where recorded soundtracks are masked and original intentions of filmmakers dismissed. For instance, avant-garde composer Mauricio Kagel wrote a new score for Un Chien Andalou (1929) to replace Bunuel and Dali’s stipulation to use discs of Wagner’s Liebestod from Tristan und Isolde and a tango. Carl Davis masked Charlie Chaplin’s recorded score for ‘silent movie’ Modern Times (1936) in order to play the music live with an orchestra, and Philip Glass wrote music for Dracula (1931), which treated the quiet early sound film as if it were a ‘mostly’ silent film, obliterating the atmospheric silences that dominate the film.

Case Study: Metropolis (1927, Fritz Lang, UFA) As I noted earlier, there are a handful of films that have proven to be perennial favourites for musicians supplying new music for silent films. There have been many musical soundtracks created for Fritz Lang’s Metropolis (1927) by many different artists, including, but not limited to (Fig. 7.1): 1975—The BBC version with an electronic score by William Fitzwater and Hugh Davies. 1984—The Video Yesteryear VHS release contained an original score by Rosa Rio on the Hammond organ. 1984—The version produced by Giorgio Moroder. This was restored, tinted and had intertitles removed and subtitles superimposed over the images. This was the most complete version at the time, running at 80 minutes. It was compiled and overseen by Moroder, who also supplied

7

NEW MUSIC FOR OLD MOVIES

131

an electronic score, as well as wrote pop songs for the film which were sung by Pat Benatar, Bonnie Tyler, Jon Anderson, Adam Ant, Cycle V, Loverboy, Billy Squier and Freddie Mercury. The ‘Original Soundtrack’ remains available on CD and while this version of the film had a cinematic release and a short release on VHS, it has never been made available on DVD. 1986—Enno Patalas’s scholarly restoration, using newly discovered documentation, and featuring Gottfried Huppertz’s musical score from 1926. Huppertz had played the piano during the shooting of Metropolis as well as providing the first score. This was a leitmotiv-based orchestral score, which included the music for the mass of the dead (the Dies Irae) at some points as well as fragments from Wagner and Richard Strauss. 1998—Peter Osborne’s electric keyboard score, which often sounds quasiorchestral, for the JEF/Eureka 139-minute DVD version (released in the UK only). The music has never been made available on CD. 2001—Jeff Mills’s version. The successful DJ provided an electronic score which was used for a few screenings, but has been made available only on CD and never on DVD. 2002—Art Zoyd’s version. The French avant-garde band, who defies description but owe something to progressive and jazz rock performed music for screenings. The music is available on CD but never has been commercially available as a DVD. 2002—The digital restoration by archivist Martin Koerber, which included a re-recording of Gottfried Huppertz’s original score for the film from 1927. This was the Kino International DVD release, which included more material and lasted 124 minutes. 2008—There was massive excitement about the discovery of the fullest version yet in Buenos Aires. Martin Koerber restored the film again. The initial screenings in Berlin and Frankfurt used a score performed live by the Alloy Orchestra. The ensemble had been founded in 1981 to play music for Metropolis, yet they seemed an anomalous preference as the film was being presented as a historical object and Huppertz’s score would have been the clear choice. As if to emphasize this, the later cinematic release with synchronized music and the DVD release used a new re-recording of Huppertz’s score. 2009—Jean-Michel Danton’s touring stage show called ‘Metropolis Revived’, which included live music and lightshows.

This list illustrates the wide range of approaches and artists involved in supplying music for Metropolis. Some of these ‘versions’ of the film have a radically different character from others. In his ‘BFI Classic’ book on Metropolis, Thomas Elsaesser stated:

132

K. J. DONNELLY

Fig. 7.1 Metropolis

The desire to perform Metropolis, instead of putting it in a critical or historical perspective, is largely responsible for lending new life to the vision of Lang and von Harbou. … To some, such treatment and especially the New Wave music track compounded heresy with blasphemy, adding a special cynicism to iconoclastic insouciance. But it was a bold move, … Moroder’s score, perhaps by its deliberate anachronisms, induces one to discover Lang’s images afresh …7

Film scholars are usually aficionados of film and, therefore, unlikely to espouse such a position. They are more likely to be affronted by the audacity of a disco producer assuming that he can ‘improve’ a film by one of the Century’s best directors. This is a brave statement by someone with an investment in film history. In the 1980s, the Moroder version must have sounded brand new and very ‘now’, but the music dated extremely quickly. It now gives the impression of being older than the more ‘timeless’ Huppertz version (making Metropolis an eighties film rather than a twenties film). Moroder’s choice of singers dates the film markedly while his electronic incidental music has worn better. Yet, the very periodspecific sound of the songs lends a certain quality to the film that makes it perhaps even stranger. Elsaesser’s notion of the sense of modernity regained for the film, now makes the Moroder Metropolis appear as a

7

NEW MUSIC FOR OLD MOVIES

133

constituent of what is now known as ‘retro-modernism’, and related cultural subgenres such as 80s postmodernism and ‘steampunk’, aesthetics that mix the modern and the archaic, but are particularly interested in outdated ideas of modernity. It was not necessarily alien to ally pop music and silent film. The Moroder version was not ‘out of the blue’, as there had been an insistent trace of silent film culture that remained around the edges of popular culture, and there was a notable strain that reappeared intermittently in pop and rock music. Examples just before Moroder’s version abound: Kraftwerk recorded the song Metropolis on The Man Machine (1979), record covers used stills from silent films (examples from just before Moroder’s release abound: Be Bop Deluxe’s Live in the Air Age [1977] from Metropolis, Hugh Cornwell and Robert Williams’s Nosferatu [1979] from the film of the same name, and Bauhaus’s Bela Lugosi’s Dead [1979] from The Cabinet of Doctor Caligari). A more immediate area where the silent film aesthetic had persisted was in avant-garde cinema. Avant-garde filmmakers such as Andy Warhol often made films that lacked synchronized sound, consisting solely of images. These usually had some music (live or on disc added when they were screened. Even some of Stan Brakhage’s films—which are usually screened in silence—have had live accompaniments recently by Sonic Youth, as have some of Harry Smith’s films. While avant-garde films could be shown with no sound accompaniment, they often would be accompanied by discs or by live music. Furthermore, there is a marginal tradition in cinema including Mel Brooks’s Silent Movie (1976), or Guy Maddin’s films that use silent or early sound aesthetics (Careful [1992], Dracula: Pages from a Virgin’s Diary [2002] and My Winnipeg [2007]) and ranging even as far as pop videos (such as Rob Zombie’s Living Dead Girl or the Kaiser Chiefs’s I Predict a Riot ). One might even argue that contemporary cinema retains a trace of silent cinema in sequences where there is no dialogue, particularly montage sequences and action sequences, where image track and soundtrack (often dominated by music) take the fore. Indeed, Manvell and Huntley suggested that the extensive music and image sequences of 2001: A Space Odyssey (1968) were returning to the modes of silent film.8 However, it is within the bounds of avant-garde cinema that the aesthetic remained the strongest. Indeed, the distance between recorded image and live sound fed the avant-garde tradition of disjunction, in this case between sound and image, allowing the sound sometimes to perform a role akin to commentary on the film. This

134

K. J. DONNELLY

is not far from the potential of any silent film accompaniment. Indeed, historian, conductor and composer Gillian B. Anderson noted that during the silent era live music “… could become an almost living, independent character commenting on the pantomimed drama on screen”.9 This is, of course, also the case for recent musical accompaniments to silent films. While arguably all soundtracks comment on the films, some make more strident or perplexing comments perhaps than others, and those whose ‘fit’ is least apparent seem to reveal the most clear audio critique of the visual.

New Synergy, New Psychology Perhaps the most out of the ordinary, jarring or disagreeable marriages of film images and new music provide the most comment from music to film. Perhaps they are most critical in the ‘gap’ between the film images and the added music. Some examples appear to have a significant distinction between sound and image, to ‘go too far’ in their transposition of the films to new contexts. However, their score for Eisenstein’s Battleship Potemkin was a surprise. Instead of their characteristic electropop as accompaniment, they instead mostly wrote music for a mostly fairly traditional sounding orchestral accompaniment to the film. In fact, there is little about much of the music that gives a clue that it was written by the Pet Shop Boys.10 A more radical event was DJ Spooky’s performance/DVD Rebirth of a Nation (2005) was the ‘remix’ of D.W.Griffith’s classic film Birth of a Nation (1915)— a film attributed with the ‘birth of modern cinema’ as well as being clearly racist. Spooky not only adds a musical soundtrack but also a voice-over commentary on the film’s representations. His remix goes further, removing and repeating sections of the film and using superimposed graphics on the screen, all in aid of a critical discourse on the film that includes the film. Since then, he has gone on to provide the score for KinoLorber’s DVD release of The Cabinet of Dr.Caligari in 2015. A marriage more in the spirit of the director’s intentions was KTL’s score for Victor Sjöström’s The Phantom Carriage (1922) is highly effective but highly unusual. They are a due of American guitarist Stephen O’Malley and British electronic musician Peter Rehberg. Their music is based on droning continuous sound, lacking notable melody or harmonic changes. Its austerity and bleakness are harrowing enough, but allied to Sjöström’s horror film, the effect is almost overwhelming. O’Malley is

7

NEW MUSIC FOR OLD MOVIES

135

most recognized as a member of Sunn0))), the drone avant-garde/heavy metal band whose live show aims at sonic effects and is far remove from concerts of songs by popular musicians. A similarly innovative move involved British musician Geoff Smith who has produced a number of scores for silent films played alone on hammered dulcimers. These include Faust (1926), The Cabinet of Doctor Caligari (1919) and Häxan (1922), and Smith’s music was performed alone on three sets of standing dulcimers, with some singing and electronic looping effects. A review of his performance at a screening of Häxan at the Barbican, London, in July 2007 noted that: “Smith’s soundtrack might lack a certain dynamic …, but a lingering musical illusion of drifting in and out of consciousness that effectively mirrors Häxan’s dreamlike qualities”.11 The music is certainly unlike most other music for films, although this relates more to timbre (instrumental sound) than to the way the music functions, as it tries to enhance the moods inherent in the images and the narrative development more than it attempts to impose its own aesthetic on the films. Vsevolod Pudovkin’s Storm Over Asia—The Heir of Genghis Khan (1928) was performed at the National Film Theatre, London, on 21 July 2001, with live musical accompaniment by the band Yat-Kha. YatKha is an acclaimed Tuvan Yenisei throat singer and his musical ensemble has mixed aspects of traditional music with Western pop and rock. While there is some ethnic veracity to this event, matching Pudovkin’s images of Mongols with Mongolian music, the sounds were perhaps too distinctive to not take attention away from the screen. An event called ‘Within Our Gates: Revisited and Remixed’ was staged as part of ‘Black History Month’ at Ithaca College, New York in 2004. It was inaugurated by a performance of Fe Nunn’s specially written music for the screening of Oscar Micheaux’s Within Our Gates . This included live music from jazz quartet, baroque clarinet solo, African dancing and djembe drumming. It also embraces digital live mixes, including a VJ (video jockey mixing images), and spoken word performances that are derived from academic theoretical and analytical analysis of the film. With contributions from the Body and Soul Ensemble and the Ida B. Wells Spoken Word Ensemble. Anna Siomopoulos and Patricia Zimmermann comment: The goal of this project was to rethink the exhibition of politically significant films and to encourage a contemporary audience to engage critically

136

K. J. DONNELLY

with one particularly important film, Within Our Gates (1920). In order to create a new reception context for a groundbreaking silent film, we used live music, digital technology, and spoken word performances; we hoped that this new presentation of the film would provoke audiences to see the cultural continuities and discontinuities between different technologies, and the political implications that these technologies have at different moments of social history.12

While this event seems essentially about community, technology is clearly central to the event and, judging by the number of constituent parts, film is merely a component. However, it is the central component around which all else coheres. Siomopoulos and Zimmermann continue: … the music and words had a dialectical relationship to each other; the music released the images from silence and the past, while the spoken word operated as a distancing device to pull the spectator out of the film and into larger historical, theoretical, and critical debates. … Based on our research into black exhibition practices during the silent period, we decided that the music for our performance would not function subserviently to the filmic text or narrative but as an equal. In other words, we sought to destabilize the film text, reanimate film reception, and complicate film spectatorship through music, spoken word, and multiple voices.13

With Ida B. Wells’s script spoken by performers in front of the film screen, this recontextualization of Micheaux’s film allowed for other elements to take the fore, and despite the prominence given to the spoken word, there is a lot of music, not functioning “subserviently”. Less analytical and more plainly spectacular was the JMD Project, JeanMichel Danton’s stage show of ‘Metropolis Revived’ in 2009. Its publicity boasted (ambiguously) about updating the film: “‘Metropolis Revived’ is a progressive masterpiece that keeps listeners asking for more, this artist has a more refined controversial approach to the message he portrays through his music”.14 The JMD Project re-situated Lang’s film in a multimedia extravaganza, and rather than extending the film, it sounds like it reduced the film to a part of something else. Not only was the event based around multiple screens and light shows but there was, according to the publicity, “a disco afterwards!”15 In fact, it seemed to have far more in common with large rock festivals or massive stadium concerts than with cinema screenings as they normally are understood. What the JDM Project underlined is just how far Metropolis has ceased to be a film by Fritz Lang

7

NEW MUSIC FOR OLD MOVIES

137

produced at a certain historical juncture, and how far it has become an acknowledged part of popular culture, boiled down, reduced, composted into a few important, defining images that summarize particular ways of thinking about things. Music changes the psychological and emotional landscape of films. Therefore, different music yields a different film. Film historians have written unproblematically about silent films untethered to musical accompaniment as if they were complete unities, consensual objects. A focus on (radically) different scores suggests that they are not. The soundtrack is more than merely a ‘bolt-on’ to the ‘film’ but more an essential part of the film experience. Indeed, DVDs with two different soundtracks offer good value as they provide, in effect, two separate films.

Conclusion: Audiovisual Meditations on History We might divide these silent film events in a different manner, using broad and non-exclusive categories (films and their music can belong to more than one designation): (i) SCHOLARLY HISTORY (ii) EMPATHETIC HISTORY (iii) ISSUE-BASED HISTORY (iv) ‘POPULIST’ HISTORY Scholarly history often has little concession to contemporary taste and tends to focus on ‘authenticity’, and fidelity to the historical context of the film’s initial release. Scholar-conductors like Gillian B. Anderson have reconstructed scores from historical sources, using primary information and often exploiting the musical compendiums of the period to furnish scores as close to what might have been as possible. In historiographical terms, this is closely related to the processes of traditional academic history, and films screened along these lines usually have a strong sense of ‘historical veracity’ and scholarly responsibility (as curators of culture). Thomas Elsaesser noted “the phantom ‘first night’ of archivists and historians …”,16 and indeed, this notion seems an important guide for this approach. Coming from a similar point of view, empathetic history is essentially interested in how it must have felt for audiences at the time of the film’s

138

K. J. DONNELLY

release. While historical veracity is often important, it is not as important as the emotional tone of the music and some anachronisms may be acceptable in the service of the film. One might argue that many current pianists, such as Ben Model and Neil Brand, provide music that broadly sounds as if it might have been played at the time although both are happier with emotionally engaging music at the expense of scholarship. This is not to deem their music ahistorical—far from it—but to note that mild compromise of historical veracity is acceptable for the cause of current accessibility. Some have gone further and the rhetoric of many silent film accompanists focuses more on audience emotion than historical authenticity. For instance, Stephen Horne avers that, “I think that music will play an increasingly important role in presentations of silent films. When I talk to younger audience members, it is usually the ‘performance’ aspect of the event which most excites them. It is as though live music dispels their preconceptions, enabling them to engage with what they are watching”.17 Emphasis on engagement or immersion in the screening of such films is a very current concern. While this is definitely still a form of history, it has been evident in the wave of conversions of museums into ‘experiences’, with costumed attendants that act out the past including visitors (such as the short-lived Museum of the Moving Image in London, which was accused of ‘dumbing down’ complex historical material for children’s consumption). In Shivers Down Your Spine: Cinema, Museums and the Immersive View, Alison Griffiths discusses interactive and immersive museums as part of the same culture as early and pre-cinema and IMAXes.18 The concluding section of the book is concerned with the debate about whether museums should be for ‘elitist’ sober education or ‘cheapened’ popular spectacle, which sums up one of the central contemporary arguments in dealing with historical objects. Issue-based history takes historical objects and narratives and imbues them with contemporary concerns. Thus any sense of a proper or ‘full’ history is replaced by one of partizan approach and acceptance of myriad points of view on the past. This is what Richard Handler and Eric Gable have described in their book on living museum Williamsburg as constructionist ‘social history’, where the lens of current issues defines the past.19 In terms of silent films, the ‘Within Our Gates: Revisited and Remixed’ event and DJ Spooky’s remix of Birth of a Nation as Re-birth of Nation are excellent instances of the original films being recontextualized by selfconscious current and critical theories and practices. Certainly, the latter case embodies the notion of critique, and in cases like these the staging

7

NEW MUSIC FOR OLD MOVIES

139

might appear less respectful of the films allied to a sense that the past is problematic (and that audiences are unable to be critical on their own). The addition of words that were not part of the original is often crucial to such ‘deconstructions’ or expansions. Siomopoulos and Zimmermann noted the dialectical relationship of images and sounds, while the overall Within Our Gates event was designed to inculcate a complex response and a critical stance in the audience.20 Seemingly taking a fundamentally differing perspective, populist history attempts to reformat and make accessible knowledge about the past to an audience who is assumed to need it structured in a ‘modernized’ and ‘popular’ manner. Instances of this abound in contemporary popular discourses of history and museums, although the ‘Horrible Histories’ series of books are perhaps the most successful example. The Moroder Metropolis would have to be the prime illustration of this notion, along with the JMD Project’s ‘Metropolis Revived’. The latter appears unproblematically to take Metropolis as an object that needs radical updating. Blair Davis describes how he found the addition of modern music allowed students to engage more with the films, as it made the film less alien through its incorporation of music from ‘their’ culture.21 The first two categories (scholarly history and empathetic history) are concerned essentially with history (how it was, how it felt to experience it at the time), while the second two categories (issue-based history and ‘populist’ history) are more concerned with the ‘modern’ context of the films. The last two are more interested in the present than the past, in some cases more obviously than others. While there are many other forms that history can take, music and silent film seems to fall under these rubrics (indeed, in some instances more than one simultaneously). The key is not the images (‘the film’) but the music added to that film. Music for silent films manifests a ‘debate’ on the nature of history and objects sourced in the past; debates the types of history. Historiographic process is evident in each film, and indeed they might be seen as part of a ‘debate’ about how film heritage should be used. Indeed, we might note that there is an undeclared battle for heritage, with more at stake than many might think. Should silent films (and the past more generally) be revered as untouched historical objects or experienced as ‘living’ culture? Should they be the stuff of glass cases in museums or of ‘reformatted’ commerce? Culture can take both but will be in trouble should one side of the debate win a conclusive victory over the other. While film (and musical) heritage ought not to allow an

140

K. J. DONNELLY

‘extinction’ of films accompanied in a historical manner, it cannot afford to remove the imaginative and virile culture of film accompaniment as a critical or developmental process.

Notes 1. Rick Altman, Silent Film Sound (New York: Columbia University Press 2004), p.4. 2. Altman, “The Silence of the Silents” in The Musical Quarterly, vol. 80, no. 4, Winter 1996, p.648. 3. See Michael Hammond discussing his and the band’s approach in “Cowboys, Beggars and the ‘Deep Ellum Blues’: Playing Authentic to Silent Films” in K.J.Donnelly and Ann-Kristin Wallengren, eds., Today’s Sounds for Yesterday’s Films: Making Music for Silent Cinema (New York: Palgrave, 2016), pp.81–104. 4. Advertisement in The Wire, 2008, p.87. 5. ‘BritishSilents’ website. http://britishsilents.wordpress.com/mus icians/ [accessed 20/4/2010]. 6. Gillian B,Anderson, Music for Silent Films 1894–1929: A Guide (Washington: Library of Congress, 1988), xv. 7. Thomas Elsaesser, Metropolis (London: BFI, 2000), pp.58–59. 8. Roger Manvell and John Huntley, The Technique of Film Music (London: Focal Press, 1975), p.253. 9. Anderson, op.cit., xxvi. 10. See Beth Carroll’s discussion in “Soviet Fidelity and the Pet Shop Boys” in K.J.Donnelly and Ann-Kristin Wallengren, eds., Today’s Sounds for Yesterday’s Films: Making Music for Silent Cinema (New York: Palgrave, 2016), pp.122–137. 11. Edwin Pouncey, review in The Wire, no. 281, July, 2007, p.66. 12. Anna Siomopoulos, Anna and Patricia Zimmermann,”Silent Film Exhibition and Performative Histories: The Within Our Gates Project” in The Moving Image, vol. 6, no. 2, 2006, p.110. 13. Ibid., pp.110–111. 14. Reverbnation website: http://www.reverbnation.com/page_o bject/page_object_bio/artist_744271 [accessed 2/6/2010]. 15. Ibid. 16. Elsaesser, op.cit., p.58. 17. ‘BritishSilents’, op.cit.

7

NEW MUSIC FOR OLD MOVIES

141

18. Alison Griffiths, Shivers Down Your Spine: Cinema, Museums and the Immersive View (New York: Columbia University Press), 2008. 19. Richard Handler and Eric Gable, The New History in an Old Museum: Creating Williamsburg (Durham, NC.: Duke University Press, 1997). 20. Siomopoulos and Zimmermann, op.cit., p.110. 21. Blair Davis, “Old Films, New Sounds: Screening Silent Cinema with Electronic Music” in Canadian Journal of Film Studies, vol. 17 , no. 2, Autumn 2008, p.92.

CHAPTER 8

From Video Game Music to Film Music: Silent Hill

While film and video games are not the same thing, there is a common core of audiovisual perception, technology and aesthetic techniques, and essence at the ‘join’ of moving image and sound. The Silent Hill franchise relatively effortlessly moved from video game to film and illustrates this explicitly. The adaptation to film brings something of video game audio to film, yet the games themselves already evinced a significant influence from horror films. In audiovisual culture sound can often adopt a more ‘musical’ character. Indeed, sound tends to take on an aesthetic character, in other words acquiring a distinctly musical aspect. This is beyond evident in the Silent Hill games and films, where not only can sounds take on an emotional character but music can also adopt a cold, mechanical character more akin to the traditional character of sound effects.

Silent Hill Adaptation Silent Hill is a successful and remarkably singular series of video games produced by Japanese company Konami. On the face of it, it concerns a third-person character’s navigation through the misty ghost town of Silent Hill, under threat from various grotesque creatures while attempting to solve a central mystery. It is a ‘third person survival horror game’, where the player’s avatar explores a three-dimensional navigable environment, © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025 K.J. Donnelly, Hearing Eyes, Seeing Ears, Palgrave Studies in Audio-Visual Culture, https://doi.org/10.1007/978-3-031-73913-2_8

143

144

K. J. DONNELLY

solves puzzles, collects items, stays alive and strives to solve a central mystery. The game has a persistent subtext of psychological disturbance, suggesting that it may all be a fantasy of the protagonist. The gameworld oscillates without warning between a seeming normality and the terrifying ‘Otherworld’ of fire and imminent threat by enemies (including the massive ‘pyramidheads’). The music and the entire sound design for each of the early games were written and produced by composer Akira Yamaoka, a specialist in video game music. The game has appeared for a number of different gaming platforms. These are the principal Silent Hill games as of 2024: Silent Hill (protagonist Harry) (1999) [for Playstation, later versions ported for PC] Silent Hill 2 (James) (2001) [Play Station 2, PC, Xbox] Silent Hill 3 (Heather) (2003) [PS2, PC] Silent Hill 4: The Room (Henry) (2004) [PS2, PC, Xbox] Silent Hill 0rigins (Travis) (2007) [PSP handheld] Silent Hill: Homecoming (2008) (Alex) [PS3, XboX, PC] Silent Hill: The Escape (2008) [iOS] Silent Hill: Shattered Memories (2009) (Harry, again) [Wii, PS2, PSP] Silent Hill: Downpour (2012) (Murphy) [PS3, XboX] Silent Hill: Book of Memories (2012) (choosable character) [PSVita]

The films are based on the earliest most iconic games. The first Silent Hill film, directed by Christophe Gans, was released in April 2006, while the second film followed in 2012. Silent Hill: Revelation was made in 3D and directed by Michael J. Bassett. The first concerns Rose, who takes her adopted daughter back to Silent Hill and finds out about her past. The second is about teenager Heather, who discovers she is not who she thinks she is and visits Silent Hill to find the truth. Silent Hill (2006) is a direct attempt to translate the video games to film, but significantly, this transposition takes place around the original music which is retained from the Konami game as a skeleton upon which to rebuild the character of the game into film form. The game music thus embodies an ‘essence’ of the Silent Hill character. Indeed, the film’s director Christophe Gans initiated the film project by creating a ‘demo’ which he sent to Konami. Its visuals were cut to the game’s music, so the music was given for the project: it was there from the very start.1 Game players will have heard this music constantly. Indeed, the repetitive nature of video game playing means that pieces of game music can

8

FROM VIDEO GAME MUSIC TO FILM MUSIC: SILENT HILL

145

be heard more repeatedly than any other music. Thus, the music needs certain qualities to allow for repeated listenings. Equally, though, the situation of game playing—and its aim at psychological immersion—allows music to insinuate itself significantly into the game player’s mind, a situation which seems particularly apt in that Silent Hill ’s world appears to emanate from the neurotic mind of the game’s central character. Indeed, the constant dislocation of sound and image establishes the sense of aberrant psychology at the heart of the game (Fig. 8.1). When it came to the film adaptations, the filmmakers were faced with a familiar problem: how to translate the game’s elements into a film format, enhancing and adding ‘cinematic’ elements while retaining the essence of the game’s character. A crucial decision was that the music and soundscape from the games would be adopted as the foundation upon which to build the films. The experience of the Silent Hill films, and notably their sounds and music, will clearly be different for an experienced game player from the manner in which it will be apprehended by someone coming to the film ‘cold’. The films of Silent Hill work differently for aficionados of

Fig. 8.1 Silent Hill 2

146

K. J. DONNELLY

the game: most notably, the music triggers a memory (and a feeling) of the world of the game.2

Silent Hill Game Sound and Music The music and the sound design for each of the games were written and produced by Akira Yamaoka, a specialist in video/computer game music.3 He was trained at art college rather than music college (so he was not a ‘classically trained’ musician). He joined Konami as a roster composer and musician at the age of 25 in 1993. The video game industry tends to have music by ‘workers’ who are contracted, either short term or long term, to the game developing company. This is similar to the heyday of the Hollywood Studio system, where music was produced industrially, rather than the more recent system of freelancing composers like in contemporary film production. It very much retains the character and rhetoric of an industry rather than seeing itself as ‘art’. Yamaoka produced not only the music but also the sound effects for the Silent Hill games, credited not only with being composer but also sound designer, sound producer and director. By the time of Silent Hill 3 in 2003, Yamaoka had become the game producer for the PC version of the whole game.4 The game’s music includes ‘indie rock’-type songs, atmospheric ambient music and noisy music that sounds much like an amalgam of machine-like and natural sounds. Although there are some female vocals and electric guitars, the vast majority of the music clearly was produced on computer software. This is not only evident in the clear and simple textures, but also in the principle of the loop as the musical structural basis of most pieces. Apart from the indie-type songs, some of the music is reminiscent of the subgenre of dark ambient or darkwave (particularly groups such as Bocksholm), while some of his other music is vaguely reminiscent of Japanese artist DJ Krush. Also influential was Brian Eno’s vaguely programmatic Ambient 4: On Land (1982), which evinced a lack of purposeful progression or development, alongside an evacuation of melody/harmonic movement. This form of stasis foregrounds texture and sonority and inspired later so-called ‘Dark Ambient’ music (such as Cold Meat Industry artists Raison D’Etre, Desiderii Marginis and Deutsch Nepal). It also takes something from ‘industrial’ rock music such as Nine Inch Nails, who used loops and mechanical sounds on the mini album Broken (1992).5

8

FROM VIDEO GAME MUSIC TO FILM MUSIC: SILENT HILL

147

Yamaoka’s incidental music for Silent Hill 2 and Silent Hill 3 has a notable ‘trip hop’ flavour particularly in the first game,6 as well as including a darker and more noise-based score. This latter music is not ‘score’ in the traditional sense for games or for films. Indeed, it includes austere ‘noise’, often mixed with echoed notes, deep chords and metallic sounds, although these also often appear in isolation and in succession. This is at least partly due to the structure of the score, as tracks are overlayed and not so obviously comprising relatively short loops of music. One of the sonic characteristics is the use of disjointed rhythms, which sound like the functioning of broken machines.7 Indeed, the video games intermittently use the music of an ‘avant-garde’ character, which at times sounds closer to what we would expect of sound effects. Indeed, the ‘musical sound design’ of the games and films is remarkable in that supernatural events involve sounds that are neither clearly sound effects nor music (Fig. 8.2). The Silent Hill game soundtracks evince an overlap between incidental music and sound effects, delivering a soundscape of ambiguous ambience. David Bessell noted that in video games ‘… the boundaries between music and sound effects start to become more blurred’.8 This was far from uncommon, indeed something of a video game tradition, perhaps partly due to sounds and music being produced by the same chip and using similar sonic component parts. Silent Hill, as a matter of strategy, aims to

Fig. 8.2 Silent Hill 3

148

K. J. DONNELLY

put the player in a state of some confusion and the soundtrack ontology is one of a number of representational points of ambiguity. In The Terror Engine, his book about the Silent Hill game, Bernard Perron notes: “… the games of Silent Hill are really ‘audio–video’ games, and it is important to note, the audio precedes the video in the expression. [It was] … designed with sound in mind (which is not always the case in video games)”.9 The games exploit the phenomenon of ‘acousmatic’ sound, which Michel Chion discusses in Audio-Vision, using Pierre Schaeffer’s definition, as sound with an unapparent source or origin.10 There is an essential ambiguity to such sounds. Their origins are immediately obscure, although their source may be understood later. In psychological terms, such sounds are perceived as a potential threat in that they hang in uncertainty for the perceiver. The use of these makes for disturbing gameplay as the player is uncertain of what these sounds indicate and where they are located. However, the protagonist/avatar has a radio which indicates imminent attack through crackling with static, while the appearance of the diegetic siren announces the onset of the transformation from normality to the nightmarish ‘Otherworld’ and sets the player in anticipation of being attacked. These are occasional but there is a more regular wash of sonic atmosphere, mixing music and what appear to be sound effects. Zach Whalen discusses the crucial acousmatic effect of the radio in the game, where an imminent threat is signalled by a burst of static from the radio. Below the static sound, the dominant base sound in Silent Hill is a chilling ambient wash which throbs with the sound of machinery and sirens. The volume level of this ambient sound is low, but its ubiquitous presence keeps the player on edge and sets an ominous tone for the visual environments of both worlds of Silent Hill. Its mechanical tone also blends smoothly with the machine-produced static of the radio such that the sonic texture of the atmosphere remains consistent.11

In Silent Hill, evil is associated sonically with the appearance of white noise. In terms of sound, white noise represents a primordial soup out of which other sounds can emerge. Significantly, it also suggests a mental state of being overwhelmed by ‘noise’, and marks a homology of the ‘immersion’ at the heart of the game. In the game, the appearance of static on the radio carried by the protagonist warns that monsters are approaching. Similarly, in the first film, there is sonic distortion on Chris’s

8

FROM VIDEO GAME MUSIC TO FILM MUSIC: SILENT HILL

149

(Sean Bean) telephone message from Rose (Radha Mitchell). The sound mixes static and break-up. Also in this film, there is a sound reminiscent of that generically used to represent radiation, which appears first when Rose begins to look for Sharon (Jodelle Ferland) in the main street. This latter sound is ambiguous about its status. There is nothing indicating that it is diegetic in origin, or indeed non-diegetic. This sound appears to come from an ‘elsewhere’, manifesting an irruption from the unconscious, or at least a sonic concretization of a fundamental psychology of fear. The sounds and music emanate from the white noise like life from a ‘primordial soup’ of basic elements and the monsters from traces of earlier culture or repressed memories. This sonic continuum provides something of a general ‘sonorous envelope’ that cocoons the game player.12 While the whole of the Silent Hill universe might be construed as an interior fantasy, inside the disturbed mind of the protagonist, sound plays an essential role in the game’s aim to homologize interior states. Music is usually conceived as having conventional melodic, harmonic and rhythmic structures. Here, much of the ‘music’ lacks these and comes across as ambience or sound effects, in a continuum of sounds that are sustained and loop-based, making a bed of continuous sound alongside occasional periodic interjections.13 Zach Whalen notes a lack of dynamics and lack of melody, which suggests something like the ‘filler material’ in Classical film scores.14 Incidental music in video games often avoids distinct dynamics or gradual and logical development, as this might not match the game event which it accompanies but Silent Hill is in some ways exceptional. Rob Bridgett suggests that this use of visual uncertainty allows the soundtrack to pervade forward and indicate the “unseen world of danger that may be lurking within the darkness”.15 The intermittent movement between ‘normality’ and the ‘Otherworld’ allows for a movement between standard aesthetics and the transmission of an idea of the ‘abnormal’ through non-standard approaches to aesthetics. The sound world is the counterpart and equivalent of the visual dreamscape in the games, consisting of empty buildings and fog-wreathed streets. The soundtrack is landscape-like, enveloping and unsettling, mixing both beauty and ugliness. The outside is dominated by distant acousmatic bubbling sounds and other indistinct sounds. However, the insides of buildings tend to be very quiet, indeed sometimes completely silent.

150

K. J. DONNELLY

Silent Hill Film Sound and Music The Silent Hill films owe much sonically to their video game counterparts: they ‘port’ a number of game music aspects into the film situation. The first film’s music consists of a pre-existing repertoire of music, from which effective pieces could be drawn. That repertoire is, of course, the music written for each of the four games by Akira Yamaoka, cementing the film and game soundtracks.16 Pieces were chosen from this reservoir of Silent Hill sound and then they were fitted to new contexts in the film. However, it is worth noting that they were used for different situations.17 For example, the piece called ‘Magdalene’ is used for repose at the resolution of the film’s concluding massacre, yet in Silent Hill 2 game, the piece plays after the death of James’s female companion. In the film Silent Hill it offers a moment of reflection after the demise of the evil cult, while in the game it overlays the continuation of the game with a sense of sadness for her seeming death. Both situations certainly call for some ‘repose’, and similarly the music for the so-called ‘boss battle’ at the conclusion was also used for the climactic final fight in the film. The film was not ‘scored’, however, and, therefore, music to a lesser or greater degree articulates the image. Pre-existing music inspires images to be cut to ‘fit’ its dynamics, and one might even argue that the pieces of music, arranged in succession, provide the temporal and motoric direction of the film as much as any narrative impetus. However, the film’s music makes the film a fragmentary experience. It contains little in the way of clear and memorable repetition, and little sense of a sonic continuity. In other words, the score is not orchestral, does not consist of songs and does not follow a particular musical style or genre. Yet, for someone who has played these games and interiorized the game music, the experience is a constant evocation of memories of the game and feelings associated with the game. The music becomes like a long finger probing the memory and gaining direct access to feelings stemming from the game experience. Strangely, on the film’s credits, the incidental music was credited solely to Canadian composer Jeff Danna. Akira Yamaoka, the actual composer of almost all the incidental music that appeared in the film, instead was credited as ‘executive producer’ of the film itself. Although initially seeming like some sort of bizarre contractual anomaly, this suggests that music’s role in the transition from small to large screen was more important than its merely incidental status. Much of the film appears simply to ‘port’ Yamaoka’s game soundtrack into the film, to simply compile and

8

FROM VIDEO GAME MUSIC TO FILM MUSIC: SILENT HILL

151

cut it into the film like a ‘temp track’.18 As Yamaoka’s pieces tend to be atmospheric loops devoid of sudden changes in dynamics,19 there is a temptation to think of the film’s incidental music as being a divide between distinct ‘pieces’ (that were written by Yamaoka) and stingers or points where music appears to match action (by Danna). The soundtrack is unlike traditional film scores and separate sound effects. In terms of music, there is no action matching and the recordings appear much more like ‘pieces’ with their own integrity (including songs) rather than as film ‘cues’ which are crafted to fit action. Silent Hill mixes up noisy austere pieces from Silent Hill 1 noise, using mechanical loops and scraping sounds, and more conventional musical pieces derived directly from the first four games. In the first film, there is sonic distortion on Chris’s telephone message from Rose. The sound mixes static and break-up. Similarly, the ‘radiation’ sound appears first when Rose enters Silent Hill’s main street in search of her daughter Sharon. The status of the sound is ambiguous. There is nothing indicating that it is diegetic in origin, or indeed non-diegetic. This sound appears to come from an ‘elsewhere’, manifesting an irruption from the unconscious, or at least a sonic concretization of a fundamental psychology of fear (Fig. 8.3).

Fig. 8.3 Silent Hill: Revelation

152

K. J. DONNELLY

The second film, Silent Hill: Revelation, moves away from building its sonic world around sound and music taken directly from the game. However, it uses a recognizable sound world and short excerpts from music in the games, usually developed into something closer to film music cues. Indeed, the film’s music is much more like a series of ‘variations’ on and developments of Yamaoka’s original music from the game. While it remains sound-based and often appears to lack traditionally ‘musical’ aspects, it most clearly was written as a score for the film. Like the first film, Silent Hill: Revelation’s many visions and dream sequences allow for unconventional sonic as well as visual aesthetics. For instance, the opening dream sequence set at a fairground incorporates prominent slashing noises on edits, a mickey-moused ‘stinger’ noise matching the large toy rabbit moving its head. Indeed, many movements are accompanied by sounds that follow the logic of sound effects but break the convention of sound effects. For example, the armless monster that is the first to threaten Heather at the school is accompanied by an electrostatic sound. This appears to be diegetic sound as it matches the monster’s bodily movement, however, there is a question mark over its status. It may simply be a musical emphasis, akin to a stinger. Similarly, in the manikin storeroom before the attack of the manikin creature, Heather’s tense wanderings are accompanied by atmospheric sounds, which are not clearly either score or diegetic ambience. There are intermittent echoed sounds, scraping sounds and sustained single notes, which appear to be part of the disturbing environment although they could equally be construed as music. Early in the film a day-dream sequence near street stalls with a clown evinces isolated and exaggerated single sounds with the film focusing in detail on a succession of individual aspects of what appears to be a straightforward but strange day. Although this is not uncommon in certain films, here it might be an aspect of the film’s 3D production, where exaggerated, foregrounded images are matched by a similar process in the soundtrack. Each sound is given a strong sense of detail with space around it supplied by electronic reverb effects. As the second film is based on the game, three themes from Silent Hill 3 are used in Silent Hill: Revelation. The film opens with Never Forgive Me Never Forget Me, which appears a few times later in different arrangements. Innocent Moon, a repetitive piano theme appears regularly in the first film including when Chris is looking on the internet for information about Silent Hill, when Rose first enters Silent Hill and when he senses but cannot see Rose passing him. It appears in Silent Hill: Revelation

8

FROM VIDEO GAME MUSIC TO FILM MUSIC: SILENT HILL

153

when Chris sees Rose in the mirror, when Heather (Adelaide Clemens) has visions in Midwich High School and at the end of the film. ‘Clockwork Little Happiness’ is based on a mechanical rhythm and keyboard chords. In addition to these, Promise (Reprise) from Silent Hill 2 is also used. In the first film, it appears as a theme for Sharon and Alessa (both played by Jodelle Ferland). In Silent Hill: Revelation, it appears when Rose looks at the drawing by her younger self (Sharon), when Alessa appears at Midwich Elementary School, in the empty Sacrificial Chamber, and as Rose sees Alessa in Brookhaven Hospital. The transposition of the video game sound world to two films illustrates differences between the two media but perhaps more clearly illustrates the correspondences between film and video game. Films have certainly influenced video games and video games now are asserting themselves on film. The Silent Hill films are a rarity in so directly using music that derives from different gaming functions (the interactive character of games means that music often loops and overlays and is not fixed) in a film context. They certainly ‘work’ as film music but excelling in the ‘scene setting’ atmospherics that are central to both the films and the games. The sounds are similar to the game, making for a distant ambience of acousmatic noises, against which there are isolated, foregrounded sounds. There are sounds for actions (loud, featured sounds analogue to the film’s 3D effects) and sounds that are ‘just there’ (as its ambient soundscape). The soundtrack is derived directly from the game, but it also evinces some distinct film conventions. There is less in the way of unexplained ambient sound and more in the way of direct synchronized sound effects that are louder than the background continuum of sound/music. The sequence at Brookhaven Asylum includes distant acousmatic shouts and groans, presumably from the inmates but this is never made specific. The soundtrack is ambiguous about whether it is music or sound effects. It consists of rumblings and resonances, echoed sounds and what could be diegetic sounds of electricity. However, these sounds have more of the character of music in that they appear used as much if not more for their physical texture as signifying ability. Some of the sustained tones clearly are more like music, yet these are indifferent to the events on screen and merely provide a disturbing ambience.

154

K. J. DONNELLY

The Integrated and Unified Soundtrack: The Game-Film Continuum Video game sound and music, while having some similarities with film sound and music are not the same.20 Some game music draws very directly upon the orchestral film scoring tradition, in terms of both sounds and functions, but other game music emanates decidedly from its own traditions, particularly where concerns are with general atmosphere and interaction rather than music fitting the precise temporal requirements of on-screen developments.21 In terms of game music’s relationship to film music, there tends to be an assumption of some key similarities (and inspirations) that have entered game music from film music (and also its close relation to television music). However, the interactive nature of video games means that they cannot be ‘scored’ in the manner of fixed-time, linear films or plays. In other words, the game player controls how long a certain ‘scene’ might take. Should they so wish, they can take their time and loiter at a certain point in the game. Thus, the nature of games disallows the use of significant ‘cues’ in the filmic sense. Apart from the occasional ‘cut scene’, the game’s temporal dimension is controlled by the player and thus cannot easily be matched or underscored by the music.22 Instead, the music logic of the game is that of looping, where a piece will repeat during an extended playing time, until the triggering of the next piece of music or silence by the character’s progress in the game world. While such gameplay cannot be scored in the traditional sense, ‘cut scenes’ owe a great deal to film. These are pre-constructed sequences triggered by the player arriving at a certain point in the game, and which are not controlled by the player in any way. In the vast majority of cases, these look very similar to films and in effect are the same as animated films. Consequently, they are scored precisely and provide sometimes a rare opportunity for game music composers to write music to match action and dynamics on screen. Most of the time, they have to provide music in the form of loops, which can be repeated endlessly to accompany certain sections of the game. Thus, the music needs to be sensitive to certain aspects that have not been important in other forms of music. In recent times, there has crept into film-making a certain ambiguity about what is ‘music’ and what is ‘sound effects’, breaking the strict demarcation between the two that characterized dominant forms of filmmaking post-production.23 We should not forget, however, that there was

8

FROM VIDEO GAME MUSIC TO FILM MUSIC: SILENT HILL

155

always a strong convention that film scores would bolster diegetic sounds, which is embodied in its most crass form by ‘mickeymousing’ imitation of screen activity. In a seeming reverse of this, it might be noted that in some films sound effects now are becoming ‘film music’, losing their tethering to on-screen causality in favour of an ‘emotional’ function and ‘musical’ organization within films as an aesthetic rather than a representational object. According to James Lastra when discussing film, “… the founding gesture of sound design … [is] the complete severing of sensory experience from representation …”.24 Losing simple representational values sound effects become ‘texture’. Indeed, one might argue that horror film music and sound design have influenced the sound of video games and that in turn has fed back into horror films. While the Silent Hill films are a well-developed example of how nondiegetic music and diegetic sound effects can be merged, in a way, it has always been impossible to fully separate musical score from sound effects in audiovisual culture. Music has always mimicked, emphasized or suggested certain sounds in the diegesis. Similarly, sound effects in films are regularly of more importance than simply fostering the illusion of the diegetic world on screen constructed by the film or video game. Sound effects regularly have symbolic or emotional values that outweigh any simple representational status. To compound this further, music has a tradition of taking inspiration from the natural world and mimicking natural sounds, and in more recent years mimicking mechanical sounds such as trains and gunfire, for example. Technological developments have enabled and encouraged the mixing of sounds and music. Video game sound and music have greatly been enabled by the use of musical software, such as sampling software and digital sound equipment developed originally for use in music. It often has been produced quickly and easily.25 Digital audio workstations (DAWs), also known as sequencers or ‘softstudios’, allow the easy integration of music and sound effects. Indeed, many of the composers for computer games are also the game’s sound designers. These softstudios have everything run through them (recording, sequencing, temporal editing and precise adjustment, even synchronization to image). Logic, Cubase, Reason, Ableton Live, FL Studio, Orion and ProTools, to name probably the most prominent, can all be used as the hub of a home computing set-up, with some being more sophisticated than others (Cubase 4, for example, allows musical scoring directly to images). The vast majority of the music and diegetic sound in the Silent Hill games was clearly

156

K. J. DONNELLY

produced on a DAW. This is not only evident in the clear and simple textures, but also in the principle of the loop as the musical structural basis of most pieces.26 Digital technology (and the process of ‘digital signal processing’) has enabled a concerted focus on the qualities of sound itself. Such developments in musical sequencing software (often the centre of ‘Music Technology’) have inculcated composers with an awareness of sound and an ability to manipulate it electronically: sound for sound’s sake—for example, adding reverb, habitually using filters and placing sounds in a stereo mix. These musical skills are not directly related to the traditional virtues of conventional training of composers for producing traditional music for traditional ensembles (what is often called ‘classical training’). As an essential part of these digital developments, the use and manipulation of sound samples have become central. This means that raw sound recordings are wielded as starting material, which allows the integration of recorded ‘sound effects’ with music.27 Contemporary sound designers by and large are using the same (or very similar) software as musicians, and often have some musical knowledge, or at the very least approach sound in a manner tuned-in to music. Inspired at least partially by the game’s sound design, the film Silent Hill tends to concentrate on ambient sound which emphasizes the uncertainty of traditional status between diegetic sound effects and nondiegetic music. This not only provides a highly distinctive sound world but also furnishes a level of sonic ambiguity and uncertainty to the Silent Hill landscape and soundscape. Considering that most of the music in the first Silent Hill film consists of pre-existing pieces, we might expect a certain ‘disconnection’ between image track and soundtrack, yet the asynchrony evident in the first film far exceeds expectation. Indeed, the dominance of musical asynchrony invokes a feeling of unease, a feeling of disconnection and uncertainty. In addition to this, the lack of repeated ‘themes’ of many traditional scores encourages a feeling of schizophrenia (where each successive second is made anew) rather than intermittent return and the repose offered by the use of repeated themes or related musical material. The music embraces a wealth of disturbing and disconcerting sounds. The ambient sounds include deep drones, sub-bass-like rumbles, distant clunked metal, sounds like bowed metal/microphone feedback and a plenitude of electronic echo and reverb. The regular dissolution of the traditional distinction between non-diegetic incidental music and diegetic

8

FROM VIDEO GAME MUSIC TO FILM MUSIC: SILENT HILL

157

sound effects often leads to a seemingly unified sonic field, often as a musically inspired integrated score. While this is not unique, it has a history of isolated instances, such as the Barron’s ‘electronic tonalities’ for Forbidden Planet (1956, Fred. M. Wilcox), where sounds are strictly not sound effects or incidental music, but both simultaneously, or perhaps more pertinently in David Lynch’s Eraserhead (1977), where Alan Splet’s disturbing background ambiences appear to be a diegetic sound effect although have more of an effect akin to that of unsettling incidental music.28 Silent Hill ’s locations inspire almost constant music and ambient sound, the presence of which serves to demonstrate the sonic importance of Silent Hill’s ‘otherness’, and the imminent presence of threats and uncertainty. This last point is crucial. We hear a metallic sound and think it is probably non-diegetic, until the metallic ‘pyramidhead’ monster appears. This questions the status of the diegesis and, significantly, adds to a sense of ambiguity in environment through confusion of sound and image. Similarly, there are regular bass rumbles (almost sub-bass rumbles) on the film’s soundtrack. Are these diegetic? That information is never furnished by the film. The disturbing effect of such ambiguous sound is discussed in Audio-Vision by Chion when discussing ‘acousmatic’ sound.29 Their origins are immediately obscure, although it may become apparent later. In psychological terms, such sounds are perceived as a potential threat in that they remain an uncertainty for the perceiver. Now, on one level, this discussion might seem naïve. Austere music might well sound like ‘sound effects’ but there is art music that sounds like this as well. I am aware of this—but there is a tradition of sound effects in film (and video/computer games), and these scores engage those traditions more than they come from outside (from art music). A number of recent films offer very rich sonic landscapes that work on their own independently of their film. This might be related to the tradition of programmatic music, illustrating vistas and places through sound, a tradition reinvigorated by certain ambient music and new age music. However, there might also be an input from sound art, which has been a burgeoning area of the art world over the past couple of decades. In terms of film, Sergei Eisenstein discussed the notion of ‘nonindifferent nature’, where the setting is more than simply a realistic backdrop but becomes an emotionally charged landscape.30 Indeed, the most complex of audiovisual art includes elements with more than simple relevance and valences.

158

K. J. DONNELLY

The Silent Hill games are premised less upon action and excitement than atmosphere, with extended sections of potentially aimless wandering about through a deserted town shrouded in mist. According to the film’s screenwriter Roger Avery, the first film focused on atmosphere to the point where the initial script looked thin and lacking potential backers.31 Like the games, the film concentrates on attempting to build an intensely immersive atmosphere—potentially at the cost of narrative and characterization. There is less in the way of functional (signifying) music, while focusing on atmosphere rather than action, movement and direction, is further enabled by a sense of stasis in music. Therefore, like sound effects, the music tends to be tied to place rather than mobile like a traditional score. Silent Hill ’s focus on atmosphere aims at a high degree of emotional involvement in the game through a strong sense of immersion, with often limited vision but enveloping sound. Akira Yamaoka notes that one of game’s central aims was to “… to get more in-depth into the subconscious of the characters …”32 Indeed, the games appear to delineate an interior (mental) landscape, and the mixture of music and sound veers away from conventional representational duties in order to constitute a sense of interiority.33 In the games and films, there is a high degree of sound that is not directly representational. This is not to suggest it is without function. It is crucial in the provision of space as well as having a ‘geographical function’, that of delineating the particularities of a place. Silent Hill as a location has a bizarre sense of space. Music works as a spatial aid to sound design that does not match the visual illusion of 3-D space—sounds can often lack reverb, echo or presence. Thus, a progression through the game is not only a progression through a series of distinct spaces (hotel, hospital, park, amusement park), but also through a series of distinct musical pieces. Hence the difficulty of building a thematic and coherent experience from the music alone. Often now bracketed under the term ‘delay’, echo is a more extreme version of reverberation. Reverberant spaces have reflective surfaces which bounce sounds off them. This means that a small reverberant space will give a certain amount of isolation to a sound, with a slightly harsh edge of reverb that makes it stand out slightly. The larger and more reverberant the space, the more a sound will echo, to the point where there are distinct repeats of sound (sometimes called ‘slapback’ in the recording industry). However, the use of electronic echo and reverb is often more of an emotional effect than one of communicating the ‘reality’ of a

8

FROM VIDEO GAME MUSIC TO FILM MUSIC: SILENT HILL

159

screen space. It can often indicate more of a state of mind rather than representing diegetic ‘reality’. According to Peter Doyle, “… echo as encountered by the human listener is an uncanny phenomenon, as if the sound has been emitted by the mass that reflects it”.34 He goes on to suggest that there is something of the transcendent and divine, particularly in echo as used in film. Indeed, one could argue that electronic reverb and echo embody technology as a psychological state in audiovisual culture. Video games largely have inherited their use of echo and reverb from film and radio, where it often indicated memory or dream. Since the 1930s, processing of sound effects such as reverberation (colloquially known as ‘reverb’) have been used to figure interior states, either for memories of spoken words, as voices in the mind or as indication of supernatural activity. The sound effect of echo, which normally involves some sort of repeated sound reflection, tends to have a more ‘realistic’ usage, although its extreme use indicates either massive space or a situation of abnormality. A constant ambience is evident in the films more than in the games, although arguably it is derived from the logic of the games. It is not continuous, and changes constantly, but almost always there is a continuum of vague sound-music-noise present. In most cases it illustrates a sense of deep space through echo and reverb, more often than not at a similar distance rather than rendering a succession of sounds at different distances. In Silent Hill games and films there is an almost constant use of subtle reverb and echo effects whether in Silent Hill or in the Otherworld. This indicates the abnormality of these spaces, while suggesting that they take place in the mind of the protagonist. However, electronic reverb and echo is less of an indicator of diegetic space than it is a musical aspect of the soundtracks. The sense of events taking place at distance is disturbing but constructed as something of a continuum in many sequences and manifests a form of aural accompaniment rather than delineators of the gameworld and film space. This can lead to something of a mismatch between the expansive, reverb-drenched music and sounds and misty (what should be sound-muffled) images.

160

K. J. DONNELLY

Conclusion The two Silent Hill films (2006, 2012) were constructed around the existing soundtracks of the source video games, particularly the early ones. As a consequence, they have highly singular soundtracks, on the one hand cementing the films’ relationship to the games while on the other sometimes appearing bizarre in relation to mainstream film soundtrack norms. In the games, Silent Hill ’s world appears to emanate from the neurotic mind of the game’s central character. This translates to the films. Indeed, the constant dislocation of sound and image establishes the sense of aberrant psychology at the heart of the game. The Silent Hill films follow the games in having a soundtrack that although it adopts some film aspects, reverses some of the film conventions, most notably in having sound effects that have an emotional cast and music that is unemotional and mechanical, like an inanimate object. There is a degree of equivalence between sonic ambience and images/ ideas. The deserted town of impossible geographies and mysterious apparitions involves electronic delay on sounds and music, while the relentless mist and falling ash inspire continuous sounds that simply remain rather than develop. Echo and reverb here are less components of representation than very direct emotion and psychology, in other words not communication so much as pure aesthetics. These are characteristics not only of the unified soundtrack but also of all sounds being organized in a musical manner and having essentially emotional valence.

Notes 1. As stated by Gans in his DVD commentary. Silent Hill. Fox Home Entertainment, 2006. B000GHRCEE. 2. Furthermore, the importance and status of the game’s music is evident in CD releases of selections from each game, and also on a DVD called ‘Lost Memories—The Art and Music of Silent Hill ’, which includes music from the first three games, as well as trailers, artwork and production material. 3. Yamaoka noted that: “Every sound and every line of sound that is in the game is done by me. And, I make all my own sound effects …” “GDC 2005: Akira Yamaoka Interview” Game Informer magazine. 2005. www.gameinformer.com/News/Story/200503/ N05.0310.1619.39457.htm [accessed 07/03/2007].

8

FROM VIDEO GAME MUSIC TO FILM MUSIC: SILENT HILL

161

4. Earlier, he had worked on Castlevania: Symphony of the Night (1997), where he programmed drums, and supplied songs for a number of games including Dance Dance Revolution (1999) and Beatmania IIDX (2002). For those games, Yamaoka predominantly composed songs, although he has composed all the music for Road Rag e (aka Speed King ) (1998). 5. Nine Inch nails leader Trent Reznor went on to produce the soundtrack for the video game Quake (1996), while member Chris Vrenna, supplied the music for the game American McGee’s Alice (2000). 6. Yamaoka (2005) refers to a ‘Bristol-influence’ (trip hop) in Silent Hill 2, moving to ‘more American’ pop songs in Silent Hill 3 and then back to a ‘more British sound’ in Silent Hill 4: The Room. Game Informer magazine, op.cit. 7. The ambient sounds and mechanical loops suggest a musical notion of the ‘post-industrial’, figuring the ghost of the industrial past, with Silent Hill as ghost town, possibly an old mining town, and thus, this ‘industrial music’ embodies nostalgia for blue collar work. 8. David Bessell, “What’s That Funny Noise?: An Examination of the Role of Music in Cool Boarders 2, Alien Trilogy and Medievil 2” in Geoff King and Tanya Krzywinska, eds., ScreenPlay: Cinema/ Videogames/Interfaces (London: Wallflower, 2002), p.139. 9. Bernard Perron, Silent Hill: The Terror Engine (Ann Arbor: University of Michigan Press, 2012), pp.87–8. 10. Michel Chion, Audio-Vision: Sound on Screen (New York, Columbia University Press, 1994), p.71. 11. Zach Whalen, “Film Music vs. Game Music: The Case of Silent Hill” in Jamie Sexton, ed., Music, Sound and Multimedia: From the Live to the Virtual (Edinburgh: Edinburgh University Press, 2007), p.76. 12. Cf. Claudia Gorbman’s use of the term ‘sonorous envelope’ (originally taken from Didier Anzieu) to describe one of the central functions of film underscore. Claudia Gorbman, Unheard Melodies: Narrative Film Music (London: BFI, 1987), p.63. 13. In a recent dedicated article about sound in Silent Hill 2 Andy Kelly notes, “The audio design is rarely talked about with the same adoration as the wonderfully dark story and surreal, twisted art style, but it’s just as important. … Can you imagine a major publisher taking a risk on such a bizarre, slow-paced game today?

162

14. 15.

16.

17.

18.

19.

20. 21.

22.

23.

24.

K. J. DONNELLY

… Whenever I listen to [Yamaoka’s score] I’m instantly transported to those misty streets.” Andy Kelly, “How Sound Makes Silent Hill 2 a Horror Masterpiece” in PC Gamer, no.262, February 2014, pp.126–7. Whalen, op.cit., p.77. Rob Bridgett, “Dynamic Range: Subtlety and Silence in Video Game Sound” in Karen Collins, ed., From Pac-Man to Pop Music: Interactive Audio in Games and New Media (Aldershot: Ashgate, 2008), p.130. Florian Mundhenke, “Resourceful Frames and Sensory Functions—Musical Transformations from Game to Film in Silent Hill ” in Peter Moormann, ed., Music and Game: Perspectives on a Popular Alliance (Wiesbaden: Springer VS, 2013), p.117. Of course, this is nothing particularly new to film. Famously, Ridley Scott took sections of Jerry Goldsmith’s score for Alien (1979) and shifted them to sections for which they had not been written. Temp tracks might be approached as a musical foundation plan for films, setting out a structure of action and emotion before the film is constructed. Good examples would include Stanley Kubrick’s 2001: A Space Odyssey (1968) and Quentin Tarantino’s Reservoir Dogs (1991). Some of the music has been remixed and some re-recorded. As the film’s credits inform us, this took place at Metalworks Studio, Toronto, although Kevin Banks’s credit as ‘music editor’ suggests that there nevertheless was a need to cut pieces together. Cf. ‘Games are not film! But …’ section in Karen Collins, Game Sound (Cambridge, Mass: MIT Press, 2008), p.5. The notion of ‘gameworld’ denotes the distinct character of the illusory world and space on screen. It is similar to film’s illusory diegesis, although it also includes the limitations upon and ‘feel’ of playing the game. It can match the action through certain activities triggering the appearance of certain music or signalling developments in the ongoing music. See, for example, Danijela Kulezic Wilson, “Sound Design is the New Score” in Music, Sound and the Moving Image, vol. 2, no.2, 2008, pp.127–131. James Lastra, “Film and the Wagnerian Aspiration: Thoughts on Sound Design and History of the Senses” in Jay Beck and Tony

8

25.

26.

27.

28.

29. 30.

31.

FROM VIDEO GAME MUSIC TO FILM MUSIC: SILENT HILL

163

Grajeda, eds., Lowering the Boom: Critical Studies in Film Sound (Chicago: University of Illinois Press, 2008), p.135. While mainstream films tend to use orchestral scores, DAWs are often used to sketch what the score will sound like when played by an orchestra. For video/computer games DAWs tend to produce the final product. Yamaoka’s Equipment on the early games was Emagic Logic, Emagic EXS24, Emagic amt8, Phrazer Infinity, Roland JD-800, Roland JP-8080, Roland TB-303, Roland MKS-80, Roland DJ70, Roland VP-9000, Sequential Circuits Prophet 5, Oberheim OB-MX, Ensoniq VFX, Akai VX-600, Korg Prophecy. There were no acoustic drums and guitars except on one or two tracks. These sounds have a strong ‘retro’ sense to more recent listeners. Indeed, one might argue that in aesthetic terms this has been ongoing for some decades, particularly if one thinks of the rock tradition which includes records like the Shangri Las’s Leader of the Pack (1965) with its motorcycle sounds or the recordings of animals at the conclusion of The Beatles’s Good Morning (from Sgt.Pepper’s Lonely Hearts Club Band [1967]), or the extensive use of field recordings on Pink Floyd’s Dark Side of the Moon (1973). These among others helped establish a solid tradition that denies any significant divide between ‘music’ and ‘sound effects’. In fact, in its lack of regular synchronization and background continuity, Splet’s ambience resembles many ‘horror’ game soundtracks. Interestingly, Lynch’s regular production designer Carol Spier worked on Silent Hill. Chion, op.cit., p.71. Evident in his discussions of ‘nonindifferent nature’, the ‘musicality of landscape’ and the ‘musicality of color and tone’. Sergei M.Eisenstein, Nonindifferent Nature: Film and the Structure of Things, translated by Herbert Marshall (Cambridge: Cambridge University Press, 1987). Roger Avery stated that, like Gans he is a video game player and was impressed by Gans’s knowledge of Silent Hill. He notes that it was “… a very difficult script for the studio to accept. … we had long moments where seemingly nothing happens. It’s all atmosphere …” Paul Davidson, “Silent Hill: A Movie Made by Gamers” [interview with screenwriter Roger Avery] in IGN ,

164

K. J. DONNELLY

13 October 2005. http://uk.movies.ign.com/articles/658/658 291p1.html [accessed 03/04/07]. 32. Akira Yamaoka, “GDC 2005: Akira Yamaoka Interview” Game Informer magazine. www.gameinformer.com/News/Story/200 503/N05.0310.1619.39457.htm [accessed 07/03/2007]. 33. This is a process that is evident perhaps in some ‘space rock’, such as Klaus Schulze’s Mirage (1977) or Brian Eno’s foundational ambient album Ambient 4: On Land (1982). 34. Peter Doyle, Echo and Reverb: Fabricating Space in Popular Music Recording, 1900–1960 (Middletown, Conn: Wesleyan University Press, 2005), p.108.

CHAPTER 9

The Triple Lock of Synchronization

Contemporary audiovisual objects unify sound and moving image in our heads through the locking together of the screen and speakers/ headphones. The synchronization of these two channels remains one of the defining aspects of contemporary culture. Video games follow their own particular form of synchronization, where not only sound and image, but also player input form a close unity.1 This synchronization unifies the illusion of movement in time and space and cements it to the crucial interactive dimension of gaming. In most cases, the game’s dedicated software assembles the whole, fastening together sound and other elements of the game, allowing skilled players to synchronize themselves and become ‘in tune’ with the game’s merged audio and video. This constitutes the critical ‘triple lock’ of player input with audio and video that defines much gameplay in digital games. This chapter will discuss the way that video games are premised upon a crucial link-up between image, sound and player, engaging with a succession of different games as examples to illustrate differences in relations of sound, image and player psychology. There has been surprisingly little interest in synchronization, not only in video games but also in other audiovisual culture.2 In many video games, it is imperative that precise synchronization is achieved or else the unity of the game world and the

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025 K.J. Donnelly, Hearing Eyes, Seeing Ears, Palgrave Studies in Audio-Visual Culture, https://doi.org/10.1007/978-3-031-73913-2_9

165

166

K. J. DONNELLY

player’s interaction with it will be degraded and the illusion of immersion and the effectiveness of the game dissipated. Synchronization can be precise and momentary, geared around a so-called ‘synch point’, or it might be less precise and more continuous but evincing matched dynamics between music and image actions, or connections can be less clear. Four types of synchronization in video games exist. The first division, precise synchronization, appears most evidently in interactive sounds where the game player delivers some sort of input that immediately influences audiovisual output in the game. Clearest where diegetic sounds emanate directly from player activity, it also occurs in musical accompaniment that develops constantly in parallel to the image activity and mood. The second division, plesiochrony, involves the use of ambient sound or music which fits vaguely with the action, making a ‘whole’ of sound and image, and thus a unified and immersive environment as an important part of gameplay. The third strain would be music-led asynchrony, where the music dominates and sets time for the player. Finally, in parallel path asynchrony, music accompanies action but evinces no direct weaving of its material with the on-screen activity or other sounds.

Synching Everything Up It is important to note that synchronization is both the technological fact of the gaming hardware pulling together sound, image and gamer, as well as simultaneously being a critically important psychological process for the gamer. This is central to immersion, merging sensory stimuli and completing a sense of surrounding ambience that takes in coherently matched sound and image. Now, this may clearly be evident in the synchronization of sound effects with action, matching the world depicted on screen as well as the game player’s activities. For instance, if we see a soldier fire a gun on screen we expect to hear the crack of the gunshot, and if the player (or the player’s avatar) fires a gun in the game, we expect to hear a gunshot at the precise moment the action takes place. Sound effects may appear more directly synched than music in the majority of cases, yet accompanying music can also be an integrated part of such events, also matching and directing action, both emotionally and aesthetically. Synchronization holds together a unity of audio and visual, and their combination is added to player input. This is absolutely crucial to the process of immersion through holding together the illusion of sound and vision unity as well as the player’s connection with that amalgamation.

9

THE TRIPLE LOCK OF SYNCHRONIZATION

167

Sound provides a more concrete dimension of space for video games than images, serving a crucial function in expanding the surface of its flat images. The keystones of this illusion are synch points, which provide a structural relationship between sound, image and player input. Sync points unify the game experience as a perceptual unity and aesthetic encounter. Writing primarily about film but with relevance to all audiovisual cultures, Michel Chion coined the term ‘synchresis’ to describe the spontaneous appearance of synchronized connection between sound and image.3 This is a perceptual lock that magnetically draws together sound and image, as we expect the two to be attached. The illusory and immersive effect in gameplay is particularly strong and when sound and image are perceived as a unity. While we, the audience, assume a strong bond between sounds and images occupying the same or similar space, the keystones of this process are moments of precise synchronization between sound and image events.4 This illusion of sonic and visual unity is the heart of audiovisual culture. Being perceived as an utter unity disavows the basis in artifice and cements a sense of audiovisual culture as on some level being a ‘reality’.5 The Gestalt psychology principle of isomorphism suggests that we understand objects, including cultural objects, as having a particular character, as a consequence of their structural features.6 The notion of ‘shared essential structure’, where certain structural features elicit an experience of expressive qualities, and these features recur across objects in different combinations, accounts for the common pairing of certain things: small, fast moving objects and high-pitched sounds; slow-moving music and static or slow-moving camerawork with nothing moving quickly within the frame and so on. Isomorphism within Gestalt psychology emphasizes a sense of cohesion and unity of elements into a distinct whole, which in video games is premised upon a sense of synchronization, or at least ‘fitting’ together unremarkably, matching if not obviously then perhaps on some deeper level of unity. According to Rudolf Arnheim, such a “structural kinship” works essentially on a psychological level,7 as an indispensable part of perceiving expressive similarity across forms, as we encounter similar features in different contexts and formulations. While this is, of course, bolstered by convention, it appears to have a basis in primary human perception.8 One might make an argument that many video games are based on a form of space exploration, concatenating the illusory visual screen space with that of stereo sound, and engaging a constant dynamic of both

168

K. J. DONNELLY

audio and visual movement and stasis. This works through isomorphism and dynamic relationships between sound and image that can remain in a broad synchronization, although not matching each other pleonastically blow for blow.9 A good example here would be the first-person shooter Quake (1996, GT Interactive) where the player sees their avatar’s gun in the centre of the screen and has to move and shoot grotesque cyborg and organic enemies. Trent Reznor and Nine Inch Nails’ incidental soundtrack consists of austere electronic music, dominated by ambient drones and treated electronic sounds. It is remarkable in itself but matches well to the visual aspects of the game world. The player moves in 3-D through a dark and grim setting that mixes an antiquated castle with futuristic high-tech architecture, corridors and underwater shafts and channels. This sound and image environment is an amalgam, often of angular, dark-coloured surfaces and low-pitched notes that sustain and are filtered to add and subtract overtones. It is not simply that sound and image fit together well but that the tone of both is in accord on a deep level. Broad synchronization consists not simply as a mimetic copy of the world outside the game (where we hear the gunshot when we fire our avatar’s gun) but also where general cohesion of sound and image worlds is derived from perceptual and cognitive horizons as well as cultural traditions. In other words, the cohesion of the sound with the image in the vast majority of cases is due to structural and tones similarities between what are perhaps too often approached as utterly separate channels. However, apart from these deep (rather than surface) level similarities, coherence of sound and image can also vary due to the degree and mode of synchronization between the two.

Finger on the Trigger While synchronization may be an aesthetic strategy or foundation, a principle that produces a particular psychological engagement, it is also essentially a technological process. Broadly speaking, computer CPUs (central processing units) have their own internal clock that synchronizes and controls all its operations. There is also a system clock which controls things for the whole system (outside of the CPU). These clocks also need to be in synchronization.10 While matters initially rest on CPU and console/computer architecture—the hardware—they also depend crucially on software.11

9

THE TRIPLE LOCK OF SYNCHRONIZATION

169

The principle of the synch point where image events and player inputs trigger developments in the music is a characteristic of video game music. Jesper Kaae discusses video games as ‘hypertext’, consisting of nodes and links, which are traversed in a non-linear fashion.12 ‘Nodes’ might be understood as the synch points that underpin the structure of interactive video games, and have particular relevance for the triple lock of sound and image to player input. Specific to video games is how the disparate musical elements are triggered by gameplay and combined into a continuum of coherent development for the player’s experience over time. Indeed, triggering is the key to development in such a non-linear environment, set in motion by the coalescence of the player’s input with sound and image elements. A player moving the avatar into a new room, for example, can trigger a new piece of music or the addition of some musical aspects to existing looped13 music. Michael Sweet notes that triggered musical changes can alter emotional state, general atmosphere, change the intensity of a battle, indicate a fall in the player’s health rating, indicate an enemy’s proximity and indicate successful completion of a task or battle.14 Triggered audio can be a simple process, where some games simply activate a loop of repeated music that continues ad infinitum as accompaniment to the player’s screen actions, as in Tetris (1989, Nintendo). However, triggered audio often sets in train more complex programmes where music is15 varied. Karen Collins effectively differentiates between types of triggered audio in video games. ‘Interactive audio’ consists of ‘Sound events that react to the player’s direct input’ like footsteps and gunshots, whereas ‘adaptive audio’ is ‘Sound that reacts to the game states’ such as location, mood or health.16 The former relates to precise gameplay while the latter is not so directly affected by player activity. In terms of music in video games, while much works on a level of providing accompanying atmosphere in different locations, for example, and so is adaptive, some programming allows for developing music to be triggered by a succession of player inputs (such as proximity of enemies). Adaptive music is particularly effective in sophisticated action role-playing games with possible multiple paths. Good examples would be later The Elder Scrolls series games such as Jeremy Soule’s music for Oblivion (2006) and Skyrim (2011, both Bethesda). A particularly complex form of synchronization comes from branching (horizontal resequencing) and layering (vertical remixing) music to fit momentary developments in the gameplay instigated by the player.17 The process of pulling together disparate musical ‘cues’ involves direct joins, crossfades

170

K. J. DONNELLY

and masks, with a dedicated programme controlling a database of music in the form of fragmentary loops, short transitions and longer musical pieces of varying lengths. These sophisticated procedures yield constant variation, where music is integrated with the experience of the player. This means momentary change, synchronized precisely to events in the game, and this precise matching of musical development to action on screen owes something to the film tradition of ‘mickeymousing’ but is far more sophisticated.18 The relationship of image, player input and soundtrack is retained at a constantly close level, controlled by the programming and with momentary changes often not consciously perceived by the player. Tradition has led to strong conventions in video game audio and the relationship between sound and image. Some of these conventions are derived from other, earlier forms of audiovisual culture,19 while some others are more specific to game design. Synchronization is fundamental for video games, but the relationship between sound and image can take appreciably different forms. It might be divided into four types: precise synchronization to gameplay, plesiochrony, forcing gameplay to fit music and asynchrony. 1) Player-Led Synchrony Player-led synchronization has a succession of synch points where player input aligns precisely with sound and image. Player input can change screen activity, and this renders musical developments in line with the changes in game activity. This is what Collins calls ‘interactive audio’ and this principle is most evident in interactive sound effects where the game player provides some sort of input that immediately has an effect on the audiovisual output of the game. For instance, the pulling of a gun’s trigger by the player requires a corresponding immediate gunshot sound and requires a corresponding resultant action on screen where a target is hit (or not) by the bullet. This is a crucial process that provides immersive effect, making the player believe on some level in the ‘reality’ of the gameplay and game world on screen. It correlates with our experience of the real world,20 or at least provides a sense of coherent world on screen if it does not resemble our own. This triple lock holds together sound and image as an illusory unity but also holds the player in place as the most essential function. Indeed, the continued coherent immersive illusion of the game is held together by the intermittent appearance of such

9

THE TRIPLE LOCK OF SYNCHRONIZATION

171

moments of direct, precise synchronization. The coherence of the experience is also aided by synchronized music, which forms a precise unity of visuals on screen, other sounds and gameplay activity. Music in such situations is dynamic, following player input to match location, mood and activity. This is reactive music that can change in a real-time mix to the action, depending directly on some degree of variable input from the player. It lacks the linear development of traditional music and indeed, each time a particular section of a game is played, the music might never be exactly the same. An interesting case in point is Dead Space (2008, EA Redwood Shores), which has a particularly convoluted and intricate approach to its music. The game is set in the twenty-sixth century, where engineer Isaac has to fight his way through the mining spaceship Ishimiura that is filled with ‘Necromorphs’, who are the zombified remnants of its crew. To destroy them, Isaac has to dismember them. Played in third person, Dead Space includes zero gravity vacuum sections, and puzzle solving as well as combat. Jason Graves, the game’s composer, approached the game events as a drama like a film. He stated: “I always think of this the way I would have scored a film …but it’s getting cut up into a giant puzzle and then reassembled in different ways depending on the game play”.21 So, the aim is to follow the model of incidental music from film,22 but in order to achieve this, the music needs to follow a complex procedure of real-time mixing. While some games only offer a repetitive music on/off experience, Dead Space offered a more sophisticated atmospheric and immersive musical soundtrack. Rather than simply branching, the game has four separate but related music streams playing all the time. These are ‘creepy’, ‘tense’, ‘very tense’ and ‘chaotic’. The relationship between these parallel tracks is controlled by what the game designers in this case call ‘fear emitters’, which are potential dangers anchored in particular locations in the game world. The avatar’s proximity to these beacons shapes and mixes those four streams, while dynamically altering relative volume and applying filters and other digital signal processing. This means that a constant variation of soundtrack is continually evident throughout Dead Space.23 Rather than being organized like a traditional film score, primarily around musical themes, the music in Dead Space is built around the synchronization of musical development precisely to avatar activity, and thus player input in relation to game geography and gameplay (Fig. 9.1).

172

K. J. DONNELLY

Fig. 9.1 Dead Space

2) Plesiochrony Plesiochrony aims not to match all dynamic changes of gameplay, but instead to provide a general ambience, making a unity of sound and image, and thus an immersive environment for gameplay. Player input is less important and might merely be at a level of triggering different music by moving to different locations. The term describes a general, imprecise synchronization.24 Plesiochrony works in an isomorphic manner (as discussed earlier), matching together atmosphere, location and general mood. Music and image fuse together to make a ‘whole’, such as a unified environment, following the principles of being isomorphically related to atmosphere, location and mood. This might be characterized as a ‘soft synchrony’ and corresponds to Collins’ notion of ‘adaptive audio’. The music modifies with respect to gameplay in a broad sense, but neglects to change constantly in direct response to a succession of player inputs. The music in these cases becomes part of the environment and becomes instituted in the player’s mind as an emotionally charged and phenomenologically immersive experience. The music is often simply triggered and plays on regardless of momentary gameplay. However, it nevertheless accomplishes an important role as a crucial part of the environment and atmosphere, indirectly guiding and affecting player activity. Indeed,

9

THE TRIPLE LOCK OF SYNCHRONIZATION

173

perhaps its principal function is a general furnishing of ‘environmental’ aspects to the game, emphasizing mood, tone and atmosphere.25 For instance, in Quake (1996, GT Active), the music at times quite crudely starts and stops, often with almost no interactive aspect. It is simply triggered by the player’s avatar entering a new location or by imminent combat. The score has something of the quality of diegetic ambience and at times could be taken to be the sound of the location. However, the sounds do not change when the avatar becomes immersed underwater, indicating that it is outside the game’s diegesis. While 3D games like Quake followed a model evident in most first-person shooters (FPSes), other 3D games have adopted different approaches. Indie game The Old City: Leviathan (2015, Postmod) is not based on skilful fighting action or thoughtful puzzling. It is a first-person ‘walking game’ where a detailed visual environment is open to the player’s exploration. This game engages with a margin of video game history, games that are about phenomenological experience rather than progressive achievement and gameplay in the conventional sense. The music was a featured aspect of The Old City: Leviathan’s publicity and the lack of gameplay allowed music to be foregrounded. As a counterpart to the visuals, the extensive and atmospheric music soundtrack is by Swedish dark ambient industrial musician Atrium Carceri. The game’s texture is emphasized by the player’s slow movement around the city location in first person, which allows and encourages appreciation of the landscape. Indeed, the game developers were obsessed with images and sounds: on their promotional website the discussion alights on the difficulty of visually rendering puddles. More generally, the Postmod website states: “Players have the option to simply walk from start to finish, but the real meat of the game lies in the hidden nooks and crannies of the world; in secret areas, behind closed doors …”. The music is not only an integrated part of the experience, but also follows a similar process of being open to exploration and contemplation as ambient music tends to be quite ‘static’ and lacks a sense of developmental movement. The fact that there is little real gameplay, apart from walking, gives music a remarkable position in the proceedings, what might be called ‘front stage’. There is no need for dynamic music, and the music has the character of Atrium Carceri’s other music, as atmospheric ambience.26 The music is equivalent to landscape in an isomorphic manner. It is like a continuum, develops slowly and has no startling changes. The aim is at enveloping ambience, with a vaguely solemn mood that matches the player’s slow movement around the large,

174

K. J. DONNELLY

deserted cityscape. In a way, the game ‘fulfils’ the potential of the music, in that its character is ‘programmatic’ or ambient music. While the music appears somewhat indifferent to the player, an interactive relationship is less important here as the music is an integrated component of the game environment (where the world to a degree dominates the player), and music functions as triggered isomorphic atmosphere. Less concerned with gameplay and more directly concerned with embodying environment, it lacks notable dynamic shifts and development. This de-centres the player and makes them a visitor in a large dominant and independent sound and image-scape. However, there is a coherence to the game’s world that synchronizes sound and image on a fundamental level. 3) Music-Led Asynchrony In the first configuration of asynchrony, music articulates and sets controls for action and gameplay. Collins and others are less interested in this form of game audio, perhaps because initially it appears to be less defining and less specific to video games than ‘dynamic audio’. However, I would suggest that asynchrony marks a significant tradition in game audio. With games that are timebound, music often can appear to set the time for gameplay. This is most clear, perhaps, with music-based games like Guitar Hero (2005, Harmonix) or Dance, Dance Revolution (1998, Konami), where the fixed song length dictates the gameplay. A fine example is the games corralled as the WiiFit (2007, Nintendo) programme, a massively popular series of exercises that mix the use of a dedicated balancing board with the console’s movement sensor to appraise performance. The music for each task is functional, providing a basic atmosphere but also providing a sense of aesthetic structure to the often repetitive exercises.27 Each piece is fairly banal and does not aim to detract from concentration, but provides some neutral but bright, positive-sounding wallpaper (all major keys, basic melodies, gentle rhythmic pulses and regular four-bar structures). In almost every case, the music articulates exercise activity. It is not ‘interactive’ as such, and the precise duration of each exercise is matched by the music. In other words, each piece of music was written to a precise timing, and then the game player has to work to that. They should not, or cannot, make any changes to the music, as might be the case in other video games. The WiiFit has a number of exercise options available, each of which lasts a

9

THE TRIPLE LOCK OF SYNCHRONIZATION

175

relatively short period of time. These are based on yoga, muscle exercises or synchronized movement. There are also some balancing activities which are closest to traditional video games and include ski jumping and dance moves to precisely timed movements. The crucial point for the player is that they can tell when the music is coming to an end, and thus when the particular burst of exercise is about to finish. Repetition of the same exercises causes the player to know the music particularly well and to anticipate its conclusion. This is clearly a crucial psychological process, where the player is forced to synchronize themselves and their activities directly to the game, dictated by the music. Similarly, in Plants vs Zombies (2009, PopCap), changes in the musical fabric are based on the musical structure and requirements, not the gameplay. It is a tower defence game where the player defends a house using a lawn (in the initial version), where they plant anthropomorphic defensive vegetables to halt an onrush of cartoon zombies. Each level of the game has a different piece of accompanying music, which comprises electronic approximations of tuned percussion and other traditional instrument sounds. The relation of music to action is almost negligible, with the regularity of the music furnishing a rigid and mechanical character to gameplay. The music simply progresses, almost in parallel to the unfolding of the game, as a homology to the relentless shambling movement of the zombies. The first levels take place during the day on the front lawn of the player’s unseen house and are accompanied by a piece of music that simply loops. The recorded piece mechanically restarts at the point where it finishes, irrespective of the events in the game.28 Because the music is not subject to interruptions from dynamic music systems, the music is free to feature a strong, regular rhythmic profile. The cue is based on a tango or habanera dance rhythm. The very regularity of dances means that, as an accompaniment to audiovisual culture, they tend to marshal the proceedings, to make the action feel like it is moving to the beat of the dance rather than following any external logic. In Plants vs. Zombies, this effect is compounded by the totally regular structure of the music (based on four-bar units, with successive melodies featuring particular instruments: oboe, strings and pizzicato strings, respectively).29 There is a clear sense of integrity from the music’s regular rhythmic structure in the face of variable timings in the gameplay. The harmony never strays too far from the key of a minor (despite an F 7th chord), and the slow tango rhythm is held in the bass line, which plays chord tones with the occasional short chromatic run between pitches. However, the continuity

176

K. J. DONNELLY

of this music is halted by the player’s successful negotiation of the level, which triggers a burst of jazz guitar to crudely blot out the existing music. Plants vs Zombies ’ music conceivably could fit another game with a profoundly different character. Having noted this, the cue’s aptness might be connected most directly with the game on a deeper level, where the music’s regularity relates to the unceasing regularity of the gameplay. The cue is not synchronized to the action, apart from the concluding segment of the level, where a drumbeat enters as an accompaniment to the existing music, appearing kinetically to choreograph movement through grabbing proceedings by the scruff of the neck as what is billed on screen as a “massive wave of zombies” approaches at the conclusion of each level. The assumption is that the beat matches the excitement of action (and chaotic simultaneity on screen). However, again, if we turn the sound off, it does not have a significant impact of the experience of the game and arguably none at all on the gameplay. In summary, the time of the music matches the game section’s pre-existing structure, and the player has to work to this temporal and dynamic agenda rather than controlling it (Fig. 9.2). 4) Parallel Path Asynchrony

Fig. 9.2 Plants vs Zombies

9

THE TRIPLE LOCK OF SYNCHRONIZATION

177

Unsynchronized music can also have a relationship of indifference to the game, and carry on irrespective of its action. It might simply coexist, as a presence of ambiguous substance, and might easily be removed without significantly impairing the experience of the game. This situation is relatively common for mobile and iOS games and other games where sound is unimportant. Here, the music is not integrated strongly into the player’s experience. This relationship of asynchrony between the music and gameplay is embodied by this form of ‘non-functional’ music that adds little or nothing to the game and might easily be removed or replaced. Such music often has its own integrity as a recording and can finish abruptly, when the player concludes a game section (either successfully or unsuccessfully), sometimes interrupted by another musical passage. This owes something to the tradition of earlier arcade games, and although it may seem crude in comparison with the processes of dynamic music, it is nevertheless an effective phenomenon and thus persists. The way this non-interactive music carries on irrespective of gameplay makes it correspond to so-called ‘anempathetic’ music, where music seems indifferent to the on-screen action. Anempathetic music has been theorized in relation to film, but clearly has relevance in all audiovisual culture. Michel Chion discusses such situations, which have the potential to ‘short-circuit’ simple emotional congruence to replace it with a heightened state of emotional confusion.30 Some games simply trigger and loop their music. Although ‘interactive dynamic music’ can be a remarkable component of video games and is evident in many so-called ‘four star’ prestige video games, in its sophisticated form, it is hardly the dominant form of video game music. Disconnected ‘non-dynamic’ music exemplifies a strong tradition in game music, which runs back to the arcade.31 While some might imagine this is a retrogressive format, and that arcade games were simplistic and aesthetically unsophisticated, this earliest form of video game sound and music remains a highly functional option for many contemporary games. For instance, this is highly evident in iOS and other mobile games, and games which lack dynamics and/or utilizing highly restricted spatial schemes such as simple puzzle games or games with heavily restricted gameplay. A good example is the action platformer Crash Bandicoot (1996, Naughty Dog), which has a sonic backdrop of kinetic music emphasizing synthesized tuned percussion, while earcons/auditory icons and sound effects for game events occupy the sonic foreground. The music changes with each level, but has no requirement to enter a sophisticated process of

178

K. J. DONNELLY

adaptation to screen activity. Yet in purely sonic terms, the game achieves a complex interactive musical melange of the game score in the background and the highly musical sounds triggered by the avatar’s activities in the gameplay. Yet the game can happily be played without sound. The musical background can add to the excitement of playing, evident in arcade games such as jet ski simulator Aqua Jet (1996, Namco), which required the player to ‘qualify’ through completing sections in an allotted time. The music merely formed an indistinct but energetic sonic wall behind the loud sound effects. Similarly, in driving game Crazy Taxi (1999, Hitmaker), ‘game time’ ticks down and then is replenished if the player is successful. Here, songs (including punk rock by The Offspring) keep going, irrespective of action. The music can chop and change when a section is finished, and when one song finishes another starts. The situation bears resemblance to the Grand Theft Auto series (1997-, DMA, Rockstar, Digital Eclipse) with its radio station of songs that are not synchronized with action. Candy Crush Saga (2012, King) is an extremely successful mobile game of the ‘match three’ puzzle variety. Music is less than essential for successful gameplay, although the game’s music is highly effective despite many people playing the game ‘silent’. The merest repetition of the music can convince the player of its qualities.32 It is simply a looped recording and in no way dynamic in relation to the game. The music has a fairly basic character, with a swinging 12/8 rhythm and clearly articulated chord changes (from I-IV-IVm-I-V) at regular intervals. However, one notable aspect of the music is that at times it is extremely rubato, speeding up and slowing down, which helps give a sense that the music is not regimented, and the player indeed can take variable time with making a move. The player’s successive moves can sometimes be very rapid and sometimes taken at their leisure. Whatever happens, the music carries on regardless. However, it is intriguing that the music contains a moment of drama that pulls slightly outside of the banal and predictable framework. The shift from major to minor chord on the same root note is a surprising flourish in a continuum of highly predictable music. This exists perhaps to make the music more interesting in itself yet this (slightly) dramatic change signalled in the music is not tied to gameplay at all. It is subject to the random relationship between music and gameplay: the music can suggest some drama where there is none in the game. It might be argued that this applies a moment of psychological pressure to the player, who might be taking time over their next move, or it may not.

9

THE TRIPLE LOCK OF SYNCHRONIZATION

179

However, while the music is a repeated recording with no connection to the gameplay, oddly it has a less mechanical character than some game music. The rubato performance and jazzy swing of the music can appear less strictly regimented than some dynamic music, which needs to deal in precise beats and quantified rhythms to hold together their disparate stem elements.33

Conclusion The process of ‘triggering’ is the key to the whole process of synchronization in video games and maybe noticeable for the player or not. In some cases, the trigger may simply inaugurate more variations in the musical accompaniment, but in others it can cue the beginning of a whole new piece of music. A certain activity embarked upon by the player triggers a change in the game audio. This locks the synchronization of the player’s activity with sound and image elements in the game. Indeed, this might be taken as one of the defining aspects of video games more generally. This interactive point functions as a switch where the player triggers sound activity, such as where moving an avatar to a new location triggers changes in the music and ambient sound.34 This physical, technological fact manifests the heart of the sound and image synch that holds video games together. However, we should remember that rather than simply being a technical procedure, crucially it is also a psychological one. It initiates and signals a significant change in the gameplay (more tense gameplay, the start of a new section of play, etc.). There has been much writing by scholars and theorists of video game music about interactive audio.35 However, ‘interactivity’ presents severe limits as a theoretical tool and analytical concept. Similarly, an analytical concept imported from film analysis used for dealing with video game audio—the distinction between diegetic and non-diegetic—also proves similarly limited in its relevance and ability as a means of analysis. Instead of these two, a division of music through attending synchronization might be more fruitful for video game analysis. This would register the essential similarities between, for example, non-diegetic incidental music and diegetic sound environment for a particular location. Similarly, the dynamic changes that might take place in an interactive musical score might also follow a similar procedure in diegetic sound effects, when a player moves an avatar around a particular on-screen location. Analytical distinctions of music in video games should not be determined purely by

180

K. J. DONNELLY

the mode of production. This has led to the focus on interactive video game music, which may well not be phenomenologically perceived by the player. This risks a focus on the underlying production, the coding level, to the detriment of addressing the phenomenological experience of the surface of gameplay.36 The electro-mechanical perfection of time at the centre of modern culture endures at the heart of video games, where their mechanistic structures work invisibly to instil a specific psychological state in the player.

Notes 1. With some games a slight lack of synchrony between sound and image, between a player’s input and the illusion of the event on screen, can be tolerable. For instance, with point and click narratives or detection games the player can mentally compensate for the discrepancy. However, the overwhelming majority of video games require rapid response to player input. What sometimes is called ‘input lag’ or ‘latency’, when a button is pressed and the in-game response to that activity is not immediate, can utterly ruin the gaming experience. 2. I have discussed this in detail is relation to film and television in Occult Aesthetics: Synchronization in Sound Film (New York: Oxford University Press, 2014). Other relevant writing includes Jeff Rona, Synchronization: From Reel to Reel: A Complete Guide for the Synchronization of Audio, Film and Video (Hal Leonard Corporation, 1990), and Michael Sweet, Writing Interactive Music for Video Games: A Composer’s Guide (London: Addison Wesley, 2014), pp.28–29. 3. Michel Chion, Audio-Vision: Sound on Screen, Claudia Gorbman, ed. and trans. (New York: Columbia University Press, 1994), p.5. 4. Chion’s synchresis matches the ideas of Lipscomb and Kendall, both of which note perceptual ‘marking’ by synch points. S.D.Lipscomb and R.A.Kendall, “Sources of Accent in Musical Sound and Visual Motion” in the Proceedings of the 4th International Conference for Music Perception and Cognition (Liege: ICMPC, 1994), pp.451–2. 5. Frans Mäyrä notes the surface and coding reality beneath. He discusses the ‘dual structure of video games’, where “players access both a ‘shell’ (representational layers) as well as the ‘core’ (the

9

6.

7.

8.

9.

10.

11. 12.

13.

14. 15.

THE TRIPLE LOCK OF SYNCHRONIZATION

181

gameplay).” Getting into the GameQueryQuery: Doing Multidisciplinary Game Studies” in Bernard Perron and Mark J.P.Wolf, eds., The Video Game Theory Reader 2 (New York: Routledge), p.317. Rudolf Arnheim, “The Gestalt Theory of Expression” in Mary Henle, ed., Documents of Gestalt Psychology (Los Angeles: University of California Press, 1961), p.308. Rudolf Arnheim, Art and Visual Perception: A Psychology of the Creative Eye (Los Angeles: University of California Press, 1974), p.450. For further reading about games and Gestalt theory consult K.J.Donnelly, “Lawn of the Dead: The Indifference of Musical Destiny in Plants vs. Zombies ” in K.J.Donnelly, William Gibbons and Neil Lerner, eds., Music In Video Games: Studying Play (New York: Routledge, 2014), p.160; Ingolf Ståhl, Operational Gaming: An International Approach (Oxford: Pergamon, 2013), p.245; and Mark J.P.Wolf, “Design” in Bernard Perron and Mark J.P.Wolf, eds., The Video Game Theory Reader 2 (London: Routledge, 2008), pp.343–4. In other words, rather than the ‘mickeymousing’ (the music redundantly repeating the dynamics of the image activity), there is only a general sense of the music ‘fitting’ the action. Randall Hyde, “The Art of Assembly” quoted at Computer Science. https://cs.stackexchange.com/questions/63449/exactlyhow-many-clocks-does-a-computer-consists-of-and-how-do-theysynchronize [accessed 21/8/2018]. MMORPGs and games for multiple players require effective synchronization to hold the shared game world together. Jesper Kaae, “Theoretical Approaches to Composing Dynamic Music for Video Games” in Karen Collins, ed., From Pac-Man to Pop Music: Interactive Audio in Games and New Media (Aldershot: Ashgate, 2008), p.77. Richard Stevens and Dave Raybould, The Game Audio Tutorial: A Practical Guide to Sound and Music for Interactive Games (London: Focal Press, 2011), p.112. Sweet, 2014, p.28. Ibid., p.36.

182

K. J. DONNELLY

16. Karen Collins, Game Sound: An Introduction to the History, Theory and Practice of Video Game Music and Sound Design (Cambridge, Mass.: MIT Press, 2008), p.126. 17. Tim van Geelen, “Realising Groundbreaking Adaptive Music” in Karen Collins, ed., From Pac-Man to Pop Music: Interactive Audio in Games and New Media (Aldershot: Ashgate, 2008), pp.93–102. 18. Tim Summers, Understanding Video Game Music (Cambridge: Cambridge University Press, 2016), p.190. 19. Ibid., p.176. 20. Alison McMahan, “Immersion, Engagement and Presence. A Method for Analyzing 3-D Video Games” in Bernard Perron and Mark J.P.Wolf, eds., The Video Game, Theory Reader (London: Routledge, 2003), p.72. 21. Ken McGorry, “Scoring to Picture” in Post Magazine, November 2009, p.39. http://beta.jasongraves.com/press-files/PostMagazineNov 2009.pdf [accessed 5/7/2018]. 22. Mark Sweeney notes that the game has two musical sound worlds: a neo-romantic one in cut scenes and a modernist one inspired by Twentieth-Century Art music (of the sort used in horror films) which works for gameplay. The latter is reminiscent of Penderecki’s music as used in The Exorcist (1973) and The Shining (1980). “Isaac’s Silence: Purposive Aesthetics in Dead Space” in Michiel Kamp, Tim Summers and Mark Sweeney, eds., Ludomusicology: Approaches to Video Game Music (Sheffield: Equinox, 2016), pp.190,192. 23. Don Veca, the game’s audio director created a scripting system he called ‘Dead Script’, which was on top of low-level audio drivers and middleware. An important aspect was what he called ‘the creepy ambi patch’, which was a grouping of sounds that constantly reappear but in different form, pitch-shifted, filtered and processed. This was also controlled by the ‘fear emitters’ but appeared more when no action or notable events are happening. Paul Mac, “Game Sound Special: Dead Space” in Audio Media, July 2009, 2–3. http://beta.jasongraves.com/press-files/am_gam eSpecial09small.jpg [accessed 2/3/2015]. 24. For more discussion of plesiochrony, see. Donnelly, Occult Aesthetics, 2014, pp.181–3.

9

THE TRIPLE LOCK OF SYNCHRONIZATION

183

25. Gernot Böhme points to atmosphere as a form of integrated, concrete relationship between human and environment. The Aesthetics of Atmospheres, edited by Jean-Paul Thibaud (London, Routledge, 2017), p.14. 26. Of course, this is importing wholesale music of a style that is not particular to video games yet fits these particular games well. 27. Composers include Toru Minegishi (who had worked on The Legend of Zelda games and Super Mario 3D World), Shiho Fujii and Manaka Tomigana. 28. Although the player’s actions trigger musically derived sounds, forming something of a random sound ‘soloing’ performed over the top of the musical bed, it is difficult to conceive this as a coherent piece of music. 29. The opening tango section comprises 16 bars, followed by eight bars of oboe melody (the same four repeated), then an orphan four-bar drop-out section leading to pizzicato strings for eight bars, followed by the same section repeated, with added sustained strings for eight bars and finally, a section of piano arpeggios of 16 bars (the same 4 repeated four times), after which the whole piece simply repeats. 30. Chion, op.cit., pp.8–9. 31. Donnelly, “Lawn of the Dead”, op.cit., 2014, p.154. 32. The ‘repetition effect’ tests music’s durability, although the cumulative effect of repetition can persuade a listener. I hated the music at first but after playing the game, ended up appreciating it. 33. ‘Stems’ are musical parts that work through fitting intimately together rather than on their own. Arguably, the process of using stems comes from digital musical culture, where it is easy to group together recorded music channels in so-called ‘submixes’. 34. Sweet notes that these points are called ‘hooks’, which describe the commands sent from player input, through the game engine to the audio, so named for the game ‘hooking into’ the music engine. Sweet, 2014, p.28. 35. Not only Karen Collins has written about different forms of interactive audio, but also Michael Lieberman, who notes that there might be three broad categories of music interaction: ‘Linear’ (which cannot be changed by the player), ‘Reactive’ (which is triggered by player actions) and ‘Proactive’ (where the player must follow the game). “Interactivity and Music in Computer Games” in

184

K. J. DONNELLY

Peter Moormann, ed., Music and Game: Perspectives on a Popular Alliance (Berlin: Springer, 2012), pp.47–8. 36. Frans Mäyrä notes that players, “… access both a ‘shell’ (representational layers) as well as the ‘core’ (the gameplay).” Frans Mäyrä, “Getting into the Game: Doing Multi-Disciplinary Game Studies” in Bernard Perron and Mark J.P.Wolf, eds., The Video Game Theory Reader 2 (New York: Routledge, 2009), p.317.

CHAPTER 10

Doctor Who: Britain’s IRCAM and Darmstadt

Doctor Who has been one of the longest running British television shows. Beginning in 1963, having a hiatus in 1989 and then a rebirth in 2005. These two periods of the programmes might be understood as an ‘Old Testament’ and ‘New Testament’. The older period is notable for its startling use of sound and music, and creative sound design. In 1973, the BBC reclassified the sound of protagonist the Doctor’s time and space travelling device, the TARDIS, taking off and landing as ‘music’, crediting Brian Hodgson with its composition.1 Although this was not noted at the time, it was remarkable as a rare acknowledgement of how music and sound effects can be conceived in similar terms. One rarely noted aspect of Doctor Who is the startling incidental music and sound design for the serial, which had some of its roots in more experimental practices. Although the BBC thought of it as a Saturday teatime filler programme, Doctor Who was a symbolic herald of the corporation’s modernity and the repository of the television medium’s imagination more generally. Yet the sound and music had to carry out a lot of the ’imagining’ for a programme beset with cheap sets, basic lighting and unconvincing special effects. As such, it had a more prominent role than music in many other areas of television, inspired by having to span the galaxy and all of time and space.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025 K.J. Donnelly, Hearing Eyes, Seeing Ears, Palgrave Studies in Audio-Visual Culture, https://doi.org/10.1007/978-3-031-73913-2_10

185

186

K. J. DONNELLY

A number of Doctor Who episodes from the 1960s are ’missing’, destroyed by the BBC. The fact that some avid fans recorded the sound on magnetic tape during their broadcast has meant that the BBC has been able to release ’soundtrack’ versions of these missing episodes with some bridging narration to retain coherence. If one listens to these soundtracks, they often demonstrate not only how well the music works for the series, but also how well the soundtracks work as objects within themselves, without the missing visuals.2 The enigmatic ‘Watcher’ in Doctor Who Magazine noted: Doctor Who was inextricably linked with the medium of sound even when it was a television programme. Television, as media scholars have long pointed out, evolved from the manifestly non-visual tradition of broadcast radio. Using the same production techniques and often the same production people, television effectively came into being as radio with added pictures. Although the boundaries are less clear today, in many ways television still has more in common with radio than with cinema.3

This is borne out by the success of the soundtracks of 1960s Doctor Who episodes divorced from their accompanying images. One important reason is that the sound effects and incidental music are so evocative, and that there is also a distinct ambiguity about what constitutes ’music’ and what might be heard as sound effects. Doctor Who regularly functioned as a showcase for sound/music technology, while science fiction more generally offers a unique scope for screen music. As the genre is all about possibilities, the music is all about musical potential. The music in Doctor Who has a massive range. It needs to span the galaxy, to go way beyond music for the rest of television, as well as masking the deficiencies of the programme’s production values.4 Although the music was written and recorded very quickly, tending towards the functional in relation to on-screen activity—it had a determined edge, derived from experimental music culture.

Relative Histories (1963–89) A musical history of the first period can cut across the ’consensual’ history of Doctor Who, which in popular literature on the subject usually follows the succession of different Doctors (starting as William Hartnell, he ‘regenerated’ to Patrick Troughton, Jon Pertwee, Tom Baker,

10

DOCTOR WHO: BRITAIN’S IRCAM AND DARMSTADT

187

Peter Davison, Colin Baker and Sylvester McCoy), or even histories that follow the succession of producers or production and transmission seasons. With respect to the programme’s sound effects, its ’special sound’ was produced by the BBC Radiophonic Workshop, and more precisely by Brian Hodgson from 1963 to 1972, and then by Dick Mills, from 1972 to 1989. Cutting across all these divisions, we might divide Doctor Who (over its initial twenty-six-year run 1963–89) into four musical periods: 1. 1963–9, seasons one to five 2. 1969–80, halfway through season six to season seventeen 3. 1980–4, seasons eighteen to twenty-two 4. 1986–9, seasons twenty-three to twenty-six In its early years, Doctor Who used much in the way of avant-garde musique concrete, startling sounds produced through the manipulation of recordings on magnetic tapes. It also showcased strange sounds produced by basic electronic devices (oscillators, ring modulators, filters), which were most apparent in Delia Derbyshire’s arrangement of the Doctor Who title music. During the 1960s, the programme used music by a number of freelance composers, some of whom were respected in art music circles or had reputations for producing avant-garde music. For example, Tristram Cary produced music for ’The Daleks’ (1963–4) and ’Marco Polo’ (1964), among others, while Humphrey Searle, who was a British pioneer in using Schoenberg’s twelvetone compositional method, scored ’The Myth Makers’ (1965), although with a rather traditional score. Some of Cary’s musical cues were reused in later stories (his avant-garde sounds from ’The Daleks’, for example, would resurface in ’The Rescue’ [1965]) and ’The Ark’ [1966]), a standard practice in television music, unlike its more opulent cousin in the cinema.5 Similarly, during the 1960s Doctor Who used a great deal of stock or library music, which was cheap and had been recorded for use in general contexts rather than for a specific dramatic moment or specific programmes. However, while Doctor Who used library music by composers such as the confusingly named French composer Roger Roger and Martin Slavin, whose Space Adventures was used as a theme for the Cybermen in ’The Tenth Planet’ (1966), ’The Moonbase’ (1967) and ’Tomb of the Cybermen’ (1967). ‘Art music’ was also utilized. Some music by respected British (but Spanish-born) avant-garde composer Roberto Gerhard was used a number of times in

188

K. J. DONNELLY

different stories, as were sounds by the Baschet brothers (also known as les structures sonores or les sculptures sonores), derived from their sound sculptures. This opening musical period runs from 1963 to 1969 (seasons onefive) and is marked by a tendency to use avant-garde music in the popular context of television. This insertion of minority culture into mass culture makes the programme’s sound profile most distinct from contiguous programmes. The fact that the music of avant-garde art music composers was utilized makes Doctor Who remarkable in itself, providing a highly singular sound world through the use of extremely dissonant and unpopular music of the sort that rarely got heard at the time outside highbrow university music departments or occasional minority-interest concerts. The longest stable phase of Doctor Who’s music from the first period ran from 1969 to 80, from halfway through season six to season seventeen. During this time, Doctor Who’s music was written overwhelmingly by Dudley Simpson, who was responsible for scoring 310 episodes. The scores usually comprised a relatively small amount of music performed by a small ensemble of traditional musicians which was usually electronically ’enhanced’ by the BBC Radiophonic Workshop to make it sound less conventional and more otherworldly. From 1963 to 1970, music had been composed blind rather than to the moving images. This changed in 1970, with the onset of the adoption of timecoding on video playback, which had been brought in from Germany.6 This development allowed precise composition and ’through composed’ music that matched the momentary dynamics of on-screen action. Yet over the years, due to the exigencies of rapid production, plenty of the music for Doctor Who was written without recourse to images. Among the most impressive of Dudley Simpson’s scores were ’Pyramids of Mars’ (1975) and ’Horror of Fang Rock’ (1977), both of which had the sort of quality usually associated with expensive film scores, yet retained a strong sense of the alien and the unsettling through the Radiophonic Workshop’s treatments and Simpson’s fragmented music, which was based on short and thematically unconnected musical cells. His music was written to fit action and indeed was probably one of the best examples of precise underscore in television, despite on occasions including some quite crass ’mickeymousing’, where action was mimicked by the music. For example, towards the end of episode two of ’The Talons of Weng Chiang’ (1977), the music simply doubles the screen activity as props are thrown down on the Doctor from the back of the stage. This tendency in Simpson’s scores is more overt

10

DOCTOR WHO: BRITAIN’S IRCAM AND DARMSTADT

189

during his final years on the programme when Doctor Who was shifting from a suspense-based ethos towards a broader and more comedic tone. Simpson’s scoring during seasons sixteen and seventeen reflects that—in effect, he was responding to the scripts’ and productions’ new emphasis. Seasons eight and nine in 1971 and 1972 experimented by having Simpson’s scores, and at times other composers, such as Malcolm Clarke, Geoffrey Burgon and Carey Blyton, scoring occasional stories yet these served only as isolated hiatuses in Simpson’s impressive tenure as the musician for Doctor Who throughout the decade. However, by season eleven he was being denied the customary aid from the Radiophonic Workshop and accompanied his small-ensemble pieces with a synthesizer he had bought himself.7 Producer John Nathan-Turner’s wholesale redesign of Doctor Who in 1980 included a notable reorientation of the programme’s music: Simpson was no longer required as composer, and instead the music was produced in tandem with the sound effects for the programme by members of the BBC Radiophonic Workshop. This approach began with season eighteen and carried on until season twenty-two in 1984. One significant effect was to integrate music and sound effects in the programme, almost all of which were produced electronically, giving Doctor Who a remarkable and unique sound signature among television programmes. The integration of music and sound effects created a sense of holistic ’sound design’, where the soundtrack was conceived as a unity rather than elements that can be mixed in a complementary and conventionally hierarchized manner. Employees of the Radiophonic Workshop who produced a regular run of scores included Peter Howell, Paddy Kingsland and Roger Limb. Of these scores, one of the most interesting was Roger Limb’s music for the last outing of Peter Davison as the Doctor, ’The Caves of Androzani’ (1984). This demonstrated an interest in the specificity of sounds, inaugurating with some highpitched synthetic wails that sound electronic rather than attempting to sound like traditional instruments, while the story uses copious amounts of understated music for percussion alone. Limb’s music is premised upon building atmosphere and follows a through-composed pattern, matching the development of momentary dynamics of the drama. Perhaps a more characteristic score was Peter Howell’s for ’The Leisure Hive’ (1980), which marked itself as the clarion call of a new era not only through its achievement of striking sounds through the use of novel synthesizers such as the Yamaha CS80, but also through the use of a revamped title

190

K. J. DONNELLY

theme that replaced Delia Derbyshire’s musique concrete sounds of the original with the seemingly more ’modern’ sounds of integrated keyboard synthesizers. After having music produced ’in house’, Doctor Who then reverted to its earlier division of labour, where the Radiophonic Workshop created sound effects and a small roster of freelance composers made the music. This final period ran from 1986 to 1989, seasons twenty-three to twentysix, and included music by Keff McCulloch, Dominic Glynn and Mark Ayres. All the music was produced on electronic keyboards. McCulloch’s music was largely pop music inspired, although some of it had more of the character of muzak (McCulloch had been a pop musician in Pickettywitch and the Climax Blues Band). It tended not to follow the dynamics of screen action and in stories such as ’Paradise Towers’ (1987) habitually was controlled rather crudely by the sound editor turning its volume up and down. Dominic Glynn produced music that owed something to the developments in popular synthesizer music at the time, and has since gone on to produce electronic dance music. Mark Ayres, on the other hand, produced music that was more in keeping with the dominant traditions of music on Doctor Who, most notably Dudley Simpson’s scores during the 1970s. At times, Ayres’s music is vaguely reminiscent of a ’silent cinema piano’ accompaniment, providing continuous development and following the dynamics of the on-screen action intricately, moment by moment (the most impressive music along these lines probably being Ayres’s rich and varied score for ’The Greatest Show in the Galaxy’ [1988–9]). Such a division of Doctor Who’s run into four segments largely follows organizational lines. Yet there are other ways, other formations that might be used to divide up the programme’s history over its initial twenty-sixyear run on television. Types of sound, techniques and sonic hardware provide a very convincing set of divisions across this period. Even though traditional small musical ensembles were used from 1963 through to 1980, developments in musical technology feature strongly, arguably in a defining manner, in different periods of the series. The 1960s marks a primitive but innovative wielding of cutting-edge musical technology. The BBC Radiophonic Workshop was only able to produce effects using basic magnetic tape technology and rudimentary electronic devices, such as the ring modulator, which multiplied one signal by another. This was wielded effectively to create the distorted sound of the Dalek voices (and was later used to modify ’organic’ scores such as Carey Blyton’s for ’Death

10

DOCTOR WHO: BRITAIN’S IRCAM AND DARMSTADT

191

to the Daleks’ [1974] and Geoffrey Burgon’s for ’Terror of the Zygons’ [1975]). The early 1970s were characterized by the appearance of keyboardoperated modular synthesizers, most notably the BBC’s massive prototype EMS Synthi 100, which was nicknamed ’the Delaware’ due to its housing in Delaware Road, Maida Vale, in London. This was showcased on season eight in 1971, where wholly synthesizer scores were first used in a systematic manner (electronic scores had appeared in season four using the Radiophonic Workshop’s ’Multicolortone Organ’). The late 1970s saw the increasing proliferation of preset keyboard synthesizers. By the time that Doctor Who’s music was being produced solely by the BBC Radiophonic Workshop, a number of ’off-the-peg’ keyboard synthesizers were being used, such as the Oberheim OBX, the Roland SY2 and the Yamaha CS80.8 These were much easier and quicker to programme. In the early to mid-1980s, ’samplers’ became prominent. These were new digital synthesizers that recorded (’sampled’) and then treated sounds rather than building them up through subtractive synthesis (the use of oscillators and filters, as in older generations of hardware). Also in the early 1980s, there were frequency modulation synthesizers, where one sine wave is modulated by another (as in the Yamaha DX7) and by the mid-1980s, wavetable synthesizers which could mix waveforms. Sampling hardware proliferated in the wake of the appearance of the Fairlight CMI in 1981. The music for Doctor Who in the later 1980s often used integrated preset digital keyboard synthesizers, such as Keff McCulloch’s Emulator and Prophet 5, the former of which regularly made use of sampled orchestral stabs, sprinkled liberally throughout McCulloch’s scores. It is not too difficult, therefore, to follow technological developments through the sort of sounds evident in Doctor Who. Technological determinism may be unfashionable, but it proves an attractive explanation for the programme’s sonic development over time. It is certainly not controversial to suggest that hardware tends to set limitations and encourage certain procedures of construction. Doctor Who provided a showcase for cutting-edge sound technology and musicians who were given scope to try things out that went far beyond the bounds of possibility for the vast majority of television. In this sense, the programme provided a more immediate window for musical technology than pop music or art music. The aim of its music was to express ’the otherworldly’ and the principal way of achieving this was, and often still is, through the use of the most recent sonic technology.

192

K. J. DONNELLY

These fresh, new sounds signify ’the future’, or at least as near to the future as possible.

BBC Radiophonic Workshop The BBC Radiophonic Workshop was founded in 1958 and disbanded in 1998, under John Birt’s leadership of the BBC, after a few years of being run down. It produced all the sound effects (including voice effects) for the whole run of Doctor Who (1963–89) and produced music as well from 1980 until 1986, among its other duties. In addition to this, as I’ve already noted, the Workshop ’treated’ Dudley Simpson’s music, through adding and developing the sounds produced by an ensemble of conventional instruments. According to one of their number, Malcolm Clarke, ’We provide the basic scenery, the emotional climate, the temperature, the environment in which things are happening’.9 There are two useful sources of information about the BBC Radiophonic Workshop. The first is the book by its one-time Head, Desmond Briscoe, and the second is the BBC 4 documentary Alchemists of Sound. The former supplies copious information about equipment and facts about the organization, while the latter is more concerned with assessing the importance of the Radiophonic Workshop. While the documentary’s brief is to be welcomed, care should be taken in such reappraisals not to overemphasize certain aspects. The documentary tends to focus on the Workshop as an avant-garde entity connected to art music practice, and perhaps this historical revisionism reflects the BBC’s desire to ’trade up’ some of its more successful elements in an era of almost continuous questioning of the BBC’s public-funded status. Nevertheless, and despite its mass media context, the BBC Radiophonic Workshop should be seen as the British manifestation of an international drive of experimentalism that embraced IRCAM (Institut de recherche et coordination acoustique/musique) in Paris (which included Pierre Boulez), Darmstadt Ferienkurse summer schools in West Germany (including Karl-Heinz Stockhausen) and the Columbia-Princeton Electronic Music Center in the USA (including Vladimir Ussachevsky and Milton Babbitt). It is beyond doubt that the BBC Radiophonic Workshop took inspiration on the one hand from tapebased musique concrete inspired by French pioneer Pierre Schaeffer, and on the other from technological developments often tied to these experimental centres. Yet the BBC Radiophonic Workshop was very much an experiment within the mainstream. This makes it extraordinary

10

DOCTOR WHO: BRITAIN’S IRCAM AND DARMSTADT

193

not only by the standards of musical experimentation, but also in that it was a nearly unique insertion of the avant-garde into a highly popular context. Indeed, Doctor Who includes striking examples of extreme and challenging music consumed as a mass medium. In the sense of mainstream music, some of Doctor Who’s music was produced by distinct outsiders. A fine example is Workshop member Delia Derbyshire, officially one of the Workshop’s ’studio managers’, who had achieved her position through a technical channel rather than specifically as a musician. Similarly, maverick experimentalist Tristram Cary, who was never a part of the Workshop, came to provide music for Doctor Who on the back of his experiments in radio and electronics probably more than his background in scoring a handful of films, including Ealing Studios’ The Ladykillers (1955). He went on to found the pioneering electronic music studio at the Royal College of Music in London, despite having a career that lacked the sort of institutional support enjoyed by most respected composers and music academics. The sound effects of Doctor Who were always striking. These were credited as ’Special Sound’ on screen and the series used sound in an iconic way far more than other television programmes. One could argue that sound ’starred’ rather than simply being there to convince audiences of the veracity of screen representations. In addition to this, certain sounds were reused constantly and thus had more status than simply ’sound effects’. The relaunched 2005 series of Doctor Who has demonstrated great awareness and respect for these ’sonic stars’, with returning effects including the TARDIS materialization/dematerialization, the Autons’ handgun and the throbbing ambience created for the Dalek control room. So distinctive and foregrounded were Doctor Who’s sound effects that several recordings were available commercially as well as the programme’s music. A disc called Doctor Who Sound Effects was released in 1978 and included sounds such as ’Dalek hatching Tanks on Skaro’, ’Sutekh Time Tunnel’ and ’Styre’s Scouting Machine (Approach, Stop, Search, Depart)’, among others. More recently, the BBC released two CD recordings of many of these sound effects (re-released by Mute in 2005). Yet these were more than simply sounds, often functioning thematically or even emotionally—like music. For example, each episode of ’Fury from the Deep’ (1968) began with a heartbeat-like sound created by Brian Hodgson, which had been developed from a multi-track recording of a heel squeaking on linoleum (and was also used to represent the Nestene consciousness in ’Spearhead from Space’ [1970]).10

194

K. J. DONNELLY

From season eighteen (1980–1), all the music in Doctor Who was produced electronically by the BBC Radiophonic Workshop. However, season eight (1971) was an experiment where all the musical scores were realized electronically by the Workshop after having been written by Dudley Simpson. This phase included the synthesized supernatural wind sound that dominates the first episode of ’The Daemons’ (1971). Tristram Cary noted that he “did quite a lot of the sound effects too”,11 while Dudley Simpson commented on Cary’s electronic music: “you couldn’t tell the difference between the sound he made and the sound they made for effects”.12 The television work of the Radiophonic Workshop evolved from its origins in the manifestly non-visual tradition of radio rather than the dominant visual tradition of film. Yet, Doctor Who’s sound and music situation should be noted as ’sound design’; indeed, this innovation was the forerunner of the current sense of ’sound design’ in feature films, particularly the creative sound design of the 1990s. Rather than being premised upon noise-reduced multispeaker set-ups, the Radiophonic Workshop’s Doctor Who soundtrack in the early 1980s reflected an integrated and holistic approach. We might conceive of an aural version of the dominant regime of television drama, that of ’naturalism’, where a minimal visual language attempts to erase any sense of style and instead presents what an audience reads as a seamless window on the world.13 Such audio naturalism would probably be characterized by a strict divide of incidental music and diegetic sound effects, or at least a strong demarcation between the two, but this was not the case on Doctor Who.

The Space Between Functionalism and Experimentation While the BBC has been more than happy to emphasize its experimental music credentials in relation to the Radiophonic Workshop (see the aforementioned BBC’s ’Alchemists of Sound’ documentary), it should never be forgotten that that the musicians produced relatively cheap functional music. This was underlined by the fact that cutting the programme’s music budget was an easy and common production economy. In season six, for example, there were two stories that contained absolutely no music. From Doctor Who’s inception, the Radiophonic Workshop took up some of the slack, as they were not included as part of the programme’s music expenditure. Consequently, there was always something of a mix

10

DOCTOR WHO: BRITAIN’S IRCAM AND DARMSTADT

195

of sound effects and music. An example of this is Brian Hodgson’s piece called Thal Wind, which appeared in ’The Daleks’ (1963–4) as a supplement to Tristram Cary’s music. Nominally at least, this was a piece of ambient sound effect, although when heard contiguously with Cary’s austere avant-garde music, it sounds more conventionally musical. Later in the 1960s, Brian Hodgson’s ’sound effects’ increasingly included an aspect of incidental music, in some cases (such as ’The Krotons’ [1968– 69]) with no credit, while in others, Hodgson and the Radiophonic Workshop were given a full credit (’The Wheel in Space’ (1968]). Economy was, and still is, paramount in the vast majority of television productions. In the 1970s, Doctor Who, although having an original score for each episode, had them performed mostly by small ensembles (usually of less than ten musicians) and augmented by electronics from the BBC Radiophonic Workshop.14 Dudley Simpson asserted that “You can’t ask the costume department to use tissue paper. And you can’t ask the graphics department to use thin lines instead of thick ones … but you can ask the composer to write for two players instead of four”.15 Simpson had to make as much variation in sound as possible with a very limited palette, and regularly included an organ to give more body to the sound, a brass instrument or two to boost the middle register and percussion to add more colour.16 Yet music could still prove important, adding vitality and atmosphere where it was sometimes sorely required. Doctor Who has often had a somewhat undeserved reputation for a budget that fell short of the programme’s imagination. Here the music became an essential area of support, compensating for cheap sets, effects, monsters and saturation lighting. As an essential part of this process, music was thus able to attempt more in the way of overt experimentation than might have been expected. It had to do what the other elements in the programme could not, to make locations convincing and imagine the unimaginable for Doctor Who. In musical terms, Doctor Who produced some moments that should go down in television history as significant. Perhaps the first of note was during ’The Web Planet’ (1965), where recordings of sound-music derived from the Baschet brothers’ sound sculptures was used. These French artists (with collaborator Jacques Lasry) produced esoteric sounds that were at home in art galleries but had never been heard on primetime British television before. The same recordings were also used later on ’Galaxy Four’ (1965). In fact, the Baschets had been approached in

196

K. J. DONNELLY

1963 to provide Doctor Who’s theme music but had proved too expensive (Fig. 10.1).17 The close relation between individual composer/performer and the music can allow a certain scope. A most unique score for a television programme was composed and performed by Malcolm Clarke of the BBC Radiophonic Workshop for the Doctor Who story ’The Sea Devils’ (1972). Clarke’s music is startling in its range of obtrusive electronic timbres and relative melodic paucity. In fact, this programme manifests an interesting point in time when mainstream television (momentarily) embraced avant-garde music. This electronic music was a featured aspect of the programme rather than simply being an unheard musical backdrop. The shocking new sounds, conjured from the BBC’s brand-new prototype EMS synthesizer, mixed music and sound effects, and presented uncomfortable sounds to a substantial early evening audience on Saturdays in a way not duplicated in Britain before or since. The score was a last-minute replacement, as another Radiophonic Workshop member,

Fig. 10.1 ‘The Sea Devils’

10

DOCTOR WHO: BRITAIN’S IRCAM AND DARMSTADT

197

John Baker, was unable to write the score commissioned.18 The Television Companion, an authoritative source on the serial, notes that: Clarke … came up with a score that can best be described as experimental. It is in effect a collection of atonal sounds that punctuate action, in some parts melodic but in others simply background noise (for example a low bubbling for the sequences set in the submarine). Opinions differ as to the merits of this approach, but one thing that is certain is that no-one who watched ’The Sea Devils’ can possibly fail to miss what is arguably its most striking aspect.19

The score had been commissioned from the BBC Radiophonic Workshop to save money, rather than hire Simpson and his musicians.20 The malignant ’sea devils’ of the story’s title are accompanied by a theme derived from the Dies Irae, the medieval mass of the dead. This is potentially one of the few recognizable melodic aspects of Clarke’s score that makes external reference. The complexity of this sort of television underscore allows for musical references and the opportunity for the music to signify on different levels: one for the viewer who is only a casual listener to the score and one for the more attentively listening viewer. What is immediately striking is how timbre—the qualities of the sounds themselves—is more important than melody, harmony or rhythm. What is also apparent is that there is difficulty in differentiating what might be non-diegetic music from what appears to be diegetic sound effect. The music and sound form is derived largely from technological capabilities. Malcolm Clarke was literally ’let loose’ on the brand-new prototype synthesizer, ’the Delaware’, which Radiophonic Workshop Head Desmond Briscoe noted was ’probably the most advanced synthesizer in the world’.21 There are isolated precedents to Doctor Who’s sound on film, most notably Louis and Bebe Barron’s ’electronic tonalities’ for Forbidden Planet (1956), and also Jerry Goldsmith’s electronic music for the Africa section of The Illustrated Man (1969). But Doctor Who’s boldness in terms of incidental music’s sound was unique to television during the 1960s and early 1970s. Science fiction shows such as The Twilight Zone (1959–65), The Outer Limits (1963–5), Star Trek (1966–9) and Raumpatrouille (1966) all used much more traditional music. The most common process for creating television underscores is through the use of smaller ensembles, solo instruments and composers

198

K. J. DONNELLY

recording their own music on electronic keyboards—good examples being Doctor Who and The X-Files (1993–2002). There is thus a tendency towards sparer textures than in film music. This is owing first to the cost restrictions on television production in relation to mainstream film production, and secondly, to television’s more intimate character. One year after ’The Sea Devils’, Pink Floyd released their best-selling album Dark Side of the Moon (1973). This not only seemed to evince a form of ’sound design’, mixing songs and sound effects, but also showcased another EMS synthesizer, the VCS3. It is interesting to note that both the BBC and Pink Floyd endorsed the use of synthesizers from a British company, Peter Zinovieff’s EMS, rather than use the more internationally prominent Moog synthesizer. EMS’s instruments potentially had more capabilities through facilities for careful calibration, although this required quite leisurely programming time and EMS synthesizers failed in the long run to match the popularity of more immediate Moogs. A sense of modernity is often tied to technology, and this is truer for music than for many other areas of culture. Technology offers the promise of the ’future’, with its developments always incomplete and tied to a discourse of progress, no matter how covertly, but perhaps most obviously through the cycle of (hardware) redundancy. As I pointed out earlier, new, unfamiliar sounds appear to be forward looking, modern through being ’futuristic’, and Doctor Who made much of its singular sound profile. This mixture of music and sound effects amounted to a musically inspired sound design. In the 1980s, it was even the beneficiary of early broadcasts in NICAM stereo, allowing for a spatialization of sound. Interestingly, in the late 1970s the Musicians’ Union in the UK waged a campaign against the use of synthesizers, which it characterized as doing musicians out of a job. On the one hand, this appealed to the possibility of synthesizers to replicate the sounds of other instruments, but on the other it played to the suggestion that such things were ’devices’ manipulated by ’operators’ rather than instruments used to make music as such. The Radiophonic Workshop, although employing a number of musicians, had its origin in studio technology and management rather than the musical areas of the BBC.

10

DOCTOR WHO: BRITAIN’S IRCAM AND DARMSTADT

199

Conclusion On rare occasions, you can see and hear what Doctor Who would have been like with more conventional music. An isolated example is in the episodes scored by Carey Blyton, most notably in ’Death to the Daleks’ (1974), where the Dalek appearances are converted into a comic turn by jocular saxophone music. In fact, a whole dimension of the programme is lost through its lack of musical ’alienness’, despite some attempts to treat Blyton’s music with minimal electronics. Doctor Who more characteristically represented the future or the otherworldly through cutting-edge sound technology and techniques that by their rarity would sound unfamiliar to the vast majority of the audience. Until the programme’s return from hiatus in 2005, that is, where, as Matt Hills notes, the more conventional dramatic score aimed at ‘mainstreaming’ the programme through the sort of music that was more similar to contemporaneous American science fiction television shows.22 The music for the earlier years of Doctor Who retains a certain cultural prominence. While it may not be the subject of detailed analyses in university music departments, it has a place in popular history through an increased commercial availability in recent years. Many of the BBC’s DVD releases (for the 1980s programmes) have isolated scores, allowing the music to be listened to along with the images but without the hindrance of dialogue or sound effects. This experience underlines how far sound effects and music interacted. In its pioneering years, Doctor Who married art music experimentation and the exigencies of cheap television drama. Even when the music had a less than experimental character, it used the sort of synthetic sounds that were only appearing furtively in the margins in television at the time. Significantly, Doctor Who blurred the distinction between music and sound effects, reconceiving screen sound as a unity that included both ’musical’ sound effects and music that made some sound effects. In this way, it prefigured the current process of sound design in feature films. This was partially the result of Doctor Who providing a space that allowed the extremes of experimental music, previously only open to highbrow musical elites, into the homes of Britain’s wider populace. Indeed, activities centred around the BBC Radiophonic workshop were arguably equivalent to the celebrated musical experimentation centres in other countries, such as IRCAM, Darmstadt and Columbia-Princeton. Yet the context for experimentation was radically different. When introduced to Radiophonic Workshop staff at the

200

K. J. DONNELLY

Centenary Conversazione of the Institution of Electrical Engineers at the Royal Festival Hall, Queen Elizabeth II questioned the status of the music.23 Her response matched that of the population, that the work of the Radiophonic Workshop was not necessarily recognizable as music but was distinctly recognizable as Doctor Who.

Notes 1. David J.Howe, Mark Stammers and Stephen James Walker, Doctor Who: The Seventies. London: Virgin, 1994), p.54. 2. There was a desire to complete the soundtracks with images, with some—sometimes unofficial—versions adding still images (‘Telesnaps’) and in later years the BBC commissioning animators to put images to these sound recordings. 3. The Watcher, “It’s the End, but …” Doctor Who Magazine, No. 292, 2000, p.50. 4. Doctor Who has and had a strong fan culture, even during its hiatus period. The music was clearly considered important and valued by the audience as nearly 20 commercial recordings were made available of music and sound effects before the Millennium. 5. K.J.Donnelly, “Tracking British Television: Pop Music as Stock Soundtrack to the Small Screen”. Popular Music, vol. 21, no. 3, 2002, pp.333–335. 6. Dallas Jones and Gary Leigh, “The Musicians: Dudley Simpson” [interview] in Gary Leigh, ed., The DWB Interview File: The Best of the First 100 Issues, no. 2. (Brighton: DreamWatch, 1994), p.110. 7. Benjamin Cook and Tom Herron, “Bohemian Rhapsodies” [interview with Dudley Simpson] in Doctor Who Magazine, no. 330, 2003, p.29. 8. Mark Ayres, sleeve notes to Doctor Who at the BBC Radiophonic Workshop Volume 4: Meglos and Full Circle. Music by Peter Howell and Paddy Kingsland (BBC WMSF6053-2, 2002). 9. mb2 l. “BBC Radiophonic Workshop” at www.mb2l.co.uk/ether. net/radiophonics/quotes.shtml [accessed May 2004]. 10. Andrew Pixley, “The DWM Archive: Fury From the Deep” in Doctor Who Magazine, no.277, 1999, p.18. 11. Tristram Cary, “The Musicians: Tristram Cary” in Gary Leigh, ed., The DWB Interview File: The Best of the First 100 Issues, no. 2. (Brighton: DreamWatch, 1994 [orig.1988]), p.18.

10

DOCTOR WHO: BRITAIN’S IRCAM AND DARMSTADT

201

12. Jones and Leigh, op.cit., p.109. 13. Lez Cooke, British Television Drama: A History (London: BFI. 2003), p.64. 14. Jones and Leigh, op.cit., p.112. 15. Cook and Herron, p.28. 16. Ibid., p.28. 17. Andrew Pixley, “Do You Want to Know a Secret?” in Doctor Who Magazine Special Edition: The Complete First Doctor, 2004, p.14. 18. Howe, Stammers and Walker, p.51. 19. Howe and Walker, op.cit., p.306. 20. Andrew Pixley, “Archive Feature: Series LLL, The Sea Devils” in Doctor Who Magazine, no. 192, 1992, p.29. 21. Desmond Briscoe and Roy Curtis-Bramwell, The BBC Radiophonic Workshop—The First 25 Years (London: BBC, 1983), p.129. 22. Matt Hills, Triumph of A Time Lord: Regenerating Doctor Who in the Twenty First Century (London: I.B.Tauris, 2010), p.179. 23. John Tulloch and Manuel Alvarado, Doctor Who: The Unfolding Text (Basingstoke: Macmillan, 1983), p.13.

CHAPTER 11

Experimental Music Video: “Why Do I need TV When I’ve got T-Rex?”

“Why Do I need TV when I’ve got T-Rex?” asked Mott the Hoople and David Bowie’s song All the Young Dudes (1972), in a seeming dismissal of the mundanity of television in the face of the novel excitement of electrified popular music. Yet television was to form a highly creative alliance with music, although the music video as an aspect of experimental or avant-garde television has received surprisingly little attention in the frequent and wide-ranging discussions on the topic. This is particularly surprising since in the last decades of the twentieth century many avantgarde techniques became evident (and some filmmakers worked) in music video, profoundly altering the way that music appeared on television. Considerations of television still suffer from ocularcentric assumptions that prioritize the image despite the fact that television has always been an audiovisual medium, often having a close relationship with its cousin, the radio. However, until the advent of NICAM (near instant companded audio multiplex) stereo broadcasts in the late 1980s, television in Britain suffered from poor sound quality that hindered the development of its sonic aspect. Furthermore, television had only embraced these new techniques around its edges, often aiming for audiovisual economy and dominated by convention, whether the talking head of news presentation or the naturalism of television drama.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025 K.J. Donnelly, Hearing Eyes, Seeing Ears, Palgrave Studies in Audio-Visual Culture, https://doi.org/10.1007/978-3-031-73913-2_11

203

204

K. J. DONNELLY

Music television is a broad area encompassing not only programmes devoted to popular music but also the ways in which music is presented on television. These are not the same, as popular music is often presented on programmes not specifically devoted to music, such as talk shows or children’s programmes. This chapter focuses on the presentation of popular music on television, specifically the pop promo or music video, rather than the dedicated music television programme. It thus focuses on experimentation with a ‘televisual unit’, whose migratory malleability has led to the development of specialist channels (MTV being the first), appearance on VHS, DVD and Blu Ray compilations, as well as increasingly important ‘content’ on the internet and mobile technologies. In this sense, the pop promo is a particularly important component within a digitized media environment. Yet, if the discussion here is not necessarily about music television, many of the videos it discusses appeared on a range of music television programmes on British television: from the populist Top of the Pops (BBC1, 1964–2006) to the more ‘serious’ The Old Grey Whistle Test (BBC2, 1971–87); from the mixed appeal (and video-only format) of The Chart Show (Channel 4/ITV, 1986–98) to the more specialist, lo-fi aesthetic of Snub TV (BBC2, 1988–90). This chapter attempts to trace the development of the pop promo (which developed out of music television) and focus on some of its more experimental moments. In particular, it focuses on its links with the avant-garde and the implications of translating avant-gardist strategies into such a format. By and large, the historical avant-garde was primarily concerned with the development of new thought, and new political and social structures through innovative cultural forms and style. With the decline of the political left in the last couple of decades of the twentieth century, this political brief gave way, in much audiovisual work, to new techniques that were developed as a means in themselves rather than as a means to an end. In the 1980s, as significant space opened up for pop songs, avant-garde practice emerged on television as their visual accompaniment. The immediate context, however, was not welcoming. A fear of alienating audiences led to an audiovisual conservatism, rather than a sense of technical development. Pop videos were not only aimed at a youth audience that was already (in theory at least) alienated from large areas of television output, but also needed to grab attention quickly in a crowded marketplace, while also expressing something of the ‘individuality’ of the musical artist in question. Once MTV had become a dedicated channel in 1981 (1987 in

11

EXPERIMENTAL MUSIC VIDEO: “WHY DO I NEED TV WHEN …

205

Europe), and needed promos to fill its homogeneous space, a new opportunity for audiovisual culture opened up and, indeed, became the ‘cutting edge’ of audiovisual culture at the time. Since then, ‘music television’ has expanded into a galaxy of dedicated channels. It should be noted that pop video and music television are not synonymous and that there is a fringe of promos that rarely or never appear on broadcast television. As popular music is a multi-million dollar industry, it is perhaps no surprise to find that it is one of the most conservative areas of culture, hostile to the presence of the avant-garde in its domain. Yet since the advent of the counterculture in the 1960s, it has developed its own ‘avantgarde’ fringe while otherwise embodying Adorno and Horkheimer’s concept of a standardized culture industry.1 In my discussion here, the term ‘avant-garde’ is used to mean a certain approach that foregrounds cutting-edge techniques in relation to both the image and sound. Music video has a heterogeneous and inconsistent cultural presence: it belongs to the mainstream and also mixes the mainstream with more avant-garde tendencies. It is certainly not hard to find promos whose banality perfectly reflects the lack of adventure in mainstream pop music. On the other hand, there are promos that transcend a direct visual translation of the song, integrating a variety of visual devices that create an extraordinary accompaniment to the music. Just as the advent of counterculture generated a cultural underground in the late 1960s, pop music itself has had a persistent ‘underground’, a shifting marginal zone of music that regularly thought of itself as something more than simply a sonic product for consumption. It is, unsurprisingly, in this sub-generic area of popular music that the avant-garde tendency in promos can predominantly be found. The avant-garde impetus in promos manifests itself as a self-conscious parading of audiovisual technique, in a manner that traditionally has been associated with the film-making avant-garde. However, the techniques lose their initial political and cultural context and in most cases become merely an attraction in themselves, simply allowing for the differentiation of products in a crowded and competitive market. In order to identify avant-garde tendencies in promos, we should not only think of a promo as simply an advertisement for a commercial recording, but also as an object in itself, with its own integrity.

206

K. J. DONNELLY

Promo History The music video developed from hybrid and heterogeneous roots: some within television itself, some developing out of the needs of the music industry. In the first instance, British television tended to include only the most mainstream popular music and British pop shows on television have by and large been an outgrowth of the traditional popular music entertainment format, such as Top of the Pops and even more up to date shows in the 1970s such as Lift Off With Ayshea (Granada, 1969–74) and Marc (A vehicle for Marc Bolan) (Granada, 1977). One of the first British music shows to vary this blueprint was The Old Grey Whistle Test (BBC2, 1971–87). It was self-consciously serious and, although primarily based on live performances by musicians in the studio, album tracks were also played in the programme. These tracks posed a problem for the producers: how should they accompany music, with no access to the group and little budget? They solved the problem through the use of old black and white animated films, such as Felix the Cat cartoons or 3D animation by Ladislaw Starewicz, which were run simultaneously with the song, irrespective of notable synch points or dynamic matching. While there was a strange discontinuity between modern sound and antique images, the aesthetic effect was often quite polished despite the random basis of the tie between music and visuals, and illustrated the malleability of image and sound, and the relative autonomy, in certain circumstances, of one from the other. The idea of the pop promo had its origins in the short films made by television programmes to accompany pre-recorded songs. This strategy had advantages for both sides: the group need not be present during the programme, nor, indeed, be paid as much as they would for a live performance. Quickly, the musicians themselves and other organizations realized the worth of these films to promote a specific release. One of the earliest was by The Kinks for their song Dead End Street in 1966, while over the next couple of years, Pink Floyd had promotional films made for them by the Central Office of Information and Pathé News. The Beatles made their own films in an attempt to fill a punishing schedule of television appearances, and, from 1966, these became the only way to see the group in action.2 The previous year, they made a number of promos with Joe McGrath for songs including Daytripper, We Can Work It Out and I Feel Fine. In 1967, they made promos with Swedish director Peter Goldmann, which ranged from simple representations of live performances to

11

EXPERIMENTAL MUSIC VIDEO: “WHY DO I NEED TV WHEN …

207

startling stylistic devices such as images running backwards, most notably Paul McCartney jumping ‘up to’ a tree in Strawberry Fields Forever.3 The Beatles’ Magical Mystery Tour (1967) should be seen as an experimental (and extended) pop promo for television, rather than, misleadingly, as a musical film. Its production was partly inspired by Paul McCartney’s interest in experimental film and the cinéma-vérité/ direct cinema-inspired sequences in Magical Mystery Tour which were similar to the style being used at the time by American filmmakers such as the Maysles brothers.4 It certainly was an uncharacteristically bold move for the BBC. It got a negative reception and, despite its colourful nature, was first broadcast by the BBC in black and white (26 December 1967).5 Magical Mystery Tour was certainly an expensive promo, made with big money and television station backing. Later promo developments were made on a smaller scale and often facilitated through technological developments. Paul McCartney went on to make a number of short ‘avant-garde films’,6 while John Lennon made films with Yoko Ono that still have an avant-garde status more than thirty years later. Magical Mystery Tour should be seen in the context of late 1960s arts crossfertilization, which inspired the founding of The Beatles’ Apple films, and allowed Yellow Submarine (1968) to find a popular context for radical visual design. During the 1970s, rock music became more ‘serious’ and certainly more serious about itself. This was reflected in television programmes like The Old Grey Whistle Test and rock opera films like Tommy (1975) and Lisztomania (1975)— both directed by Ken Russell. Rock ‘dinosaurs’ sought to aggrandize themselves through the cinema screen’s epic capabilities rather than the limited vision and sound available on the television set. However, this is not to suggest that television programmes embraced more serious ‘rock’ at the expense of more easily consumed ‘pop’. Rock programmes were often premised upon group performances, while the use of film clips allowed more leeway for avant-gardism on screen, yet the vast majority of programmes were visually as banal as their songs were prosaic. In 1975, Queen’s promo for Bohemian Rhapsody was an important element in the massive success of the single. Interestingly, the success of the video arguably set the group on a course away from being a progressive rock band to being a far more commercially oriented pop band in the next decade.

208

K. J. DONNELLY

Progressive rock self-indulgence was shunted aside by the crudity, directness and lack of pretension in the wave of punk rock‘ that overtook Britain in the late 1970s and found some outlet on television. After the break-up of the Sex Pistols and the dissipation of punk, John Lydon (the erstwhile Johnny Rotten) formed Public Image Limited (PiL). They led the way in terms of product format. Their second album Metal Box (1979) was an aluminium film can containing three 12-inch 45 RPM discs, which had a better audio response than a normal 33 1/3 record. This group was at the forefront of what Greil Marcus called ‘Britain’s postpunk pop avantgarde’.7 In 1981, they rhetorically declared that they were no longer a musical group in the traditional sense, but a ‘communication corporation’ who would be involved in video making and film soundtracks.8 In fact, they did little in this direction, although their artistic aspirations led to a bizarre riot-inducing performance behind a screen at a New York nightclub in 1981. The group projected video onto a screen while hiding themselves behind it, clearly upsetting an audience happier with a conventional concert performance. PiL also made a notable appearance on The Old Grey Whistle Test in February of 1980. Their performance of Careering, which was energetic but showcased involved tuneless singing of arcane and ambiguous lyrics, a lack of conventional song form, highly repetitive bass and drums backing and seemingly random guitar and synthesizer noises led presenter Annie Nightingale to conclude their appearance by noting that it was one of the most remarkable performances ever seen on the programme. Despite PiL’s intentions, they never realized the audiovisual potential of their rhetoric, although in the late 1970s and early 1980s, they certainly brought some avant-garde musical techniques into the popular domain, including the performance of Death Disco to a bemused audience on the extremely popular British weekly television pop music chart show Top of the Pops on 12 July 1979. The predominantly teenage audience was no doubt shocked by the surface sheen of mainstream pop conventionality being ruptured not only by PiL’s punk appearance, but also by the tuneless song that included an out-of-tune guitar quotation from Tchaikovsky’s Swan Lake. Remaining at the cutting edge, in 1986 they released a promo called Video as an accompaniment to a single called Single and an LP called Album.9 The first dedicated outlet for music videos was MTV, which started broadcasting in the USA in 1981 and in Europe six years later. In its early incarnation, MTV was quite experimental, allowing a flow of videos

11

EXPERIMENTAL MUSIC VIDEO: “WHY DO I NEED TV WHEN …

209

without any mediating presenter, although it later gave way to more conventional programmes and scheduling. However, promos at this time might be characterized as what Raymond Williams called an ‘emergent form’,10 preparing to become dominant in the near future. When MTV first started it had trouble filling its schedules as there were barely enough promos in existence to allow for significant variation. They developed in response to this already existing and needy audiovisual broadcasting space, which allowed an incredible scope and leeway for operation in terms of style. The early years of music video were plagued by a crude ‘literalism’ as the image attempted as far as possible to double the words of the song and, as most promos are conceived as illustrative of the songs, this impetus has never gone away, although it has become far less pronounced. There was also at this time an increasing use of Quantel Paintbox visual effects, which manipulated images and added graphics using early digital techniques.11 Music videos came to be the place to see such cuttingedge technological capabilities, although they quickly became overused. Baltimora’s Tarzan Boy (1985) illustrated its affordances well, which was created with multiple images and blue backgrounds. It resembled certain avant-garde films, such as the abstract animated films by Len Lye and Norman McLaren that used saturated colour to obscure rather than enhance the image. Tarzan Boy also used the principle of simultaneity, rendering the singer one of a number of competing elements and undermining any sense that the image simply expressed the production of the music. The aesthetic possibilities of Paintbox were showcased in a BBC television series called Painting in Light in 1986, which allowed artists such as Richard Hamilton and David Hockney to demonstrate its capabilities. The irony was that its capabilities were already more than evident in mainstream music video.

Promos and Their Makers With respect to promos, the avant-garde impetus manifests itself as a foregrounding of audiovisual technique. This is more common than might be expected, due to the desire for differentiation in a competitive marketplace. Thus, techniques become an attraction in themselves rather than a subversion of the norms of mainstream audiovisual culture or a means to a conceptual end. According to some, the use of specific and cutting-edge

210

K. J. DONNELLY

film techniques for their own sakes marks a travesty of avant-garde sentiments. For others, it underlines the collapse of avant-garde culture into a postmodern play of aesthetics that have lost their original meanings and contexts. Promos quickly began to develop a strong stylistic tendency to use montage, involving fast cutting, jump cuts and collapsing spatial cohesion with discontinuities along the lines of other audiovisual traditions, such as Experimental music video and television action movies or avant-garde film. They also adopted dramatic camera movement, probably derived from the film musical, that became the ‘pop promo style’ of mainstream Hollywood films in the 1980s. There was also a tendency to lapse into a series of visual clichés, such as slow motion shots of flying doves, images using saturated colour and shot with special lenses or fairly crude effects from early digital programmes (such as Quantel paintbox) that could change image and graphics effects on videotape. A ‘pop surrealism’ proliferated: a radical juxtaposition of seemingly disparate and illogical images. Influences came not only from surrealist art but from abstract filmmakers such as McLaren and Lye, in the riot of kinesis, colour and movement reflecting the dynamics of the music that the visuals complemented. The influence of the avant-garde can be divined in a persistent rejection and negation of the characteristic transparency of mainstream television (and film), premised upon an illusionistic world on screen without a foregrounded visual style. However, similar stylistic characteristics were noticeably present in television commercials, arguably music video’s close cousin. Examples of ‘pop surrealism’ include David Bowie’s Ashes to Ashes (1980, directed by David Mallet), which reached number one in the British charts, and included strident colour adapted and solarised with the sort of technology soon to be agglomerated and sold commercially as Quantel Paintbox. It embraced bizarre and obscure imagery, including Bowie and others in white pierrot costumes walking in front of a mechanical digger. Similarly, Ultravox’s Vienna (1981), which was directed by Australian Russell Mulcahy, included footage of tarantulas walking on people’s faces and children playing as a string quartet, mixed in with nostalgic images of neo-classical buildings in Vienna. Both examples illustrate the surreal imagery evident in contemporary mainstream videos, in heavy rotation on prime-time shows, such as Top of the Pops, that featured top ten music. Two of the most influential figures in the production of music videos in Britain in the 1980s and 1990s were Kevin Godley and Lol Creme.

11

EXPERIMENTAL MUSIC VIDEO: “WHY DO I NEED TV WHEN …

211

Both had been very successful as part of the top international pop group 10cc in the 1970s. After they had developed a device called the ‘Gizmo’ with which to bow an electric guitar and make interesting sounds, they took a break from 10cc to produce a demonstration record. The record took on a life of its own and became the sprawling concept triple album Consequences (1978). While their work with 10cc had exhibited as much irony and pop literacy as might be possible in a top chart act, this concept album showed an altogether more experimental side to the duo. Consequences was notoriously unsuccessful and for years was spoken of as a limited case of artist and industry excess. Godley and Creme persisted with their musical career as a duo and moved rapidly into producing their own promo videos for singles. The first to bring them to prominence as video makers was for the single Cry (1985), where a succession of closeups of faces transformed into one another while lip-synched to the words of the song. This was startling to audiences at the time, as these were precise fades in and out to give smooth transitions, predating the advent of digital morphing, a ‘trick’ that became overused in the 1990s. Cry showed how close the aesthetics of promos could be to that of small-scale experimental films. The song itself was very conventional but the video elevated it to another plane, imbuing a much stronger sense of the song words through the series of facial expressions across a wide range of male and female archetypes. As music video makers they worked in the mainstream but also with more challenging material, for instance, U2’s Even Better than the Real Thing (1992). The video had a prominent place in U2’s rebranding with its postmodern concern with information overload and the contradictions of politics and pop music. The spine of the video is imagery of U2 performing the song with the camera mounted on a large wheel that allows it to travel in a circuit above the group and below their feet on a glass floor, thus avoiding a succession of angles on the action. The effect is highly disorientating, particularly with fast cutting, flash frames and the interpolation of footage of U2 tribute band Doppelganger, who, in the mêlée of the fast-paced action and images, appear to be the real U2. For the guitar solo section of the song, the promo changes mode to show a rapid selection of TV news images in smaller frames within the screen. In stylistic terms, Even Better than the Real Thing is a remarkable visual accompaniment to U2’s song, utilizing a vast range of stylistic techniques in pursuit of informational overload.

212

K. J. DONNELLY

Blondie’s Eat to the Beat (1979) was publicized as the ‘first video album’, released as an object with images as well as a music disc. In the next decade, the British group The The released the album Infected (1986), with purpose made videos, as a ‘video album’, which was broadcast in its entirety on Channel 4, although in a late-night format and at a time when the channel was self-consciously offering more minority-interest programming. Apparently an aesthetic unity, it was in fact a succession of videos directed by different people, including Mark Romanek and Tim Pope, who regularly worked with The Cure. Three songs, Mercy Beat, Heartland and Infected were directed by Peter Christopherson, an important figure in the British avant-garde music scene as a member first of Throbbing Gristle and then Coil. He had worked with Derek Jarman on a number of occasions including the collaboration with Coil on the music for the television film The Angelic Conversation (1985), while Jarman had also filmed Throbbing Gristle in T.G.: Psychic Rally in Heaven (1981). Christopherson made some avant-garde-inspired videos for American group Nine Inch Nails, whose collection he directed, and included Edison’s short reality film Electrocution of an Elephant (1903) (Fig. 11.11. 11.1).12 Arguably, the most interesting music videos, with the most impetus from the avant-garde, were in ‘independent’ music that was hardly related

Fig. 11.1 Nine Inch Nails’ Closer

11

EXPERIMENTAL MUSIC VIDEO: “WHY DO I NEED TV WHEN …

213

to ‘top 10’ pop music culture. Some examples illustrate the direct influence of the cinema’s avant-garde and artistic wing. Clock DVA produced a video for their recording The Hacker (1992), which is premised upon rapid cutting, to match the rapid continuous quaver rhythm of the music. This yields a highly stylized effect that, on the one hand, removes the emphasis from the fairly banal black and white images while, on the other, renders the whole into something resembling a piece of ‘Op Art’. Indeed, this video is very difficult to watch, and presumably impossible to watch for those suffering from epilepsy. The promo for Propaganda’s Dr Mabuse (1984) was directed by Anton Corbijn using some stylistic tropes taken from European art cinema. Shot in black and white, featuring gothic castles and monks with dramatic shadows crossing the screen, it copied the visual style and iconography of German Expressionism. And, in clear reference to the effect performed by Jean Marais in Jean Cocteau’s Orpheé (1949), singer Claudia Bruecken goes into and through a mirror. Its title and theme were derived from the series of films directed by Fritz Lang. Corbijn was a stills photographer who specialized in portraits of pop personalities, and the video exhibits a strong aesthetic sense, in addition to the references to film history. The House of Love’s Destroy the Heart (1987) employed a camera panning continuously on its axis, as had Laura Mulvey and Peter Wollen’s avant-garde film Riddles of the Sphinx (1977). For Mulvey and Wollen, this technique sidestepped the limits of conventional space and time creation in the cinema. For the House of Love, this technique undermined the format’s dominant sense of frontal onstage performance and, further, undermined the idea of choreography for the camera, as the group was shown in undramatic poses rather than mugging for the camera as was and is the norm of music video.

Art and the Promo During the 1980s, there was a significant crossover between British art cinema and music videos. One of the most significant British art film directors, Derek Jarman, had already used popular music in his feature films. The Tempest (1979) concluded with a song sequence of Stormy Weather that parodied traditional musicals. Jubilee (1977) had exploited developments in punk rock and used several punk musicians in acting roles as well as showing music being performed, most notably Siouxsie and the Banshees playing in a studio (although it was actually Siouxsie singing with Adam and the Ants—minus Adam). Jarman went on to make

214

K. J. DONNELLY

a promo for the Smiths, called The Queen is Dead (1986) after their album, accompanying a few songs with blown up 8mm footage. This technique, that Jarman had been working with for a number of years, made for vague images but an unusual texture. Although the promo was clearly not made to be played in heavy rotation on music television as the image quality worked against the promo’s commercial potential, some of it was, bizarrely screened on the most mainstream (and most conservative) of British pop programmes, Top of the Pops. Jarman also made the video for The Pet Shop Boys’ It’s A Sin (1987), which on the surface looked like it used the same set and costumes as his feature film Caravaggio (1986). He also made the promos for that group’s Violence (1986) and Rent (1987), and Suede’s The Next Life (1993). One of Jarman’s assistants on his feature films was John Maybury, who, particularly in the wake of Jarman’s death, was expected by many to take on his mantle as the leading progenitor of British art cinema. Instead, he made a career in the 1980s as a promo/music video maker, having worked with Jarman and the Pet Shop Boys (on It’s a Sin). Maybury went on to make many promos, including Each and Everyone (1984), Native Land, Mine (1984) and These Early Days (1984) for Everything but the Girl; Dreams (1992) for The Cranberries; Buffalo Stance (1989), Kisses on the Wind (1989) and Money Love (1992) for Neneh Cherry, and most notably, Nothing Compares 2 U for Sinéad O’Connor (1990). This last promo doubtless played an important part in the song’s success on both sides of the Atlantic and earned popular acclaim for Maybury. He was still a teenager when he worked on Jarman’s Jubilee, later working on Jarman’s The Last of England (1987) and War Requiem (1988), where he directed a sequence. Along with Cerith Wynn Evans, Maybury was the leading light of the ‘New Romantic’ wave in British art and filmmaking, which was highly exotic and inspired by filmmakers such as Jean Cocteau and Kenneth Anger. This movement made an impact in 1981 at the exhibition ‘A Certain Sensibility’ at the Institute for Contemporary Art in London. Maybury often used super 8 film and video for his short films, which include Is Like a Melody (1981) and Pagan Idolatory (1984). He later went on to make feature films and, despite the amount of money available to promo makers, Maybury appears now to have forsaken that world for the more art cinema-based career that had looked likely in the early 1980s. At the same time, another of the New Romantics went into making music videos: John Scarlett-Davies also worked with Jarman and Scratch Video (an influential political video collage group) in the 1980s.

11

EXPERIMENTAL MUSIC VIDEO: “WHY DO I NEED TV WHEN …

215

His music video work was certainly more mainstream and included Wood Beez (Pray like Aretha Franklin), Absolute and The Word Girl for Scritti Politti, Who Needs Love (Like That) and Heavenly Action for Erasure and Good Morning Britain for Aztec Camera. American-born twin brothers Stephen and Timothy Quay, often known as ‘the Brothers Quay’, became notable British-based avant-garde filmmakers in the late 1970s and 1980s, making television advertisements and music videos as well as more esoteric, personal films. They made part of Peter Gabriel’s acclaimed video for Sledgehammer (1986), other sections being Experimental music video and television made by Stephen R. Johnson, and Nick Park, who later went on to make animated films such as Creature Comforts (1989) and the Wallace and Gromit series. The Quay brothers also made music videos for His Name is Alive (Are We Still Married? [1991] and Can’t Go Wrong Without You [1993]) and Sparklehorse (Dog Door/Heloise [2001]). In both cases, their work dovetailed with less-mainstream pop music and they produced promos that were not out of keeping with the highly singular style of their other films. The highest profile music video collaboration between the art world and the promo world is, perhaps, Blur’s Country House (1995) directed by fine artist Damien Hirst. One of the so-called ‘Young British Artists’ that came to national notoriety in the 1990s, Hirst also belonged to the conceptual pop group Fat Les with comedian Keith Allen and Blur’s bass guitarist Alex James. Another artistic collaboration was director Chris Cunningham’s startling promos for Aphex Twin, Come to Daddy (1997) and Windowlicker (1999). Both used CGI technology to be highly dramatic, and in the case of the second promo comic, effect. Along with other music video directors Michel Gondry and Spike Jonze, Cunningham is now seen as a ‘respectable’ filmmaker, having released a DVD that brings together his promos, packaged as short films rather than ‘merely’ as music videos (as The Work of Director Chris Cunningham). Indeed, these releases mark a clear recognition of the art of some contemporary promos. U2’s succession of promos demonstrates how far some mainstream rock groups have endeavoured to connect with more esoteric film-making (and other) practices. In addition to Godley, a succession of highly interesting filmmakers made promos for U2 and have stayed on the cutting edge of audiovisual culture. These include Wim Wenders, Anton Corbijn, Jake Scott, director of the film Plunkett and Maclaine (1999), Mark Pellington, Jonas Åkerlund, Neil Jordan and Donald Cammell.13 The

216

K. J. DONNELLY

Fig. 11.2 U2’s (Even Better Than) The Real Thing

interface between pop/rock groups and filmmakers not only demonstrates an alliance between those who choose to work in esoteric and personal areas but also shows how many artists, in the loosest sense of the term, saw potential in promos from which they gained creative status beyond simply advertising a pop record (Fig. 11.2).

Conclusion While some theorists of music video have embraced the notion of its radical potential,14 others have conceived music television channels merely as ‘image radio’.15 While some promos have gained critical status from the involvement of ‘respectable’ film directors, in many cases creative and challenging work has all too readily been ignored. It is not hard, however, to see references to, or similarities with, certain aspects of a fine art-based avant-garde in music videos (for example, Franz Ferdinand’s Take Me Out (2004) that included visual montage aspects that were clearly influenced by some of Kurt Schwitters’s ‘Merzbild’ work). If the avant-garde has fallen into decline since the mid-twentieth century, in its place has developed a postmodern culture of blank quotation and an aesthetic devoid of purpose that has lost touch with an original context and thus a primary or intrinsic meaning. According to Fredric Jameson:

11

EXPERIMENTAL MUSIC VIDEO: “WHY DO I NEED TV WHEN …

217

“In a world in which stylistic innovation is no longer possible, all that is left is to imitate dead styles, to speak though the masks and with the voices of the styles in the imaginary museum”.16 This famous passage was doubtless at least partly inspired by the proliferation of promos in the early 1980s, and their most visible culture of visual pastiche. Music videos have created a space in which avant-garde aesthetics are co-opted for commercial purposes. This cuts both ways. Although, on the one hand, the traditions of avant-garde subversion and challenge to the norms of mainstream audiovisual culture are diverted, on the other, this mainstream culture has to try to integrate material that has traditionally been kept well outside its solidly demarcated boundaries. It can clearly be argued that the avant-garde has been institutionalized and tamed and music video is now an established format that, despite pressure to differentiate product, tends by and large towards docile, even banal audiovisual styles and ideas. ‘Generic’ music videos are heavily standardized, with repeated elements de rigeur as iconic accompaniments to their associated genre of music. Prime examples are heavy metal videos, which overwhelming embrace a format that is based on showing a group performing on stage, and rap videos, many of which show a rapper frontally gesturing towards the camera while accompanied by seemingly adoring female ‘groupies’. Of course, not every video in these genres follows this blueprint, but it is striking how many do. The possibilities that still remain ‘outside’ generic production are, all too often, devoted to the musician’s public image or ‘creative’ pretensions. New developments in promo style owe much to technological advances, such as cheaper filming equipment and video imaging effects, or to institutional innovations, most notably the advent of MTV, and a host of other channels, such as The Box and VH1 and VH2. We stand at a point where platforms of dissemination and modes of production are changing, becoming more accessible and immediate. The way television has been broadcast through radio waves, the way it has been decoded and the way that television sets work, have all undergone radical changes in recent years. ‘Internet television’ is considered to be a certainty for the future, as the logical conclusion of the process of ‘convergence’ on to a single consumption platform. Music video as a medium has been at the cutting edge of these developments. In terms of the past, MTV was one of the first television channels to be used widely in public spaces. In terms of the future, not only has music video been endemic on the internet for some years but also some of its more experimental aspects have already

218

K. J. DONNELLY

translocated there. A key example is ‘mashup pop promos’: the amateur practice of adding images to music. This new form begins to question the hierarchy of music and image, with one of the most prominent instances being music added to television images of the 9/11 attack on the World Trade Centre. These ‘unofficial’ promos have lost the requirement to act as an advertisement for a piece of music, and thus point to a new, less polished future for the promo in a shifting televisual landscape.

Notes 1. Theodor Adorno and Max Horkheimer, “The Culture Industry: Enlightenment as Mass Deception” in Simon During, ed., The Cultural Studies Reader (London: Routledge, 1993), p.30. 2. Robert Neaverson, The Beatles Movies (London: Cassell, 1997), p.120. 3. Ibid., p.4. 4. The Maysles already had worked with the group on the documentary Yeah Yeah Yeah: New York Meets The Beatles (1964). 5. Many have disdainfully pointed out that the BBC broadcast this colourful film first in black and white rather than colour. Yet this was due to the fact that the BBC was changing from a 405 line system to a 625 line system. BBC1 was still using the 405 line system and could not, therefore, be received in colour. Magical Mystery Tour was broadcast for a mass audience on BBC1. Its second broadcast was on the 625 line BBC2, where it went out in colour but to a very small number of people who owned colour 625 line television sets. 6. Ian Peel, The Unknown Paul McCartney: McCartney and the Avant Garde (London: Reynolds and Hearn, 2002). 7. Greil Marcus, In the Fascist Bathroom: Writings on Punk, 1977–92 (London: Penguin, 1992), p.108. 8. Gavin Martin, ‘Company Lore and Public Disorder’, New Musical Express, 14 March 1981, p.31. 9. Rather than having filmmakers produce a documentary (or ‘rockumentary’) or concert film, some musicians in the wake of punk rock decided to make their own films. This was enabled by the increased availability of video equipment. Good examples were The Cure, who made a short film called Carnage Visors in 1981, and Bauhaus, who made an elliptical narrative film in 1982.

11

EXPERIMENTAL MUSIC VIDEO: “WHY DO I NEED TV WHEN …

219

10. Raymond Williams, Marxism and Literature (Oxford: Oxford University Press, 1977), pp.131–2. 11. Adrian Wilson, “How Quantel’s Paintbox Revolutionized TV Graphics 40 Years Ago” in TV Tech, 9 November 2021 [accessed 17/02/2024]. 12. His pop promo work runs from mainstream artists (Erasure’s A Little Respect, Level 42’s To be With You Again and Van Halen’s Can’t Stop Loving You) to more esoteric, left field music (Nine Inch Nails’s Wish, and March of the Pigs, Ministry’s New World Order, and Just One Fix, Rage Against the Machine’s Freedom and Front 242’s Rhythm of Time). This exhibits the full spectrum of the pop promo video from mainstream pop teenagers Hanson to austere industrial groups whose music is aimed at anything but chart success. 13. Godley made Even Better than the Real Thing (1992), One (1992), Numb (1993) and Stuck in a Moment (2001). Wenders made Night and Day (1990), Stay (1993) and The Ground Beneath Her Feet (2000). Corbijn made Pride (third version, 1984), Please (1997) and The Hands That Built America (2002). Jake Scott made Staring at the Sun (1997), Mark Pellington made One (second version, 1992), Jonas Åkerlund made Walk On (second version, 2001) and Beautiful Day (2000), Neil Jordan made Red Hill Mining Town (1987) and Donald Cammell made Pride (1984). 14. E. Ann Kaplan, Rocking Around the Clock: Music Television, Post Modernism and Consumer Culture (London: Routledge, 1987). 15. Andrew Goodwin, Dancing in the Distraction Factory: Music Television and Popular Culture (Minneapolis: University of Minnesota Press, 1992); Michel Chion, AudioVision: Sound on Screen (New York: Columbia University Press, 1994). 16. Fredric Jameson, ‘Postmodernism and the Consumer Society’ in Hal Foster, ed., Postmodern Culture (London: Pluto, 1985), p.112.

Afterword

The interplay between music and visual elements can evoke a range of intense emotional responses and complex cognitive interpretations, making audiovisual media a powerful tool for storytelling and audience engagement. This dynamic is evident in the way music can reinforce or subvert visual narratives, creating layers of meaning that enrich the overall viewing-listening or gaming experience. Technological developments and changes in practices have allowed for increasing (and increasingly complex) music use, both inside media and outside. The increased ‘musicalization’1 of culture has been partially enabled by audiovisual culture and feeds in a positive feedback loop that escalates the expectation of having images to accompany music and the expectation of having music, and often prominent featured music, in all areas of audiovisual culture. This book is a testament to the untenability of analytical approaches to audiovisual culture that simply analyse the music or understand film as at heart being a ‘visual culture’. Indeed, theorizations of music and the moving image have been beset by a desire to understand the audiovisual as an addition of discrete musical elements to images (and vice versa) rather than as the genetic level integration of sound and image elements. Music is no simple supplement (or vice versa). Artists working with audiovisual media instinctively know and work to the situation we are told by neuroscience: that sound and image are not perceived as distinct but unified. Rather than thinking of these channels as separate they cohere © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2025 K.J. Donnelly, Hearing Eyes, Seeing Ears, Palgrave Studies in Audio-Visual Culture, https://doi.org/10.1007/978-3-031-73913-2

221

222

AFTERWORD

into a single entity. So, a complex merged signal rather than idealized and discrete ‘music’ and ‘the film’ (or other media), which, conceptually and physically, can be added together in a simple operation. This process might be understood as one signal modulating or changing the other, and while pulling them apart may work well for analysis it does not approximate the effect on us. This ‘modulation’ idea is endemic in production, where the notion that a poor film can be ‘saved’ by the application of effective music has its counterpart in the more evident process where a mediocre song can be redeemed commercially by the images in its music video. Music certainly ‘aestheticizes’ the image, making it become something less of a recording of what was in front of the camera and more of an emotional experience. Of course, much of the time audiovisual culture may follow a fairly banal path, whereas at others it might exploit the sensual and cognitive possibilities available. Indeed, it is clear that much film and television correspond with what Eisenstein, Pudovkin and Alexandrov railed against in their “Statement on Sound”— ‘filmed drama’—rather than exploiting the opportunities of the marriage of sound and image.2 Indeed, dialogue sequences with precise synch in reverse shots illustrate how sections of asynchrony (where we are shown a character’s facial expression as they listen to the other speaker who is momentarily off screen) appear but are anchored by neighbouring shots of precise synchronization of speaking heads. This is but one example of the ‘common core’ of audiovisual culture, where a number of shared aspects and techniques partly unify different media. While this limited repertoire is inevitable when dealing with electronic sound and images on flat screens, crucially the nature of moving images married with sounds coheres around the relationship between the two. This book, which reprints and updates diverse writings of mine about music in audiovisual culture, has chapters that share a central focus on the intricate relationship between music and audiovisual media, examining how this dynamic has evolved across different formats, including film, television, video games and music videos. The underlying theme is the exploration of how music interacts with visual elements to create cohesive narratives and emotional experiences, often through synchronization and a dynamic relationship between the two media which is merged at a level of aesthetics, perception and cognition. It invokes issues to do with synchronization and asynchrony and how these relate to stable notions of depicting an illusory diegetic reality and normality in audiovisual culture, with the contradictions and confusions of sound’s place in narrative space,

AFTERWORD

223

and to do with how the ‘integrated soundtrack’ blurs what were once solid categorical distinctions between music and sound effects, diegetic ‘reality’ and commentary/emotional cueing. A recurring concept is the ‘occult aesthetics’ of audiovisual synchronization, where the alignment (or intentional misalignment) of sound and image creates deeper abstract connections and emotional responses. This theme of precise synchronization and asynchrony is explored through the analysis of specific genres and media, such as the experimental use of music in silent films, the unique soundscapes of the Silent Hill series, and the ground-breaking work of the BBC Radiophonic Workshop in Doctor Who. Moreover, the chapters emphasize the shift towards more ‘musicalized’ soundtracks in various media, influenced by trends in ambient and experimental music. This shift is seen in both the resurgence of silent film screenings with new musical scores and the increasing use of music video aesthetics in television and film.The first four chapters of this book set out a particular approach to music in audiovisual culture, while the later chapters look at more specific instances of the capabilities and affordances of music in audiovisual culture. The introduction gave a sense of how the relationship of the moving image and music has been, and might be, conceived, while “Occult Aesthetics” focuses on the crucial, defining element of audiovisual culture: the synchronization of image and sound. This often ignored aspect also is crucial in aesthetic relationships and signification. Of course, the merged signal comes into being no matter what sounds and images are put together and no matter how they are synchronized. The following chapter, “Gymnasium for the Senses”, builds upon the insights provided by the McGurk Effect to investigate the artificial construction of audiovisual spaces, which play directly to the limitations and particularities of human sound and image perception. “Extending Film Aesthetics” considers the reciprocal influence between film (and other audiovisual) music and other media forms, noting an increasing ‘musicalization’ of soundtracks and of audiovisual objects more generally. The first four chapters of this book set out a particular approach to music in audiovisual culture, while the later chapters look at more specific instances of the capabilities and affordances of music in audiovisual culture. The introduction gave a sense of how the relationship of the moving image and music has been, and might be, conceived, while “Occult Aesthetics” focuses on the crucial, defining element of audiovisual culture: the synchronization of image and sound. This often

224

AFTERWORD

ignored aspect also is crucial in aesthetic relationships and signification. Of course, the merged signal comes into being no matter what sounds and images are put together and no matter how they are synchronized. The following chapter, “Gymnasium for the Senses”, builds upon the insights provided by the McGurk Effect to investigate the artificial construction of audiovisual spaces, which play directly to the limitations and particularities of human sound and image perception. “Extending Film Aesthetics” considers the reciprocal influence between film (and other audiovisual) music and other media forms, noting an increasing ‘musicalization’ of soundtracks and of audiovisual objects more generally. The book’s later chapters confirm the profound impact of music on visual media across different historical and cultural contexts. They examine various instances of this interplay, revealing how audiovisual media harness music’s emotive power to enhance storytelling and viewer engagement. For example, the integration of music in film often follows the basic tenets of Hollywood’s Classical Film Score model, where the musical score underpins the narrative, accentuating emotional beats and guiding audience perception. Despite the fragmentation of this model in recent years, its foundational principles continue to influence contemporary practices, demonstrating the enduring power of music in shaping filmic experiences. Similarly, in television, the use of music can range from the more straightforward, functional approach seen in many dramas to the innovative and experimental use in series like Doctor Who. The latter’s employment of avant-garde electronic music by the BBC Radiophonic Workshop exemplifies how unconventional musical choices can compensate for limitations in visual effects, thereby enriching the narrative and expanding the show’s imaginative scope. Video games present another fascinating domain where music plays a crucial role. The synchronization of sound, image and player input—in what I have called the ‘triple lock’—is fundamental to the immersive experience of gaming. In games such as the Silent Hill series, sound design extends beyond mere background music to become a central element of the game’s psychological and emotional landscape. The soundtracks of these games, characterized by their emotionally charged sound effects and mechanical music, challenge traditional cinematic norms and create a uniquely unsettling atmosphere that is integral to the highly immersive gameplay experience. Music videos, particularly those from the 1980s and 1990s, often employed avant-garde audiovisual techniques, bridging the gap between

AFTERWORD

225

experimental art and popular media. This period saw a melding of innovative visual styles with popular music, resulting in a new form of audiovisual expression that captivated audiences and had a significant influence on films. In conclusion, the later chapters of the book illuminate the multifaceted nature of audiovisual culture, emphasizing the integral and paramount role of music in this interplay. By examining different media forms, the chapters reveal how music, when allied with moving images and other sounds, engages with complex discourses of representation and signification. Whether through conventional styles or experimental approaches, music’s ability to evoke emotion and impart meaning remains a substantial force in audiovisual culture. When integrated with moving images and other auditory elements, it can engage deeply with intricate discourses related to representation and signification. Meaning can be intentional or unwitting, and this engagement often occurs subconsciously or semi-consciously, illustrating how music’s emotional appeal and dynamic qualities can create powerful effects. Music, even when adhering to highly conventional or stereotypical styles, can evoke strong emotional responses and impart nuanced meanings. This phenomenon highlights the inherent potency of music held within audiovisual contexts, even if it appears banal and unattractive. Noël Coward’s famous line from his 1930 play Private Lives, “It’s extraordinary how potent cheap music is”, aptly encapsulates this dynamic. In the realm of audiovisual culture, music’s potential to be transformed by visual elements, and reciprocally, to transform these elements, underscores its significant impact. Music in this context is not merely an accompaniment but a decisive integral component that can enhance, alter or even redefine the visual narrative and other audiovisual elements, gaining a critical potency from the fundamental level of integration of audio and visual components.

226

AFTERWORD

Notes 1. Cf. Tobias Pontara and Ulrik Volgsten, “Musicalization and Mediatization” in Olivier Driessens, G¨oran Bolin, Andreas Hepp and Stig Hjarvard, eds., Dynamics of Mediatization: Institutional Change and Everyday Transformations in a Digital Age (New York: Palgrave, 2017), pp.247-248; Benjamin Kr¨amer, “The Mediatization of Music as the Emergence and Transformation of Institutions: A Synthesis” in International Journal of Communication, no.5, 2011, p.471. 2. Sergei Eisenstein, Vsevolod Pudovkin and Grigori Alexandrov, “Statement on Sound” in Richard Taylor, ed., and transl., S.M.Eisenstein: Selected Works, volume 1, Writings 1922-1934 (London: BFI, 1988), p.113

Index

0–9 8-bit music, 10 9/11 attack on the World Trade Centre, 218 10cc, 211 1960s series Star Trek, 74 2001: A Space Odyssey (1968), 133 2004 Re-Birth of a Nation, 86 A Aarseth, Espen, 66 Ableton Live, 76 Acousmatic, 3 Acoustic ecology, 32 Across 110th Street (1972), 88 Adaptive audio, 172 Added value, 7 Addinsell, Richard, 56 Southern Rhapsody, 56 Adorno, Theodor, 218 African American popular music, 12 Åkerlund, Jonas, 215 Akinesthesia, 33 Altman, Rick, 4

Amadeus , 111 Ambient 4: On Land (1982), 146 Ambient Music, 50 American International Pictures, 91 American McGee’s Alice (2000), 161 Anderson, Gillian B., 127 Anempathetic music, 177 Anger, Kenneth, 214 Antonioni, Michelangelo, 35 The Passenger (Professione Reporter, 1975), 35 Apocalypse Now (1979), 70 Apophenia, 31 Apple Logic, 76 Aqua Jet (1996), 178 Armstrong, Craig, 51 Arnheim, Rudolf, 41, 45, 167 Artemyev, Eduard, 70 Art Zoyd, 127 Ashes to Ashes (1980), 210 Assassins Creed (2007, Ubisoft), 60 Assassins Creed Unity (2014, Ubisoft), 60

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2025 K.J. Donnelly, Hearing Eyes, Seeing Ears, Palgrave Studies in Audio-Visual Culture, https://doi.org/10.1007/978-3-031-73913-2

227

228

INDEX

A Stroll Through the Abandoned City (on Kapnobatai from 2005), 59 Asynchrony, 40, 176 Atrium Carceri, 58, 173 Avant-garde, 188 Ayers, Roy, 95 Ayres, Mark, 190 Aztec Camera, 215

B Babbitt, Milton, 192 Baltimora, 209 Barrow Hill: The Dark Path (2016, Shadow Tor studios), 61 Bart, Lionel, 109 Battleship Potemkin, 134 BBC Great Expectations (1999), 56 BBC Radiophonic Workshop, 188 BBC television series Peaky Blinders (2013–22), 51 Becce, Giuseppe, 9 Kinothek (published in 1919), 9 Beck, Jay, 70 Beggars of Life, 126 Belton, John, 19, 27 Ben Model, 127 Bennett, Richard Rodney, 51, 56 Bessell, David, 147 Bitches Brew (1970), 95 Black Belt Jones (1974), 92, 93 Black Caesar (1973), 90 Black Water (2008), 39 Blacula (1972), 93 Blake’s Seven (1978–1981), 75 Blaxploitation films, 11, 87 Blom, Eric, 55 Blondie, 212 Blur, 215 Blyton, Carey, 189 Bohemian Rhapsody, 207 Bonnie and Clyde (1967), 91

Bordwell, David, 18 Boulez, Pierre, 192 Bounce Inducing Effect, 5 Bowie, David, 203 Brakhage, Stan, 133 Branching (horizontal resequencing), 169 Brechtian alienation (Verfremdungseffekt ), 37 Bresson, Robert, 71 Brideshead Revisited (1981, Granada), 56 Bridge, Frank, 55 Bridgett, Rob, 162 Briscoe, Desmond, 192 Bronnt Industries Kapital, 127 Bronson (2009), 37 Brooks, Peter, 22 Brother’s Gonna Work It Out , 94 Brown, Geoff, 106 Browning, Tod, 24 Freaks (1932), 24 Brown, James, 89 Brown, Joe, 110 Bruno, Giuliana, 58 Buhler, James, 19 Burch, Noel, 19 Burgon, Geoffrey, 189 Burnand, David, 76 Butterworth, George, 55 Bye, Matti, 127

C ’Camera’ or ’set’, 7 Cameron, James, 123 Titanic (1997), 123 Cammell, Donald, 215 Candy Crush Saga (2012), 178 Can Hieronymous Merkin Ever Forget Mercy Humppe and Find True Happiness? (1969), 118

INDEX

Caravaggio (1986), 214 Careless Rapture (1939), 107 Carli, Philip, 129 Carlos, Wendy, 126 Carpenter, John, 77 Escape from New York (1981), 77 Prince of Darkness (1987), 78 Cary, Tristram, 187 Castlevania: Symphony of the Night (1997), 161 Change from analogue to digital media, 76 Cherry, Neneh, 214 Chicago (2002), 122 Chion, Michel, 1 ‘Chiptune’ music, 10 Choreograph movement, 176 Christopherson, Peter, 212 Chu Chin Chow (1916), 107 Clancy, Tom, 60 The Division 2 (2016, Massive Entertainment), 60 Clarke, Malcolm, 189 Classical Film Score, 9 Classical Hollywood Cinema, 9 Cleopatra Jones (1973), 92 Clint Bakajanian, 81 Clinton, George, 89 Clock DVA, 213 Cocteau, Jean, 214 Coffy (1973), 92 Cognitive dissonance, 33 Cohen, Annabel, 51 Coil, 212 Collins, Jim, 19 Collins, Karen, 20, 169 Congruence, 51 Cooder, Ry, 50 Coppola, Sofia, 51, 70 Marie Antionette (2006), 51 Corbijn, Anton, 213 Cotton Comes to Harlem, 90

229

Country House (1995), 215 Coward, Noel, 105 Crash Bandicoot (1996), 177 Crazy Taxi (1999), 178 Cry (1985), 211 Cunningham, Chris, 215

D Dance, Dance Revolution (1998), 174 Dance Dance Revolution (1999), 161 Danna, Jeff, 150 Dark Side of the Moon (1973), 198 Dark Was the Night Cold was the Ground, 64 David Ireland, 51 Davis, Blair, 141 Davis, Miles, 88 Dead End Street in 1966, 206 Dead Space (2008), 171 Dear Esther (2012, The Chinese Room), 58 Delius, Frederick, 55 Denisoff, R. Serge, 83 Dennett, Daniel, 42 Derbyshire, Delia, 187 Destroy the Heart (1987), 213 Detroit 9000 (1973), 92 Diegetic and non-diegetic sound, 4 Digital audio workstations (DAWs), 155 Digital signal processing (DSP), 76 Dirty Harry (1971), 95 Doctor Dolittle (1967), 118 Doctor Who (1963–1984), 13, 75, 185 Dolby sound, 71 Donnelly, K.J., 65 Doyle, Peter, 159 Dr Mabuse (1984), 213 Dynamic audio, 174

230

INDEX

E Earth Wind and Fire, 87 Easy Rider (1968), 91 Eat to the Beat (1979), 212 Edzard, Christine, 111 Eisenstein, Sergei, 3, 16, 157 Electric pianos, 94 Electronic reverb, 152 Elephant (2003), 38 Elfman, Danny, 52 Ellis, Vivian, 108 Elsaesser, Thomas, 137 Emotional sound effect, 56 Empathetic history, 137 ‘English Pastoral’ art music, 55 Eno, Brian, 50, 146 Ambient 4: On Land (1982), 50 Enter the Dragon (1973), 95 Erasure, 215 Evans, Cerith Wynn, 214 Even Better than the Real Thing (1992), 211 Everything but the Girl, 214 Expresso Bongo, 109

F Fairlight CMI digital synthesizer, 77 Fantasia in 1940, 71 Far from the Madding Crowd in 1967 and 2015, 51 Fat Les, 215 Felix the Cat, 206 Ferdinand, Franz, 216 ‘Figure and ground’, 8 Film adaptations, 145 Flinn, Caryl, 17 Floyd, Pink, 71, 198 Dark Side of the Moon (1973), 71 Forbidden Planet (1956), 157, 197 Force of Evil ’s (1948), 67 Forman, Milos, 111

Formby, George, 123 Foxy Brown (1974), 92 Friday Foster (1975), 92 Funkadelic, 89 A Funny Thing Happened on the Way to the Forum (1966), 110

G Gabriel, Peter, 215 Gans, Christophe, 80, 144 Gay, John, 106 The Beggar’s Opera, 106 Gay, Noel, 108 Gee, Grant, 65 Patience (After Sebald) (2012), 65 Gerhard, Roberto, 187 Gestalt principles of proximity, 28 Gestalt Psychology, 8 Giacchino, Michael, 81 Gilbert and Sullivan, 106 Glamorous Nights (1935), 107 Glass, Philip, 130 Glynn, Dominic, 190 Godard, Jean-Luc, 37 Weekend (1967), 37 Godley and Creme, 211 Godley, Kevin, 210 Goldmann, Peter, 206 Goldsmith, Jerry, 197 Gondry, Michel, 215 Gone Home (2018, Fulbright/ Blitworks), 58 Goodwin, Andrew, 219 Gorbman, Claudia, 9 Grand Theft Auto series (1997), 178 Graves, Jason, 171 Gregson-Williams, Harry, 81 Grier, Pam, 92 Griffin, Julian, 91 Griffith, D.W., 134 Birth of a Nation (1915), 134

INDEX

Griffiths, Alison, 138 Guinness, Alec, 110 Guitar Hero (2005), 174 Gumshoe (1971), 124 Gunning, Tom, 23 Guy, Stephen, 124 H Hamilton, Richard, 209 Hammond, Michael, 140 Hancock, Herbie, 88 Happens (1967), 83 Hardy, Thomas, 51 Harper, Graeme, 64 Haslinger, Paul, 80 Häxan (1922), 127 Hayes, Jason, 81 Head Hunters (1973), 95 Healthy dynamic experience, 47 Heldt, Guido, 17 Hemmings, David, 83 Hirst, Damien, 215 His Name is Alive, 215 Historically accurate versions, 128 Hitchhiker’s Guide to the Galaxy (1981), 75 Hockney, David, 209 Hodgson, Brian, 185 Holst, Gustav, 55 Horrocks, Jane, 113 Howard, Josiah, 91 Howarth, Alan, 77 Howell, Peter, 189 Human hearing, 29 Huppertz, Gottfried, 131 Hutch, Willie, 94 Hutmacher, Fabian, 20 I Image radio, 216 Immersion, 148

231

In a Silent Way (1969), 95 InFAMOUS Second Son (2014, Sucker Punch), 60 Inside the City (on Phrenitis from 2009), 59 Integrated soundtrack, 9 Interactive audio, 169 ‘Interactive’ or ‘dynamic’ music, 10 IRCAM, 13 Isomorphism, 3, 167 It’s A Sin (1987), 214 J James, M.R., 48 The Mezzotint (2021), 48 Jameson, Fredric, 216 Jarman, Derek, 212 Jarre, Maurice, 49 Jean-Michel Danton, 136 Jeremy Soule The Elder Scrolls series of games, 10 John Scarlett-Davies, 214 Johnson, Blind Willie, 64 Johnson, J.J., 93 Johnson, Stephen R., 215 Jonze, Spike, 215 Jordan, Neil, 215 Jordan, Randolph, 43 Jubilee (1977), 213 Ju-On: The Grudge (2003), 70 K Kaae, Jesper, 169 Kalinak, Kathryn, 9 Kamp, Michiel, 20 Kassabian, Anahid, 17 Keff McCulloch, 190 Kelly, Jim, 95 Kelly, Tadhg, 58 King, Geoff, 10 Kirby, Philip, 59

232

INDEX

Köhler, Wolfgang, 31 Koven, Mikel, 91 Krzywinska, Tanya, 10 Kubrick, Stanley, 6, 65 A Clockwork Orange, 6 Eyes Wide Shut (1999), 65 Kulezic-Wilson, Danijela, 19, 162 L Lacan, Jacques, 43 Landscapes in audiovisual culture, 47 Lang, Fritz, 213 Last Days (2005), 43 Lastra, James, 70, 162 Launder and Gilliat, 107 The Story of Gilbert and Sullivan (1953), 107 Layering (vertical remixing), 169 Lean, David, 49 Lawrence of Arabia (1962), 49 Leigh, Mike, 107 Topsy-Turvy (1999), 107 Let Me In (2010), 81 Lewin, Kurt, 84 Lewis, Hannah, 19 Library music, 187 Library sounds, 47 Lift Off With Ayshea (Granada, 1969–74), 206 Lionel Festinger, 34 Lisztomania (1975), 111, 207 Little Voice, 113 Lloyd Webber, Andrew, 105 London’s West End, 12 London’s West End theatres, 105 Lost (2004–2010, Bad Robot-ABC), 81 Lost Highway (1997), 84 Louis and Bebe Barron, 197 Lumière brothers, 21 L’arrivée d’un train en gare de la Ciotat (1895), 22

Lydon, John, 208 Lye, Len, 209 Lynch, David, 32, 157 Eraserhead (1977), 157 Lost Highway (1997), 32

M Mackintosh, Cameron, 112 Magical Mystery Tour (1967), 207 Magnum Force (1973), 95 Manovich, Lev, 83 Man with a Movie Camera, 128 Marc (A vehicle for Marc Bolan) (Granada, 1977), 206 Marcus, Greil, 208 Martin, Paul, 20 Maybury, John, 214 Mayfield, Curtis, 87, 94 McGrath, Joe, 206 McLaren, Norman, 209 Me and My Girl , 108 Mel Brooks’s Silent Movie (1976), 133 Méliès, Georges, 22 Mera, Miguel, 76 Metal Gear Solid series, 81 Metropolis , 130 Meyer, Leonard B., 26 Micheaux, Oscar, 135 Within Our Gates , 135 Mickeymousing, 155, 170 Midsomer Murders , 52 Moonraker (1979), 49 Moral occult, 22 Moroder, Giorgio, 130 Mostel, Zero, 110 Mount Vernon Arts Lab’s album, 72 MTV, 204 Mulcahy, Russell, 210 Mulvey, Laura, 213 Munday, Rod, 20

INDEX

’Musicalization’ of the film soundtrack, 11 Music television, 204 Musique concrete, 11, 187

N Nathan-Turner, John, 189 Natural history documentaries, 47 Neumeyer, David, 17 Neuroscience, 29 Newley, Anthony, 106, 109 NICAM (near instant companded audio multiplex) stereo broadcasts, 203 Niebur, Louis, 85 Nightingale, Annie, 208 Nine Inch Nails, 168 Nonindifferent nature, 57, 157 Novello, Ivor, 107 ‘Novel’, radical versions, 128 Novotny, Lawrence, 91 Nyman, Michael, 52

O Oblivion (2006), 169 O’Connor, Sinéad, 214 Oh What a Lovely War!, 110 Oliver!, 109 Operetta, 106 Orpheé (1949), 213 [O T H E R] (2008), 80 Our Man in Havana (1959), 114

P Paddy Kingsland, 189 Page, Gene, 93 Pareidolia, 30 Parker, Jim, 52 Park, Nick, 215 Parks, Gordon, 95

233

Pate, Johnny, 98 Perceptual phenomena, 5 Perron, Bernard, 148 Phantom fundamentals, 31 Plants vs Zombies (2009), 175 Plasketes, George, 83 Player-led synchronization, 170 Plesiochrony, 166 Plunkett and Maclaine (1999), 51 Point Blank (1967), 36 Ponty, Jean-Luc, 127 Pope, Tim, 212 Pop promo, 204 Pordenone, 127 Prendergast, Mark, 50 Priestley, J.B., 109 The Good Companions (1956), 109 Principle of simultaneity, 28 Progressive rock, 207 Propaganda, 213 Propellerheads Reason, 76 Pro Tools, 77 Psycho (1960), 27 Psychological trauma, 33 Public Image Limited (PiL), 208 Pudovkin, Vsevolod, 135 Storm Over Asia—The Heir of Genghis Khan (1928), 135 Punk rock, 208 Q Quake (1996), 161, 168, 173 Quantel Paintbox, 209 Quatermass and the Pit , 72 Queen, 207 R Rapée, Ernö, 9 Encyclopedia of Music for Pictures (published in 1925), 9 Raumpatrouille (1966), 197

234

INDEX

Rausch, Andrew J., 102 Rayner, Jonathan, 64 Rebirth of a Nation (2005), 134 Reed, Graham, 42 Repeated listenings, 145 Revell, Graeme, 80 Reznor, Trent, 161, 168 Rhapsody (1949, Palace), 108 Rice, Tim, 110 Jesus Christ Superstar, 110 Rich, Robert, 80 Stalker (1995), 80 Riddles of the Sphinx (1977), 213 Rodgers and Hammerstein, 108 Roger Limb, 189 Romanek, Mark, 212 Rorschach blot test, 31 Rossini, Gioachino, 6 William Tell Overture, 6 Rowntree, Richard, 97 Ruffles, Tom, 40 Russell, Ken, 109, 207

S Savage! (1973), 94 Schaeffer, Pierre, 3, 192 Schafer, R. Murray, 32 Schifrin, Lalo, 95 Schizophonia, 32 Schwitters, Kurt, 216 Scott, Jake, 215 Scratch Video, 214 Searle, Humphrey, 187 Seeks and finds patterns, 30 Sense of spaces, 4 Severin, Steven, 127 Sex Pistols, 208 Shaft (1971), 88, 90, 97 Shaft’s Big Score, 95 Sheba Baby (1975), 92 Shellard, Dominic, 108, 113

Shermer, Michael, 42 Silent cinema, 12, 125 Silent Hill (2006), 12, 144 Silent Hill games, 143 Silent Hill: Revelation, 144 Silent Hill 4: The Room, 161 Silvers, Phil, 110 Simpson, Dudley, 188 Siouxsie and the Banshees, 213 Sjöström, Victor, 134 The Phantom Carriage (1922), 134 Skyrim (2011), 169 Sledgehammer (1986), 215 Sly and the Family Stone, 89 Smith, Jeff, 4 Something Funny Happened on the Way to the Forum, 110 Sondheim, Stephen, 110 Song sequences, 9 sonorous envelope, 149 Sony Play Station (PS1), 10 Sosin, Donald, 127 Soule, Jeremy, 10, 169 The Elder Scrolls series of games, 10 Soundtrack albums, 68 Space 1999 (1975), 43 Spadoni, Robert, 23 Spellbound (1945), 53 SPK, 80 Splet, Alan, 157 Spooky, D.J., 86, 134 S.T.A.L.K.E.R.: Call of Pripyat (2010), 60 S.T.A.L.K.E.R.: Shadow of Chernobyl (2007), 60 Starewicz, Ladislaw, 206 Starsky and Hutch (1957-79, Spelling-Goldberg/ABC), 103 Star Trek (retrospectively relabelled ‘The Original Series’, 1966–69), 63, 197 Steele, Tommy, 109

INDEX

Steinberg Cubase, 76 Stereo sound, 4 Stilwell, Robynn J., 17 Stockhausen, Karl-Heinz, 192 Storr, Anthony, 42 Suede, 214 Sugar Hill (1974), 92 Summers, Tim, 20 Sunset Boulevard (1950), 36 Superfield, 69 Super Fly (1972), 89, 94 Sweeney, Mark, 20 Sweet, Michael, 169 Sweet Sweetback’s Baadasssss Song (1971), 88 Synch points, 167 Synchresis, 3 Synchronization, 3, 23, 166 Synergetic effect, 5

T Tarkovsky, Andrei, 60 Stalker (1979), 60 Tarzan Boy (1985), 209 Technological determinism, 69 Technology, 4, 67 Television library music, 52 Temp track, 151 Tetris (1989), 169 The Alloy Orchestra, 127 The Angelic Conversation (1985), 212 The Baschet brothers, 188 The BBC, 185 The Beatles, 206 The Boy Friend, 109 The Bridal Party, 126 The Cabinet of Dr.Caligari (Das Kabinett des Doktor Caligari, [1919]), 127 The Chart Show (Channel 4/ITV, 1986–98), 204

235

The Cranberries, 214 The Dancing Years (1939), 107 The Day the Earth Stood Still (1951), 53 The Dodge Brothers, 126 The ‘forgetting’ of mediation, 14 The Golden Compass (2007), 22 The Grass is Greener (1960), 123 The Hacker (1992), 213 The House of Love, 213 The Illustrated Man (1969), 197 The Kinks, 206 The Ladykillers (1955), 193 The Lambeth Walk (1939), 108 The Lone Ranger (1949–57, ABC), 6 The Mack (1973), 92, 94 The McGurk Effect, 1 The Monstrous Soul (1992), 79 The Music Lovers (1970), 111 The Next Life (1993), 214 The Night of the Demon (1957) (known as Curse of the Demon in the USA), 79 The Odessa File (1974), 124 The Old City: Leviathan (2015, Postmod), 58, 173 The Old Grey Whistle Test (BBC2, 1971–87), 204 The Outer Limits (1963–5), 197 The Pet Shop Boys, 214 The Place Where the Black Stars Hang (1994), 79 The Quay brothers, 215 The Queen is Dead (1986), 214 Theremin, 53 The Rocky Horror Picture Show, 111 The Séance at Hobs Lane (2001; re-released in 2007), 72 The Smiths, 214 The Strange World of Gurney Slade, 119 The Stranglers, 52

236

INDEX

The Tales of Beatrix Potter (1971), 111 The TARDIS, 185 The Tempest (1979), 213 The Towering Inferno (1974), 98 The Twilight Zone (1959–65), 197 The Vanishing of Ethan Carter (2014, The Astronauts), 58 The Wizard of Oz (1939), 27 The X-Files (1993–2002), 198 Three Cases of Murder (1955), 48 Timbre, 51 Tippett, Michael, 56 T.N.T. Jackson (1974), 92 Together Brothers (1974), 92, 96 Tol’able David, 126 Tommy (1975), 111, 207 Toop, David, 24 Top of the Pops (BBC1, 1964–2006), 204 Triggered audio, 169 Triggering, 179 Trip hop, 147 Truck Turner, 94 Twin, Aphex, 215 U U2, 211 Uematsu, Nobuo, 10 Final Fantasy series of games, 10 Ultravox, 210 Uncle Josh at the Moving Picture Show (1902), 22 Unified signal, 5 Urry, John, 64 Ussachevsky, Vladimir, 192 V van Beethoven, Ludwig, 38

Moonlight Sonata, 38 van Leeuwen, Theo, 18 Van Peebles, Melvin, 90 Vernallis, Carol, 20 Vienna (1981), 210 Visual music, 72 Vrenna, Chris, 161 W Walker, Andrew, 102 Walking simulator, 58 War Requiem (1988), 214 Watson, Chris, 102 Wayne, Jeff, 71 War of the Worlds (1978), 71 Weis, Elisabeth, 19 Wenders, Wim, 50, 215 Paris Texas (1984), 50 Westerkamp, Hildegard, 38 Beneath the Forest Floor (1992), 38 Türen der Wahrnehmung (‘Doors of Perception’) (1989), 38 Whalen, Zach, 148 White, Frances, 38 Walk Through Resonant Landscape #2 (1992), 38 WiiFit (2007), 174 Williams, Brian, 78 Williams, Ralph Vaughan, 55 Williams, Raymond, 209 Willie Dynamite (1974), 93 Winters, Ben, 17 Womack, Bobby, 88 Women in Love (1969), 111 World of Warcraft , 81 Y Yamaoka, Akira, 146 Yellow Submarine (1968), 207