Music Video Games: Performance, Politics, and Play 9781501308529, 9781501308536, 9781501308512, 9781501308499

Table of contents :
Cover
Contents
List of Figures
List of Tables
Acknowledgments
Introduction—Taking Note of Music Games
PART ONE Preludes and Overtures
1 SIMON: The Prelude to Modern Music Video Games
2 Mario Paint Composer and Musical (Re)Play on YouTube
3 Active Interfaces and Thematic Events in The Legend of Zelda: Ocarina of Time
4 Sample, Cycle, Sync: The Music Sequencer and Its Influence on Music Video Games
PART TWO Virtuosi, Virtues, and the Virtual
5 Consumerism Hero: The “Selling Out” of Guitar Hero and Rock Band
6 Beat it!—Playing the “King of Pop” in Video Games
7 Virtual Jam: A Critical Analysis of Virtual Music Game Environments
PART THREE Concerts, Collaboration, and Creativity
8 Guitar Heroes in the Classroom: The Creative Potential of Music Games
9 Rocksmith and the Shaping of Player Experience
10 Rhythm Sense: Modality and Enactive Perception in Rhythm Heaven
11 Pitching the Rhythm: Music Games for the iPad
Afterword—Toadofsky’s Music Lessons
Glossary of Music and Gaming Terms
About the Contributors
Author Index
Game Index
General Index

Citation preview

Music Video Games

APPROACHES TO DIGITAL GAME STUDIES Volume 4

Series Review Board (Alphabetically) Mia Consalvo, Massachusetts Institute of Technology James Paul Gee, Arizona State University Helen Kennedy, University of the West of England Frans Mäyrä, University of Tampere Toby Miller, University of California, Riverside Torill Mortensen, IT University Copenhagen Lisa Nakamura, University of Illinois Gareth Schott, University of Waikato Mark J. P. Wolf, Concordia University

Series Editors Gerald Voorhees, University of Waterloo Josh Call, Grand View University Katie Whitlock, California State University, Chico

Music Video Games Performance, Politics, and Play

EDITED BY MICHAEL AUSTIN

Bloomsbury Academic An imprint of Bloomsbury Publishing Plc

Bloomsbury Academic An imprint of Bloomsbury Publishing Inc 1385 Broadway New York NY 10018 USA

50 Bedford Square London WC1B 3DP UK

www.bloomsbury.com BLOOMSBURY and the Diana logo are trademarks of Bloomsbury Publishing Plc First published 2016 © Michael Austin, 2016 All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage or retrieval system, without prior permission in writing from the publishers. No responsibility for loss caused to any individual or organization acting on or refraining from action as a result of the material in this publication can be accepted by Bloomsbury or the author. Library of Congress Cataloging-in-Publication Data A catalogue record for this book is available from the British Library. ISBN: HB: 978-1-5013-0852-9 PB: 978-1-5013-0853-6 EPDF: 978-1-5013-0849-9 EPUB: 978-1-5013-0850-5 Series: Approaches to Digital Game Studies Cover design by Clare Turner Typeset by Integra Software Services Pvt. Ltd.

To Paul

Contents List of Figures  ix List of Tables  xi Acknowledgments  xii

Introduction—Taking Note of Music Games 1 Michael Austin

PART ONE  Preludes and Overtures 23 1 SIMON: The Prelude to Modern Music Video Games 25

William Knoblauch 2 Mario Paint Composer and Musical (Re)Play on YouTube 43 Dana M. Plank 3 Active Interfaces and Thematic Events in The Legend

of Zelda: Ocarina of Time 83 Stephanie Lind 4 Sample, Cycle, Sync: The Music Sequencer and Its Influence on Music Video Games 107 Michael Austin

PART TWO  Virtuosi, Virtues, and the Virtual 125 5 Consumerism Hero: The “Selling Out” of Guitar Hero

and Rock Band 127 Mario A. Dozal

viii

CONTENTS

6 Beat it!—Playing the “King of Pop” in Video Games 153

Melanie Fritsch 7 Virtual Jam: A Critical Analysis of Virtual Music Game Environments 177 David Arditi

PART THREE  Concerts, Collaboration, and Creativity 195 8 Guitar Heroes in the Classroom: The Creative Potential

of Music Games 197 David Roesner, Anna Paisley, and Gianna Cassidy 9 Rocksmith and the Shaping of Player Experience 229 Daniel O’Meara 10 Rhythm Sense: Modality and Enactive Perception in Rhythm Heaven 251 Peter Shultz 11 Pitching the Rhythm: Music Games for the iPad 275 Nathan Fleshner Afterword—Toadofsky’s Music Lessons 297 William Cheng Glossary of Music and Gaming Terms  About the Contributors  308 Author Index  312 Game Index  316 General Index  320

304

List of Figures Figure 1.1 An original model SIMON game. SIMON® &

©2015 Hasbro, Inc. Used with permission 30 Figure 2.1 Screenshot of Mario Paint (1992) 47 Figure 3.1 Ocarina command melodies in The Legend of Zelda: Ocarina of Time 89 Figure 4.1 Screenshot from Chiptune Runner 116 Figure 4.2 Screenshot from My Singing Monsters 118 Figure 9.1a Rocksmith 2014 startup menu 233 Figure 9.1b Marketing image for Rocksmith 2014; courtesy Ubisoft 233 Figure 9.2 Blur, “Song 2,” excerpt from chorus 236 Figure 9.3 Muse, “Plug in Baby,” excerpt from introduction 238 Figure 9.4a The XX, “Islands,” excerpt from outro 241 Figure 9.4b The Arctic Monkeys, “RU Mine,” excerpt from outro 242 Figure 10.1 Cues to grab a piece of candy or swat a spider, and the two-spider warning 262 Figure 10.2 The basic pattern in “Shiwake” / “Packing Pests,” and the basic pattern in its sequel, “Shiwake 2” 263 Figure 11.1 Soundrop with two sounding bars drawn (screenshot from iPad). Arrows added to highlight the trajectory of the balls 283

x

LIST OF FIGURES

Figure 11.2 The octave divided into semitones.

The space between each pitch represents the distance of 100 cents 290 Figure 11.3 Circadia, Level 6 (screenshot from iPad) 292 Figure A.1 (Left) Tadpole tablature at Melody Bay and (right) endless opportunities to recompose final melody 298 Figure A.2 Toadofsky’s symphony, mm. 1–6 (my transcription) 299 Figure A.3 Three examples of how players might complete the melody of mm. 7–8 299

List of Tables Table 2.1

Available sounds in Mario Paint (1992) 48 Table 2.2 Top ten YouTube results for “Mario Paint Composer” on March 2, 2015, by view count 64 Table 2.3 Top ten YouTube results for “Mario Paint Composer” on March 2, 2015, by rating 65 Table 3.1 Major game events in The Legend of Zelda: Ocarina of Time 87

Acknowledgments I

am extremely fortunate to have received the support and encouragement of many people throughout the completion of this project. First and foremost, I wish to thank the contributors for sharing their great ideas and insight with me and for the time and hard work they have invested in this collection; of course, this project would not have been possible without them. I also owe a debt of gratitude to the series editors—Gerald Voorhees, Josh Call, and Katie Whitlock— the series review board, and other anonymous reviewers for their encouragement, constructive criticism, and diligent review of the manuscript and its preceding drafts. I especially want to thank Katie Gallof, Mary Al-Sayed, Michelle Chen, Balaji Kasirajan, and everyone at Bloomsbury for their assistance in making this dream a reality. I took up my position at Howard University as I began working on this project, and I would like to thank all of my colleagues in the School of Communications, especially those in the Department of Media, Journalism, and Film, for providing such an stimulating and collegial space in which to work on this project. I would also like to thank the organizers and participants of the 2nd Annual North American Conference on Video Game Music in Ft. Worth, Texas, its European predecessor, hosted annually by the Ludomusicology Research Group, and the participants of the Society for Music Theory’s Film and Multimedia Special Interest Group’s special session on video games. Although there are far too many colleagues and friends to thank by name, I want to specifically thank my family and James Buhler, Juan Chatta, William Cheng, Karen Cook, William Gibbons, Lee Hartman, Michiel Kamp, Jesse Kinne, Judi Latta, Neil Lerner, Jeffrey McKnight, Bishetta Merritt, Kiri Miller, Amber Neuroth, Steven Reale, Yanick Rice-Lamb, Candice Shannon-Lewis, Tim Summers, Mark Sweeney, Sue von Rautenkrantz, and Sonja Williams for their inspiration and words of encouragement.

Introduction—Taking Note of Music Games Michael Austin

A

ttending a performance in a concert hall can be an exhilarating experience, especially when attending a classical concert. Imagine walking into a crowded auditorium, surrounded by people dressed in their finest clothes, obviously prepared to enjoy an evening of culture and masterful performance. Each person settles quietly into his or her seat as the orchestra takes the stage. The audience claps as the concertmaster steps out from just offstage to prepare the orchestra for the concert. After the orchestra is tuned, the audience applauds even louder as the players stand when the conductor and soloist walk from backstage to center stage. The conductor stops quickly to shake the concertmaster’s hand on her way to the podium. After a few bows (and perhaps the National Anthem), the conductor raises her baton as the crowd waits in hushed anticipation for the music to begin. With a commanding downbeat from the conductor, the orchestra begins to play the first few melodies and themes of the concerto, but the music quickly winds down to a quiet whisper in preparation for the soloist’s grand entrance. With a quick glance to the conductor, the soloist raises her instrument, takes a deep breath and begins: orange, red, red, blue, yellow, yellow, green. As she presses colored buttons on the neck of her plastic guitar controller, the music soars. The soloist slips in and out of the music, astounding the audience with her breathtaking technique. Breaking concert protocol a few minutes later at the cadenza, she jumps to her feet, and with a few tugs on the whammy bar, a flurry of button clicks, and rapid flicks of the strum bar, she totally shreds on the plastic guitar controller. Although this scenario sounds preposterous (and probably is), it is not entirely outside of the realm of possibility. Concert producers

2

MUSIC VIDEO GAMES

Tommy Tallarico and Jack Wall have developed an incredibly popular concert series called “Video Games Live” wherein Emmanuel Fratianni conducts top-tier orchestras, such as the Los Angeles Philharmonic and the National Symphony Orchestra, in playing music from World of Warcraft (Blizzard Entertainment 2004), Tetris (Nintendo 1984), Sonic the Hedgehog (Sega 1991), Final Fantasy (Square 1987), and many other video games and game series. Almost ten years ago, students at the University of Maryland, College Park formed the Gamer Symphony Orchestra—a student-run orchestra and chorus that only performs music from video games. And there is, in fact, a concerto written for plastic guitar controller. Inspired by Guitar Hero 2 (Harmonix 2006), award-winning composer Shiva Feshareki wrote GH Konzert—the retro for Nintendo Guitar Hero and Mixed Ensemble, premiered by YouTube star, filmmaker, musician, and world champion Guitar Hero player Freddie Wong in the basement of London’s Royal College of Music in 2011 (Hewett 2011). Every day, amateurs and aficionados alike stage performances in pubs, living rooms, and parents’ basements across the world for crowds of adoring fans, small groups of friends, or for personal enjoyment with music video games. Of course, games such as those from the Rock Band (Harmonix) and Guitar Hero (Harmonix) series are certainly considered to be music video games, but many other games in this genre are not as cut-anddry. Broadly defined, music video games are those in which the formal elements of the game (rules, rhetoric, dynamics, etc.) are musical in nature. This musicality is apparent when the most meaningful interaction with the game is musical; for example, players interact with rhythm games, such as Guitar Hero or Rock Band, by pressing colored buttons in rhythm to the music heard. Nonetheless, there are also puzzle games, racing games, and “endless runners” that are just as musical. So before we discuss music games in more detail and uncover their finer points, perhaps the initial question is: What makes a game “musical”?

Play Music and video games have more in common that one might initially think, but we should begin with the most obvious similarity: Music is, at its core, a playful thing. Ian Bogost points out this clear connection,

INTRODUCTION—TAKING NOTE OF MUSIC GAMES

3

writing: “Music and games share a fundamental property: both are playable, offering their listeners and operators and expressive experience within the framework of melody and rhythm” (Bogost 2011, 36). But as with other things we play (such as sports, “dress up,” and especially video games), the seriousness of music is often overshadowed by its ludic nature. In Truth and Method, Hans-Georg Gadamer argues that play is not necessarily a bad thing but is, in fact, what gives art its meaning, describing how “play itself contains its own, even sacred, seriousness” (Gadamer 1975, 107). He explains: The player himself knows that play is only play and that it exists in a world determined by the seriousness of purposes. But he does not know this in such a way that, as a player, he actually intends this relation to seriousness. Play fulfills its purpose only if the player loses himself in play. Seriousness is not merely something that calls us away from play; rather, seriousness in playing is necessary to make the play wholly play. Someone who doesn’t take the game seriously is a spoilsport. The mode of being of play does not allow the player to behave toward play as if toward an object. The player knows very well what play is, and that what he is doing is “only a game”; but he does not know what exactly he “knows” in knowing that. (Gadamer, ibid., original emphasis) According to Gadamer, this immersion is how we experience art (and, in this case, music): through play. Likewise, we usually experience the artistry of a game through playing it. Johan Huizinga, also valued play as a defining characteristic of the work of art but makes a different claim regarding music’s nature. In Homo Ludens, one of the first major works to examine the value of play as a critical aspect of culture, Huizinga describes musical elements, like the elements of play, as those that lie “outside the reasonableness of practical life…[it] has nothing to do with necessity or utility, duty or truth” (Huizinga 1938, 158). He goes on to write: Musical forms are determined by values which transcend logical ideas, which even transcend our ideas of the visible and the tangible. These musical values can only be understood in terms of the designations we used for them, specific names like rhythm

4

MUSIC VIDEO GAMES

and harmony which are equally applicable to play or poetry… But whereas in poetry the words themselves lift the poem, in part at least, out of pure play into the sphere of ideation and judgment, music never leaves the play-sphere… In the enjoyment of music, whether it is meant to express religious ideas or not, the perception of the beautiful and the sensation of holiness merge, and the distinction between play and seriousness is whelmed in that fusion. (Huizinga 1938, 159) For Huizinga, it seems as if meaning in music is derived from its phenomenological experience as play. In Man, Play, and Games, Roger Caillois delineates several characteristics of play that apply equally to play and to games: Free, Separate, Uncertain, Unproductive, Governed by rules, and Make-believe (Caillois 2001, 9–10). With the exception of extreme circumstances, playing music is fun and meant to be a diversion (even for working professional musicians), and it is an activity separate from the other duties of the day. Setting aside time to relax and play a video game is no different (except, of course, when playing serious games, learning games, or other types of games wherein the boundaries between play and work seem to blur). Playing music, especially for fun, can be uncertain and even make-believe, giving the singer or instrumentalist freedom to experiment or even improvise or compose new music; likewise, video games allow players to unlock new levels previously unknown to them, and they introduce players to fantasy worlds and narratives that are often far from the realities of day-to-day life. While Caillois’s assertion that playing music should be “unproductive” is debatable (even perhaps erroneous), there is a sense that playing both games and music allow the player to suspend their sense of duty to achieve a particular outcome. Yes, it is important to reach the end of a piece of music without playing a wrong note, or defeat a particular boss, or earn a higher score, but the unproductive nature of play allows for the option to always try again. And finally, both games and music are governed by rules. Not only are games governed by the stated rules (e.g., navigate Mario to the end of each level, avoid enemies and obstacles, collect coins, and do so within 300 s), but games are also regulated by rules that are left unstated (Mario must usually abide by the laws of gravity, for example). Music

INTRODUCTION—TAKING NOTE OF MUSIC GAMES

5

is also bound by a corresponding set of explicit and implicit rules: Pianists are taught to sit a certain way and to play with “proper technique,” playing the music the “correct way” (itself governed by traditions, part-writing rules, harmonic conventions, etc.). Describing the important function of rules in play, Caillois further delineates two very different types of play. The first, paidia, or “spontaneous manifestations of the play instinct” (Caillois 2001, 28), is present both in improvisatory musical practices, such as “jam sessions” and playing jazz, and in playing music games that give players the freedom to create music through gameplay. The second, ludus, is the more strictly organized play of performing music by rote or note-for-note from the page or the mastery of a rhythm game level achieved by executing the requisite movements to perfection.

Video games as musical instruments We demonstrate our understanding of the world based on the ways in which we classify the things within it. Oftentimes, our taste for video games, our willingness to buy and play them, our evaluation of these games, and our ultimate desire to study them academically, depends upon the categories in which we place them or the ways we perceive such categorization. In addition to the aforementioned topics, this volume also explores ways in which we classify music video games. One way that is particularly helpful in understanding how people perceive and use music video games is to examine these games as we would examine musical instruments. When setting out to decide whether or not a music video game can be a musical instrument, we must ask ourselves: “What is a game?” Even before the codification of Game Studies as an academic field, this question has plagued scholars thinking seriously about ludological activities, and as with popular music scholarship, issues of “the real” and “the virtual” are often central to this discussion. In the introduction to his book Half-Real: Video Games between Real Rules and Fictional Worlds, Jesper Juul writes: Video games are two rather different things at the same time: video games are real in that they are made of real rules that players

6

MUSIC VIDEO GAMES

actually interact with; that winning or losing a game is a real event. However, when winning a game by slaying a dragon, the dragon is not a real dragon, but a fictional one. To play a video game is therefore to interact with real rules while imagining a fictional world and a video game is a set of rules as well as a fictional world. (Juul 2011, 1) Taking issues with this definition in his keynote address at the 2009 Digital Games Research Association conference, titled “Video Games Are a Mess,” Ian Bogost discusses a bit of the history of the ontological investigation of video games, the ludology vs. narratology debate, writing that the ludology/narratology question may have appeared to look like this: Is a game a system of rules, or is a game a kind of narrative? But really, it amounted to something more like this: Is a game a system of rules, like a story is a system of narration? (Bogost 2009, original emphasis) Later, he writes: Game studies means not just studies about games-for-players, or as rules-for-games, but also as computers-for-rules, or as operational logics-for computers, or as silicon wafer-for-cartridge casing, or as register-for-instruction, or as radio frequencies-for-electron gun. And game is game not just for humans but also for processor, for plastic cartridge casing, for cartridge bus, for consumer… and so on. (Bogost 2009) It is from this type of “flat ontology” that I want to situate our examination of music video games. I certainly do not intend to settle any esoteric questions about the essential nature of video games, but I do think that an object-oriented approach toward video games would be fruitful in discussing the possibility that music video games can, in fact, be musical instruments—or that they could at least be investigated as such. Organology is the classification of musical instruments and the study of their design and use. Much like video games, music can be

INTRODUCTION—TAKING NOTE OF MUSIC GAMES

7

described as both a system of rules and a narrative, but unlike musical instruments, video games-as-objects do not really have a codified system of classification that describes their use and mechanics. Classification of games into genre comes closest to performing this task. Most scholars agree that genre is most useful when it categorizes games based on the modes of interactivity they engender (Myers 1990) or facilitate (Wolf 1997). Others find that genre is much more multifaceted and can group games based on their function (how or why we use them), their themes or representational characteristics (sci-fi, fantasy, military, etc.), or by their ludic function (perspective, mechanics, etc.) (Voorhees et al. 2012). This approach is particularly fitting for music video games since, as mentioned previously, they transcend one single genre. For example, Rayman Legends (Ubisoft Montpellier 2013) is classified as an action platform video game (or platformer) because the avatar (Rayman) must jump from platform to platform to advance in the game. The same game is also considered to be an “endless runner,” not because the game is endless—it does have goals to reach at the end of each level—but because the game’s avatar literally runs continuously throughout the game and his forward trajectory limits a player’s ability to explore a level. The ability of a player to collect currency (called “Lums”) is an important part of gameplay, and this task is accomplished most efficiently if the player matches Rayman’s actions in rhythm to the game’s nondiegetic music, making Rayman Legends a music game, too. The most famous classification system for musical instruments, the Hornbostel-Sachs system, is based upon the materiality of the instrument and how this object vibrates to produce sound. Idiophones (instruments such as cymbals or gongs, which vibrate themselves to produce sound), membranophones (instruments that produce sound by a vibrating membrane), chordophones (instruments with vibrating strings), and aerophones (which produce sound with vibrating columns of air) are the traditional categories of classification. Other categories, such as water-based hydraulophones, plasmaphones (that make sound using plasma), analog instruments that are amplified or triggered electronically or those that use electronically driven oscillators (electrophones), corpophones (instruments for which the human body serves as a resonator), and hyperspace or non-physical instruments (called quintephones), have been added more recently.

8

MUSIC VIDEO GAMES

In On Concepts and Classifications of Musical Instruments (one of the very few books published on organology in the last thirty years), Margaret J. Kartomi begins with a chapter titled “Any Classification Is Superior to Chaos,” borrowing the phrase from anthropologist Claude Lévi-Strauss, who argued that “any classification is superior to chaos and even a classification at the level of sensible properties is a step towards rational ordering” (Lévi-Strauss 1966, 15). She makes a distinction between schemes that are imposed by observers (“observer-imposed”) and those that arise from the culture in which they exist (“culture-emerging”). Perhaps the closest parallel to these types of taxonomies in video games is genre. Genre Studies is a subfield of Film Studies and Literature Studies that, of course, addresses issues of genre in these artforms. By observing characteristics that are similar among films or literary texts, groups can be formed of objects that share these qualities, and thus, observations and sometimes assumptions can be made about these objects based upon their classification. Genre gives the audience a set of expectations. For example, in viewing a film from the Western genre, one can assume that he or she will encounter cowboys, a conflict between the “good guys” and the “bad guys” in the wilderness or on the frontier, harmonicas and guitars, and other elements (many of which are clichés). These schemes are observerimposed because the viewer notices them and expects them in films advertised as Westerns; they are also culture-emerging because these characteristics are based in the history, culture, and folklore of the Wild West. While all of these conventions may be found in most John Wayne movies, they may not be found in Star Wars, a film which director George Lucas argues is a Western. Although dividing video games into genres is a fruitful endeavor, there are many reasons why such classification is also troublesome. In his book Film Theory: An Introduction, Robert Stam identifies several problems with genre analysis that could be extended to the examination of video game genres. The first concern is extension, meaning that extending a label to include a certain characteristic could be too broad to be useful to analysts; on the other hand, a narrow focus that only includes a few specimens would not really constitute a genre. Secondly, normativisim, or “having a preconceived a priori idea of what a genre

INTRODUCTION—TAKING NOTE OF MUSIC GAMES

9

film should do” (Stam 2000, 128), can disqualify films (or games) from a genre with which they share many characteristics. Stam’s third issue with genre analysis is the tendency to make genres monolithic, whereby films can classified as fitting into one genre only. The inclination toward fitting video games into distinct pigeonholes is felt quite poignantly with music games. Rarely is the point of a music game to simply make music; rather, games that most consider as “music games” are often games from other genres with musical elements. For example, Chiptune Runner (Evil Indie Games 2013) is an endless runner game that also functions as a step sequencer. Dance Dance Revolution (Konami 1998, hereafter “DDR”) is a music game, a dance game, and because of the physicality involved in gameplay, it could also be classified as an exercise game. In his chapter in this volume, Daniel O’Meara argues that even games squarely within the music game genre, such as Rocksmith (Ubisoft 2012) and BandFuse: Rock Legends (Realta Entertainment Group 2013), can sometimes blur the lines between casual video games and serious educational games or even simulation games. The fourth problem Stam identifies is biologism, that is, the tendency to essentialize a genre, giving it a life cycle of teleological development toward an ideal, with distinct periods of maturity and decline when styles change. New films are then compared to these idealized forms and are critiqued based upon how well (or not) they conform to this model. Despite these issues raised by Stam and others, generic labels are beneficial to games, even if on a superficial level. Many players have gaming styles and preferences that are highlighted in various genres of video games; game publishers capitalize on these categories (even inventing new ones when necessary) as a way to market and sell games to targeted audiences. Clara Fernández-Vara writes, “At times mentioning the genre or family the game belongs to becomes shorthand for explaining the core mechanics” (Fernández-Vara 2015, 94). In other words, games that share a genre often share the same rules and operating procedures, so a player can usually presume a basic understanding of how to play games simply based on their familiarity with other games in the same genre. For example, the core mechanics of a first person shooter are clear from the game’s inclusion in that genre—move around within a game world, aim at targets, and shoot at them. From a ludomusicological standpoint, similarities

10

MUSIC VIDEO GAMES

and differences among the characteristics of games that constitute a genre are often interesting starting points for investigation, especially when a game deviates from the normal expectations. Beyond genre, other scholars have sought to organize video games based on player interactivity. Unlike films or literary works, the audience of video games enjoys the benefit of interacting with the game, and in doing so, directly affects the ways in which many of the games characteristics are expressed. In his essay Transcribing Musical Worlds; or, Is L.A. Noire a Music Game?, Steven Beverburg Reale points out that there are four salient, yet abstract, features of Guitar Hero (Harmonix 2005) related to player interactivity that classify it (and other titles with similar features) as a music game (even games that are less obviously musical): (1) an ideal musical object, (2) a visual representation of the ideal musical object, (3) a means by which the player can interact with the visual representation of the musical object, and (4) a means by which the game communicates success or failure in realizing the ideal musical object (Reale 2014, 79). David Saltz’s notion of interactivity is equally object-oriented but points more toward the gaming system itself. He suggests that an input device must capture movement, the computer must interpret the input, and feedback must be given to the player (Saltz 1997, 118). Matthew Lombart and Jennifer Snyder-Duch also advocate for object-oriented interaction but with a longer list of requirements that influence the level of possible interactivity: “the number of inputs, the number and type of characteristics that can be modified, extent of change possible, speed with which the medium responds to the user input, and degree of correspondence between input and response” (Lombard and Snyder-Duch 2001; quoted in Collins 2013, 10). This interaction corresponds to musical instruments quite easily, especially concerning the development of the instrument’s “interface.” An instrument’s interface for input—whether it be bore holes in a transverse flute or intricate interlocking Boehm system of keywork for modern flutes—allows for prompt interaction, allows the player to modify the notes being played, and provides direct response in the sound produced based upon the keys pressed or holes covered. Likewise, the audience and the intended performative aspects of a game have a direct influence on design and performance in video

INTRODUCTION—TAKING NOTE OF MUSIC GAMES

11

games; for example, compare the size and weight of a violin and imagine the performative differences these characteristics afford compared to an organ that weighs several tons and is played from a loft in the back of a church sanctuary, hiding the organist, the console, and a large majority of the organ’s pipes from view. In similar ways, this aspect corresponds to DDR and the height of the dance platform, and the way in which it is designed for spectators to gather around the players in an arcade. This public performance is different perhaps than Rock Band (Harmonix 2007), which is designed to facilitate a public performance, although an arguably smaller one (in the player’s home), and is certainly different from music games designed for personal mobile devices that are usually only viewed by the players themselves. New musical devices have varying degrees of success in penetrating the conceptual boundary between instrument and non-instrument, and frequently their path into the instrument domain is unexpected from the perspective of design intentionality. The issue is further confused by a layer of artistic interpretation, exploding the possible definitions of “instrument” to virtually any conceivable artifact that can involve sound (including the absence of sound). “Instrument” can thus refer to a traditional acoustic device, a controller with no specific mapping, a software program that maps control input to musical output, or can be synonymous with a musical piece itself, in which the interface (including its physical component) is integrated with musical sound output in the composer’s expressive intent. (Malloch et al. 2006, 1) The player’s interaction with an instrument also directly correlates with Karen Collin’s notion of “kinesonic synchresis.” Sound in video games and musical instrument playing are driven by interaction. A player-generated event in a video game results in a sound (Mario’s squeaky jump over a hole in the ground, the auditory feedback from a selection on a menu, or the notes being “played” on a guitar controller for a rhythm game) much the same way pressing keys on a piano, or the valves of a trumpet, will result in a particular note. Sometimes this music is automatic and controlled less by the player—such as “over world” music, the drones of a bagpipe, or the automated

12

MUSIC VIDEO GAMES

notes on a step sequencer. Collin’s notions of kinesonic fidelity apply to instruments too; for example, Collin describes playing Wii Tennis (Nintendo 2006) and questions whether the volume and pitch of the “pop” of the tennis ball should correspond to the relative vigor of the swing of her controller (Collins 2013, 36). Sometimes sound is mapped to action, sometimes it is not. By design, an acoustic piano has kinesonic fidelity as the volume of the notes correspond to the velocity of the key press; on the other hand, a note on the pipe organ will sound the same whether the organist plays it with a light, emotionless touch, joie de vivre, or an angry and forceful stab of the finger. Hornbostel-Sachs categories are further divided into sub-categories based on how the instrument is either struck, plucked, blown, or bowed, referring to the ways in which instrumentalists interface with the instrument. Likewise, players interface with games such as Guitar Hero and Rock Band using instrument-shaped controllers (or controllers loosely based on “real” instruments). Games such as Sound Shapes (Sony Computer Entertainment 2012) utilize the standard console controller. Dance Dance Revolution employs a square dance pad controlled with the player’s feet, and other music games, such as TouchMix FX (Gamevil 2013), operate based on the same principle, except players use their thumbs to interact with the game using the touch screen of their mobile device. Beyond the physical, haptic interfaces of the game, music video games feature various forms of interactivity that divide this genre further into subgenres. These forms of interactivity can be boiled down simply to matching, making, mixing, and metonymy. While a case can be made for including a particular music game in several of these categories, each provides its own distinct mode of player interaction. The most well-known subgenres of music games are examples of matching games. Rhythm-matching games (e.g., DDR, Guitar Hero, Rock Band) require players to perform a particular action to the rhythm of the music they hear, and pitch-matching games require players to react to changes in pitch. For example, players of SingStar (Sony Computer Entertainment Europe 2004) or Karaoke Revolution (Konami 2003) sing along with a popular song into a physical microphone; the game then analyzes the pitch of each note that the

INTRODUCTION—TAKING NOTE OF MUSIC GAMES

13

player sings, and a player succeeds by correctly matching pitch with the original song. Title-matching or artist-matching games, such as SongPop (FreshPlanet 2012), are quiz-type games that ask players to race others in hearing a short music clip and correctly matching it to its title or performing artist.1 Mixing games allow players to arrange pre-packaged sounds in a particular way to create music. In the webgame Incredibox (So Far So Good 2009), players drag and drop musical samples onto the t-shirts of cartoon characters to create a cappella beatboxing band, and certain combinations of sounds unlock bonus sounds. Much like Incredibox, other games, such as Chiptune Runner, are also sequencers of sorts in which the arranging and mixing of sounds is pivotal to gameplay. Musical-making games allow players to actually compose music of their own (even if only to a limited degree). As Stephanie Lind discusses in her chapter in this volume, players of The Legend of Zelda: Ocarina of Time (Nintendo 1998) write their own short melody to be played by Link on his ocarina when he needs to summon the scarecrow, Bonooru, at integral points within the game. While this compositional freedom is rare among video games, there are some “sandbox-type” video games that facilitate composition with fewer constraints. Dana Plank discusses the participatory practice of creating music using Mario Paint Composer (Unfungames.com 2007) in her chapter, “Mario Paint Composer and the Musical (Re)Play on YouTube.” Metonymy is a figure of speech in which someone or something is referred to using a related concept or other correlated object or person. For example, “the White House” can be used to refer to the President of the United States or his or her administration and “the Pentagon” will often refer to the United States military. Metonymic music games are symbolically musical because their gameplay is not expressly musical but rather alludes to musicians, music making, or the music industry. In her chapter “Beat It! Playing the King of Pop in Video Games,” Melanie Fritsch discusses video games that feature pop icon Michael Jackson; many consider these games to be music games even though the mechanics or rules of the games include musical activities. Instead, Michael Jackson symbolically stands in as a metonymy for music. In the serious game Music Inc. (UK Music

14

MUSIC VIDEO GAMES

and the Intellectual Property Office 2014), players manage a record label, deciding how much money he or she spends on publicity, branding, stylists, etc. The point of the game is to teach players about the effects of illegal file-sharing on the music industry—a lesson that players quickly learn when they invest lots of time and in-game money to produce a record only to have it pirated. Although gameplay only really involves making strategic business decisions, it is considered a music game because of its thematic subject matter. Emerging video games, platforms, and control interfaces will continue to inspire, or necessitate, new taxonomies and systems of video game classification, so the genres and subgenres proposed here are not meant to encompass any and all possible iterations of the music video game. The values inherent in classification systems continue to be scrutinized as value systems evolve and influence the ways in which music is experienced, composed, and studied in video games. So if music video games really are instruments, how do we play them?

Performance I will never forget the first time I saw someone play the rhythm arcade game Dance Dance Revolution. I grew up in a small town in the Texas Panhandle, so the only opportunity to go to an arcade was on high school trips to Amarillo during the inevitable lunch break at Westgate Mall, where “Tilt” (the mall’s arcade) was strategically located in the food court. My classmates and I stood in awe as teenagers (who probably should have been in class at that particular time of day) played the game. Not only did they stomp their feet on the “Step Zone” when the corresponding arrow scrolled by on the screen, they did so with gusto, inserting occasional claps, shouts, and impressive upper-body dance moves. And as the crowd grew more excited, the players responded with even more remarkable feats, dancing flawlessly to levels with a higher tempo and even adding an occasional flip over the safety bar that was mounted behind them. Although there is a now home version of DDR, this particular music game was designed for audiences. We had previously seen people who were remarkably good at playing pool,

INTRODUCTION—TAKING NOTE OF MUSIC GAMES

15

or pinball, or Street Fighter (Capcom 1987), or even the ubiquitous claw machine game, but never before had we seen anyone perform a game before and certainly not with that level of virtuosity. In fact, some of my friends were embarrassed to even be seen in the mall’s arcade, but these guys had come to put on a show with their acrobatic and energetic dance moves, hoping to draw an audience from the crowd of field trippers and mall walkers who happened to pass by. DDR, Rock Band, and Guitar Hero competitions sometimes fill stadiums with onlookers, proving that the performative aspects of gameplay—and even the audience—are fundamental elements of some music games. Kiri Miller reminds us that playing music or playing music video games is a collaborative activity, and we often play along with others. In playing games, she writes that we: … build on (and build up) relationships among game designers, players, choreographers, dancers, writers, composers, directors, performers, and audiences, as well as marketers, publishers, and other commercial mediators… each player collaborates with the game designers to turn code into a virtual performance, while remaining aware that millions of other players have engaged in the same endeavor. (Miller 2012, 5) Even when no one else is playing a music game with us, we are still part of an ensemble. And not only is playing a music game a cooperative undertaking among its players and those who created and sold it, it can most certainly be a performative experience that depends upon the development of a meaningful relationship between the player and the audience gathered to watch him or her play and with the original performers of the song used in the music game. Many games are designed to facilitate performance and, as with role-playing games, help players assume and perform new personas. Karen Collins suggests that “games themselves have become the media through which music is expressed. They provide the visual context of performative expression in the case of virtual performances, the lyrical content in the case of covers and filk songs, and the instruments in the case of chiptunes and software and hardware hacking. This use of games as musical instrument is

16

MUSIC VIDEO GAMES

leading to new innovations in nongaming musical practice” (Collins 2013, 120).2 She also explains that these performances challenge our notions of live and recorded, real and virtual, authentic and fake: Games and virtual worlds have helped lead to new forms of sonic performance that questions notions of liveness and authenticity. Players engage in a variety of simultaneous performative activities that become a form of play. Performance in itself, in other words, becomes a playful part of the game experience. Through promoting performance, these types of games strengthen the role-playing aspects of the game, encouraging a kind of rock performative excess and rewarding the player for these gestures (raising the guitar to enter Star Mode and so on). Shy players may in this way mimic the kinds of performative activities that they witness popular stars enact and though this physical mimicry may be able to live that performance mentally. (Collins 2013, 66) Kiri Miller also addresses this disjunction between the real and virtual in what she calls a “schizophonic performance” (Miller 2012, 85–6). Borrowing the word schizophonic, a term that refers to the peculiarity and confusion that can possibly be created when we hear a recorded sound that has been separated from its original source, from Canadian composer and acoustic ecologist R. Murray Schafer, Miller applies this term to performances to describe the uneasiness some feel when experiencing a performance in Guitar Hero or Rock Band, especially when visual or sonic cues do not match expectations. They may hear Blue Öyster Cult playing “(Don’t Fear) The Reaper,” but rather than seeing images of the band members performing the song as one might expect, they see generic avatars performing instead. They also hear the addition of sampled cheers from an artificial crowd and other extra sounds from the game itself that alert the player each time that he or she has earned bonuses or made mistakes during gameplay. Although the world of popular music has a long history of sifting out the real from the “fake,” authentic performances within video games are even harder to qualify. What constitutes an authentic musical performance in a video game? Does it even matter?

INTRODUCTION—TAKING NOTE OF MUSIC GAMES

17

Reading the score and taking score The aim of this volume is to address these and many other questions that are important to the appreciation and understanding of the genre of music video games. It is divided into three sections: Preludes and Overtures; Virtuosi, Virtues, and the Virtual; and Concerts, Collaboration, and Creativity. Because the authors and their analytical methods come from a wide variety of musical and non-musical backgrounds—from media studies and communications scholars to musicologists, music theorists, and practicing musicians, and avid gamers to the occasional casual player—each interdisciplinary section includes a wide array of chapters that speak to a particular overarching theme or controversy surrounding music video games.

Preludes and overtures The chapters in this first section of this volume deal with the prehistory of music games, various forms in which they are currently found, and ways in which people extend the use of these games to create, enjoy, and distribute music. In the volume’s first chapter, William Knoblauch examines what is arguably the first music game: SIMON (Milton Bradley 1978), a handheld music memory game made popular in the late part of the twentieth century. Dana M. Plank discusses the development of Mario Paint Composer, the rise of virtuosic players (or better yet, programmers), and the musical praxes that surround the game regarding composition and online distribution in the form of YouTube videos. She addresses issues of emerging performance practice, participatory culture, online networks, and fan labor surrounding games in the Mario Paint series. Though her discussion of The Legend of Zelda: Ocarina of Time, Stephanie Lind discusses ways in which a player experiences—and performs—the game’s music and considers ways that music and music-making practices influence the game’s plot, mechanics, and the player’s immersion within the game. In the last chapter in this section, I draw connections between music games and musical instruments, focusing particularly on games in which music sampling and

18

MUSIC VIDEO GAMES

sequencing are central to the game’s mechanics, level design, and overall experience.

Virtuosi, virtues, and the virtual Video games serve a much larger purpose than to simply function as entertainment. Although it is expected from the most serious and violent of video games, even the most benign and fun-toplay video game instills ideologies, prejudices, and other values within players and audiences through the procedural rhetoric they present. The chapters in this section focus on the axiology of music video games. In “Consumerism Hero: The ‘Selling Out’ of Guitar Hero and Rock Band,” Mario Dozal investigates notions of authenticity and authentic performance, discussing ways in which these values may or may not translate well from the real world into video games. Melanie Fritsch introduces games that are musical through representation, focusing particular attention on games that are musical because they include the image and likeness of Michael Jackson. She considers the aspects of Jackson’s career that have had the greatest influence on these games—his music, his life, or his legacy—and ways in which Jackson’s public persona and controversial lifestyle affect the reception of games that use his image. In his chapter, David Arditi investigates the political economy of music video games by discussing ways in which they are used to appropriate, commodify, and consume music in contemporary culture.

Concerts, collaboration, and creativity The focus of the final section of this volume is on pedagogy and using video games as a means through which music can be taught and learned. In “Guitar Heroes in the Classroom: The Creative Potential of Music-Games,” David Roesnser, Anna Paisley, and Gianna Cassidy share the initial findings from the first phase of their study on teaching music to and cultivating creativity within children by introducing music video games into primary school curricula. Daniel O’Meara

INTRODUCTION—TAKING NOTE OF MUSIC GAMES

19

examines the use-value of Rocksmith in learning guitar by discussing the game’s reception history, mechanics, idiosyncrasies, and intrinsic approach to guitar pedagogy though level design. Peter Shultz introduces the notion of “rhythm sense” in games in the Rhythm Heaven (Nintendo) series. In his chapter, Shultz describes the ways in which players of Rhythm Heaven (Nintendo 2008) are instilled with a sense of rhythm through playing various game levels and interacting with other elements of the game. Finally, Nathan Fleshner offers practical suggestions (and their theoretical underpinnings) about the ways in which music games can be introduced into the music theory classroom.

Glossary of music and gaming terms This volume is designed to be equally useful to scholars, students, and players of both music and games alike. We also understand that music video games lie at the intersection of several academic fields that have highly specialized vocabularies that do not often overlap, and when they do, the same term can mean something completely different depending upon the disciplinary context in which it is used. Included at the end of this volume is an interdisciplinary glossary of music and gaming terms to assist readers in becoming more familiar with the specialized concepts, industry jargon, and conventional (and non-conventional) terminology associated with music video games. Whether this is your first experience with music video games or you are a seasoned maestro, we hope this volume will “hit all the right notes” as you engage with each chapter and the ideas presented therein.

Notes 1 Because these games are often social games that appear on Facebook or via mobile smartphone app, this “race” is sometimes performed asynchronistically. So, rather than playing the game in real-time with another player, a user wins by beating the recorded time of his or her challenger.

20

MUSIC VIDEO GAMES

2 Filk songs are folk songs based on fantasy or science fiction themes. They are a type of fan labor of which video games characters or tropes are sometimes the topic. Filk songs are often performed in “filk circles” (much like drum circles) that are formed at conventions and also challenge notions of authenticity.

References Bogost, Ian. 2009. “Video Games Are a Mess.” Last modified September 3. http://bogost.com/writing/videogames_are_a_mess/. Bogost, Ian. 2011. How to Do Things with Videogames. Minneapolis, MN: University of Minnesota Press. Caillois, Roger. 2001 [1958]. Man, Play and Games. Translated by Meyer Barash. Urbana, IL and Chicago, IL: University of Illinois Press. Collins, Karen. 2013. Playing with Sound: A Theory of Interacting with Sound and Music in Video Games. Cambridge, MA: The MIT Press. Fernández-Vara, Clara. 2015. Introduction to Game Analysis. New York: Routledge, 107. Gadamer, Hans-Georg. 2013 [1975]. Truth and Method. Revised translation by Joel Weinsheimer and Donald G. Marshall. New York and London: Bloomsbury Academic Press, 107. Hewett, Ivan. 2011. “Guitar Hero Rocks the Classical World.” The Telegraph. December 15, 2011. Accessed June 5, 2015. http://www. telegraph.co.uk/culture/music/classicalmusic/8956886/Guitar-Herorocks-the-classical-world.html. Huizinga, Johan. 1938. Homo Ludens. London: Routledge. Juul, Jasper. 2011. Half-Real: Video Games between Real Rules and Fictional Worlds. Cambridge, MA: The MIT Press. Lévi-Strauss, Claude. 1966. The Savage Mind. Chicago: University of Chicago Press. Malloch, Joseph, et al. 2006. “Towards a New Conceptual Framework for Digital Musical Instruments.” In Proceedings of the 9th International Conference on Digital Audio Effects (DAFx-06), Montreal, Canada (September 18–20, 2006). Miller, Kiri. 2012. Playing Along: Digital Games, YouTube, and Virtual Performance. New York and London: Oxford University Press. Myers, David. 1990. “Computer Game Genres.” Play & Culture 3: Fictional Worlds 3: 286–301. Reale, Steven Beverburg. 2014. “Transcribing Musical Worlds; or, Is L.A. Noire a Music Game?” In Music in Video Games: Studying Play, edited by K.J. Donnelly, William Gibbons, and Neil Lerner, 77–103. New York: Routledge.

INTRODUCTION—TAKING NOTE OF MUSIC GAMES

21

Saltz, David. 1997. “The Art of Interaction: Interactivity, Performativity, and Computers.” Journal of Aesthetics and Art Criticism 55 (2): 117–27. Stam, Robert. 2000. Film Theory: An Introduction. Malden, MA: Blackwell Publishers, Inc. Voorhees, Gerald A., Josh Call, and Katie Whitlock. 2012. “Series Introduction—Genre and Disciplinarity in the Study of Games.” In Dungeons, Dragons, and Digital Denizens: The Digital Role-Playing Game, edited by Gerald A. Voorhees, Joshua Call, and Katie Whitlock. 1–10. New York: Continuum Press. Wolf, Mark J.P. 1997. “Inventing Space: Toward a Taxonomy of On- and Off-Screen Space in Video Games.” Film Quarterly 51 (1): 11–23.

Games Blitz Games. Karaoke Revolution. [Xbox 360, et al.]. Konami: Tokyo, 2003. Blizzard Entertainment. World of Warcraft. [PC]. Blizzard Entertainment: Austin, 2004. Capcom. Street Fighter. [Arcade]. Capcom: Osaka, 1987. Evil Indie Games. Chiptune Runner. [iOS/Android]. Evil Indie Games: 2013. FreshPlanet. SongPop. [iOS, et al.] FreshPlanet: New York, 2012. GAMEVIL Inc. TouchMix FX. [Android]. GAMEVIL Inc.: Seoul, 2013. Harmonix. Guitar Hero. [PlayStation 2]. RedOctane: Mountain View, CA, 2005. Harmonix. Guitar Hero 2. [PlayStation 2/Xbox 360]. Activision: Santa Monica, CA, 2006. Harmonix, et al. Rock Band. [Xbox 360, et al.]. MTV Games/Electronic Arts: New York, 2007. Konami. Dance Dance Revolution. [Arcade, et al.]. Konami: Tokyo, 1998. Milton Bradley. SIMON [Handheld Game]. East Longmeadow, MA, 1978. Nintendo. Wii Tennis [Nintendo Wii]. Nintendo: Kyoto, 2006. Nintendo EAD. Legend of Zelda: Ocarina of Time. [Nintendo 64, et al.]. Nintendo: Kyoto, 1998. Nintendo SPD/TNX Music Recordings. Rhythm Heaven. [Nintendo DS]. Nintendo: Kyoto, 2008. Pajitnov, Alexey, Nintendo Research & Development 1, et al. Tetris. [NES, et al.] Nintendo, et al.: Moscow, 1984. Queasy Games/SCE Santa Monica Studios. Sound Shapes. [PlayStation 3, et al.]. Sony Computer Entertainment: Tokyo, 2012.

22

MUSIC VIDEO GAMES

Realta Entertainment Group. BandFuse: Rock Legends. [Xbox 360/ PlayStation 3]. Mastiff: Tokyo and San Francisco, 2013. SCE London Studio. SingStar. [PlayStation 3/ PlayStation 4]. Sony Computer Entertainment: Tokyo, 2004. Sega. Sonic the Hedgehog. [Sega Genesis]. Sega: Tokyo, 1991. So Far So Good. Incredibox. [Microsoft Windows/OS X]. So Far So Good: Saint-Étienne, 2009. Square. Final Fantasy. [NES]. Square: Tokyo, 1987. Ubisoft Montpellier. Rayman Legends. [Microsoft Windows, et al.]. Ubisoft: Montreuil, 2013. Ubisoft San Francisco. Rocksmith. [Xbox 360, et al.]. Ubisoft: Montreuil, 2013. UK Music. Music Inc. [iOS/Android]. Aardman: Bristol, 2014. Unfungames.com. Mario Paint Composer [Super Famicom/SNES]. Unfungames.com:1992.

PART ONE

Preludes and Overtures

1 SIMON: The Prelude to Modern Music Video Games William Knoblauch

B

efore Dance Dance Revolution and Karaoke Revolution, before Rock Band or Guitar Hero, before Wii Music (Nintendo 2008), Mario Paint Composer, or Miracle Piano Teaching System (The Software Toolworks 1990), before any digital music games—there was SIMON. Created by Ralph Baer, manufactured by Milton Bradley, and played by millions, SIMON is one of the most successful handheld electronic games of all time. A combination of intuitive design, challenging gameplay, and broad appeal for players of all ages makes SIMON more than a retro handheld game. SIMON paved the way for numerous electronic music games. It thrived in the late 1970s, an age of emerging video game consoles; survived the early 1980s, years featuring SIMON clones and brain game fads; and established conventions that late 1990s games adopted and which live on in today’s music video game era. More than an industry history of SIMON, this chapter examines how one game set the stage for a generation of music memory games. SIMON remained relevant over four decades, enduring shifts The author would like to thank Nathan Bean, Marty Goldberg, and Anna Seidel-Quast for their comments and help with this chapter. Photo credit to Lucas Brehm for the SIMON image; thanks also to research assistant Shaela Morin.

26

MUSIC VIDEO GAMES

in American gaming technology and popular culture. From SIMON ’s development and distribution in the late 1970s, the game’s rise to cultural prominence in the 1980s, and its continued marketability into the 1990s and beyond, SIMON has a longevity seldom seen in a highly competitive and rapidly changing gaming industry. Even today, thirty-seven years after its invention, SIMON continues to challenge and delight gamers. This is the story of SIMON.

Ralph Baer: The father of video games and inventor of SIMON SIMON was the brainchild of Ralph Baer. Considered by many to be the “father of video games,” Baer was born in Germany in 1922. He was a fourteen-year-old student in Cologne when Hitler came to power (Garrido 2011). Like many Jewish citizens who faced Nazi persecution, Baer and his family fled increasingly violent Nazi pogroms (government-sponsored persecutions). Many Jewish immigrants fortunate enough to cross the Atlantic greatly impacted American science, technology, and culture. Some contributed to the Manhattan Project—the program that designed and detonated the first atomic bomb. Others found work in the growing post-war Military Industrial Complex. In places like the Brookhaven National Laboratory, the Massachusetts Institute of Technology (MIT), and the RAND Corporation, they worked on Cold War defense initiatives. It was also in these labs where the earliest video game prototypes were designed and tested. After emigrating to the United States in 1938, Baer worked long hours in a factory job before being drafted into the U.S. Army in 1943, where he became an intelligence officer. After the war, Baer used his G.I. Bill funding to earn a degree in television engineering from Chicago’s American Television Institute of Technology. In 1966, he became chief engineer at Sanders Associates, a military electronics firm in Nashua, New Hampshire. It was here that Baer helped to design interfaces that could interact with television sets—a major breakthrough in video game history (Burnham 2001, 55). Baer was perhaps the first to realize TV’s gaming potential. He recalls: “It wasn’t

SIMON: THE PRELUDE TO MODERN MUSIC VIDEO GAMES

27

until I came along and recognized that there were already over 80 million TV sets in US homes—and that they might be used as displays for playing games—that the concept of home videogames became a practical reality” (Burnham 2001, 17). Soon, Baer was working for Chicago-based toy manufacturer Marvin Glass & Associates while continuing to consult with bigger companies like Coleco and Magnavox, all of which were gaining interest in electronic gaming. Baer’s break came at Magnavox, where he designed his “game box” (later labeled the “Brown Box”) prototype in 1969. The unit was revolutionary. By using two dials to control the path of a digitized dot on a television set, Baer had created the first commercially viable game interface. The distinction is important. Some game historians have asserted that video games were byproducts of America’s Cold War defense initiatives. To be sure, governmentsponsored scientists had already achieved tool-controlled pixilation: at Brookhaven, physicist William Higinbotham designed Tennis for Two (William Higinbotham 1958), a precursor to Pong (Atari Inc. 1972, arcade version); at MIT, bored electronic engineers created the proto-shooter Spacewar! (Steve Russell et al. 1962) (Van Burnham, 28–42). Like these pioneers, Baer did benefit from some Cold War funding while working as a defense contractor; however, he saw little connection between military applications and the birth of the video game industry. Instead, he cited his independently designed “Brown Box” (Model #ITL200) as the first commercially viable video game prototype.1 In 1972, Magnavox turned Baer’s “Brown Box” into the Odyssey, the world’s first home video game console. The device seems primitive by today’s standards. Players simply controlled two moveable dots on a blank screen. Odyssey’s graphics were essentially non-existent, so the system came with television overlays to entice the imagination. Another Baer breakthrough, a “light gun” that interacted with television screens, was equally innovative. Despite a considerable advertising campaign, Odyssey did not live up to Magnavox’s expectations—a failure Baer blames on the company’s marketing strategy.2 The Odyssey was soon overshadowed by competitor Nolan Bushnell’s new game, Pong (Atari Inc. 1974, home version), and his company, Atari. All told, the Odyssey sold modestly—200,000 units by some accounts, 330,000

28

MUSIC VIDEO GAMES

by others—but Atari dominated the market, first with Pong, and then with their “Video Computer System” (VCS, or model 2600). Magnavox discontinued the original Odyssey in 1975, replacing it with the Odyssey 2. Magnavox later sued Bushnell’s Atari Corporation over copyright infringement, winning a $1.5 million lawsuit. In the years that followed, Magnavox had similar litigation successes with other game corporations, netting over $100 million.3 Whatever its initial sales, Odyssey’s impact was huge in that it expanded the commercial applications of a ubiquitous technology: television. By the 1970s, network TV had become the most common form of home entertainment. As media scholar James Baughman notes, “no technological innovation before or since—not newspapers, the telephone, radio, cable television, personal computers, or even indoor plumbing—achieved such overwhelming popularity in so short a span.” When Baer conceived of the Odyssey, close to 90 percent of Americans owned televisions. But high numbers did not mean high quality, and by the late 1950s television was a medium drowning in mediocrity (Baughman 2007, 1–3). Once thought of as a device to cultivate high culture, TV’s reliance on quiz shows and soap operas dispelled such hopes. In 1961, Federal Communications Commission president Newton N. Minow assessed TV airwaves as a “vast wasteland” of mindless entertainment (Watson 1990, 18–35). It was Baer’s new device that turned this vast wasteland into fertile new ground for entertainment. The Odyssey made the video game era possible.

SIMON and handheld electronic gaming SIMON  ’s roots trace back to November of 1976, when Baer attended the Music Operators of America show in Chicago and first encountered an Atari Touch Me cabinet (Atari Inc. 1974). Initially released in June 1974, Touch Me was a “waist-high [42 inch] cabinet with four large, dark ‘buttons’ facing the player on its top.” It was on this “nearly horizontal surface” that four buttons “lit up in random sequences. It was the player’s job to follow the light sequence by pressing the appropriate buttons.” Baer liked the idea but hated the

SIMON: THE PRELUDE TO MODERN MUSIC VIDEO GAMES

29

execution. “The machine issued truly awful, raucous accompanying sounds,” he recalled. Still, Touch Me made an impression. Baer later discussed the game with Howard Morrison, a partner at Marvin Glass & Associates. Together, they decided that the concept for a music memory game was good, but that Touch Me was “visually boring” and sounded “miserable.” By 1977, they had designed a similar game entitled “Follow-Me.” It differed from Touch Me in that it had better sound and replaced the mono-lit lights with four different colors (Baer 2005, 108–9). Back at his Nashua lab, Baer collaborated with a young software engineer, Lenny Cope, to transfer this music memory game idea onto a microchip. The two designers settled on a Texas Instruments’ TMS1000 microprocessor but technical limitations slowed development. As Cope plodded away on programming code, Baer designed the physical prototype. What he most wanted to avoid was Touch Me’s annoying noises. Baer settled on a four-tone scale after referencing his child’s Compton Encyclopedia, which explained that the Bugle could play numerous songs with only four tones. He picked G, C, E, and G—essentially the tones that make up a C-major chord with an extra G below the root of the chord—to synch with the game’s flashing lights.4 When Baer presented his prototype to Milton Bradley, reps liked what they saw but suggested some changes. Initially titled “Feedback,” Milton Bradley reps suggested changing the game’s name to SIMON, shorthand for the command and response children’s game “Simon Says.” One top-level assistant, Dorothy Wooster, demanded more game features to raise replay value. It was a request with precedent. Many early electronic handheld games, and even video game consoles, featured “difficulty” toggle switches to increase challenge and extend replay value; in time, SIMON ’s variable skill levels became one of the toy’s defining features. Finally, Baer’s square prototype was changed to a smooth and rounded saucer-like design featuring four flashing buttons colored red, blue, green, and yellow. With these final flourishes, SIMON ’s design was complete. The game’s patent was filed in July 1977, and SIMON was officially issued U.S. patent No. 4,207,087 in June 1980 (Gershman 2003, 9).

30

MUSIC VIDEO GAMES

FIGURE 1.1  An original model SIMON game. SIMON® & ©2015 Hasbro, Inc. Used with permission.

SIMON ’s early notoriety came from some curious, and at times unexpected, publicity. Milton Bradley first marketed SIMON as a futuristic game with flare, an image they promoted by premiering the game at New York’s famed nightclub, Studio 54. America’s disco craze—forever captured in the 1977 film Saturday Night Fever— remains an iconic part of 1970s culture. Studio 54 was the king of all discotheques, an exclusive club featuring famous Hollywood clientele and flaunting excess and drug use (a gigantic drug spoon adorned the dancefloor’s ceiling). Milton Bradley’s publicity stunt worked. With a four-foot diameter model of the game floating above Studio 54’s dance floor, its giant buttons flashing with the music, SIMON seemed tailor-made for a Disco lightshow.5 SIMON got some more

SIMON: THE PRELUDE TO MODERN MUSIC VIDEO GAMES

31

publicity later that year in director Stephen Spielberg’s film Close Encounters of the Third Kind. The film, which follows extraterrestrials’ interactions with humans, ends with an alien craft that communicates not with words, but though a five-note tone sequence. One extra tone didn’t dispel comparisons. Baer himself recalled that Spielberg’s “spaceship (a round saucer) looked for all the world like a big SIMON, or vice versa; and it communicated with the earthlings by emitting sounds, a sequence of tones that also resembled SIMON ’s… talk about serendipity!” (Baer 2005, 172–3). Such publicity helped Milton Bradley sell an estimated 750,000 SIMON units at roughly $20–$40 per unit (Gershman 2003, 9). This curious combination of celebrity discotheques and alien spacecraft certainly helped promote Baer’s game, but the secret to SIMON’s success was its broad appeal. Whether playing alone, with a friend, or at a party, gamers young and old could enjoy SIMON. Jack Fox, then-Mattel’s director of marketing and public relations, knew that adults purchasing electronic toys for their kids often “end up playing with them themselves.” He humbly recalled, “when I play [SIMON] against my son, he consistently beats me. It’s frustrating.”6 SIMON proved to be “the first electronic game that could be played by more than one person, and played at a party, and the four bright colors were visually appealing.”7 Furthermore, it was “challenging without being frustrating, and could be played at different levels of difficulty.” Such high praise suggests SIMON ’s appeal across the generation gap and to gamers of different skill levels (Harmetz 1981). In addition to changing difficulty settings, SIMON ’s toggle switches allowed for three different types of games. In Game I, “Simon Says,” players repeat an arithmetically increasing sound and light pattern; this is the iteration most commonly associated with SIMON. In Game II, “Player Adds,” gamers select “what signals to play and the order in which you want to play them.” In other words, you create a unique pattern that arithmetically increased each turn. Game III, “Choose Your Color,” was a “round robin” battle for two, three, or four players, with each person playing their own Game II but passing SIMON on after each turn. No matter the game, SIMON ’s “Razz” tone informed players when they misremembered a pattern. It was a lot of variety for a game with only four buttons. With difficulty levels of eight,

32

MUSIC VIDEO GAMES

fourteen,  twenty, and thirty-one randomized patterns, SIMON had considerable replay value (“SIMON Instruction Manual” 1979, 4–6). This combination of broad appeal and replay value helped SIMON achieve success, but in an age of comparatively expensive video game consoles, its low price tag also didn’t hurt. Price competitiveness helped SIMON become the hit toy of the 1978 and 1979 holiday seasons. Frequently, new shipments of the game sold out at retailers, such as Toys-R-Us, on the same day (Bruske 1978). “We’ve never experienced anything like this,” commented Milton Bradley’s George Ditomassi on the 1978 shopping season. “It started out as a blessing but now it’s driving me crazy. We allocate these units, and people hear that there are 15 at a particular store and they show up half an hour before the place opens and run to the toy department and get into a brawl over who’s going to get them” (Zito 1978). Consumers in Los Angeles purchased damaged SIMON units at full price just to have it under their Christmas trees (Harmetz 1981). “SIMON has been sold out at Toys-R-Us for more than a month,” warned Washington Post’s business reporters during the run up to the last Christmas of the decade (Knight and Wright 1979). SIMON spurred a handheldelectronic-game boom that was outpacing video game console sales. Newsweek took note, featuring SIMON alongside another handheld game, Merlin (Parker Brothers 1978), on its December 11, 1978 cover. SIMON ’s success prompted numerous imitators. Atari re-released Touch Me, the failed music memory game that first inspired Baer, this time as a handheld game that looked and sounded remarkably similar to SIMON. So did the Sears’ game Follow Me (Sears 1979), which promised a SIMON-like experience at half the price. The imitation was so obvious that Follow Me advertisements admitted that it was not “the first or the only handheld computer game that challenges your memory and powers of deduction,” and that “there are others on the market that play virtually the same way.”8 Tiger Electronics embraced its SIMON plagiarism with the aptly named 1979 handheld Copycat (Tiger Electronics 1979), which also featured four colored buttons, a round (octagon) disc shape, and prominent toggle switches. The Ideal Toy Company even recruited Baer for a true SIMON follow-up, the 1979 toy Maniac (Ideal Toy Company 1979), a far more challenging handheld game that combined sound, rhythm, melody, and numbers to challenge players’ memory. Baer later admitted that perhaps

SIMON: THE PRELUDE TO MODERN MUSIC VIDEO GAMES

33

Maniac was too difficult, stating: “you have to want to play games to want to play Maniac.”9 Faced with these imitators, Milton Bradley offered two true SIMON follow-ups: Super SIMON (Milton Bradley 1979), which extended gameplay by doubling the buttons (eight instead of four), and a portable Pocket SIMON (Milton Bradley 1980), which appeared in 1980 (Blake 1980). Even during this handheld electronic game surge, late 1970s toy industry leaders recognized the video game market’s potential and continued to release home consoles. Mattel’s Intellivision, Texas Instruments’ TI/99, and Milton Bradley’s Vetrex boasted innovative graphics and more games; the Fairchild Channel F, the Sinclair ZX81, the BBC Micro, Colecovision, and the Commodore 64 all battled Atari for market share. Milton Bradley even hybridized portable and console technologies with the Microvision, the first handheld game system with swappable cartridges—and with a price tag of $30 it was far less expensive than home consoles (Zito 1979). Japanese playing card company Nintendo offered its own series of affordable handheld games with its Game & Watch series. These handhelds aimed to mimic SIMON ’s success, to carve out a market share with an inexpensive alternative to high-priced consoles. The Game & Watch series was the brainchild of Gunpei Yokoi, a Nintendo toy designer who hatched the idea during a morning commute where he watched one bored passenger playing with a pocket calculator. In many ways, Yokoi’s designs were similar to Baer’s in that both developers embraced inexpensive, readily available technology. For Baer, a trained electronics engineer, game design did not rely on new and expensive emerging technologies but on creatively utilizing readily available parts and techniques. Yokoi designed games that could be made with inexpensive and abundant calculator parts, all at a fraction of the cost needed for console development. Yokoi labeled this design philosophy “Lateral Thinking of Withered Technology,” or the use of “mature technology in novel or radical applications.” It was an approach realized in his Game & Watch series, especially in the adoption of Liquid Crystal Displays (LCDs), a technology that was both “well-understood” and “relatively inexpensive” (McFarran 2014, 27–8). Low production costs meant lower price tags, and for thrifty shoppers seeking holiday gifts in the recession plagued early 1980s, handheld games stood out.

34

MUSIC VIDEO GAMES

Despite the recession, SIMON continued to sell. It still retailed for under $40, making it a popular and relatively inexpensive alternative to high-priced consoles. By 1981, three years after the ’78 handheld boom, sales of SIMON, Super-SIMON, and Pocket SIMON were projected at 1.5 million units (Salmans 1981). It was an impressing showing, but could SIMON survive in such a rapidly changing game landscape?

SIMON in the 1980s and beyond As the 1970s gave way to the 1980s, Americans embraced a resurgent Conservative movement, which flaunted conspicuous consumption, consumerism, and avarice. It was a decade of “Greed is Good,” of Wall Street’s young urban professionals, or Yuppies. It was also a breakthrough decade for the computer industry. What began in the mid-1970s from a few upstarts—notably Bill Gates’ Micro-soft (later Microsoft) and Steve Jobs’ Apple Computer—had by the 1980s invigorated the economy with a personal computer boom. These undercurrents of Wall Street excess and Silicon Valley savvy led to a curious embrace of “nerd” culture. Gone were the good intentions of 1960s Baby Boomers and 1970s disco-era narcissism. In the 1980s, American pop culture embraced big brains and big bucks. In popular music, Thomas Dolby had a nerdy electro-pop hit with “She Blinded Me with Science,” while Timbuk 3 bragged about getting good grades in their song “The Future’s So Bright, I Gotta Wear Shades.” On the big screen, viewers cheered on the bespectacled underdogs in Revenge of the Nerds, well-intentioned MIT students in Real Genius, and hormonally driven geeks in Weird Science. On television, the improvised inventions of MacGuyver and antics of wunderkind Doogie Howser, M.D. entertained millions. In the 1980s, you could be both brainy and badass. 1980s “nerd chic” evolved, in part, from a 1970s gaming culture that challenged player intelligence. The mid-1970s analog puzzler Rubix Cube (Rubiks Brand, Ltd. 1977), for example, became a sort of an informal intelligence test—something to be played, tested, and mastered for casual (and later, formal) competitions. In the electronic

SIMON: THE PRELUDE TO MODERN MUSIC VIDEO GAMES

35

toy market, numerous learning and brain games promoted fun with a purpose. There was 2-XL, a talking robot that ran on 8-Track cassette tapes and offered courses in math, history, and general information. Texas Instruments’ Speak & Spell challenged children with games to improve spelling and pronunciation skills, while their Little Professor (TI 1976) and Dataman (TI 1977) games presented math equations. The Parker Bros.’s handheld Merlin and Lakeside Toy Company’s Computer Perfection (Lakeside Toy Co. 1979) both tested player memory skills. In the late 1970s, “brain games” garnered big bucks.10 SIMON fit into this “brain game” fad. 1980s ads boasted the machine’s ability to set players apart from their less-skilled competitors. Print ads presented SIMON as a measure of intelligence with taglines such as: “Test your concentration with this challenging game.”11 TV ads provided similar messages. In one European ad, a group of adults play SIMON at a party in a friendly competition of wits. An American commercial showed a group of pre-teens gathered around a showdown between a leather-jacket clad “Johnny” and SIMON. A classmate warns “nobody beats SIMON.” Unfazed, Johnny hands his leather jacket to a girl while a narrator explains: “SIMON is the challenge you’ve been waiting for, because it takes coordination of hand and mind just to play the game. And if you get very good at SIMON, great rewards await you.” When Johnny wins, the crowd erupts with approval and the young girl follows him home. The message was clear: If you were “smarter” than SIMON, you might win adoration from friends and peers.12 The idea that electronic games could improve brainpower gained some credibility in the 1990s. “Games Improve Brainpower,” one 1994 Calgary Herald headline read, listing SIMON as a prime example (Craven 1994). Later that decade, American pizza chain Little Caesars created their own coin-operated SIMON-like memory game, the Little Caesar’s Brain Teaser, which featured four buttons colored just like SIMON (red, yellow, green, and blue) and replicated the game’s mechanics. This in-store unit promised to test your “memory retention level” in those minutes before your pizza (or “Pizza Pizza”) was ready. Play well enough, and you might win an appetizer. Popular perceptions of gaming for brain power may have helped keep SIMON relevant in the 1980s and beyond. When toy industry giant Hasbro took over Milton Bradley in 1984, they gave SIMON a

36

MUSIC VIDEO GAMES

makeover, replacing the black plastic casing with a clear see-through plastic shell. The Pocket SIMON also got a redesign. Even into the 2000s, new iterations emerged. The SIMON Trickster (Milton Bradley 2005) incorporated LEDs and incorporated mini-games, such as “Simon Bounce” swapped color patterns mid-game, adding yet another element of challenge, “Simon Surprise,” which removed the color from the buttons (an ironic touch, as it replicated the colorless design of Atari’s Touch Me that Baer sought to avoid), and “Simon Rewind” reversed playback (now players had to play the pattern backwards). In 2012, SIMON Flash (Hasboro 2012) featured four moveable squares with lit LEDs to provide numerous game variations, including “Simon Lights Off,” which tasked players with moving blocks into prescribed patterns; of course, it also included “Classic Simon.” In 2014, Hasbro released SIMON Swipe (Hasbro 2014), which replaced buttons with swipe-able touchscreens. On the Apple iPad, numerous downloadable SIMON apps continue to challenge a new generation of players.13 While tech keeps changing, the challenges remain basically the same, much as they were over three decades ago when SIMON first appeared above the dance floor of Studio 54. Looking back, Baer’s game proved to be far more durable than disco.

SIMON’s legacy SIMON established numerous design and gameplay conventions that are still used in today’s music video games. First, consider Baer’s design choice of colorfully bright buttons, an element mimicked in peripherals for Dance Dance Revolution (or DDR) and Guitar Hero. In arcade versions of DDR, players dance atop a brightly lit four button pad—essentially a SIMON-like platform. In Guitar Hero, game controllers are modeled to look like smaller versions of real instruments, such as Fender Stratocasters or Gibson SG model guitars—except for the five brightly colored buttons on the fret boards. Second, like Baer, many of today’s music video game developers embrace Yokoi’s “Lateral Thinking of Withered Technology” philosophy for peripherals. Home console versions of DDR come packaged with a relatively inexpensive soft plastic floor control pad, one similar to the Nintendo “Power Pad,” a peripheral input device that dates back

SIMON: THE PRELUDE TO MODERN MUSIC VIDEO GAMES

37

in American markets to 1988. Similarly, Guitar Hero and Rock Band utilize withered technologies—their toy guitar, keyboards, and mic controllers come equipped with Universal Serial Bus (USB) inputs and Musical Instrument Digital Interfaces (MIDI), technologies that date back to the 1990s and 1970s (respectively). Baer, who believed that new technology was not necessary to achieve fun, would have applauded these cost-effective choices. “Never mind the graphics and all that jazz,” Baer advised in a 2013 interview; game designers should just “make games!” (Barton 2013, 134). Today’s music video games certainly look and sound more advanced than their 1970s ancestors, yet modern games still rely on SIMON-era conventions of difficulty toggling. In the early 1980s, arcade conventions of high score competition and difficulty switching were on the decline. As video games evolved, platformers like Super Mario Bros. or “sandbox” games such as Grand Theft Auto (BMG Interactive 1997) suggested a medium largely becoming teleological in nature—these were games in which players derive fun from exploring a world or progressing toward a finish point. Yet even recent music games, including DDR, Guitar Hero, and Rock Band, still rely on the handheld-era conventions of difficulty toggling, head-to-head challenges, and short-form gameplay. Like SIMON, modern music video games feature varying difficulty settings for broad appeal and to derive replay value from a variety of game modes. SIMON ’s longevity provides another lesson for the music video game market. The game’s success shows that in an industry of rapidly evolving technology, inexpensive toys can still nab market share. Back in 1983, Atari and its competitors had flooded American markets with too many games, and the bottom fell out of the market. The video game industry did not recover until the release of Nintendo’s Entertainment System in 1985. What followed was a cycle of console wars. In the 1990s, Sega’s 16-bit Genesis System battled the Super Nintendo; then the 32-bit Sega Saturn competed with Nintendo’s N64; Sony’s PlayStation and the Sega Saturn introduced CD-ROM technology. With each new console came brags of better graphics and smoother gameplay but also new surges in price. Yet throughout the decade, relatively inexpensive music games continued to sell. Consider the successes of Bop-It (Hasbro 1996), a reflexive handheld game, or Loopz (Mindscape 1990), a motion memory game—both big

38

MUSIC VIDEO GAMES

hits in the 1990s. In a video game market plagued by booms and busts, small and inexpensive handheld games still appeal to consumers. SIMON also contributed to the growing popularity of brain games. SIMON was a measure of eidetic skills, or the ability to recall images or sounds after only short-term exposure. Scholarship suggests that those early SIMON ads, which promoted the game’s brain enhancing abilities, were not without merit. In a 2006 edition of The Journal of Behavioral and Neuroscience Research, Mathew H. Gendle and Michael R. Ransom argue that SIMON “holds potential value as a rapid and portable measure of working memory span in adults” (Gendle and Ransom 2006, 1). Such research has fueled a recent online boom in testing, and improving, memory through gaming. Websites such as Lumosity promote the strengthening of memory through gaming, while books including Jane McGonigal’s Reality Is Broken and Ian Bogost’s How to Do Things with Videogames explain “why games can make us better.” These ideas, of better thinking through brain games, were a part of SIMON ’s popularity.14 Sadly, SIMON ’s creator Ralph Baer passed away on December 6, 2014. Memorialized in Smithsonian magazine, in his ninety-two years Baer had patented over 150 inventions (Edwards 2006). In 2006 he was awarded the National Medal of Technology for his pioneering work on the Odyssey, which helped birth the video game industry. Four years later, Baer gained entry into the National Inventors Hall of Fame. These were fitting accolades. In 2013, the industry that Baer helped to launch netted an estimated $90 billion dollars worldwide.15 In his later years, Baer continued to be outspoken about his place in video game history. Yet SIMON remains his most iconic invention. In large part, the game’s success came from a combination of addictive gameplay, broad appeal, and affordability. SIMON showed that games need not feature the latest and greatest technology to be successful and that electronic games could endure in the video-game-console era. Perhaps, most importantly, SIMON ’s success suggests that a great idea, well executed, can compete with fad games, gimmicks, or graphics. As Milton Bradley’s executive vice president, John W. O’Donnell, rightfully professed: “There will probably never be another SIMON (‘Electronic Games Race’ 1980).”

SIMON: THE PRELUDE TO MODERN MUSIC VIDEO GAMES

39

Notes 1

For arguments that Cold War funding fueled gaming, see Halter (2006). Baer disagreed with these kinds of assertions. See Barton (2013, 134).

2

Baer remembers that Odyssey’s marketing methods implied that the console only worked with Magnavox television sets, which was not the case. The misconception likely hurt the console’s sales.

3

200,000-unit estimate found in Burnham, Supercade; the 330,000unit estimate found in Martine (2014), online at: http://www. nytimes.com/2014/12/08/business/ralph-h-baer-dies-inventorof-odyssey-first-system-for-home-video-games.html (accessed February 20, 2015; hereafter referred to as “Baer NYT Obituary”).

4

Other versions of the game use an A-major triad, with the notes E, A, C#, and E; Baer, Videogames: In the Beginning, 171–3.

5

See “Electronic Games and Toys,” ad by Milton Bradley Electronics, online scan at: http://www.plaidstallions.com/mb/ (accessed March 6, 2015; hereafter referred to as “Electronic Games and Toys”).

6

Bruske (1979).

7

Harmetz (1981), (hereafter referred to a Harmetz, “New Bradley Game Tests Fickle Market.”)

8 Another SIMON clone was a featured level on Nintendo’s 1980 release Flagman (Nintendo 1980). See “Your Choice of Electronic Numbers Game or Follow Me”. 9

“Maniac Electronic Game, 1979.”

10 “Electronic Learning Aids”. 11 “Electronic Games and Toys.” “Adult” ad online at https://www. youtube.com/watch?v=gqcf01pGDbU (accessed March 6, 2015); “Kid” ad online at https://www.youtube.com/watch?v=KVzZVYtt64I (accessed March 6, 2015). 12 See “Simon Through the Years” from Board Game Geek, online at: http://boardgamegeek.com/thread/1028920/simon-through-yearsreview-dickclarkfan1 (accessed March 14, 2015). 13 Ibid. 14 On Video Games’ potential and significance, see: Bogost ((2011); McGonigal (2011); Gee (2007); Goldberg and Larsson (2015). 15 Bayer NYT Obituary.

40

MUSIC VIDEO GAMES

References Baer, Ralph H. 2005. Videogames: In the Beginning. Springfield, NJ: Rolenta Press. Barton, Matt. 2013. Honoring the Code: Conversations with Great Game Designers. New York: CRC Press. Baughman, James. 2007. Same Time, Same Station: Creating American Television, 1948–1961. Baltimore, MD: Johns Hopkins University Press. Blake, Harriet L. 1980. “Does Santa Run on a 9-Volt Battery? Toying with Christmas.” Washington Post, E1, December 14, 1980. Bogost, Ian. 2011. How to Do Things with Videogames. Minneapolis: University of Minnesota Press. Bruske, Ed. 1978. “2,001 Toys—A Space Odyssey; Toying Around”. Washington Post, K1, November 26, 1978. Bruske, Ed. 1979. “This Way to Toyland.” Washington Post, E1, November 25, 1979. Burnham, Van. 2001. Supercade: A Visual History of the Videogame Age, 1971–1984. Cambridge, MA: The MIT Press. Craven, Joan. 1994. “Games Improve Brainpower, Give Quality Time.” Calgary Herald, November 24, 1994. Edwards, Owen. 2006. “The No-So-Simple Simon Proved the Young Were Swifter than the Old.” Smithsonian Magazine, September 2006. “Electronic Games and Toys.” Milton Bradley Advertisement. Accessed March 6, 2015. http://www.plaidstallions.com/mb/index.html. “Electronic Games Race.” New York Times, 180, December 14, 1980. “Electronic Learning Aids,” in Sears Catalog Wishbook 1979, page 657. Accessed March 15, 2015. https://www.flickr.com/photos/ wishbook/69926559/in/set-1360453. Gee, James Paul. 2007. What Video Games Have to Teach Us About Learning and Literacy. New York: Palgrave Macmillan Trade. Gendle, Mathew H., and Michael R. Ransom. 2006. “Use of the Electronic Game SIMON as a Measure of Working Memory Span in College Age Adults.” Journal of Behavioral and Neuroscience Research 4: 1–7. Gershman, Jacob. 2003. “Beeping, Burping Memory Games.” New York Sun, sec. 9, May 27, 2003. Goldberg, Daniel, and Linus Larsson, eds 2015. The State of Play: Sixteen Voices on Video Games. New York: Seven Stories Press. Halter, Ed. 2006. From Sun Tzu to Xbox: War and Video Games. New York: Thunder’s Mouth Press. Harmetz, Aljean. 1981. “New Bradley Game Tests Fickle Market.” New York Times, Section D, 4. September 23, 1981.

SIMON: THE PRELUDE TO MODERN MUSIC VIDEO GAMES

41

Knight, Jerry, and Chapin Wright. 1979. “Companies Placing Chips on Electronic Game Boom; High-Tech Toy Sales Booming.” Washington Post, F1, December 9, 1979. “Maniac Electronic Game, 1979.” Smithsonian’s National Museum of American History website. Accessed March 12, 2015. http:// americanhistory.si.edu/collections/search/object/nmah_1302006. Martine, Douglas. 2014. “Ralph H. Baer, Inventor of First System for Home Video Games, Is Dead at 92.” New York Times, December 7, 2014. McFerran, Damien. 2014. “Game & Watch.” In Retro Gamer: Videogames Hardware Handbook: 1977 to 1999, eds Sherwin Coelho and Hannah Kelly, 27–8. Bournemouth, Dorset, UK: Imagine Publishing, Ltd. McGonigal, Jane. 2011. Reality is Broken: Why Games Make Us Better and How They Can Change the World. New York: Penguin Books. Milton Bradley Company. 1979. “SIMON Instruction Manual.” 4–6. Ralph Baer: The Father of Videogames, directed by Nestor Garrido. 2011. Caracas, Venezuela: Videojuegos Geracion X, C.A., DVD. Salmans, Sandra. 1981. “Christmas is a Video Game.” New York Times, Sec. 3, 1. December 6, 1981. Watson, Mary Ann. 1990. The Expanding Vista: American Television in the Kennedy Years. New York: Oxford University Press. “Your Choice of Electronic Numbers Game or Follow Me.” Sears Catalog Wishbook 1979, page 667. Accessed March 11, 2015. https:// www.flickr.com/photos/wishbook/69927062/in/album-1360453. Zito, Tom. 1978. “Wired-Up Wizards: Toys and Games that Tease the Brain”. Washington Post, D1, December 19, 1978. Zito, Tom. 1979. “ZIP! ZIP! ZOUNDS! Gearing up for a $1 Billion Electronic Toy Bonanza.” Washington Post, B1, February 27, 1979.

Games Atari Inc. Pong [Arcade]. Atari Inc.: Sunnyvale, CA, 1972. Atari Inc. Pong [Arcade]. Atari Inc.: Sunnyvale, CA, 1974. Blitz Games. Karaoke Revolution. [Xbox 360, et al.]. Konami: Tokyo, 2009. DMA Design, et al. Grand Theft Auto. [MS-DOS, et al.]. BMG Interactive: New York, 1997. Harmonix. Guitar Hero. [Playstation 2]. RedOctane: Mountain View, CA, 2005. Harmonix, et al. Rock Band. [Xbox 360, et al.]. MTV Games/Electronic Arts: New York, 2007. Hasboro. Bop-It [Handheld Game]. Pawtucket, RI, 1996.

42

MUSIC VIDEO GAMES

Hasboro. SIMON Flash [Handheld Game]. Pawtucket, RI, 2012. Hasboro. SIMON Swipe [Handheld Game]. Pawtucket, RI, 2014. Higinbotham, William. Tennis for Two. [Analog computer/oscilloscope]. William Higinbotham: Upton, NY, 1958. Ideal Toy Company. Maniac [Handheld Game]. Queens, NY, 1979. Konami. Dance Dance Revolution. [Arcade, et al.]. Konami: Tokyo, 1998. Lakeside Toy Co. Computer Perfection [Handheld Game]. Minneapolis, MN, 1979. Milton Bradley. Pocket SIMON [Handheld Game]. East Longmeadow, MA, 1980. Milton Bradley. SIMON Trickster [Handheld Game]. East Longmeadow, MA, 2005. Mindscape. Loopz [Handheld Game]. Novato, CA, 1990. Nintendo. Flagman [Handheld Game]. Kyoto, 1980. Nintendo EAD Group 2. Wii Music. [Wii]. Nintendo: Kyoto, 2008. Parker Brothers. Merlin [Handheld Game]. Beverly, MA, 1978. Russell, Steve, et al. Spacewar! [PDP-1]. Steve Russell, et al.: Cambridge, MA, 1962. Sears. Follow Me [Handheld Game]. Chicago, IL, 1979. Software Toolworks. The Miracle Piano Teaching System. [NES, et al.]. The Software Toolworks: West Sussex, 1990. Texas Instruments. Little Professor [Handheld Game]. Dallas, TX, 1976. Texas Instruments. Dataman [Handheld Game]. Dallas, TX, 1977. Tiger Electronics. Copycat [Handheld Game]. Vernon Hills, IL, 1979. Unfungames.com. Mario Paint Composer [Super Famicom/SNES]. Unfungames.com: 1992.

2 Mario Paint Composer and Musical (Re)Play on YouTube Dana M. Plank

W

hen I was a little girl, I was given a Fisher Price xylophone as a toy. Though it lacked the flashing lights and “batteries sold separately” label of many playthings of the 1980s and 1990s, its cheery row of rainbowcolored bells made a sweet sound synonymous with early childhood and nursery rhymes. The toy treated musical play as a safe haven in which all of the pitches sounded good, all of the tones equally bright and inviting. And yet something was missing. Attempting to figure out how to play songs from my then-favorite film, The Sound of Music, I found that “DoRe-Mi” was within reach, but I could not seem to play “Sixteen Going on Seventeen,” no matter which bells I struck. The toy spans a diatonic major octave in the key of C major. Though I lacked the musical vocabulary to express my difficulty, I knew that there must have been missing tones— notes that belonged in-between the existing bells. I would later learn to refer to those “extra notes” by name in piano lessons: sharps and flats, which added color and life to the musical texture. I relate this anecdote because I was reminded of my frustrations several years later, when my brother brought home Mario Paint (Nintendo 1992) for the Super Nintendo Entertainment System. The game might be classified as a rudimentary computer graphics program— akin to Microsoft Paint—rather than as a traditional video game. Mario Paint incorporated drawing and animation tools and an interface to write

44

MUSIC VIDEO GAMES

simple background music for the player’s resultant artistic creations. Like the Fisher Price xylophone, however, the composing tool was limited to a strictly diatonic palette. It also suffered from a relative lack of control over dynamics, rhythmic subdivisions, tempo, and form. However, the game’s music mode experienced an unexpected renaissance when standalone recreations and revisions of the composing tool began to be developed and distributed as freeware online. Mario Paint Composer (2007)—or MPC, as it is called by enthusiasts—particularly flourished with its expansion of the original program’s capabilities, creating an active online community focused on creating and sharing arrangements of popular music, video game music, and classical art music made entirely in the program. The interest in this software seems puzzling at first; even with several modifications and improvements, the program is quite limited as an audio sequencer. The community has had to develop certain tricks and workarounds as they seek to encode ever-more difficult pieces of music in MPC, and these arrangements often take upwards of 20 hours of labor spread out over weeks or months. This chapter will explore the music-making practices surrounding Mario Paint, writ large—the limitations of the original in-game composing tool as well as the expanded capabilities of Mario Paint Composer— and seek to answer several questions about the MPC YouTube video phenomenon of the past eight years. The labor surrounding these videos is surprising to the outside observer: Why do these creators spend so much time and effort on compositions that sound like they belong in a 1990s video game? Why choose MPC over more sophisticated audio programs? Do these creators toil in hopes of rewards in the form of views, likes, comments, and praise from the community, or for some other, intrinsic satisfaction? Exploring this question draws on issues of fandom, participatory culture, and nostalgia.

Composing in a straightjacket: Mario Paint and diatonic constraint Mario Paint was released on August 1, 1992 in North America, introducing a peripheral controller that fundamentally changed the player’s tactile experience with the SNES console: a two-button

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

45

computer-style mouse.1 Sweeping her hand across the hard, gray plastic mouse pad, the player interfaced with the system in an entirely new way (even if the rigid mouse pad was later abandoned for the superior traction of the nearest rug or floor). From the moment they opened the package, they were primed to expect a very different gaming experience from the standard fare of side-scroller platformers and top-down adventure games. The mouse suggested computerlike utility and freedom from the complex choreography of button presses on the standard input, and the game was just as novel as this accessory.2 Mario Paint has always occupied a liminal space between a traditional game and a toy. It had no set goals and no prescribed mission. Players could color wildly and aimlessly and then amuse themselves with the various eraser animations. They could painstakingly assemble custom stamps to recreate static images resembling their favorite video game levels. They could write a simple song and set it behind their art. Today, Mario Paint might be likened to sandbox games: free-form play spaces prioritizing exploration and discovery over traditional goal-oriented tasks and linear progression through a preset narrative. However, if the game cartridge was a sandbox, it proved to be a rather small one. All of the sand remained firmly inside the box, unable to spill over into other media forms through exporting and sharing one’s creations. Perhaps it is more useful to think of Mario Paint not in relation to contemporary video games but as a kind of toy, a toolkit for child-like discovery through play. As Roger Moseley and Aya Saiki have shown, Nintendo’s history as a toy company informs their approach to games, resulting in several non-traditional gaming devices and interfaces such as the Game & Watch of the 1980s. Indeed, the authors quote an interview with Nintendo’s Shigeru Miyamoto on Wii Music, a later, though equally open-ended gaming experience: When questioned by a journalist as to whether Wii Music’s main mode qualified as a game given its lack of an overarching goal, intermediate objectives, quantifiable progress, and fail states, Miyamoto acknowledged that the software was indeed not a typical game: it was “more interesting” in that it was more akin to a musical toy box. (Moseley and Saiki 2014, 56)

46

MUSIC VIDEO GAMES

Mario Paint may have had a relatively limited number of toys in its box, but these same limitations also made it friendly to the novice and easy to learn and use and enjoy. The drawing mechanism in Mario Paint is quite similar to modern computer graphics programs such as Microsoft Paint, with brushes of various sizes, textures, and patterns. The players also had access to a bank of seventy-five stamps and the ability to personalize fifteen blank stamp slots by using a small pixel-grid and saving the custom creation to the palette. They could create simple looping animations, pull up preset images to use as an electronic “coloring book,” or take a break with a mini-game in which the player uses the mouse to control a fly swatter and deter armies of invading gnats. The game contained enough interesting features to explore to encourage replay and the development of a rudimentary set of mouse-led digital drawing and composing techniques. If one looks at more sophisticated graphics programs (such as Photoshop or Gimp), or music-notation software (such as Finale and MuseScore), the skills gained in Mario Paint are transferable to the more complex platforms. Though hardly a comprehensive tool for the aspiring artist, it served as a useful device engendering freeform play: an accessible, brightly colored, and friendly introduction to the basic techniques of computer art and musical notation and sequencing. The game’s music composition mode consists of a blank staff in treble clef and a row of stamps representing fifteen different instrumental timbres. The player simply selects an instrument from the top of the screen (see Figure 2.1), places it on the desired pitch (ranging from C4 to G5), and builds up the texture until the song is deemed ready for its debut with the art. The icons do not always have an obvious semantic correlation to the sound, beyond the mimetic Cat’s meow or Game Boy’s “bloop” (see Table 2.1). It is fairly easy to speculate about some of these connections: The “twinkly” timbre of the glockenspiel seems perfectly suited to the star, the electric organ/synth pad seems to suggest a blaring car horn, and the warm twang of the bass perhaps echoes the tug of one’s heart strings. However, it is much harder to explain why the airplane is an acoustic guitar, or why Mario’s face represents a xylophone. Perhaps the icons represent unique, eclectic associations for the game designers. While all of these selections may have been meaningful and intentional, it is

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

47

FIGURE 2.1  Screenshot of Mario Paint (1992).

also possible that the desired sound set for the game’s music mode was determined in advance, rendering some choices somewhat arbitrary. After the players have selected an instrument icon, they can add up to three distinct pitches per beat to form triadic harmonies or adorn a simple melody with percussion (though this simplistic texture is ultimately somewhat limiting).3 Players can switch between triple and quadruple meter, adjust the tempo from approximately 40–240 beats per minute, add a repeat sign (though only at the end of the piece in place of the double bar), and play back the existing score.4 Despite these capabilities, the composing tool’s limitations are soon keenly felt. The program is restricted to the diatonic notes of the C major (or A natural minor) scales, and so the player attempting to recreate favorite songs by ear has to transpose from the original key and often either omit notes or encode incorrect pitches. The player cannot subdivide the rhythms in the measure in units smaller than

48

MUSIC VIDEO GAMES

TABLE 2.1  Available sounds in Mario Paint (1992) Icon

Description

Instrument represented

Mario

Xylophone

Mushroom

Drums (toms)

Baby Yoshi

Yoshi sound effect from Super Mario World

Star

Glockenspiel

Fire flower

Trumpet

Game Boy

8-bit tone (“bloop”)

Dog

Pitched woof

Cat

Pitched meow

Pig

Pitched grunt

Swan

Strings/orchestral hit

Face

Pitched Baby Coo

Plane

Acoustic guitar

Boat

Misc. percussion (e.g., bongos in lower range; stopped cymbal in upper range)

Car

Electric organ/synth pad

Heart

Bass

quarter notes; recreating eighth and sixteenth notes requires a degree of planning—multiplying the tempo by two or four and treating each quarter-note space as the new smallest note value. But with a maximum length of twenty-four measures of 4/4 or thirty-two measures of 3/4, these kinds of manipulations leave little wiggle room to compose.

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

49

Mario Paint lacked an export function, and there were no save slots for individual compositions in the music-writing tool: Clear the existing song or load one of the presets and any previous work was lost forever. This was the situation in which I found myself as a small child. In several years of play, I only created one simple, short composition that I was proud of: a solo acoustic guitar melody line in A minor. The unadorned melodic line was contemplative and slightly melancholic, but it remained locked in the game. A crude attempt to record the song to a cassette—by holding a small microphone up to the television speaker—captured the notes but distorted the timbre and feel of the original. My invented melody did not—indeed could not—develop, resolve, or change character, and so it remained an ellipsis, an incomplete thought trailing off into the limitations of the software.

Mario Paint Composer In 2007, the online flash gaming site UnFun Games released a freeware downloadable program called Mario Paint Composer.5 The MPC program isolated the audio composing tool from the original SNES game, updating the software but retaining the look and feel of the original, ostensibly to preserve a sense of mid-1990s retro gaming nostalgia.6 The MPC user would find that the process of composing in the new program was much the same as in the original Mario Paint tool: They would still maneuver icons representing instrumental timbres and pitched sound effects, placing them on a musical staff in one of the two meters and manipulating the tempo and texture to achieve the desired sound.7 Adding to the original palette of sounds found in Mario Paint, MPC v.2 introduced four new instruments to the original sound set: Boo (a harp-like plucked string), Piranha plant (electric guitar), Coin (acoustic piano), and Shy Guy (sustained bowed cello section).8 Though the Mario Paint user was already familiar with mouse input from the game’s unique peripheral controller, the computer keyboard of their Mac or PC allowed for expanded functionality: A simple shift-click command gave users the ability to add sharps and flats, opening up the entire chromatic scale and making the tool more adept at capturing

50

MUSIC VIDEO GAMES

musical nuance.9 MPC expanded the texture as well, allowing for five simultaneous sounds per beat instead of Mario Paint’s three. The user now had the ability to change dynamics with volume slider bars underneath individual beats, offering more sophistication and polish in the resultant sound.10 Another vital addition was the arranger box, which allowed the user to upload multiple MPC files and string them together into a queue for seamless playback when they were ready to record the finished product. The arranger function gave the user greater control over the form than the meager 24- or 32-measures provided on the SNES, allowing for rondo-like recurrence of material as well as expansion to longer and more complex forms. Dedicated users could also opt to change instrumentation and timbre by downloading “sound fonts,” which replaced the default.11 The program increased the pitch range of the staff as well; while the original Mario Paint only allowed for C4-G5, MPC allowed users to place pitches from A3 to C5.12 Further, the MPC user now had greater freedom in adjusting the tempo: Rather than the standard metronomic range of Mario Paint (40–240 bpm), MPC could now support tempos of 500, 2000, or even 5000 beats per minute, allowing for more complex manipulation of the rhythms of the piece. Of course, in preserving the structure of the original composing tool, several limitations still remained. Enthusiasts had to learn to compensate for these issues, creating idiosyncratic workarounds that were unique to the MPC program. Although they were granted the ability to add chromatic notes and play with dynamics and form, users still had to contend with the lack of rhythmic subdivisions. Because the users were still unable to subdivide beyond quarter notes in either meter, they learned to create the illusion of eighth, sixteenth, or thirty-second notes by treating the quarter-note division as the smallest desired note value and multiplying the tempo of the piece accordingly. This is why the MPC program expanded the tempo range to such seemingly absurd levels: to account for these issues with subdivision. Thus, MPC requires a particular kind of digital literacy: a familiarity with the idiosyncrasies of this specific piece of software and the techniques that can overcome these lingering limitations. Users must learn to think in terms of managing macro-beats, manipulating the tempo to often-absurd levels in order to achieve proper rhythmic effects. The MPC community thus imposes norms for participation that are

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

51

centered on competency not only in traditional musical notation systems but also on the unusual workarounds borne of MPC’s quirks. The fan forum site Mario Paint Hangout contains several threads elucidating and discussing these standard workarounds, from the basic manipulation of the program’s preset subdivisions to the “Low A Glitch,” “Dickspeed,” and pseudo-echo effects. The “Low A Glitch” cuts off notes abruptly, giving the user the ability to create a staccato articulation.13 “Dickspeed” involves manipulating the tempo so that the user can work around the five simultaneous notes-per-beat limit of MPC and create the illusion of a fuller texture: The player simply places the additional notes on the next closest line and speeds up the tempo to a level where the delay becomes imperceptible and the two beats sound as one.14 The echo effect is somewhat simpler to implement. It involves placing the same note at the same pitch on the next line and then lowering the volume of the second note. This technique can also be used to create an artificial lengthening of the note value. MPC does not contain complex controls over the attack, sustain, and decay, and so advanced users have had to devise such workarounds in order to achieve realistic articulation effects in their compositions. Members of the Mario Paint Hangout forums disseminate this collective knowledge, gained through months or years of personal experimentation and trial and error—through message board threads discussing every conceivable element of the program. The threads range from the basic mechanics of transcribing by ear to technical issues such as creating (or faking) triplet effects, to more subjective topics such as “what sounds go together most?” The forum serves as a repository for this fan knowledge, as the users continue to refine their virtuosic command of the program, critique each other’s work, and mentor new users.

The MPC YouTube phenomenon as a video genre, subculture, and online community MPC is more than just a fun, retro piece of freeware: It has become a full-fledged phenomenon since its debut in 2007. A search for “Mario Paint Composer” on YouTube turns up over 158,000 videos

52

MUSIC VIDEO GAMES

showcasing all manner of arrangements, homages, remixes, and original compositions. These videos are enjoyed, curated, and shared on social networks, and this activity extends beyond the MPC devotees to the broader pop cultural sphere. In seeking to understand more about the YouTube phenomenon of videos showcasing evermore-elaborate MPC compositions (and about the role of the Mario Paint Hangout forum in fostering relationships between otherwise disparate MPC enthusiasts) several questions arise. How should we categorize the content creators—the arrangers and composers— working with this program? Do these people represent a community, a subculture, a fandom, or merely a niche YouTube music video genre? Is MPC the object of fandom, or is it merely a tool or platform from which to launch musical tributes to favorite songs and artists? What, exactly, is the nature of MPC composition? At a first glance, it is tempting to refer to MPC as merely a video genre in the broader culture of YouTube, with individual creators brought together by keyword searches. Yet, the music represented in the program spans all musical genres, even if the medium is fixed. Further, there does appear to be stratification in viewership patterns between these different genres of MPC composition; for example, popular songs simply attract more views than arrangements of obscure video game music. And, as is increasingly common in open online spaces, the comments sections on the individual videos offer little insight or substantive critique, little to link one video to the next. Comments are frequently from passing spectators: praising or deriding the quality of the video, sharing memories of playing Mario Paint as a child, derailing the conversation or “trolling” with nonsequiturs and inflammatory attacks, or questioning the authenticity of the video (particularly in complex, highly skilled arrangements). In general, the comments section does not appear to be the primary space for lively critique of the art of MPC composition by the content creators. The video-sharing platform appears to be seen mostly as a vehicle for distribution, a convenient and accessible spot to host one’s video data and easily share it with others. Comments on popular videos are quickly buried under layers of general praise or trolling; YouTube’s potential to function as an archive of productive conversation is hampered by the structure and format of the comments platform. Comments fields, for all of their immediacy

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

53

and interconnectedness, appear to be reserved for more informal conversations rather than opportunities to further the form. Thus, YouTube is not the hub of the MPC community, and thinking of the MPC phenomenon merely in terms of the uploaded videos negates the social and aesthetic practices of the community and the personal labor involved in creating these texts. Though YouTube does not foster such rich dialogue, MPC composers do engage in ongoing discussions of technique, form, and function; they simply choose to do so in dedicated forums, which can preserve and archive the conversation and serve as a resource for newcomers to learn about MPC composition. The Mario Paint Hangout appears to be the main hub for the MPC phenomenon outside of YouTube: Members gather to share tips, organize collaborations, and share ongoing work. Members can critique or seek support and mentorship from more established composers, or refer to the many pages explaining the nuances of Dickspeed and other advanced techniques. It might be most productive to think of MPC as the nexus of fandom, participatory culture, and virtual community, spread laterally across several forums and using YouTube as a distribution vehicle. One might then think of MPC as a niche subculture of gamers; other than a few breakout hits, the bulk of the posted videos have less than 1,000 views. MPC may occasionally cultivate a mass appeal— particularly when videos remediate popular songs and Internet memes—but the majority seem to be directed toward a small group of video game music enthusiasts. And yet, the word “subculture” seems inadequate to describe MPC; the members do not appear to adopt any particular sartorial standards or accouterments. They share a common interest in the production of these videos but remain somewhat diverse in musical tastes and offline lifestyles. Francesca Coppa distinguishes between subcultures and communities by suggesting stronger relationships among members of the latter: Subcultures are social groups based on common interests, whereas communities reach a deeper level of involvement by creating a shared history and promoting kinship and close ties.15 If not for the Mario Paint Hangout, one might argue that MPC lacks a true sense of history and that the video creators are only superficially linked by keyword; random videos of a common interest thrown together by

54

MUSIC VIDEO GAMES

YouTube’s search algorithms. Comparing recent uploads against those from 2007 and 2008 serves as a striking testament to how use of the program has changed drastically in eight years. The group has a shared history, communicating frequently and refining their collective technical knowledge and aesthetic preferences over time. Therefore, perhaps it would be more accurate to refer to MPC as a community. Robert Kozinets describes virtual communities as social aggregates created and shaped by ongoing group discussion (Kozinets 2010, 8). These groups foster a cohesive sense of identity for the larger unit but also create spaces where individual relationships can flourish, strengthening members’ social bonds. The word “community” also implies a group of a particular size: While less than twenty members may not feel socially diverse enough for the label, groups larger than one- or two-hundred members will feel too large, fracturing into subgroups and smaller communities (Kozinets 2010, 8). Although there are over 850 registered members of the Mario Paint Hangout, many of these users are dabblers or “lurkers,” some of whom have never posted or composed with MPC and simply want to check out posts made by the more dedicated, active members.16 The greatest number of users online at one time was 71 (and according to the site’s front page, that was on November 13, 2010, at 11:53 p.m.), putting the peak activity right in a sweet spot for Kozinets’s ideal community size. The Mario Paint Hangout‘s members engage in ongoing discussions, and thus the constituents constantly renegotiate the meaning of the community as they contribute to a collective voice. Online communities foster a sense of group cohesion as the members create participatory norms, skill sets, and jargon (certainly “Dickspeed” is among these invented terms). In addition to cultivating a sense of group identity, virtual MPC community also fosters relationships on the local level between individual members as MPC users gain knowledge and skills. Kozinets describes a process by which these elements work in tandem to shape a member’s experience: the pattern of relationship development in an online community is one in which task-oriented and goal-directed informational knowledge is developed in concert with social and cultural knowledge and social relationships… fact-based information

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

55

is learned alongside knowledge of the online community’s specialized language and sensitized concepts, norms, values, rituals, practices, preferences, and the identities of experts and other group members. As personal details and stories are shared, cultural cohesiveness ripens and empathy blooms. A group structure of power and status relationships is learned. What began primarily as a search for information transforms into a source of community and understanding. (Kozinets 2010, 27) The rules of membership in the community can vary from person to person: for some, one might be considered a member only when they have created and uploaded an MPC arrangement. And yet, a large number of community members engage in conversation and debate with their peers, create playlists grouping MPC videos together, or simply serve as an eager audience for new videos without ever creating new content with the program. Mario Paint Hangout is open to members of all levels of skill and participation. There are twelve members listed as Admins, who moderate the various boards and enforce forum rules; these members appear to be chosen largely based on successful, frequent participation in the community.17 In treating MPC as an online community, we are able to situate the different kinds of participation seen on the Hangout forums and on YouTube within a broader framework of identification with the central activity and with the community itself. After all, some members will derive a higher sense of satisfaction and sense of belonging through interacting with other members; others surely prefer to take a more active role in the production of MPC videos.18 Interest in the central activity—the creation of new videos—is deeply correlated with proficiency in the program.19 However, less-proficient members can still benefit from participation in the social (rather than creative and productive) aspects of the community. This framework is useful for understanding MPC because no matter how virtuosic the compositions become, the interest in the program itself does not appear to be a central, all-encompassing aspect of members’ identities. In fact, the members treat MPC somewhat casually, at times with flippant, dismissive remarks about their labors—at least to the larger YouTube audience.

56

MUSIC VIDEO GAMES

Though MPC composers take technical proficiency seriously and are interested in producing works and videos of high quality, few are likely to refer to MPC as an integral, primary marker of identity. Rather, the program serves as a quirky medium with which to channel and express their broader interests in music, technology, and retro nostalgia. The community is certainly interested in improving their skills within the program, and they value the innovation of the more gifted members. However, they often take pains not to portray themselves as overly invested in the MPC program itself; they are not declaring and promoting this aspect of identity as central. To some extent, this is surely a rhetorical device creating the illusion of ironic distance from the act of composing or arranging. The public display—in the form of the uploaded video and its description—often underemphasizes this intense engagement with the community and the process of acquiring the skills necessary for successful participation (however you might define success in this context through views, comments, or other interactions). It is only on the MPC forums that one catches a glimpse behind the curtain. And yet, the very existence of the work and descriptions of the time and energy spent on these videos suggest powerful identification with some aspect of participation. As Francesca Coppa argues, these members may take on a number of identities within the community as their level of involvement increases, and whether or not they place a great deal of importance on these individual identities, a member can certainly “impact her sense of self and the way she engages the world” (Coppa 2014, 78). At the very least, the MPC members can derive satisfaction from understanding their place within the larger community: “There is a critical sensibility that comes from engaging culture from the inside, as someone with a defined role in the cultural ecosystem” (Coppa 2014, 78). And, occasionally, the video descriptions belie or reflect the users’ deeper investment in the process by stating how long a particular project took to complete. Cat333Pokémon spent twenty hours on Queen’s “Bohemian Rhapsody”;20 Geoff Klassen’s version of Michael Jackson’s “Thriller” took 11 hours;21 Banjaro created a custom MPC sound font for his arrangement of “Africa” by Toto and states

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

57

defensively in the video description that “I did not spend 35 hours just to have you call it fake, so you might as well not comment on that because I’ll delete it anyway.”22 Uploaders seem to equate time invested with the overall quality of the arrangement (and, perhaps, the level of dedication to the MPC community), and so they announce the total number of hours in order to impress their viewers. One would think that the most-skilled, advanced MPC composers would seek to create complex arrangements in the smallest (rather than largest) amount of time, in order to demonstrate technical fluency, even virtuosity. However, the video descriptions also typically do not disclose whether or not the uploader consulted sheet music in creating the MPC arrangement; this would certainly make a difference in the amount of time invested. A user who copies from sheet music can save a lot of time over the user who first has to transcribe the music by ear. Users on the forums are split on the issue: While many admit to using sheet music as a reference in the initial stages, just as many are adamant about creating arrangements unassisted. It is important to note how an arranger’s level of fluency in musical notation could impact the length of the process; perhaps these declarations of time spent relate to the user’s stance on the issue of sheet music. Kozinets derived a typology of member participation from Kristine de Valck (2005), consisting of newbies (“lurkers” and those who stumble across the site with only a superficial interest in MPC or the community), minglers (“networkers,” those that are there for the social ties but who are not concerned with making MPC videos), devotees (“interactors,” those who are deeply interested in developing refined skills and knowledge sets for MPC composition but who are not as involved with the more social elements of the community), and insiders (“makers,” those who both lead the community creatively and socially, who serve as the social glue and as respected, virtuoso vidders and composers) (Kozinets 2010, 33). All of these kinds of participants exist within the Mario Paint Hangout forums, and all of their contributions are important to the development of the community as a whole, but for the purposes in this chapter, I will focus on the work of the interactors and insiders, who drive the production of new videos, as well as on the content itself.23

58

MUSIC VIDEO GAMES

The MPC YouTube phenomenon as participatory culture and fandom So far, Kozinets’s description of a virtual community seems particularly well suited to MPC, but there are a few other elements that can enrich our view of this phenomenon: namely, fandom and participatory culture. YouTube is often seen as a vehicle for self-promotion, driving homespun content into pseudo-celebrity as vloggers, musicians, and comedians create videos and cultivate a dedicated following. And yet, many of the site’s users have no particular viral aspirations and do not derive satisfaction from mere view count. Jean Burgess et al. point out that it is important not to fall into the trap of simply assuming that vernacular video is organized primarily around a desire to broadcast the self. Viewed as a form of “vernacular creativity,” the creation and sharing of videos functions culturally as a means of social networking as opposed to as a mode of cultural “production.”… It is this social network function that is most noticeably absent from most mainstream media accounts of amateur and everyday content creation: the idea that the motivation for this activity might have at least as much to do with social network formation or collective play as it does self-promotion… Amateurs are represented as individualistic, self-expressive producers who are mainly interested in “broadcasting themselves,” rather than engaging in textual productivity as a means to participation in social networks. (Burgess et al. 2009, 26–30) This caveat is useful when considering the seemingly low return on the MPC community’s investment of time and energy in creating videos: A song with only a few hundred views may be seen as a failure from the broader pop cultural standpoint—but the users may not derive satisfaction from this superficial measure of popularity. While a user may seek extrinsic rewards in the form of comments, ratings, or interactions with other MPC community members after sharing the video on YouTube and the Mario Paint Hangout, he or she may instead pursue these projects for personal reasons that go beyond the desire

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

59

for external validation: to pay homage to a favorite song that has been previously neglected by the community, to explore and deepen their understanding of musical structures and timbres, or simply for the satisfaction of a job well done after hours of painstaking adjustment of the existing parameters.24 After all, as Henry Jenkins’s pervasive and influential work states, “fans benefit—often in nonmaterial ways—from what they create” (Jenkins 2014, xxvi). There are myriad reasons why the MPC community may be drawn to share their work on YouTube as opposed to merely uploading video files to the Hangout forums, and these reasons may have little to do with superficial metrics of popularity through view count. The community may seek to broadcast their work out of a desire to extend the reach of their forums, to venerate master MPC composers and teach and encourage new members, to raise exposure for their activities and create a broader context for MPC composition through linked videos that weave a dense web of interrelated content, to connect with independent MPC composers and videos that may be isolated from the forum community, and so on. This phenomenon has many facets—at least as many facets as there are individual members of the MPC community. And yet, Jenkins’s work is deeply focused not on individual members’ interpretations of their activities within and surrounding a particular fandom, but instead with collective meaning-making and social interaction between members of a particular online community. Jenkins thinks of participatory culture (such as the participatory culture enabled by YouTube as a whole) as too vast to describe the practices of the communities he studies. Rather, Jenkins situates this kind of specialized, concentrated labor as a fandom, that is, as a particular manifestation of the broader participatory culture. A focus on participatory norms treats the MPC community as inherently social and relational, exploring questions of exclusion and access.25 A focus on the participatory aspect of MPC asks how new contributions are judged and what skills and competencies are valid and valued forms of participation. But fandom studies can expand this emphasis through its focus on the production of texts—that is, on the videos themselves. Framing MPC as a fandom can be illuminating in some respects, allowing us to focus on the musical objects that serve as the central

60

MUSIC VIDEO GAMES

production activity of the community as well as on the social and aesthetic practices that govern participation. Jenkins’s early work focused on television fan communities that focused on consuming, discussing, interpreting, critiquing, socializing, and producing works (such as videos or fan fiction) around a particular series. Jenkins theorizes this activity as taking place on several levels, only some of which are germane to our conversation about MPC. Fandom encourages participatory norms in the form of critical and interpretive practices. Lucy Bennett suggests that these kinds of forums enforce a normative fan identity, transmitting “correct” ways of reading, as fans suggest what kinds of narratives are appropriate and which interpretations are legitimate.26 Understanding the interpretive practices of the group is part of an individual fan’s socialization into the community, and the enforcement of these norms fosters group identity and cohesion (Bennett 2014, 13). As Jenkins states: “Part of the process of becoming a fan involves learning the community’s preferred reading practices… Fans are concerned with the particularity of textual detail and with the need for internal consistency… Fan critics work to resolve gaps, to explore excess details and undeveloped potentials” (Jenkins 2014, 278). Certainly these statements apply to the MPC forums, where fans focus on minute details of rhythmic representation and on the potential uses of the program’s undeveloped—or perhaps merely underdeveloped—potentials. New members certainly learn the MPC community’s “preferred reading practices” by observing reactions to posted videos and taking part in ongoing discussion and critique. Additionally, Jenkins theorizes that fandom creates “particular forms of cultural production, aesthetic traditions and practices,” and that fans often speak directly to the community rather than to a broader public audience (Jenkins 2014, 279). It is true that MPC is a somewhat insular community, even within the broader YouTube platform—many videos have only a handful of views, largely made up of fellow community members. The tempo manipulations of “Dickspeed” and the articulation provided by the “Low A Glitch” certainly contribute to a unique system of aesthetic practice within the program itself. Through sheer repetition, certain MPC timbres in the default sound set have become analogues for specific real-world instruments and voices.27

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

61

And yet, Jenkins’s model falls short of capturing the MPC phenomenon in several important ways. The model was developed around television fans who watch and then respond to a particular series. The sorts of television fan communities Jenkins discusses in Textual Poachers cultivate an ongoing engagement with a series, and he explores the sorts of feedback loops created by fan labor in relation to the original product. Fans are seen as atypical media consumers due to their activity, obsession, and engagement with the objects of fandom; we can certainly argue that MPC contains a similar level of activity, obsession, and engagement (despite their video description protestations). But with which texts are the MPC composers primarily engaging? And how do fans complicate the notion of the text through other forms of textual production such as comments and forum posts? We can argue that the MPC community represents video game fans in general, or perhaps that they would describe themselves as fans of the original Mario Paint, but that depiction only takes us so far. If MPC is a fandom, then one simple question persists: What is the object of this particular fandom? Is the fandom centered on the original Mario Paint game and nostalgia for a 1990s childhood experience, albeit remediated through modern software? Is the fandom based on the MPC program itself? Is the program merely a platform—and YouTube the distribution channel—with dispersed object(s) of fandom focusing instead on the popular songs, memes, video game music, and classical pieces that are appropriated and arranged? Essentially, is MPC the locus of the activity and the object of the fandom, or merely the tool that enables the users to speak and create? The MPC program and YouTube appear to be the platforms. The texts are the individual songs that are arranged, taken out of their original pop, rock, classical, or video game contexts and rewritten with an ironic edge to play first to a group of MPC enthusiasts and then, occasionally, to a wider audience. Intense involvement and active discussion takes place largely between those who know the ins and outs of the program—quirks and its modes of address—and this discussion takes place in less public spaces than in the YouTube comments. Because of the many dimensions of this phenomenon and the number of users and types of social and aesthetic activities surrounding MPC, it may be impossible to definitively identify a

62

MUSIC VIDEO GAMES

central text for the community as a whole. MPC appears to be more of a social practice, a manifestation of participatory culture, rather than a fandom grounded by a singular love for a media franchise. To further complicate the notion of a strictly Mario Paint fandom, very few members of the MPC community attempt to create within the constraints of the original SNES composing tool. Therefore, can we truly say that this labor comes out of a love of the original game? It seems that it was only the introduction of the MPC computer program that engendered this kind of participatory cultural production, enabling the community to create the sorts of passionate, freely given fan labor that Jenkins describes. But the community is not centered around the original text, if that text is the SNES Mario Paint; if anything, it is centered on a dramatically rewritten version of the text, a “rose-colored glasses” nostalgic view of a Mario Paint that never was, a version that fans may have wished for it to be in their youth. Thus, do we describe it as a fandom of a fandom? Where would these users situate themselves? There are other problems with describing MPC as a Mario Paint fandom. For example, Jenkins suggests: “Fans are viewers who speak back to the networks and producers, who assert their right to make judgments and to express opinions about the development of favorite programs… Fandom provides a base from which fans may speak about their cultural preferences and assert their desires for alternative developments” (Jenkins 2014, 278). In the case of MPC, there is no larger media entity with which the community appears to communicate. Community members certainly do not direct their videos to Nintendo, the publishers of the original Mario Paint; instead, the fans appear to circulate the videos among themselves. Despite the usefulness of fandom in talking about the social processes and aesthetic practices surrounding participation in the community, the MPC phenomenon does not appear to arise out of enthusiasm for any particular cultural production. And yet, the online proliferation of these videos and the social and aesthetic practices surrounding MPC composition does seem to resonate powerfully with the established models of fandom. Clearly the model is incomplete, but it does capture at least a piece of the MPC puzzle. Is MPC a community? A subculture? Or a fandom? And what do these distinctions imply? Is the distinction important when none of

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

63

the members seem interested in articulating a specific group identity to the public audience, when individual creators seem (at least publically) disinterested in MPC as a marker of identity? Does the ontology of the phenomenon matter, or merely the work in the sense of material and creative labor to produce the videos as well as the social labor involved with commenting, critiquing, and connecting? Ultimately, the answer to these questions may lie in another field, but situating MPC at the intersection of several forms of online behavior—subculture, virtual communities, participatory cultures, and fandom—helps to clarify these numerous interactions that take place around the video objects themselves.

The MPC YouTube phenomenon: Videos As Jean Burgess et al. demonstrate in the third chapter of YouTube: Online Video and Participatory Culture, popularity on YouTube is a multi-faceted phenomenon, encompassing the number of total views as well as user ratings, comments, and videos generated in response to a particular upload.28 And indeed, sorting the results by views versus ratings gives a very different picture of popularity for MPC videos. Table 2.2 represents the list of the top ten YouTube results for the search term “Mario Paint Composer” filtered by view count as of March 2015. Table 2.3 sorts the same search through the lens of highest ratings; no videos appear on both lists. Speaking generally, the top-viewed videos are made up mostly of popular songs, with over 1.5 million views each. The top-viewed videos also tended to be older uploads; other than user jeonghoon95’s blockbuster arrangement of Daft Punk’s “Get Lucky” that went viral in 2013, the videos ranged from four to eight years old, suggesting that these songs accumulated views over time as curious fans searched for the original pop songs and shared them to social networks. The highest-rated videos, in contrast, were almost entirely made up of recreations of video game music, and most were posted in the last three years. The videos in the top ten had no negative ratings, but the extremely low view counts seem to suggest that thumbs-downs are tied somewhat to exposure beyond the community. Mainstream

64

MUSIC VIDEO GAMES

TABLE 2.2  Top ten YouTube results for “Mario Paint Composer” on March 2, 2015, by view count Top ten YouTube results—filtered by view count Likes/ dislikes

Ranking and title

User

Posted

Views

1. Mario Paint— Through the Fire and Flames— Dragonforce

Levi Davis

April 22, 2008

6,321, 827 60,623/1,501

2. Get Lucky—Mario Paint Composer— Daft Punk

jeonghoon95

September 3,469,935 2, 2013

51,172/1,080

3. PaintRoll’d

ihasmario

December 19, 2007

3,462,413

27,157/743

4. Thriller by Michael Jackson in Mario Paint Composer

Geoff Klassen

April 24, 2008

3,105,517

23,192/417

5. Evolution of Video Games Epic Melody (Made in Mario Paint Composer)

adolfobaez

September 2,159,180 13, 2009

34,988/478

6. Still Alive (from Geoff Klassen Portal) 2.0 in Mario Paint Composer

February 9, 1,941,355 2008

20,410/378

7. Bohemian Rhapsody Mario Paint Composer

cat333pokemon

February 23, 2008

1,802,229

14,509/675

8. *Dorkly Bits— Mario Paint Torture by Dorkly

Dorkly

March 21, 2011

1,699,429

8,631/480

9. **The BEST MARIO PAINT SONG EVER!!!!!

igotsthepower9000 May 20, 2008

1,564,172

7,826/1,562

10. Angry Video Game Nerd Them on Mario Paint

TomBobBlender

1,505,957

14, 623/349

November 16, 2007

*A parody video in which the Mario at the top of the screen reacts and responds to the song he is playing in the composer mode. ** A reposting of ihasmario’s PaintRoll’d video, with a more ambiguous title in order to make it easier to rickroll one’s gullible friends.

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

65

TABLE 2.3  Top ten YouTube results for “Mario Paint Composer” on March 2, 2015, by rating Top ten YouTube results—filtered by rating Views (3/2/15)

Likes/ dislikes

January 16, 2011

3,062

58/0

Maylson Matos

November 2, 2010

243

5/0

3. Mario Paint Composer—Those Chosen by the Planet (Final Fantasy VII)

Cool70sfreak

May 30, 2014

270

3/0

4. Mario Paint Composer—The Legend of Zelda A Link to the Past— Sanctuary (organ)

HolyDemon017

August 26, 2013

502

11/0

5. Mario Paint Composer —One by Metallica

Steven Moutray

March 12, 2010

123

1/0

6. Mario Paint Composer—Ripple Field 2 (Kirby’s Dream Land 3)

kirpys

March 8, 2011

871

11/0

7. Mario Paint Composer: Linkin Park—From the Inside

buckyboy2009

May 18, 2011

2,085

48/0

8. Mario Paint Composer Sonic the Hedgehog— Marble Zone

Shypo15

September 14, 2010

796

7/0

9. (Adele) Mario Paint Composer—Rumour Has It

Raymusicification

March 18, 2012

755

8/0

10. Space Junk Galaxy (Piano Remix) from Super Mario Galaxy on Mario Paint Composer

ElectricPopTart18

October 22, 2009

2,883

33/0

Ranking and title

User

Posted

1. Paper Mario— Normal Battle Theme—Mario Paint Composer

Skatemancory222

2. Storm Eagle on Mario Paint Composer

66

MUSIC VIDEO GAMES

crossover hits bring dislikes from “haters,” trolls, and others who stumbled across the video without any context for the MPC phenomenon. Videos posted and shared among a small number of MPC enthusiasts were more positively received because the audience ostensibly understands the norms, limitations, and potentials of the MPC program and the virtuosity required to create the videos. The most-viewed video of all time is Levi Davis’s arrangement of “Through the Fire and Flames” by Dragonforce, with over 6 million views.29 Through the Fire and Flames was popularized through its use in Guitar Hero III: Legends of Rock (Activision 2007), where the speed metal song was used as an unlockable bonus track. Davis posted his version on April 22, 2008, and states in the video description (which is styled as a FAQ answering common questions from the comments section) that the video took “[r]oughly 15 hours of overall work spread out through 5 to 7 days.” Oddly enough, Through the Fire and Flames does not appear to be a particular labor of love for Davis, despite the amount of work put into the arrangement. When asked whether he would remake the video in an updated version of the MPC program, Davis responds in the FAQ: “Probably not. I dislike the song enough as it is.” He then explains the video’s origin: “My friend actually asked me to do it as a joke so I did it.” One wonders if this description, updated several years after the original video upload, represents Davis’s changing tastes as he leaves his teens or reflects exhaustion with the video’s continued popularity; if he truly hated the song at age fourteen, then why would he spend 15 hours on a joke for a friend? Davis’s self-effacing manner might seem puzzling, but it is surprisingly common in the descriptions for MPC videos. This video uses a fairly straightforward compositional process: recreating the original song faithfully, rather than attempting an ambitious remix. He retains the original key of C minor and encodes his version at a slightly slower tempo of 750 (about 187 bpm, versus the original track’s tempo of 200 bpm), and the focus appears to be on capturing the song’s rhythms and chromatic notes as accurately as possible. Davis sticks to a simple instrumental palette using the default MPC sound set: Airplanes (acoustic guitar), Hearts (bass), Mushrooms and Boats (percussion), and the Car (organ) for vocals, as well as occasional Fire Flowers (trumpets) and Game Boys (bloops). Custom sound fonts and tempos in the 2,000–5,000 range are less common in

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

67

uploads from the first few years of the program’s existence; it is only after advanced users have become fluent in the program that they begin to incorporate new techniques to extend MPC’s capabilities. Thus, Through Fire and Flames still scans comfortably, despite the fast tempo; that is, the eye can still track how the sounds correspond to the placement of the icons.30 The second-most-watched video is jeonghoon95’s arrangement of Daft Punk’s song, “Get Lucky.”31 Posted on September 2, 2013, the 16-bit makeunder became a mainstream viral hit, attracting over 2.4 million views in a single week and an article on Wired.com.32 The video description suggests that the arrangement took two and a half months of work, “working on and off.” Hoon’s arrangement is also rather straightforward in rhythm and instrumentation, using the original sound set and Airplane (guitar), Heart, (bass), Boo (violin), Coin (piano), Mushroom and Boat (percussion), with the Game Boy (bloops) and Mario (xylophone) supplying the vocal line and harmony. Though the community had largely moved on to more complex representations of rhythm and articulation by 2013, Hoon’s video stands out for its attention to timbre and dynamics that seem to make a particularly faithful translation of the sound of the original song. Most popular MPC videos recreate popular music, such as Geoff Klassen’s version of Michael Jackson’s “Thriller,” a video with over 3 million views.33 The video annotations serve a didactic function for the MPC community by answering a few frequently asked questions, instructing newcomers as to where they can download a copy of the MPC program and explaining basic techniques, such as the shiftclick command to add sharps and flats. The annotation also informs the viewer that the arrangement took approximately 11 hours over a week’s time. Videos of popular songs, such as “Thriller,” tend to have the highest number of views overall, no doubt due to curiosity from fans of the original track. Even more recent pop hits can still garner more attention than most video game music recreations: Blahguy1818’s version of Taio Cruz’s “Dynamite” has over 100,000 hits;34 TessituraToni’s version of Lady Gaga’s “Poker Face” has over 400,000.35 Popular music tends to attract much more attention than many of the top-rated MPC videos of video game music, such as Skatemancory222’s version of the battle theme from Paper Mario (3,062 views), or Cool70sfreak’s “Those Chosen by the Planet” from

68

MUSIC VIDEO GAMES

Final Fantasy VII (270 views), because they appeal to gamers and nongamers alike and thus have the potential for crossover appeal and viral sharing. MPC user adolfobaez’s recreation of “I Get Around”36 by the Beach Boys has almost 293,000 views, but his version of “Vampire Killer”37 from Castlevania (Nintendo 1986) has 46,000 views despite posting several months earlier than the Beach Boys track. Many MPC users treat the program as an amusing hobby and infuse their works with humor.38 One device that has become an injoke in the community is the use of the Cat icon’s meow sound to represent the vocal line of a song. Sbsmith86’s version of Carly Rae Jephsen’s “Call Me Maybe” seems a particularly appropriate use of the meow. The track was catchy, lighthearted, and sweet to begin with, and the meow, while a little ridiculous, carries the melody well (while still suggesting an ironic, critical distance from the sugary teen pop hit).39 Geoff Klassen’s homage to “Funky Town” by Lipps Inc. uses Cats to great effect in the disco instrumentation, particularly to depict the three-part triadic vocal harmonies at the end of the first verse.40 While pop songs may seem well suited for this kind of parody, the device works equally well in more serious contexts, precisely because it is so absurd and unexpected. A short, but effective example is a 38-second arrangement of the “Lacrimosa” from the Mozart Requiem by KmarkJ, in which the weight and gravitas of the instrumental line—complete with dark, moody timbres of the bass, soft acoustic guitar, and organ—are undercut by the use of Cats for the vocal parts.41 The questing opening string line is depicted not with the violin effects of Piranha Plant or Shy Guy but instead with the Airplane’s acoustic guitar and Game Boy’s bloops, giving the song’s introduction a pointed, chip-tuned sound and setting up the punch line of the video. The video is a short fragment; KmarkJ gives an explanation in the video description: “Lacrimeowsa by Meowzart. He stopped after eight bars, and so did I.” The judicious use of the Cat in MPC composition—in small doses and in unexpected contexts—is the most successful in terms of comedic effect. MPC videos are also exceedingly intertextual, recreating ephemeral musical Internet memes in the MPC idiom. Famouskoifish’s version of “What Does the Fox Say?” by Ylvis is a perfect example of highlevel modern MPC composition as well as this kind of parody and pastiche.42 What is striking about famouskoifish’s arrangement

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

69

is that it uses all nineteen of the MPC instruments in the original sound font, many of which are used as extra sound effects after the appropriate line in the original lyrics.43 The video uses “Dickspeed” at a tempo of 2222, making it difficult to for the eye to follow the exact instrumentation and clusters of sounds in the expanded texture; the purple tracking bar is scrolling so quickly that the individual sounds are triggered in rapid-fire succession but sound together or simply extend the length of a given note. The arrangement is complex, sophisticated, infused with humor, and yet all of this work is merely to make a clever reference to a flavor-of-the-week viral video. After the song faded from the public imagination, this work faded into the archive as well, as fewer users search for the original song. Surely famouskoifish had reasons other than view counts in mind when putting in the time and energy to make such an homage in MPC. But crossover viral hits are hardly the most interesting or affecting MPC compositions. YouTube user Bally (whose real name is Erik Mott) has only four extant videos on YouTube but is clearly a veteran MPC arranger and a valued member of the community; his video “for regis” from December 30, 2013—a memorial for a close friend—is full of comments by other prominent MPC users such as adolfobaez. Bally seems to have had a strong presence in the community before deciding, abruptly, to take down the majority of his videos; several comments on “for regis” ask what happened to his other (presumably numerous) uploads. Despite the disappearance of some of his work, the remaining videos on the channel are a testament to Bally’s talent, musicality, and sublime MPC expertise. On April 28, 2011, Bally uploaded his version of “Asphalt Cocktail,” a frenetic piece for concert band by composer John Mackey from 2009. Mackey’s description of the piece on his website gives a good indication of the aggressive sound of the work: “Asphalt Cocktail” is a five-minute opener, designed to shout, from the opening measure, “We’re here.” With biting trombones, blaring trumpets, and percussion dominated by cross-rhythms and back beats, it aims to capture the grit and aggression that I associate with the time I lived in New York. Picture the scariest NYC taxi ride you can imagine, with the cab skidding around turns as trucks bear down from all sides. Serve on the rocks.44

70

MUSIC VIDEO GAMES

Bally’s version is shockingly fast, using an MPC tempo of 10,001 (the highest I have ever seen): the apotheosis of Dickspeed. No doubt Bally used this incomprehensibly fast tempo in order to mimic the dense, aggressive orchestration of the original, but the video is a blur of activity, progressing swiftly through 130 separate sound files in the MPC arranger over the course of five minutes. The accomplishment is stunning (several of the comments contain incredulous profanity); users praise Bally’s deft manipulation of the default sound font to capture all of the passionate, bombastic energy of the original piece. But the true achievement is in Bally’s subtle deployment dynamics and different instrumental timbres. Bally makes dramatic adjustments to the dynamics on every single microbeat, bringing clarity to the crowded texture. He seems to transcend the nineteen samples of the original sound font, creating blended and novel instrumental timbres through seamless clustering of icons. His orchestration creates the illusion of a diverse group of individual instruments rather than a flat 16-bit performing force. This virtuosic use of MPC differentiates the instruments in the ensemble and leads the listener through the piece rather than hitting them over the head with noise. Rather than dissolving into chaotic cacophony, the arrangement is a pitch-perfect interpretation and translation of the score. The achievement did not go unnoticed: John Mackey commented on the original video “This is my favorite version of ‘Asphalt Cocktail’ ever. And I wrote the thing.” He continued his praise on his personal blog, in an entry titled “Asphalt Bleep Bloop” published on April 29, 2011. Mackey writes: I really, truly love this, because it takes me back to what ALL of my music sounded like when I was writing it on a Commodore 64 back in junior high… It’s like going back in time and meeting my 1984 self and telling him to write Asphalt Cocktail. And it’s not just that Mr. Mott took the time to program this, which took him months. It’s the fact that he made it sound good. Really, really good. There’s attention to voicing, dynamics (probably the most impressive element of his interpretation), and the percussion writing. He even made the clarinet solo sound cool, going as far as to program in all of the pitch “falls” that are in the part.45

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

71

The labor involved in creating a successful MPC video (whether you define success as a particular kind of arranging virtuosity or as a measure of popularity based on views and engagement in the comments) is massive, beginning with downloading and learning the program and continuing with more elaborate projects after years of practice and improvement. The creators of this content create a compartmentalized, niche identity around MPC, whether they treat it as a silly hobby, a waste of time, a pleasant way to squander time and talent, a private passion, or simply as a social phenomenon, a familiar corner of the unknowable vastness of the Internet. The activity of composing is highly intertextual, drawing on popular culture texts to appropriate, remix, and create anew, with a healthy dose of irony and humor and a sophisticated, intuitive sense for how content and knowledge are shared and spread online. Users approach MPC from a number of skillsets. Technical knowledge of musical notation does not always translate into successful MPC arranging; there is a specific competency or program literacy that is required before participants can engage fully with the community (as evidenced by the scathing critiques of videos that are deemed too simplistic, that are not edited to create seamless cuts between sections of the song, or that contain technical errors in rhythm or pitch). What, exactly, drives these fans to pursue these arrangements, to post them to YouTube, to spend so much time on them? A knee-jerk reaction might pin this creative energy on nostalgia. Though, perhaps, nostalgia does not adequately explain the desire to produce, as the merely nostalgic can simply tinker in the SNES game, producing within the original game constraints without exporting and sharing their creations socially. Perhaps too much nostalgia shuts down the creative impulse of MPC entirely: Does the desire to retreat into the past preclude the notion of “fixing” the flaws of the original game mechanics, as the MPC program has done? Will a player invalidate their earlier experience by rewriting it with the new platform? Or will it engender a more utopian vision, creating a rose-colored version of the past as it “should” have been, a fiction that seems to re-inscribe a sense of completion and wholeness where none existed in the actual experience?

72

MUSIC VIDEO GAMES

Nostalgia or (No)stalgia? Revisionist history, MPC, and musical play Participation in an online community certainly explains the continued appeal of MPC as an expressive platform, as the users become more adept in the program and integrated into the community. But the initial impetus to make music and share it with others must, for many, arise out of the nostalgia triggered by seeing familiar icons from childhood, a desire to reconnect to a particular memory of the freedom and openness of play.46 As Sean Fenty suggests, “… classic video games have become powerful nostalgic artifacts, not only as reminders of another time and place (a tether to a longed-for past) but as yearned-for states of being, desired spaces in and of themselves—digital homes to which gamers yearn to return” (Fenty 2008, 20). Mario Paint had no rules, no time limits, no bosses to fight, no missions to complete; it handed over control to the player and allowed them to literally and figuratively orchestrate their experience. Players may remember feeling empowered by this game: the sheer excitement at the prospect of play without constraint and creation without judgment. Of course, many of us eventually discovered the boundaries and limitations of this play, but that initial spark, that moment of agency, the cusp of potential creation: This is the most powerful lure, the nostalgic “home” to which the player longs to return. Nostalgia is typically theorized as a futile endeavor, described by Fiona Smith and Mary Brown as a “… type of mourning for lives past and a longing for something that is unattainable…” (Smith and Brown 2014, 149). Perhaps this is true of Mario Paint. It may represent dashed hopes—the incompleteness of past experience—and yet nostalgia suggests a pure moment before the limitations were discovered, when it was possible to wield the power to compose and give voice to dreams. Smith and Brown are concerned with online communities of “retrophiles” who wear vintage clothes and seek to live a 1940s lifestyle, but their words perfectly encapsulate the poignant allure of Mario Paint Composer:

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

73

it is not that people necessarily have a longing for some authentic and original concept of the past itself—it may be, however, the completeness of past experience that they really desire… There is a need to re-appraise and characterize the postmodern context and it is often defined now as fragmented, insecure, a risk-culture, an acceleration of time and the collapse of time-space distanciation. (Smith and Brown 2014, 158) MPC is not just an individual reaction to postmodern culture, a solipsistic reconfiguration of memory, but a collective, active revision of history that conquers past failures and limitations. Matthew Thomas Payne defines collective memory as the power for a group to communally reshape the past, writing that “… collective memory is not interested in researching ‘objective’ histories, but rather how a group re-presents its past and how it propagates and disseminates this narrative” (Payne 2008, 53). The social dimension of this (no) stalgia, this unwriting and consequent rewriting of history, is a powerful communal experience.47 The members of the Mario Paint Hangout, and of the MPC community more generally, are creating a utopian vision of everything that Mario Paint could have been and all that it can be in the future. Mario Paint Composer gives the player agency to bring a previously incomplete experience to completion: the recreation of that remembered moment of freedom with the ability to rewrite the past experiences of Mario Paint.48 MPC promises creative potential and an opportunity to correct the memory of the game’s disappointments. MPC is a revisionist history, the unmaking of nostalgia; a (no)stalgia that delivers satisfaction to unrealized potential, the unfulfilled desires of the past. By repackaging the look and feel, the superficial aesthetic of the original experience, MPC triggers individual memories of Mario Paint, the elements that made the composing tool so compelling. Mario Paint—and by extension the improvements of Mario Paint Composer—offer the opportunity to become fluent in a form of musical notation, to create all that you can imagine and express all you hear, to translate your experience of the world into pure music. This is musical play at its most empowering.

74

MUSIC VIDEO GAMES

Notes 1

By some counts, there were over sixty games released after 1992 that supported the peripheral mouse, almost half of which were only ever released in Japan. One such list can be found in the article “List of SNES Mouse Compatible Games,” Encyclopedia Gamia: The Gaming Wiki, accessed March 21, 2015, http://gaming.wikia. com/wiki/List_of_SNES_Mouse_Compatible_games. However, the average player may have been unaware that they could use the mouse in games other than Mario Paint, as it is often an optional addition to the traditional input. In games like Jurassic Park (Ocean Software 1993), for example, the player can only opt to use the mouse in first-person sequences, and so most will simply continue to use the controller rather than switching between the two.

2

The mouse was not Nintendo’s first peripheral input device, though it remains one of the most recognizable. The original Nintendo Entertainment System launched in 1985 with an iconic orange and gray “NES Zapper,” a light gun used for games such as Duck Hunt (Nintendo 1984) and Wild Gunman (Nintendo 1985). The NES Power Pad (released in 1988) was a plastic mat fitted with several pressure-sensitive pads, which transmitted the player’s steps for running games such as World Class Track Meet (Bandai 1988). Another iconic—but ultimately failed—peripheral was the Power Glove from 1989. The glove was meant to track the player’s hand gestures and translate them into game action, but the device was notoriously difficult to use.

3

What if the player wanted to try to encode four-part vocal harmony? He/she would not have enough spaces per beat!

4

The tempo slider bar did not indicate beats per minute in Mario Paint; the player simply used the SNES mouse to drag a white bar to the left (for slower tempos) or right (for faster tempos). I arrived at this approximate tempo range with a metronome.

5

Originally housed at http://www.unfungames.com/mariopaint/, though the site is no longer active.

6 Though MPC is the most popular program, other Mario Paintderived audio sequencers do exist: the Mario Sequencer duplicates the conditions of the original game exactly and is typically used only by the most advanced users looking to challenge themselves by working within the constraints of the original game. The Triple Mario Sequencer is quite similar to the other programs but provides three staves and thus the opportunity to place nine notes per beat as opposed to the three of Mario Paint or five of Mario Paint

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

75

Composer. The Advanced Mario Sequencer is an explicit attempt to improve upon the popular MPC program, with an aim to improve the GUI and incorporate advanced editing options. 7

The Mario Paint Hangout forum FAQ by eataninja suggests that the novice composer can ignore 3/4 entirely, explaining that “[t]he 3/4 button decreases the number of subdivisions within each measure from 3 to 2, effectively creating a 3/4 measure. It’s rarely used, so don’t worry about it too much.” Poster Pseudonym eataninja, “The Official Mario Paint Composer FAQ,” last modified August 6, 2011, accessed March 28, 2015, http://mariopaintcomposer.proboards. com/thread/5427/official-mario-paint-composer-faq.

8

The first version of the composer copied the original game’s instruments verbatim; version two added the four additional sound icons that would become standard in MPC composition. Depending on the volume and tempo, the Boo can also mimic a pizzicato violin or a soft, articulate acoustic guitar. Indeed, message board posts and various Mario Paint Composer-related Wiki pages often refer to Boo as a violin. The Shy Guy—also typically identified as a violin— has more depth to the sound and suggests a cello section with sustained bow strokes.

9

These keyboard shortcuts thus represent a form of input that was not available in the original SNES game (the SNES only included a peripheral mouse, not a peripheral keyboard).

10 This is similar to the velocity function in MIDI. 11 The Mario Paint Hangout forum contains several message boards relating to sound font creation and use. Users developing new sound fonts will use these boards to troubleshoot and workshop the set, and then post a download link of the completed file for the benefit of the community. 12 The original sounds were thus expanded to this new range. 13 User eataninja’s Mario Paint Composer FAQ offers the following definition for the “Low A Glitch”: “The low A glitch allows you to shorten a note without decreasing its volume. On an EMPTY LINE, click on the border between the green volume bars and the staff. When a note of that instrument reaches the glitched note, it will abruptly cut off. The note has to be placed on an empty line to work, but you can place notes after a glitched note is placed. You can put multiple glitch notes on one line.” http://mariopaintcomposer. proboards.com/thread/5427/official-mario-paint-composer-faq. 14 User eataninja’s thread also defines “Dickspeed”: “In Mario Paint Composer, you can only place up to five notes on a line. To get around this limitation, you can multiply the tempo by sixteen and

76

MUSIC VIDEO GAMES place any excess notes on the line before or after the ‘main’ line.” http://mariopaintcomposer.proboards.com/thread/5427/officialmario-paint-composer-faq.

15 “Petty production and media consumption aside, fandom provides a mode of social organization that has the potential to move from being a subculture (that is, a social group based on common interests) to a community (that is, based on shared geography, kinship, or history).” (Coppa 2014, 78). 16 A lurker is a member of an online forum who reads or observes but does not post or otherwise participate. However, the exact definition can vary by community. Some forums—such as the Mario Paint Hangout—do not require registration with a username and password in order to read posts, and so a lurker may refer to a non-registered guest (tracked by the forum’s site visit statistics, but otherwise anonymous) or a registered user who has yet to post. Lurking does not necessarily carry a negative connotation; new members often lurk in order to learn the community’s social norms. 17 The rules are designed to generate constructive posts for the community: users are discouraged from intentionally misspelling words, provoking other users into “flame wars,” or posting oneword responses such as “lol.” Users are also discouraged from posting on “dead” boards (those that have had no new activity in over a year), posting on the incorrect board (e.g., sharing a new video anywhere but the “New release” boards), or straying from a board’s initial topic (they are instead urged to start a new discussion thread in order to keep the forum organized). 18 “… the more central is this activity to a person’s sense of identity, and the more that they believe the pursuit and development of the skill or activity is central to their self-image and core selfconcept, then the more likely this person is to pursue and value membership in a community, be it online or otherwise…” Kozinets, Netnography, 32. 19 As Kozinets suggests, it is “… a measure not only of selfidentification, but of identity and interest combined with expertise.” Kozinets, Netnography, 32. 20 Cat333Pokémon, “Bohemian Rhapsody Mario Paint Composer,” YouTube video, 5:44, last modified February 23, 2008, accessed March 1 2015, https://www.youtube.com/watch?v=eO_70TBTTiY. 21 Geoff Klassen, “Thriller by Michael Jackson in Mario Paint Composer,” YouTube Video, 3:39, last modified April 24, 2008, accessed March 1, 2015, https://www.youtube.com/watch?v=6gc1Dq3uQQ.

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

77

22 Banjaro, “Africa—Toto—Mario Paint Composer,” YouTube video, 4:33, last modified July 2, 2010, accessed March 1, 2015. https:// www.youtube.com/watch?v=C_iaGf73Bq0. 23 Of course, this is not to set aside the important role of the “passive” audience in constructing the particular participatory norms for the MPC community. As Jean Burgess et al. remind us in YouTube: Online Video and Participatory Culture, “The dominant discourses around participatory culture (including the very idea of a gap in participation) appear to frame passive engagement as a kind of lack—continuing to affirm and reward those who speak more than those who listen. Some of the excitement and energy around participatory culture was motivated by the possibility that those of us who had been limited to the role of the ‘passive’ audience could become producers, and therefore more ‘active’ participants in the media. While the affordances of the technologies and media forms associated with the participatory turn have increased the range of producers, and undoubtedly moved a significant number of people toward cultural production, continuing to value only those who produce replicates the politics of the previous system. It is important to consider consumption and audiences practices as significant modes of participation, rather than as a lack of participation—as in the widespread habit of referring to the online audience as (mere) ‘lurkers’” (Burgess et al. 2009, 82). 24 “Labor can be about professionalism or the satisfaction…that is, affective and social rather than economic rewards. For fans, there can be enormous satisfaction in creating something and sharing it with a larger community; they may be willing to accept restrictive terms set by a video sharing or social networking site in order to access a platform that allows them to circulate what they created…” (Jenkins 2014, xxvii). 25 This point is important, because MPC does appear to be a largely male-dominated community. Though admittedly YouTube usernames are ambiguous, many of the most visible and influential members of the community are male, such as Levi Davis, the creator of the highest-viewed MPC video on YouTube; Jeong Hoon, whose arrangement of “Get Lucky” became an overnight viral sensation in 2013; Adolfo Baez, known for his complex and densely orchestrated video game music medleys; and Geoff Klassen, who is known for his versions of “Still Alive” from Portal and Michael Jackson’s Thriller. Jenkins writes, “[Participation as a concept] asks us who is participating (and who is excluded from participation) and what factors limit or enable participation (whether

78

MUSIC VIDEO GAMES legal constraints around intellectual property, economic constraints over media ownership, technological constraints around media access, educational constraints in terms of access to the skills and competencies required to participate, even constraints over who has disposable time to put towards their hobbies and interests).” Jenkins, Textual Poachers, xxiii.

26 Although, as Bennett points out, the notion of a “correct” reading is not fixed; instead, the norms are negotiable and change as with the community. Further, the “correct” readings are not monoliths; even within tight-knit fan communities there exist oppositional readings of the texts, and these collisions of meaning can prove to be a fruitful area of research into an individual fan community. See Bennett (2014, 13). 27 Though, it should be noted that the use of sound fonts in more modern videos may serve as a critique or even a complete disruption of this practice. As more “realistic” or stylized electronic sounds become incorporated into the cultural landscape of MPC composition, the 16-bit sound of the original samples can seem quaint by comparison. Perhaps this movement suggests that MPC composition is shifting from a heavily ironic, nostalgic, and casual mode of music production to a more polished and professional approach to the program as an artistic platform. This notion is puzzling: If MPC composers are frustrated with the program’s limitations as an audio processor, why not move on to another program that gives more flexibility and control over the musical parameters? Why not leave the bloops and kitschy animal icons for something sleek, intuitive, and more suited to produce the kinds of music one wants to create? Why try to make MPC into something that it is not? It is possible that these users simply do not want to invest the time in learning to use an unfamiliar program, or that this reluctance stems from a belief that leaving the community is a sunk cost after years of participation and relationship-building. 28 And, as they remind us: “… these [popularity] metrics also take an active role in creating the reality of what is popular on YouTube: they are not only descriptive; they are also performative.” Burgess et al., YouTube, 41. 29 Levi Davis, “Mario Paint—Through the Fire and Flames— Dragonforce.” YouTube video, 7:56, last modified April 22, 2008, accessed February 27, 2015, https://www.youtube.com/ watch?v=hil1F2T19GY. 30 This tracking becomes increasingly difficult in videos that incorporate Dickspeed; when the tempo is above 2000, the icons whizz by in a flash of color, obscuring the MPC score for the viewer.

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

79

31 Jeong Hoon. “Get Lucky—Mario Paint Composer—Daft Punk.” YouTube Video, 4:19, last modified September 2, 2013, accessed February 12, 2015, https://www.youtube.com/watch?v=Ic01Vhiyrb4. 32 Devin Maloney, “Man’s First Try at Mario Paint Composition Results in Perfect Cover of ‘Get Lucky’,” September 9, 2013, accessed March 1, 2015, http://www.wired.com/2013/09/monday-jam-getlucky-mario-paint/. 33 Geoff Klassen. “Thriller,” https://www.youtube.com/watch?v=6gc1Dq3uQQ. 34 Blahguy1818, “Mario Paint—Dynamite—Taio Cruz,” YouTube Video, 3:30, last modified September 18, 2010, accessed February 13, 2015, https://www.youtube.com/watch?v=KtBfcWDFcsM. 35 TessituraToni, “Poker Face—Lady Gaga—Mario Paint Composer,” YouTube Video, 3:34, last modified June 12, 2009, accessed February 13, 2015, https://www.youtube.com/ watch?v=igx0tmhDmpk. It is interesting to note that MPC user Banjaro is TessituraToni’s brother. 36 Adolfobaez, “Beach Boys—I Get Around (Mario Paint),” YouTube video, 2:11, last modified February 12, 2008, accessed February 12, 2015, https://www.youtube.com/watch?v=vfHEhuYYDM4. 37 Adolfobaez, “Mario Paint—Castlevania ‘Vampire Killer’ (Stage 1),” YouTube Video, 1:10, last modified December 21, 2007, accessed February 13, 2015, https://www.youtube.com/ watch?v=EA3hg0eiTcQ. 38 Such as “Paint Roll’d,” a play on the “rickrolling” internet meme where a user tricks a friend into watching the music video for Rick Astley’s “Never Gonna Give You Up.” Ihasmario, “PaintRoll’d,” YouTube video, 1:43, last modified December 19, 2007, accessed February 15, 2015, https://www.youtube.com/ watch?v=5uZr3JWYdy8. Another great example of intertextuality in MPC is Blahguy1818’s version of “Bed Intruder,” recreating a viral video by the Gregory Brothers in which an interview with Antoine and Kelly Dodson about a home invasion and attempted rape were auto-tuned and set to music. Blahguy1818, “Mario Paint—Bed Intruder Song,” YouTube, 1:07, last modified September 15, 2010, accessed February 15, 2015, https://www.youtube.com/ watch?v=9zPxxIVBkSg. 39 The comments for this video have a singular obsession with the use of the Cat. Many of the comments are simply a string of meows or “haha.” And user robloxian awesomeness refers to yet another internet meme when he responds “TOO MANY MEOWS! AAHHH! flips table.” Sbsmith86, “Call Me Maybe—Carly Rae

80

MUSIC VIDEO GAMES Jepsen—Mario Paint Composer,” YouTube video, 3:08, last modified July 10th, 2012, accessed March 1, 2015, https://www.youtube. com/watch?v=PEzUbIOcR9Q.

40 Geoff Klassen, “Funky Town in Mario Paint Composer,” YouTube video, 3:51, last modified February 15, 2009, accessed March 1, 2015, https://www.youtube.com/watch?v=sWaxPouJ6fU. 41 KmarkJ, “Mario Paint—Lacrimosa (Lacrimeowsa) Mozart REQUIEM,” YouTube Video, 0:38, last modified April 2, 2008, accessed March 1, 2015, https://www.youtube.com/ watch?v=xOjS7e7zhT4. KmarkJ also writes new text for the work and overlays the words on the video: “Lacri-meow-sa/Feed us mouse-a/Meow-zart want us sing in Latin/But he’s dead and I’m a cat Meow.” 42 Famouskoifish, “Ylvis—The Fox (What Does the Fox Say?)— Mario Paint Composer,” YouTube 3:39, last modified September 25, 2013, accessed March 1, 2015, https://www.youtube.com/ watch?v=uiImcN4rZyc. 43 The first lines are the easiest to mimic with the original sound set: “Dog goes woof [dog woof], cat goes meow [cat meow], Bird goes tweet [swan orchestral hit] and mouse goes squeak [the face ‘coo’ on a high pitch at this tempo creates a squeak, though there is no mouse icon].” Of course, Famouskoifish is limited to nineteen sounds, and thus has to take some liberties depicting the Cow’s moo [pig grunt], Frog’s croak [Baby Yoshi, perhaps more for looks and green coloring than for the sound?], and Elephant’s toot [layers Yoshi with a Fire Flower trumpet]. Famouskoifish continues to create creative approximations of the required sounds, such as harmonized pig grunts for the seal, and the Face coo in a middle register for the fish “blub.” Famouskoifish, “Ylvis,” https://www. youtube.com/watch?v=uiImcN4rZyc. 44 John Mackey, “Asphalt Cocktail,” 2009, accessed February 1, 2015, http://www.ostimusic.com/Asphalt.php. 45 John Mackey, “Asphalt Bleep Bloop,” April 29, 2011, accessed February 1, 2015, http://ostimusic.com/blog/asphalt-bleep-bloop/. 46 Sean Fenty suggests in his chapter on gaming nostalgia that all video games represent a remembered freedom, due to the player’s ability to uniquely interact with the game world: “As an interactive medium, video games give over a great deal of control to players that other media retain in the presentation of their content. In part, it is this control and the illusion of freedom that makes play possible and video games enjoyable and memorable. Players enjoy the relative freedom that they have in a game world versus other

MARIO PAINT COMPOSER AND MUSICAL (RE)PLAY ON YOUTUBE

81

fictional worlds represented in books, television, and movies; they enjoy the control that they have over video game avatars and the struggle of learning how to succeed in the game world. It is the effort involved—the struggle to learn and overcome—that makes the games memorable, and these memories feed into the process where earlier games are idealized and game-play operates nostalgically for players” (Fenty 2008, 25). 47 And, indeed, the members of the community shape MPC to their desires without concern for the original Mario Paint forms, specifically because the program affords this kind of play. Smith and Brown point out that “[t]he romantic and aestheticized view of the past is a strong theme in retrophilic communities even to the point where historical fact is secondary to image and style.” Smith and Brown, “Negotiating Meaning,” 149. 48 Smith and Brown speak to this agency as a kind of productive escapism: “… it is an active process by which people can address and counter the notions of fragmentation, uncertainty, and instability” in our postmodern context. Ibid., 155.

References Bennett, Lucy. 2014. “Tracing Textual Poachers: Reflections on the Development of Fan Studies and Digital Fandom.” Journal of Fandom Studies 2 (1): 5–20. Burgess, Jean, et al. 2009. YouTube: Online Video and Participatory Culture. Cambridge: Polity. Coppa, Francesca. 2014. “Fuck yeah, Fandom is Beautiful.” Journal of Fandom Studies 2 (1): 73–82. De Valck, Kristine. 2005. Virtual Communities of Consumption: Networks of Consumer Knowledge and Companionship, ERIM PhD Series: Research in Management. Fenty, Sean. 2008. “Why Old School Is ‘Cool’: A Brief Analysis of Classic Video Game Nostalgia.” In Playing the Past: History and Nostalgia in Video Games, edited by Zach Whalen and Laurie N. Taylor, 19–31. Nashville, TN: Vanderbilt University Press. Jenkins, Henry. 2014. Textual Poachers: Television Fans and Participatory Culture. New York: Routledge. Kozinets, Robert V. 2010. Netnography: Doing Ethnographic Research Online. Los Angeles, CA and London: Sage. Moseley, Roger and Aya Saiki. 2014. “Nintendo’s Art of Musical Play.” In Music in Video Games: Studying Play, edited by K.J. Donnelly,

82

MUSIC VIDEO GAMES

William Gibbons, and Neil Lerner, 51–76. New York and London: Routledge. Payne, Matthew Thomas. 2008. “Playing the Déjà-New: ‘Plug it in and Play TV Games’ and the Cultural Politics of Classic Gaming.” In Playing the Past: History and Nostalgia in Video Games, edited by Zach Whalen and Laurie N. Taylor, 51–68. Nashville, TN: Vanderbilt University Press. Smith, Fiona and Mary Brown. 2014. “Negotiating Meaning in the Consumption of the Past.” Journal of Fandom Studies 2.2: 147–61.

Games Bandai Co., Ltd. World Class Track Meet. [NES]. Bandai America, Inc.: Cypress, CA, 1988. Konami. Castlevania. [Famicom Disk System/NES]. Konami: Tokyo, 1986. Neversoft, et al. Guitar Hero III: Legends of Rock. [PlayStation 2, et al.]. Activision, et al.: Santa Monica, 2007. Nintendo EAD Group 2. Wii Music. [Wii]. Nintendo: Kyoto, 2008. Nintendo Research and Development 1. Duck Hunt. [NES, et al.]. Nintendo: Kyoto, 1984. Nintendo Research & Development 1. Wild Gunman. [Arcade, et al.]. Nintendo: Kyoto, 1985. Nintendo Research & Development 1. Mario Paint [Super Famicom/ SNES]. Nintendo: Kyoto, 1992. Ocean Software. Jurassic Park. [SNES]. Jaleco/Ocean Software: Manchester, 1993. Unfungames.com. Mario Paint Composer [Super Famicom/SNES]. Unfungames.com: 1992.

3 Active Interfaces and Thematic Events in The Legend of Zelda: Ocarina of Time Stephanie Lind

T

he Legend of Zelda series has been one of Nintendo’s most popular and famous franchises since the release of the first game in 1986. Games in the series are not sequels in the traditional sense, carrying forward the plot from previous iterations, but rather feature archetypical characters and tropes within the same fictional universe. As a result, a common feature is the reuse of plot elements, characters, and musical themes.1 The treatment of musical themes within the Zelda series is particularly interesting because it features a blurring of diegetic and non-diegetic functions. Specifically, although “background” and “environmental” music is heard throughout the game, virtual instruments also exist within the game: the initiation of musical themes is an integral element of the gameplay, particularly as themes are often given leitmotivic associations with characters and locations. This chapter will examine the setting and roles of diegetic music in the most popular iteration of the Zelda series, The Legend of Zelda: Ocarina of Time for Nintendo 64 (1998), including its effect on the gamer’s sensory experience and game immersion.2 In so doing, I will also examine the interplay between diegetic and non-diegetic functions within Ocarina of Time.

84

MUSIC VIDEO GAMES

Previous scholarship, and an understanding of “diegetic” Given the game’s popularity, Ocarina of Time has been previously discussed by several game music scholars such as Zach Whalen, Karen Collins, and Leonard J. Paul. However, previous research on the game has focused less on diegetic sound (although this is briefly mentioned by both Collins 2008 and Whalen 2007) and more on sound construction and environments (Collins 2008; Paul 2013; Whalen 2004). Immersion is a frequent topic among these discussions; Whalen, in particular, argues that an approach more focused on game immersion and engagement allows for a better understanding of how players interact and communicate with games as a medium (2004). However, in this study of Ocarina of Time, a distinction between player immersion and more narrative elements of the game is difficult given that plot and action are so closely intertwined. Accordingly, this discussion will examine seemingly disparate facets of the game music: its physical interface, gameplay function, formal/analytical structure, and extra-musical associations via the concept of leitmotif. The term diegetic takes on two meanings in game and film music scholarship.3 First, film music scholar David Neumeyer discusses the origin of the term diegetic and defines the term as “sound in the universe in which the story takes place” (Neumeyer 2009, 3). While discussion of diegetic sound and music in film often focuses on temporal and agential connections (that is, how the music is perceived by and/or influences actions and characters), in video games, diegetic music may function more interactively with the audience/player given its nature as a participatory medium. Second, diegetic is sometimes interpreted as “how music supports a game narrative” (Munday 2007, 52). In Ocarina of Time, elements of both definitions are present: music and musical instruments exist within the gameworld and use of these is required to progress through the plot. Ocarina of Time features a switch between diegetic and nondiegetic musical functions, a process that is discussed by both game scholars and film scholars. Game music scholar Karen Collins

ACTIVE INTERFACES AND THEMATIC EVENTS

85

suggests that players have similar sonic responses to both selfproduced and non-self-produced sounds (Collins 2013, 56–62), and consequently a switch between these two modes can contribute to the player’s sense of immersion and emotional investment in the game. Film music scholars Neumeyer and Robynn Stilwell, on the other hand, argue that this functional switch “calls attention to the act of crossing and therefore reinforces difference” (Neumeyer 2009, 28). However, in a game setting where the plot is closely intertwined with the game’s musical themes (whose specifics will be discussed shortly), this switch creates clear connections between active and passive musicking (in the Christopher Small sense of the term).4,5 Several elements of gameplay in Ocarina of Time further support this connection. Karen Collins sees the game controller as a tool that functions as an extension of the self, which is particularly true not only in Ocarina of Time with its use of a virtual ocarina but also in other games of the Zelda series.6 Specifically in Ocarina of Time, there are physical similarities between playing an ocarina and the game controller: Both feature six holes or buttons, and the player raises and lowers fingers in specific combinations to create a change in pitch. So if both the virtual and physical instruments feature a similar playing mechanism, what makes one more “real” than the other? Jesper Juul argues that “[o]n some level it is true that these are not real instruments, but what makes them not real? The basic experience of playing these games is that if you press the buttons correctly, music appears—it feels as if you are making music” (2010, 115). He argues that this mimetic interface shifts the focus of the game from the avatar to the player, supporting Collins’ ideas of increased game immersion (Juul 2010, 117). Given the rising importance of interfaces as part of game design, particularly with the move to motion-sensing game controllers popularized through the release of the Nintendo Wii in 2006, some scholars such as Michael Liebe differentiate between objective interfaces, where the player’s physical actions mimic those within the game, and symbolic interfaces, where there is no gestural similarity between actions in real-life and their digital representation (Liebe 2013, 55–7). Ocarina of Time mixes these modes of interface, with actions such as running and fighting being symbolic but others, such as playing the ocarina, being objective. Such a mix of symbolic

86

MUSIC VIDEO GAMES

and objective interfaces mirrors the switch between diegetic and non-diegetic modes. In Ocarina of Time, interesting interactions occur between musical themes and game events. For the sake of this discussion, a trifold differentiation of game music function into linear, reactive, and proactive forms, proposed by Liebe, will be useful.7 Linear refers to non-diegetic game music that sounds with no input from the player, such as background music; reactive refers to diegetic music triggered by in-game events or actions; and proactive refers to diegetic music to which players must respond in some way (e.g., the songs of Dance Dance Revolution or Guitar Hero) (Liebe 2013, 47). While all three of these functions occur in Ocarina of Time, much of its music combines these functional roles, particularly the reactive and proactive functions: The initial presentation of each ocarina theme occurs as part of the (relatively linear) plot and is triggered by the player entering a particular location or initiating dialogue with a specific non-player character (reactive). Once this is initiated, the player must react by repeating the given melody on the controller to progress to the next step of the game (proactive). When the player later performs these melodies on the virtual ocarina, particular game actions are triggered, thus going one step beyond Liebe’s categories: Rather than the music being a response to an in-game event, the music is an in-game event. This diegetic event-triggering music will thus be called active throughout this chapter.

Gameplay and musical themes For those not familiar with Legend of Zelda: Ocarina of Time, the game is an adventure/fantasy released in 1998 for the Nintendo 64 system. The fifth game in the Legend of Zelda series, it follows the hero, Link, through a series of quests that ultimately involve defeating the evil Ganondorf in order to save the land of Hyrule and its princess, Zelda. To accomplish this, Link must find and reassemble a magic artifact called the Triforce in order to awaken six sages who grant him special villain-defeating powers. The following is a brief summary of major game events (Table 3.1):

ACTIVE INTERFACES AND THEMATIC EVENTS

87

TABLE 3.1  Major game events in The Legend of Zelda: Ocarina of Time. Ocarina song learned

Game event

Location

Earn Kokiri’s Emerald. Meet Saria and acquire the Fairy Ocarina. Learn that you have to find the three gems to reconstruct the Triforce.

Kokiri Forest

Meet Malon. Visit the Temple of Time.

Hyrule Castle Town

Meet Princess Zelda.

Hyrule Castle

Zelda’s Lullaby

Return to the Lost Woods, speak with Saria.

Kokiri Forest

Saria’s Song

Visit the graveyard to find a secret passage to the Royal Family Tomb. Get the Hylian Shield.

Kakariko Village

Sun’s Song

*Visit Malon and meet her horse, Epona.

Lon Lon Ranch

Epona’s Song

Clear Dodongo’s Cavern to Death Mountain/ earn the Spiritual Stone of Fire. Goron City Conquer Jabu-Jabu and save Princess Ruto to earn the Spiritual Stone of Water.

Zora’s Domain

Witness Princess Zelda’s kidnapping by Ganondorf. Retrieve the Ocarina of Time, which Zelda drops.

Hyrule Castle

Song of Time

Go to the Temple of Time to assemble the Triforce and retrieve the Master Sword; Ganondorf interrupts, Rauru the Sage appears, and Link is teleported to the future. Receive the Light Medallion.

Hyrule Castle Town

Prelude of Light

88

MUSIC VIDEO GAMES

Ocarina song learned

Game event

Location

**Visit Malon and acquire Epona.

Lon Lon Ranch

Visit the Forest Temple to retrieve the Forest Medallion.

Kokiri Forest

Minuet of Forest

Visit the Fire Temple to retrieve the Fire Medallion.

Death Mountain

Bolero of Fire

Visit the Water Temple to retrieve the Water Medallion.

Lake Hylia

Serenade of Water

Visit the Shadow Temple to retrieve the Shadow Medallion.

Kakariko Village

Nocturne of Shadow

Visit the Spirit Temple to retrieve the Spirit Medallion.

Gerudo Desert

Requiem of Spirit

Go to the Temple of Time to speak with the Sages. Scale Ganon’s Castle and Tower (formerly Hyrule Castle) to defeat Ganondorf.

Hyrule Castle Town

*May occur at any point after meeting Malon. **May occur at any point in ‘future’ time after vising LonLon Ranch in ‘past’ time.

Over the course of the game, Link learns twelve melodies (plus one extra song, to be discussed near the end of this chapter) on his diegetic ocarina. Each functions as a gameplay command to trigger a particular in-game event. Some melodies are mandatory for completing the main quest of the game, while others provide optional bonuses. Three roles become associated with these ocarina melodies. First, some ocarina melodies unlock doors, reveal hidden areas, or trigger interactions with non-player characters, thereby advancing the plot.8 Second, some ocarina melodies reveal bonus areas, treasure, or call for help from a friend, thus assisting the player

ACTIVE INTERFACES AND THEMATIC EVENTS

89

FIGURE 3.1  Ocarina command melodies in The Legend of Zelda: Ocarina of Time.

beyond the basic gameplay. Last, the remaining ocarina melodies allow the player to fast-travel (or “warp”) to specific locations. Titles and musical notation for these twelve ocarina commands are given in Figure 3.1.

Plot-advancing melodies Two ocarina commands act primarily to advance the plot; the first is Zelda’s Lullaby and the second is the Song of Time. Zelda’s Lullaby, the most frequently used ocarina song of the game, functions actively as the principal game-event trigger, revealing and clearing

90

MUSIC VIDEO GAMES

paths. Some examples of events triggered by Zelda’s Lullaby include earning the Great Fairy’s magic spells and special attacks (one of which is required to complete the game), gaining access to Zora’s Domain and areas of Goron City, raising and lowering the water level to access different floors of the Water Temple and the well dungeon in Kakariko Village, and unlocking a gate within the Spirit Temple. All of these actions are required to complete the main storyline. However, in addition to its active diegetic role, Zelda’s Lullaby also recurs non-diegetically: It is heard when Link interacts with Zelda or when he visits her home, Hyrule Castle Courtyard. This mixes Liebe’s reactive (triggered by location or character) and proactive (learning the melody through forced repetition) functions with the more active, event-triggering role of the ocarina commands. The second plot-advancing melody, the Song of Time, also combines reactive, proactive, and active functions. Actively, it is used to move large blue blocks that act as barriers in tunnels and dungeons; performing this melody is required in four of the temples that the player must complete within the game.9 It is also used to trigger the second half of the plot: after receiving the Ocarina of Time, which Zelda drops during her kidnapping, Link then plays the Song of Time in the Temple of Time, triggering a series of cut-scenes and the ability to switch between “past-time”, where Link exists as a child, and “future-time”, where Link exists as a young man.10 Nondiegetically, the Song of Time is heard as environmental music in the Temple of Time; the complete melody heard at this location, which begins with the six-note Song of Time on the ocarina and is then extended, sounded with synthesized male voices in an unmetered style, suggestive of a Gregorian chant to suggest the sacredness of this location within the gameworld.11

Assistance melodies Other ocarina melodies function primarily to assist the player in finding hidden treasure and to access help from non-player characters. Like the plot-advancing melodies, each of these assistance melodies are six notes long, with the last three notes repeating the first three.

ACTIVE INTERFACES AND THEMATIC EVENTS

91

Epona’s Song (shown in Figure 3.1) is a diegetic melody used to call Link’s horse. Epona’s Song is heard non-diegetically in an extended, elaborated version quite early in the game and is associated with both a location, Lon Lon Ranch, as well as the character Malon. The player first hears the melody when meeting Malon in Hyrule Castle Town and the melody recurs in later interactions with this character. It is also heard at Lon Lon Ranch, Malon’s home. In these non-diegetic contexts, the music heard is an extended version of the six-note Epona’s Song with added accompaniment and melodic variation. Sun’s Song is shown in Figure 3.1(d); this melody’s active role is to change night into day (and vice-versa), a feature that is helpful for several optional side quests and which makes it easier to kill undead enemies but is not required to complete the main quest of the game. Unlike the other plot-advancing and assistance songs, the Sun’s Song is not heard non-diegetically, and consequently, no longer version of the song is heard elsewhere within the game.

Blended roles and functions Next, two ocarina commands blend plot-advancing and assistance roles: At some moments and locations the command is mandatory to progress in the plot, while in other contexts the command triggers a non-essential “assistance” feature. Both are heard diegetically and non-diegetically. Saria’s Song, shown in Figure 3.1(e), is initially heard as background music in a longer, elaborated version of the melody within the Lost Woods, but has an assistance role as it is heard when a secret shortcut to the Lost Woods is near, a reactive function. The extended melody is also heard non-diegetically (as background music) when interacting with the character Saria. Later in the game, though, the Saria’s Song command is essential to open the path to the Forest Temple and thus functions actively as a plot-advancing melody. It can also be used actively to call Saria for advice and to reveal bonus items within the Lost Woods. This blend of plot-advancing and assistance roles is also seen in the Song of Storms (Figure 3.1(f)). A non-diegetic extended version of the Song of Storms is heard as background music within the windmill in Kakariko Village. Activating the Song of Storms diegetically creates

92

MUSIC VIDEO GAMES

a weather storm in the game; it is needed to drain the well in Kakariko Village, where an essential quest item is located, but in other cases is used to reveal optional bonus items and locations such as caves and fairies. Why employ the same music both diegetically and nondiegetically? Several reasons are possible. Van Elferen observes that “hearing a piece of music bring[s] back former experiences of listening to the same music” (2011, 31); Zelda’s Lullaby is one of the more famous melodies of the Zelda series, appearing in twelve different games, and would be well-known to those who had played The Legend of Zelda: A Link to the Past (Nintendo 1992), thus reinforcing the continuity within the series (despite its lack of direct sequentiality). Second, the repeated hearings makes the melody much easier to recall, important since the player must frequently replay it as a game command. Florian Mundhenke argues that there is a “need of relative[ly] simple structures since the cues of diegetic space (fighting, walking and collecting) already challenge the player considerably, so that music can be counterproductive if it is too dominant or sophisticated” (Mundhenke 2013, 113), and that certainly holds true in this case. Third, Collins postulates that this type of diegetic sound creates engagement with the player; games should be understood “not as texts but as sites of participation and practice where players construct meanings” (Collins 2013, ix) with both the game controller and the in-game avatar functioning as extensions of the self (Collins 2013, 41–2). She argues this is particularly the case with sound: … when players produce the sounds in a game (in the sense that they are immediately receiving feedback for their own actions), they are experiencing those sounds cognitively as “their” sounds. Because players receive immediate feedback tied to their own proprioceptive or kinesthetic actions, the sounds become a part of self rather than other. In this way, sound helps players to become a character, or perhaps more accurately, their character can become a part of their sense of self. (Collins 2013, 44)12 Van Elferen further supports this observation, postulating that video game music complements and intensifies the player’s perception of

ACTIVE INTERFACES AND THEMATIC EVENTS

93

virtuality, immersion, and flow (that is, the sense of being completely involved in the gameworld) (van Elferen 2011, 31). Interestingly, Collins’ argument for kinesthetic actions, reinforcing the player’s self-identification with their virtual avatar, mirrors comments made by Ocarina of Time’s composer, Koji Kondo. Given that Kondo is Nintendo’s most famous composer, having written the music for both the Super Mario Bros. (Nintendo) and Legend of Zelda series throughout the 1980s and 1990s, and leading Nintendo’s EAD sound team since the early 2000s, his personal aesthetic has had a strong influence on Nintendo’s sound and game design (Greening 2012). In particular, Kondo has expressed a strong desire for the music to synchronize with gameplay in order to avoid “disconnects” between the player and the game; sound and music thus are essential components in creating an immersive game experience (Paul 2013, 73). Given this belief, Kondo’s use of diegetic music throughout the Legend of Zelda series is not surprising; the active interface of the controller-as-instrument and the active function of the ocarina command melodies contribute strongly to the immersion effect.

Warp melodies Six melodies are yet to be examined; while two categories of diegetic function for ocarina melodies in Ocarina of Time have now been discussed, the other six melodies allow the player to fast-travel to designated locations. These are given in parts (g)–(l) of Figure 3.1, and each takes its name from a historical Western art music genre: Bolero of Fire, Minuet of Forest, Nocturne of Shadows, Prelude of Light, Requiem of Spirit, and Serenade of Water.13 These six warp songs do not occur non-diegetically at particular locations or with particular characters, but rather are only employed actively by the player after their initial presentation, unlike the plot-advancing songs and assistance songs. The lack of non-diegetic settings means that these melodies are heard less frequently, but this is not surprising since they are less essential to the gameplay: The warp songs act as short-cuts from one location to another, and the game can be played without ever accessing these commands should the player choose.

94

MUSIC VIDEO GAMES

Form As we have just seen, the twelve ocarina melodies seem to fall into two categories: the first category (plot-advancing and assistance melodies) blends diegetic and non-diegetic roles, whereas the second category (warp melodies) sticks exclusively to diegetic presentations. Interestingly, their musical structures show clear distinctions as well. The plot-advancing and assistance melodies are each six notes long and feature a repetition of the first three notes of each melody, whereas the warp melodies vary in length from five to eight notes and do not feature the same three-note repetition (the Bolero of Fire and Prelude of Light feature two-note repetition, while the other four warp melodies do not incorporate repetition at all in their shorter versions). This differentiation into two categories continues into the extended version of the music that is heard when learning each ocarina melody. The extended versions of the plot-advancing and assistance melodies (with the exception of the Sun’s Song, which has no extended version) conform to standard phrase lengths and structures: each features 8-bar phrases and conforms to either period or sentence forms, standard models of phrase structure in Western classical style (Caplin 1998, 9–12).14,15 The extended versions of the warp melodies, on the other hand, are more irregular. Some are not standard phrase lengths (the Requiem of Spirit, for example, features a division into 6 bars rather than 8). Most do not clearly adhere to sentence or period form, although all six feature a two-fold repetition of the opening motive: the second half of each extended melody either trails off with new material (such as in the Serenade of Water) or else features a third repetition of the opening motive followed by a single long note at the end of the phrase to signify a weak cadence (such as in the Requiem of Spirit). Why this differentiation? In the course of the game, the player must memorize each of the ocarina themes for future use (there is a reference screen should a player forget the appropriate melody; however, accessing this screen interrupts the gameplay and thereby weakens the player’s game immersion). The plot-advancing and assistance melodies are used far more frequently given that they are required at certain locations, whereas the warp songs need not be

ACTIVE INTERFACES AND THEMATIC EVENTS

95

used at all should the player so choose. Consequently, the greater repetition of the six-note plot-advancing/assistance melodies, their regularity of phrase structure, and the incorporation of these melodies into non-diegetic settings, such as background music, reinforces the player’s memorization of these tunes.16

Leitmotif While musical structure certainly aids the player’s memory of ocarina commands, narrative associations of melodies within the game also contribute to both memory and game immersion. Zach Whalen discusses Ocarina of Time in an article revolving around game immersion, flow, pleasurability, and engagement (2004). Although diegetic music is not the main focus of his paper, in the course of the discussion he identifies how diegetic music engages the player through links between environmental and action-based settings of the ocarina themes. Whalen further conceptualizes this thematic usage as “Wagner’s leitmotifs acting in reverse,” where the theme identifies a particular area of the gameworld. Many scholars have discussed the “signpost” approach to leitmotif presentation, particularly in the domain of film music study, which Whalen seems to suggest here.17 However, the musical settings in Ocarina of Time, coupled with the active interface and functional role of music as a game-command, in fact suggest the opposite: that, in Kondo’s attempt to fully integrate video/game/music, he aims for a more developmental use of thematic recurrence, a true leitmotif in the Wagnerian sense. Rod Munday supports this view, stating that “video games have aligned themselves with an aesthetic tradition of ‘mythic drama’ found in both opera and film,” and specifically likens this alignment to Richard Wagner’s compositional techniques and philosophies (Munday 2007, 58). How applicable, then, is the concept of leitmotif, and how might it influence our understanding of the interactions between diegetic and non-diegetic sounds in Ocarina of Time? Arnold Whittall presents the following definition for the term “Leitmotif” in Grove Music Online: A theme, or other coherent musical idea, clearly defined so as to retain its identity if modified on subsequent appearances, whose

96

MUSIC VIDEO GAMES

purpose is to represent or symbolize a person, object, place, idea, state of mind, supernatural force or any other ingredient in a dramatic work. This definition clearly holds for our first category of melodies, the plot-advancing and assistance melodies (Figure 3.1(a)–(f)). Using Epona’s Song as an example, we have an identity (the six-note initiating motive) that represents a place (Lon Lon Ranch), a person (Malon), and a supernatural force (the ability for Link to call his horse from anywhere). The only component of this definition missing is whether the game constitutes a “dramatic work”, but given that the game follows a relatively linear plot with characters, locations, and events, this would be difficult to deny by most definitions (especially given the fantasy theme of the game, which like Wagnerian opera employs storylines evoking a larger mythology). In addition to Whittall’s definition, many scholars further argue that leitmotif requires transformation over time. Stephen C. Meyer identifies the evolution of the term leitmotif from Wagner’s time into the present and points out that leitmotif includes both the concept of an organic growth process and of motivic transformation that reflects and articulates the drama of the work (Meyer 2012, 103). The reconfiguration of the diegetic Song of Time into the non-diegetic environmental music of the Temple of Time achieves such an effect, as previously discussed, in its transformation from six-note motive to extended melody, communicating the sanctity and religious character of the location due to its similarities to Gregorian chant style. The “organic growth process” also occurs in the first six ocarina melodies; for example, returning to Epona’s Song, the theme is modified but still retains its identity. The six-note command for calling the horse Epona becomes the first six notes of the monophonic melody sung by Malon, which then is heard as the environmental music with added harmonic accompaniment at Lon Lon Ranch. However, some might argue that these elements of growth are single transformations rather than an ongoing growth process. If we were to look further than Ocarina of Time, however, this organic growth process continues throughout the Legend of Zelda series. For example, in 2002’s The Legend of Zelda: The Wind Waker (Nintendo 2002), a harmonically modified setting of Zelda’s Lullaby is heard

ACTIVE INTERFACES AND THEMATIC EVENTS

97

when one of the main characters is revealed to be Princess Zelda in disguise; in that instance, the melody’s length is altered, set in a different key (a minor mode to suggest the ominousness of villain threatening the main characters), and re-harmonized. Furthermore, in The Legend of Zelda: Skyward Sword (Nintendo 2011), Zelda’s Lullaby is reversed, set in a different mode, and re-harmonized to become a new song called Ballad of the Goddess, which is used within the game to suggest a link between the character Zelda and the world’s supernatural forces. This suggests an expectation that at least some players will be familiar with musical themes from previous iterations of the series, and provides a layer of insider knowledge that creates a more nuanced experience of the plot. However, a scholarly approach that suggests film and game are only capable of using the “signpost” approach to leitmotif, with only simplistic associations, such as those between location and musical theme, misses this sort of nuanced interpretation. Take, for example, Munday’s interpretation of Donnelly: Both [games and film] employ the notion of the leitmotif, although in this context it is the particularly cinematic notion of the leitmotif that Wagner would have disparaged and Kevin Donnelly referred to as a “signpost” … Cinematic leitmotifs are used primarily as storytelling signposts that tell the player a certain environment is dangerous, or that a character is not to be trusted. (Munday 2007, 62) Ocarina of Time, with its organic growth process and multiple levels of signification, clearly takes on more than a signpost role. Why discuss leitmotif at all? So far this chapter has discussed the uses of diegetic sound in Ocarina of Time, and as part of this has also examined the formal structure of the ocarina command melodies, interface, the interaction of diegetic and non-diegetic roles, and game function. As previously examined, the repetitive structure of these melodies and their use through both diegetic and nondiegetic presentations are a direct result of Kondo’s desire for game immersiveness. The repetitive melodic structure acts as a mnemonic aid for the player. Consider some comments by Biancorosso, in “Memory and the Leitmotif in Cinema”:

98

MUSIC VIDEO GAMES

The basis of leitmotifs lies in the ability of music to make us recall the circumstances in which it was first heard … leitmotifs unfold, and evolve dramatically, over time, as if embodying—as much as referring to—an event person, or quality, fleshing out, in the form of sound, their presence. (Biancorosso 2013, 217–18)18 Biancorosso further postulates that the context of leitmotifs evolves over time, taking on new layers of meaning for each of their iterations within the larger work, which he calls a “reflexive play on music.” He furthermore states that this reflexive state “heightens the spectators’ appreciation of their own experience” (Biancorosso 2013, 218), which contributes to what we have been identifying as immersion and engagement. Indeed, the “layers of meaning” generated within Ocarina of Time occur not only as observed meanings, but rather directly engage the player in a fully participatory role: The game is not possible to complete if the player does not initiate these leitmotifs, each a thematic event.19 In van Elferen’s words, “the fact that the player produces [the diegetic music] means that the player is a founding part of the diegesis” (2011, 35).

Player as creator Those who are familiar with Ocarina of Time may note at this point that I have not discussed the final ocarina melody, the Scarecrow’s Song. Upon visiting a friendly scarecrow at Lake Hylia, the player is able to record their own melody, effectively using the controller to compose his or her own melody rather than simply rote-repeating a previously given tune. Later repetitions of the player’s self-generated melody causes a scarecrow to appear at hard-to-access locations, and Link is able to rappel to the scarecrow to visit previously inaccessible locations. The player therefore acts as composer, not just as imitator, going beyond the previously described active function into a new role that requires both the creation and retention of a musical object by the player (perhaps the term “cre-active” would be appropriate in this context). The player is directly engaged in creating the gameworld, an example of Collins’ belief of investment within gameplay being causative of the player’s sense of extension of self.

ACTIVE INTERFACES AND THEMATIC EVENTS

99

Indeed, Munday sees the creation of a gameworld, and the player’s immersion into this gameworld, as one of the essential roles of video game music (Munday 2007, 54). One could argue that all gamers are creators in a certain sense in that their actions are required to initiate the gameworld, and that each game decision creates a game space that (depending on the game design) may differ from that created by other players. Thus the game is a true “site of participation,” to use Collins’ terminology, and music should be no less important than any other feature in this context. Moormann discusses this, referring to the player as “co-author,” and that the game creator and consumer have a much closer relationship than in other media (2013, 8).

Conclusion Collins identifies how “there is a distinction between listening to sound, evoking sounds already made (by pressing a button, for instance), and creating sound (making new sounds)” (2013, 2), all three of which are present in Ocarina of Time. All three are also ways in which diegetic sound is processed by the player in the game: The player hears a melody, repeats it back, initiates it to trigger game events, and composes a melody of their own. Earlier, I discussed game interfaces and a differentiation of music within video games into linear, reactive, proactive, and active functions. The commonality between these concepts, as well as many of the others presented in this chapter, is immersion. Collins, in her book Playing with Sound, speculates that immersion in the game creates an empathetic game experience (2013, 95). She uses the term kinesonic congruence throughout the book to discuss this effect, and includes interfaces as one way by which immersion can be created, particularly in a role where the player sees the virtual instrument as an extension of the self: as we increase our expertise using the controller (thus reducing the feeling of mediation), the gap between person and avatar is reduced. The more adept that players become at using the device, the less they notice the controller, and the more engaged with the game they can become. (Collins 2013, 41; emphasis added)

100

MUSIC VIDEO GAMES

In the introduction to this chapter, I presented Jesper Juul’s view that a shift of focus from avatar to real-life player makes it feel as though the player is making actual music. Juul’s expanded discussion of this topic suggests a development toward procedural memory that mimics that of learning a musical instrument, further increasing the player’s sense of game immersion: Interestingly, this is quite similar to learning to play an instrument: playing sheet music on the piano at first feels exactly like playing Guitar Hero: you follow a notation telling you what to do. If you press the right keys on the piano, music appears. If you press the wrong keys, music does not appear. It is only through practice that you begin to feel a direct connection between the piano keys and the notes that come out of the piano. Subjectively, playing Guitar Hero isn’t any less real than playing a piano is when you first begin to learn to play, and this is probably why you do feel as if you are playing music, playing Guitar Hero. (Juul 2010, 115) This type of kinesthetic immersion is often discussed in relation to games such as Guitar Hero, which are almost entirely structured around virtual music-making, a genre not surprisingly called “music games.” The fact that Ocarina of Time shares this creation of procedural memory suggests that this game is not simply an adventure game with musical themes but rather blends the two genres of adventure and music games. Other scholars also discuss how immersion affects the gamer’s experience. Whalen distinguishes between immersion, which he interprets as the player’s sense of in-the-moment-ness, versus engagement, the process of the player learning the behaviors/rules of the gameworld.20 Immersion focuses on the idea of creating a new reality for the player, whereas engagement focuses on the notion that the player remains aware of the non-reality of the game universe (Whalen 2004). Munday believes that sound enriches visual information, and that it is a key component of creating a gameworld: Video games are “cocooning players in a sonic zone of protection, where they can forget about interruptions and lose themselves in the game” (Munday 2007, 55). The recurring thread between these scholars is a suggestion that the blend of diegetic and non-diegetic

ACTIVE INTERFACES AND THEMATIC EVENTS

101

musical roles in Ocarina of Time—a process that van Elferen would term “supradiegetic” (van Elferen 2011, 35)—are an attempt to create an immersive world: While the player could play the game muted, with no sound whatsoever, thematic connections between characters, locations, and actions would be lost, as would the elements of foreshadowing present in later Zelda games, such as Skyward Sword and Wind Waker. The player would thus experience a less nuanced storyline, experiencing text but not sub-text. Through listening to sound (in order to find shortcuts or by hearing the return of ocarina melodies in non-diegetic contexts), evoking sounds (by activating ocarina melodies), and creating sound (by composing the Scarecrow’s Song), the player is fully engaged and immersed in a sonic world inherently tied to the game’s narrative.

Notes 1

For example, the song “Zelda’s Lullaby” was first used in The Legend of Zelda: A Link to the Past (Nintendo 1992), but has also occurred in twelve other games from the series almost exclusively in connection to the character Princess Zelda (Zeldapedia, “Zelda’s Lullaby,” 2015).

2

The game has been ranked the best-rated video game of all time by sources such as Metacritic (2015).

3

Given that the study of video game music is a relatively new field, analysis in this area often borrows terminology and concepts from film music analysis.

4

“To music is to take part, in any capacity, in a musical performance, whether by performing, by listening, by rehearsing or practicing, by providing material for performance (what is called composing), or by dancing. We might at times even extend its meaning” (Small 1998, 9).

5

Isabella van Elferen suggests the use of the term “half-diegetic” for gameplay that blends both diegetic and non-diegetic music (such as non-diegetic background music occurring at the same time as diegetic cues to the in-game characters). However, given that this chapter extensively discusses how the function of musical themes shifts between diegetic and non-diegetic roles, I have chosen to maintain the distinction between these functions throughout the chapter (van Elferen 2011, 33).

102

MUSIC VIDEO GAMES

6

For example, in The Legend of Zelda: The Wind Waker, the player uses a stylus as a conducting baton, and in The Legend of Zelda: Skyward Sword, the player plays a virtual harp by moving the motion-sensitive game controller in a strumming gesture.

7

Liebe 2013, 47; although he acknowledges that this categorization was first proposed by Georg Spehr, Dennis Mathei, and himself.

8

The term non-player character (generally abbreviated as NPC) refers to an in-game character controlled by the computer rather than the player.

9

“Completing a level” can be accomplished in varying ways depending on the genre of game. In Ocarina of Time, like many fantasy/adventure games, the player completes the temple levels by navigating through the assigned area to find and conquer the “dungeon boss”, who requires more time and effort to defeat than a regular opponent. The rewards for completing this “boss fight” include new weapons, treasures, or other items essential to completing the storyline of the game (in this case, the player earns each of the five medallions after completing the corresponding temple level).

10 A cutscene consists of “film or animation scenes between sections of gameplay” (van Elferen 2011, 33). 11 For example, the melody emphasizes an initial perfect fifth interval, occurs in the Dorian mode, and is sounded using (synthesized) male voices, features common to a Gregorian chant. 12 Collins’ term “kinesthetic actions” refers to a physical action by the game player that triggers an in-game event. Examples include pushing a button on the controller, moving the controller in the case of motion-sensitive controllers, such as those of the Wii systems and the current generation of Sony and Xbox devices, or performing physical gestures that are recognized by motion-sensing devices, such as the Xbox 360’s Kinect sensor. Her book, Playing with Sound: A Theory of Interacting with Sound and Music in Video Games, examines at length how such gestures co-ordinate (or lack co-ordination) with game sound, a phenomenon she names “kinesonic congruence.” 13 Note, however, that these melodies do not necessarily conform to the typical features of the genres suggested by their titles. 14 The 8-bar length is, to an extent, determined by the transcriber; the melody could equally be represented by keeping the proportions but increasing the unit length of each motivic segment to produce a 4-bar phrase, 16-bar phrase, or other proportional length. However, this does not change the observation here: phrases with bar lengths that are powers of 2 (2 bars, 4 bars, 8 bars, 16 bars, and so

ACTIVE INTERFACES AND THEMATIC EVENTS

103

forth) are all generally considered to be normative phrase lengths (Berry 1966, 2, 14–15). 15 Period features two phrases whose first halves are similar and whose second halves differ, while sentence features a basic idea repeated twice, followed by fragmentation and a cadence. Of the five melodies, Zelda’s Lullaby, Epona’s Song, Saria’s Song, and Song of Storms conform to period structure, while the Song of Time is a sentence form slightly modified to account for the melody’s “free meter” style. 16 My personal experience playing the game was that the warp songs were much more difficult to remember than the plot-advancing and assistance melodies: I had to access the menu approximately three times more often when using warp melodies than when using plot-advancing and assistance melodies. However, while I believe Kondo structured the latter two melody types to make them more memorable, I do not believe he intentionally made the warp melodies less memorable; rather, I believe this was a sideeffect from his use of increased thematic variety, expanded phrase structures, and wider melodic contours. 17 For example, Munday 2007; Moormann 2010; and Meyer 2012. 18 Van Elferen further ties this idea to previous research on emotional and cognitive processes of listening (2011, 31). 19 Whalen discusses these in terms of aural “cues”, but they function at more than just a surface level in Ocarina of Time. 20 Van Elferen also discusses engagement, but using the term “magic circle” instead (2011, 30–1).

References Berry, Wallace. 1966. Form in Music: An Examination of Traditional Techniques of Musical Structure and their Application in Historical and Contemporary Styles. Englewood Cliffs, NJ: Prentice-Hall Inc. Biancorosso, Giorgio. 2013. “Memory and the Leitmotif in Cinema.” In Representation in Western Music, edited by Joshua S. Walden, 127–43. Cambridge, UK: Cambridge University Press. Caplin, William E. 1998. Classical Form: A Theory of Formal Functions for the Instrumental Music of Haydn, Mozart, and Beethoven. Oxford: Oxford University Press. Collins, Karen. 2008. Game Sound: An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design. Cambridge, MA: The MIT Press.

104

MUSIC VIDEO GAMES

Collins, Karen. 2013. Playing With Sound: A Theory of Interacting with Sound and Music in Video Games. Cambridge, MA: The MIT Press. Greening, Chris. 2012. “Koji Kondo Profile.” Game Music Online. Accessed January 23, 2016. http://www.vgmonline.net/kojikondo/. Juul, Jesper. 2010. A Casual Revolution: Reinventing Video Games and Their Players. Cambridge, MA: The MIT Press. Liebe, Michael. 2013. “Interactivity and Music in Computer Games.” In Music and Game: Perspectives on a Popular Alliance, edited by Peter Moormann, 41–62. Wiesbaden, Germany: Springer VS. Metacritic. 2015. Accessed February 20, 2015. http://www.metacritic. com/browse/games/score/metascore/all/all?sort=desc. Meyer, Stephen C. 2012. “‘Leitmotif’: On the Application of a Word to Film Music.” Journal of Film Music 5 (1–2): 101–8. Moormann, Peter. 2010. Spielberg-Variationen: die Filmmusik von John Williams. Baden-Baden: Nomos, Ediition Reinhard Fischer. Moormann, Peter. 2013. “Foreward.” In Music and Game: Perspectives on a Popular Alliance, edited by Peter Moormann, 7–9. Wiesbaden, Germany: Springer VS. Munday, Rod. 2007. “Music in Video Games.” In Music, Sound and Multimedia: From the Live to the Virtual, edited by Jamie Sexton, 51–67. Edinburgh: Edinburgh University Press. Mundhenke, Florian. 2013. “Resourceful Frames and Sensory Functions—Musical Transformations from Game to Film in Silent Hill.” In Music and Game: Perspectives on a Popular Alliance, edited by Peter Moormann, 107–24. Wiesbaden, Germany: Springer VS. Neumeyer, David. 2009. “Diegetic/Nondiegetic: A Theoretical Model.” Music and the Moving Image 2 (1) (Spring): 26–39. Paul, Leonard J. 2013. “Droppin’ Science: Video Game Audio Breakdown.” In Music and Game: Perspectives on a Popular Alliance, edited by Peter Moormann, 63–80. Wiesbaden, Germany: Springer VS. Small, Christopher. 1998. Musicking: The Meanings of Performing and Listening. Middletown, CT: Wesleyan University Press. van Elferen, Isabella. 2011. “¡Un Forastero! Issues of Virtuality and Diegesis in Videogame Music.” Music and the Moving Image 4 (2) (Summer). Whalen, Zach. 2004. “Play Along: An Approach to Video Game Music.” Game Studies 4 (1), November 2004. www.gamestudies.org/0401/ whalen/. Whalen, Zach. 2007. “Case Study: Film Music vs. Video-Game Music: The Case of Silent Hill.” In Music, Sound and Multimedia: From the Live to the Virtual, edited by Jamie Sexton, 68–81. Edinburgh: Edinburgh University Press. Zeldapedia. 2015. Accessed February 20, 2015. http://zelda.wikia.com/ wiki/.

ACTIVE INTERFACES AND THEMATIC EVENTS

105

Games Harmonix. Guitar Hero. [Playstation 2]. RedOctane: Mountain View, CA, 2005. Konami. Dance Dance Revolution. [Arcade, et al.]. Konami: Tokyo, 1998. Nintendo EAD. The Legend of Zelda: A Link to the Past. [Super NES, et al.]. Nintendo: Kyoto, 1992. Nintendo EAD. Legend of Zelda: Ocarina of Time. [Nintendo 64, et al.]. Nintendo: Kyoto, 1998. Nintendo EAD. The Legend of Zelda: The Wind Waker. [GameCube]. Nintendo: Kyoto, 2002. Nintendo EAD Group 3. The Legend of Zelda: Skyward Sword. [Wii]. Nintendo: Kyoto, 2011. Nintendo R&D Group 4. Super Mario Bros. [NES/Famicom]. Nintendo: Kyoto, 1985.

4 Sample, Cycle, Sync: The Music Sequencer and Its Influence on Music Video Games Michael Austin

A

t the most basic level, a sequencer can be defined as “a musical interface designed to record, edit and playback audio samples in pattern format for both music composition and performance” (Arar and Kapur 2013, 383). While music sequencers became increasingly popular during the 1970s and 1980s in the form of drum machines and groove boxes, they have existed long before in the form of mechanical musical toys and automatic instruments. As early as 875, the “hydraulis,” an automatic hydraulic organ, automatically played interchangeable cylinders with raised pins; automatic “mechanical carillons” in the Low Countries used a similar system in the beginning of the thirteenth century (Fowler 1967, 45). More importantly (and much like video games), automatic instruments were often dismissed as “toys” or “novelties.” Eighteenth-century music boxes use carefully placed raised pins on cylinders (much like those used in the previously mentioned water organs and carillons) that strike tuned metal tines as it spins. E.S. Votey’s 1897 invention, the “Pianola,” introduced an element of interactivity to automatic instruments by allowing a “player” to control the speed that paper rolls were fed

108

MUSIC VIDEO GAMES

through a player piano to allow the user to “interpret” the music (by adding variations in tempo, rubato, etc.). These paper rolls were the latest development in pneumatic player piano technology; sequences of holes in the roll allowed air to pass through, triggering the piano’s hammer to strike a particular string, thus sounding the appropriate note at the proper time. Disks and rolls for player pianos and pianolas were among the earliest forms of “recorded music,” and provided musical entertainment even in homes where no trained musicians lived. Throughout the twentieth century, electronic instruments, modular synths, and multitrack tape paved the way for modern sequencers. Although magnetic tape loops were used to make percussive drum sounds in the early part of the century, the protosequencers of the 1940s and 1950s, such as Raymond Scott’s “Wallof-Sound” and the Wurlitzer “Sideman” were electro-mechanically operated, usually employing rotating disk switches or stepping relays to produce a sequence of tones and artificially produced drum noises. Sequencers, as we know them today, came to fruition at the end of the 1960s with the introduction of Robert Moog’s “Moog 960” in 1968. Created as a module for his famous modular synthesizers, the 960 was among the first commercially available (and viable) sequencers. It “contained three rows of eight value knobs and allowed for a three-value sequence of up to eight steps controlled by a clock. Each of the three banks could steer three different voltage-controlled oscillators (VCO), amplifiers (VCA) and filters (VCF)” (Arar and Kapur 2013, 384). Other companies, such as the Keio Organ Company (now better known as Korg), Roland, E-mu, and Yamaha, released more developed drum machines that used sampled recordings of drum kids or electronically synthesized sounds. Step sequencers emerged in the 1970s, giving users the ability to program a sequence of eight or sixteen notes that played automatically in a continuous loop. When MIDI was introduced in the 1980s, computers expanded the number of tracks used by step sequencers to well beyond 16. Early hardware sequencers, like the E-mu Emulator II (or EII), the Roland D20 S&S (synthesizer and sequencer), and Korg’s MI, and their contemporary software counterparts (now most often taking the form of digital audio workstations, or DAWs), such as Logic Pro, Garage Band, and

SAMPLE, CYCLE, SYNC

109

Ableton Live, have provided users with the ability to record samples of music on multiple tracks, re-arrange them, change their pitch and timbre, and play them on command. Also in the 1980s, video games developed from strictly arcade attractions into home entertainment systems. Although systems like Atari existed for almost a decade before the video game crash of 1983, home videogame consoles became a big hit in the West with the North American introduction to the Nintendo Entertainment System in 1985 (and subsequently in Europe in 1986 and Australia in 1987). Musical games, such as SIMON (discussed in Chapter 2) and music-themed pinball and arcade games existed since the 1970s, but the genre of music games did not really emerge until the late 1990s with PaRappa the Rapper (NanaOn-Sha 1996), and did not garner the attention of mass audiences until the release of Dance Dance Revolution (DDR; 1998), Guitar Hero (2005), and Rock Band (2007). Video games and music sequencers even crossed paths well before music games emerged as a recognized genre. Atari ST, a personal computer released by the video game manufacturing company that was especially famous for its ability to run sequencing programs. In this chapter, I discuss the music sequencer and its many manifestations and adaptations within music video games using Chiptune Runner (Evil Indie Games 2013) and My Singing Monsters (Big Blue Bubble Inc. 2012) as case studies. These titles demonstrate the various ways in which some music video games emulate sequencers and share some of the same controls, affordances, and mechanics. Additionally, these selections demonstrate ways in which music video games function as musical instruments and facilitate musical performances and opportunities for composition.

MDA, sequencers, and music video games The interactive nature of video games presents unique challenges and opportunities regarding the ways in which we can approach and understand them. Although millions of people may purchase and play the same game, their in-game choices result in almost as many outcomes. In this way, each copy of the same video game could be

110

MUSIC VIDEO GAMES

understood as an individualized artifact in and of itself; likewise, each game changes and evolves into a different artifact each time a player plays it and reveals something new within the game. Understandably, this iterative process makes analyzing a video game a hermeneutic adventure in its own right. In their essay, “MDA: A Formal Approach to Game Design and Game Research,” authors Robin Hunicke, Marc LeBlanc, and Robert Zubek present an invaluable formal methodology for analyzing video games by taking into consideration a game’s mechanics (“the particular components of the game, at the level of data representation and algorithms”), dynamics (“the run-time behavior of the mechanics acting on player inputs and each others’ outputs over time”), and aesthetics (“the desirable emotional responses evoked in the player, when she interacts with the game system”) (Hunicke et al. 2004, 2). This approach accounts for several perspectives, especially from both the consumer’s and developer’s point of view. Hunicke, LeBlanc, and Zubek argue that “… games are more like artifacts than media. By this we mean that the content of a game is its behavior—not the media that streams out of it towards the player” (Ibid., original emphasis). In discussing common features and differences among sequencers and music video games, an analysis of the “behavior” of both artifacts is an interesting way to understand the ways in which these artifacts influence our own behavior and our interaction with them when we create them and play them.

Mechanics Regardless of whether or not these parts are digital or physically manufactured, creating games—and sequencers—involves hours of coding, programming, designing, etc.—all of which come together to create an experience to meet the needs of the end user. Lev Manovich describes the way software performs the tasks that create this experience, writing: Computer program can use a variety of components to create these performances: design templates, files stored on a local

SAMPLE, CYCLE, SYNC

111

machine, media from the databases on the network server, the real-time input from a mouse, touch screen, joystick, our moving bodies, or some other interface. Therefore, although some static documents may be involved, the final media experience constructed by software usually does not correspond to any single static document stored in some media. (Manovich 2013, 34) The mechanics of a game comprises these “moving parts,” plus the rules that govern how they interact with each other and how the player interacts with them. Core mechanics are the basic actions that make up the playing of a game. Other types of extra— or “satellite”—mechanics include “enhancement” (when a player earns a “power up,” for example), opposition mechanics such as special enemies or obstacles, and alternate mechanics (e.g., giving a player the option to swim to the end of a level or fly over the water and avoid swimming altogether) (Fabricatore 2007, 13). Taking Tetris as an example, the console, code, video screen, sound card, visual and sonic assets, and a host of other components determine how the game “works,” facilitating the ways in which a player can move various shapes (called “tetriminos”) and strategize ways to fit them together as they fall from the top of the screen. The core mechanics of Tetris are a small repertoire of movements and actions the player can make in playing the game: move the shapes to the left and right, make them fall faster, and rotate the shapes ninety degrees clockwise or counterclockwise at a time. Mechanics found in other games are noticeably absent from or inconsistent in Tetris— tetriminos sometimes float over empty gaps where they would normally be expected to fall, and other times, when tetriminos are arranged in such a way that they create a horizontal line of ten units, this line disappears and the blocks above it fall onto the surface (or into the empty space) below.

Dynamics The “moving parts” that comprise both sequencers and videogames and the ways in which they work together determine what the user can or cannot do with the artifact, and whether or not the player uses

112

MUSIC VIDEO GAMES

them for their intended purposes. These affordances are the game’s (or instrument’s) dynamics. For example, the sequencer can “store information such as pitches, note durations, dynamics, and so on, but they can record the information in ‘real time’ much like a tape recorder; unlike a tape recorder, however, they do not store sounds,” (Théberge 1997, 222) but rather, they simply access them from a computer’s memory. So, these affordances allow a sequencer’s user to reproduce or manipulate relatively short sounds in real time but do not really facilitate the archiving of musical performances. Similarly, the fact that a Guitar Hero controller only has five buttons (rather than six strings) that do not correspond to any particular notes severely limits its use as a musical instrument capable of producing all twelve tones of a chromatic scale, but they enable players of the video game to quickly and easily imagine they are playing the guitar by emulating the low versus high interface of a guitar (i.e., low notes are played by depressing strings within the frets found high up on the guitar’s neck and high notes are played using frets found further down the neck).

Aesthetics According to Hunicke, LeBlanc, and Zubeck, the aesthetics of a game are the elements that make the game “fun.” These elements describe the player’s interaction with the game. As an example, they list several words used to describe the aesthetics of a game: Sensation (game as sense-pleasure), Fantasy (game as makebelieve), Narrative (game as drama), Challenge (game as obstacle course), Fellowship (game as social framework), Discovery (game as uncharted territory), Expression (game as self-discovery), and Submission (game as pastime) (Hunicke et al. 2004, 2). These terms help to describe the interface between a player and a game’s mechanics and dynamics. For example, the Super Mario Bros. series offers players fantasy in the fictional Mushroom Kingdom, narrative in the storyline wherein Mario and Luigi defeat Bowser to save Princess Peach (formerly known as Princess Toadstool), and challenge as they defeat enemies and overcome obstacles to reach and rescue the princess.

SAMPLE, CYCLE, SYNC

113

MDA of sequencers While the MDA framework was specifically designed to describe the mechanics, dynamics, and aesthetics of video games, this outline is equally applicable to other interfaces. Although variations exist among the various types of sequencers, especially among analog sequencers and their digital counterparts, the core mechanics of sequencers remain essentially the same: to read a set of instructions that instruct the machine (or software) to play certain sound files in a certain sequence. Some sequencers digitally record live sounds by transducing physical sound waves to electrical impulses with a microphone and further converting the electrical signal to binary ones and zeroes through sampling. This signal is then often quantized so that the audible sound is made more regular and discrete. After sampling and quantization, a digital sound file is stored for later access. This process is sometimes bypassed on physical sequencers as sounds are loaded into the machine’s memory during the programming; digital sequencers and emulations can access sounds saved on peripheral computers or external memory devices, often through MIDI. When a tempo is specified, the sequencer plays the requested sound files in the sequence indicated during programming. The mechanics of a sequencer facilitate user interaction through a control surface or interface. The dynamics include all of the control parameters afforded to the user through the sequencer’s mechanics. Of course, most sequencers allow users to specify which notes or sounds to play and in what sequence; however many provide ways to control the velocity (or the relative volume) of a note, the tempo (or speed) of the music, and the timbre (or tone color) of the notes that are played. The sequencer cycles through the instructions and plays the notes in real time; if everything works as planned, the sequence of notes that are played result in a form of “automatic” music to which the player does not necessarily need to attend. Most sequencers also offer a form of feedback, assisting the user in knowing what sound to expect and when. This is usually accomplished with a series of flashing lights (that light up when a switch triggers a corresponding note), scrolling bars, or other outputs signify passing of cursor.

114

MUSIC VIDEO GAMES

Most importantly, the democratization of music making through interface design in sequencers, the aforementioned dynamics, and their relative ease of use afford the capacity for musical expression for those with little or no musical training or compositional experience. The ability to choose sounds with modular, interchangeable qualities also fosters the discovery of new sounds, new tonal combinations, new melodies, and eventually helps users create new music. Designed with the aesthetic goal of “sensation” in mind, newer physical sequencers have minimalistic, streamlined interfaces that display attractive light patterns as it cycles through a sequenced piece of music. Many of these elements are genre specific, and some that function well in one genre may be completely inappropriate or of little use within a game of another genre. This suitability is also true of music games, and especially true of music games in which the mechanics, dynamics, or aesthetics are modeled after sequencers, creating games with many of the same functions in a wide variety of forms. For example, Cosmic DJ (GL33k 2014) is essentially a very sophisticated, gamified emulation of a musical sequencer that uses narratives to advance the player from one level to another. Other games combine sequencers with other game genres. Chiptune Runner and Wave Trip (Lucky Frame 2013) are “endless runner”/ platform games wherein the platform is actually a step sequencer— the player creates levels and helps his or her avatar along by adding elements that appear as “notes” for the sequencer to play. Other platform games, such as Rayman Origins (Ubisoft Montpellier 2011) and Rayman Legends (2013), rely on the player’s ability to steer his or her avatar through the level at a particular tempo so that certain actions that are synced with the game’s music are triggered or avoided at the proper time. Similarly, My Singing Monsters is a digital pet game (much like Tamagotchi (Bandai 1996) and Nintendogs [Nintendo 2005]); as players hatch eggs and raise monsters, they sing looping music and dance as if they lived as part of a sequencer ecosystem. Isle of Tune (Happylander Ltd. 2010) is a city-building simulation/fantasy game wherein players place objects along the roads that trigger specific sounds and pitches whenever in-game cars drive past them. There are also emerging games, such as Sentris (Timbre Interactive 2014), that combine

SAMPLE, CYCLE, SYNC

115

sequencers with the challenge of puzzle gameplay. Even the most successful “rhythm” games—such as Rock Band, Guitar Hero, and DDR are, in fact, sequencer games because they share many of the same qualities in terms of aesthetics, usability, and humancomputer interaction as a music sequencer, especially considering the fact that the music in these games is essentially composed of sound samples that are played in a particular, pre-determined order. They even have a “cursor” like most sequencers; however, rather than a moving cursor passing over a stationary visualization of the musical elements, the musical elements (or icons that represent their associated movements) pass through a stationary cursor in a steady stream. Beyond these obvious surface-level similarities, music video games and sequencers have even more in common, and these similarities can be examined by comparing their MDA to that of a sequencer.

MDA in Chiptune Runner Chiptune Runner is an endless runner/puzzle mobile game for iOS and Android, released by Evil Indie Games in 2013, that shares many of the same mechanics, dynamics, and aesthetics with a step sequencer. To provide a brief game overview of the game: The player, represented by a bouncing, square smiley face (called a “hero”) advances steadily to the right on a gridded platform, acting as a cursor of sorts. The X-axis of the grid represents various notes and timbres (specified on the far right of the screen); notes are played when the hero passes over a block (or column of blocks) in that row as it advances to one space to the right along the Y-axis to each beat of the music (see Figure 4.1). The hero hops from block to block as the player creates a path for it, hoping to collect all of the stars in the level and other helpful items along the way—all while avoiding bombs, buzz saws, and holes in the ground. Since we are approaching this game from the perspective of researchers and players, we should begin our analysis by examining the aesthetic goals of the game and then focus our discussion on the dynamics and mechanics that facilitate these goals. On the

116

MUSIC VIDEO GAMES

FIGURE 4.1  Screenshot from Chiptune Runner.

surface, Chiptune Runner appears to share the same qualities as many other endless runners and puzzle games. For instance, a sense of challenge is a common quality of puzzle games that induces immersion into the game by requiring players to solve problems. In Chiptune Runner, players must add or remove blocks to either expose or create a path to stars and other collectibles as the game character passes by them. This same technique is used to avoid dangerous obstacles, since touching one results in the loss of a life (of which the player has only three). I suggest that this game also falls under the “submission” category since it is a mobile game and easy to play on a casual basis. Most germane to our discussion is the fact that this game certainly exhibits the aesthetic of expression as it offers multiple ways in which the player can create his or her own music One would initially expect that the core mechanics of the campaign mode of Chiptune Runner would be comparable to other endless runners. In this case, however, the player has little control over the hero; in fact, the only action performed by the player as a core mechanic is the placement of blocks. He or she can either tap a square to place a block or tap an existing block to remove it. There are, however, a few additional satellite mechanics, such as

SAMPLE, CYCLE, SYNC

117

collecting hearts for extra lives and potions that grant the player a limited moment of invincibility, or hopping on rockets, sending them flying toward an enemy. While placing blocks in a direct path to collect each star is the most efficient way to play the game, the player is able to place blocks virtually anywhere a block do not currently exist, and can remove blocks to his or her taste, creating sequenced music in the process. Some in-game elements over which the player has less control also influence the music. For example, touching arrows facing to the right will speed up the tempo, and conversely, touching arrows pointing to the left will slow it down. Affording further creative power to the player, the “Editor Mode” of the game allows the player to build a level from scratch, edit the tempo (measured in beats per minute), set the instrument for each column, specify the randomization of objects that will appear in that row (measured in percentages), and can even export a .wav file of the resulting music. All of these affordances correspond directly with a hardware or software step sequencer, and players can use this game to create “chiptune” music with very little experience or training.

MDA in My Singing Monsters My Singing Monsters is a simulation/world-building game reminiscent of the Tamagotchi handheld virtual pet games of the late 1990s. The objective of My Singing Monsters is to raise, breed, and collect monsters hatched from eggs that sing, play instruments, and/or dance. Each of these monsters exhibits its own characteristic tone color, musical motifs, and movements. Beyond the obvious fantasy elements of the game, the player uses the game in various modes of musical and visual expression. The player also uses a great deal of strategy in deciding on the best times to breed which monsters, which plants or other objects to place near a monster to make it happiest while managing a limited amount of space, or how to create music that others players, who view their island, will appreciate (Figure 4.2).

118

MUSIC VIDEO GAMES

FIGURE 4.2  Screenshot from My Singing Monsters.

While there is no way to “win” this game, a player completes ingame tasks (such as clearing trees and rocks or raising a particular class of monster) to earn currency (which comes in the form of coins, diamonds, and food) or experience points. The player arranges the monsters on an island and uses the currency they earn to decorate it with items and other structures to increase the happiness of the monsters. These choices in turn help the player earn more money, raise more monsters, and thus make more music as they advance in game levels. As with a music sequencer, players determine which musical elements are played (rhythm instruments, melody, and harmony); in My Singing Monsters, this function is performed by raising and placing specific monsters in a given level. Audio clips are louder or softer depending upon the player’s point of view and camera location; in other words, monsters become louder when the camera is zoomed in closer to them. All motifs are played in a particular sequence, but unlike Chiptune Runner, there is no cursor to indicate when each motif will be played. Although players cannot initially control the notes that will be sung or played by the monster, they can mute a particular monster or use their currency to purchase a “time machine” that adjusts the pitch and tempo of the music their monsters sing (a positive warp increases the music’s tempo and results in higher pitched singing, and vice-versa).

SAMPLE, CYCLE, SYNC

119

Players can also purchase a “Recording Studio” structure, stylized as a hollow tree trunk lined with sound-baffling foam, microphone, and loudspeakers, as one might encounter in a real-world recording studio. When purchasing this structure, players are greeted with a message: Prolonged contact with the human universe has spurred the invention of the magical Recording Studio! Now you can record and save your own version of a monster’s musical part, and then share a video with friends! And if you’re worried about hurting a monster’s feeling by replacing their music, don’t be! Collaboration is the very essence of music, after all! Happy recording! Once purchased, a player can use the microphone on their device to record a new sample for their monster to “sing.” The video recording option allowed players to share the new music they created with a friend. A player could ostensibly replace the motifs sung by every monster on his or her island, thus creating an entirely unique composition using his or her own audio samples, using the game as a music sampler that graphically represents sound files with monsters.

Gamifying musical performances or performing with musical games Writing about performance with various electronic musical instruments, Mark Butler discusses an interview with a DJ in Berlin, named Pacou, who laments the way in which other DJs utilize sequencers within the context of a performance: I know a lot of people that are also using Ableton, but in a very boring way. They are just clicking the parts on, in a sequential order, and from an audience point of view it looks like they are checking emails or playing Tetris or things like that. Very boring presentation. And these guys are charging money for this? (Butler 2014, 96) Likewise, Butler make reference to the manual for the Faderfox Micromodul LV1, a now-discontinued hardware sequencer controller,

120

MUSIC VIDEO GAMES

wherein the anonymous technical writer laments the lag in the development of exciting new controllers: It is 2004. Music is being made with electronic instruments that were originally intended to replace the typewriter. PC’s and Mac’s (sic) are replacing more and more synthesizers, pianos, CD players and LP turntables in the inventory of today’s musician … Since computer music should also thrive from improvisation, we … designed a controller to support the playful use of loops … Let’s face it, who wants to drive a car with a computer keyboard or a mouse? Fact is, slide controls, knobs, joysticks, buttons, and LEDs are as much a part of electronic music as a steering wheel is to a car. (cited in Butler 2014, 97)1 Perhaps the video games discussed in this chapter are filling this gap. While no one will accuse a player of checking his or her e-mail while making music with Chiptune Runner or My Singing Monsters, perhaps the “playing Tetris” argument is still valid. I admit, watching a live performance of electronic music is not very exciting in-and-ofitself, but is watching someone play a video game much different? As discussed in the first chapter of this book, games such as Guitar Hero, Rock Band, and DDR were designed with performance in mind. Their interface and game mechanics facilitate a performative quality around which game nights in local pubs and even professional tournaments are built. While it is hard to imagine a crowd of people gathered around an iPhone to watch an individual feed and breed his or her monsters in My Singing Monsters, players do share their creations online. Other players are able, though in game options, to “visit” these islands of other players and vote on the quality of the music they produced with a “thumbs up” or “thumbs down.” Even if this isn’t the most exciting way to experience sequenced music, it is much more cute—or at least a more visually stimulating—option when compared to watching a cursor pass by colored squares or watching the timer on an MP3 player count down to the end of a song. The gamification of sequencing has also made the somewhat tedious task of recording or choosing samples and arranging them into a musically coherent song into an enjoyable endeavor for even the least technically savvy of casual game players.

SAMPLE, CYCLE, SYNC

121

While the aim of this chapter was to introduce the concept of sequencers as models for some music games, it also raises several questions for further examination. For example, how much composing or performing can one really do with a video game, or even with a sequencer? Because the notes that are “played” by the performer have been pre-prepared and the arrangement of these musical elements are only triggered during a performance, are sequencer “players” or sequencing game players simply DJs or performers in their own right?2 Also of interest are the affordances and constraints present within a game and the ways in which a musician can use them to his or her advantage. How does the desire to make music affect gameplay? Does a player follow the stated or implied rules of the game at the expense of the music in order to win, or should a player sacrifice lives, time, and strategic advantages in order to produce better music? Similarly, how does space and world-building play influence the music-making process? In games such as Chiptune Runner, the most strategic path to collect stars might not necessarily be the most sonorous. Even some questions of economics come to mind: Are players willing to make in-game purchases to create “better” music sooner, or would they “earn the right” to create better music through successful gameplay and winning access to additional musical options?

Notes 1 Available online: http://www.faderfox.de/PDF/LV1_User_manual_V01. PDF. Accessed 10 June, 2015. 2 Here I use the term DJ to refer to the original disc jockeys of the early- and mid-twentieth century, not to turntablists, whose genuinely virtuosic practices emerged in the late 1960s and 1970s.

References Arar, Raphael, and Ajay Kapur. 2013. “A History of Sequencers: Interfaces for Organizing Pattern-Based Music.” In Proceedings of the Sound and Music Computing Conference 2013, Volume 2, 383–8. Stockholm, Sweden: Logos Verlag Berlin.

122

MUSIC VIDEO GAMES

Butler, Mark J. 2014. Playing with Something That Runs: Technology, Improvisation, and Composition in DJ and Laptop Performance. New York and London: Oxford University Press. Fabricatore, Carlo. 2007. “Gameplay and game mechanics design: a key to quality in videogames.” In Proceedings of OECD-CERI Expert Meeting on Videogames and Education, Santiago de Chile, Chile, November 2007. Accessed June 8, 2015. http://www.oecd.org/edu/ ceri/39414829.pdf. Fowler, Charles B. 1967. “The Museum of Music: A History of Mechanical Instruments.” Music Educators Journal 54 (2): 45–9. Hunicke, Robin, Mark LeBlanc, and Robert Zubek. 2004. “MDA: A Formal Approach to Game Design and Game Research.” In Proceedings of the AAAI Workshop on Challenges in Game AI, 1–5. San Jose, CA: AAAI Press. Manovich, Lev. 2013. Software Takes Command. New York and London: Bloomsbury Academic Press. Théberge, Paul. 1997. Any Sound You Can Imagine: Making Music/ Consuming Technology. Hanover, NH: University Press of New England.

Games Bandai. Tamagotchi [Handheld Game]. Tokyo, 1996. Big Blue Bubble Inc. My Singing Monsters [iOS, et al.]. Big Blue Bubble Inc.: London, ON, 2012. Evil Indie Games. Chiptune Runner. [iOS/Android]. Evil Indie Games: 2013. GL33k. Cosmic DJ. [Microsoft Windows, et al.]. Devolver Digital: Austin, 2014. Happylander Ltd. Isle of Tune. [iOS]. Happylander Ltd.: London, 2010. Harmonix. Guitar Hero. [Playstation 2]. RedOctane: Mountain View, CA, 2005. Harmonix, et al. Rock Band. [Xbox 360, et al.]. MTV Games/Electronic Arts: New York, 2007. Konami. Dance Dance Revolution. [Arcade, et al.]. Konami: Tokyo, 1998. Lucky Frame. Wave Trip. [iOS]. Lucky Frame: Edinburgh, 2013. NanaOn-Sha. PaRappa the Rapper. [PlayStation/PlayStation Portable]. Sony Computer Entertainment: Tokyo, 1996. Nintendo Entertainment Analysis & Development Group 1. Nintendogs. [Nintendo DS]. Nintendo: Kyoto, 2005. Nintendo R&D Group 4. Super Mario Bros. [NES/Famicom]. Nintendo: Kyoto, 1985.

SAMPLE, CYCLE, SYNC

123

Pajitnov, Alexey, Nintendo Research & Development 1, et al. Tetris. [NES, et al.] Nintendo, et al.: Moscow, 1984. Timbre Interactive. Sentris. [Linux, et al.]. Timbre Interactive: Seattle, 2014. Ubisoft Montpellier, et al. Rayman Origins. [PlayStation 3, et al.]. Ubisoft/Feral Interactive: Montreuil, 2011. Ubisoft Montpellier, et al. Rayman Legends. [Microsoft Windows, et al.]. Ubisoft: Montreuil, 2013.

PART TWO

Virtuosi, Virtues, and the Virtual

5 Consumerism Hero: The “Selling Out” of Guitar Hero and Rock Band Mario A. Dozal

I

n 2005, game publishers Harmonix and RedOctane introduced the world to Guitar Hero, a rhythm music game that provided the home user a rhythm game experience like none other. Guitar Hero allowed players to live out their rock star fantasies by positioning them as the guitarist in a rock band, and pushed the envelope of rhythm music gameplay through the use of a plastic guitar controller crafted specifically to play the game.1 Regardless of musical or guitar playing proficiency, Guitar Hero allowed players to be able to “play” songs like the Ramones’ “I Wanna Be Sedated” and the Edgar Winter Group’s “Frankenstein,” with a few simple clicks of the plastic strum bar and the push of five colored buttons on the fretboard. Guitar Hero represented the rebelliousness of rock music and redefined the mainstream rhythm game genre that had since been dominated by The author wishes to thank the following people for their support: Dr. Roberto AvantMier, for encouraging me to contribute to this collection; Dr. David Weiss, Dr. Michael Austin, and Gabriela Morales, for providing comments and guidance in the creation of this chapter; and to the Dozal family, for their unwavering support and for not throwing out the author’s unused collection of plastic instruments.

128

MUSIC VIDEO GAMES

arcade games like Dance Dance Revolution, and home console button sequence games like Parappa the Rapper and Britney’s Dance Beat (Metro Graphics 2002). By the time Guitar Hero III and Rock Band came out in 2007, rhythm games had gone mainstream and become a pop culture phenomenon. The iconic plastic guitar controllers and the Guitar Hero logo would appear in television shows, clothing, and posters, and some players would tattoo Rock Band’s instrument icons onto themselves; clothing companies even sold parody t-shirts proclaiming the wearer was a “Trumpet Hero” or “Flute Hero.”2 Even Gene Simmons would cash-in on the band rhythm game craze by releasing his own game controller, which was a mini-replica of the axebass he uses when performing as a member of the metal band KISS. In the years that followed, the band rhythm game genre would grow tremendously. In 2008, Guitar Hero World Tour featured 86 master tracks as recorded by the artist, a far cry from the 47 cover songs on the first Guitar Hero. Rock Band also made its impact in the genre by expanding the gameplay from one guitar player to a four-piece rock band, creating a party game and reflecting the social aspect of music making in the process. Fans could rejoice when their favorite bands were included in Guitar Hero and Rock Band as it meant that the day the game came out or the song became available as a digital download, they could “shred” to it just like their heroes had done before. Guitar Hero would even get its own dance music spin-off—DJ Hero (Activision 2009)—complete with its own turntable control peripheral. However, in 2011, the clicking of plastic strum bars all but ceased, band rhythm games remained on store shelves, and no amount of “Star Power” or “Overdrive” could save Guitar Hero and Rock Band from being laid to rest.3 Both Rock Band and Guitar Hero, games once so popular that the Pew Internet and Life Project found it to be the most frequently played game by teens, were now relegated to status equivalent to washed up acts who only perform at state fairs (Lenhart et al. 2008, 4). While the argument can be made that the market for rhythm music games moved on, I think it is more appropriate to consider whether or not Guitar Hero and Rock Band can be attributed partly to their status as “sell outs.” In this chapter, I will begin by reviewing relevant literature on “selling out” and authenticity with regard to music and video games in order to craft a definition specific to both series of rhythm music games. I

CONSUMERISM HERO

129

follow this with a review of existing literature examining video games as a medium and literature on the Guitar Hero and Rock Band video game series. I will then move onto presenting a theoretical framework for analysis, which includes Veblen’s (1899) concept of conspicuous consumption and Fürsich and Roushanzamir’s (2001) commodification model of communication. Finally, I will present a brief linear history of the Guitar Hero and Rock Band games that have been released on home gaming consoles and discuss their cultural impact as well as point out the specific instances in which the games “sold out.”

Selling out, authenticity, arts, and video games “Selling out” has traditionally been seen as presenting a set of moral, political, economic, or artistic ideals and then later compromising those same ideals to gain personal economic or political power. The literature on selling out has approached the concept in various artistic settings. After the death of playwright Arthur Miller, Wendy Smith offered a criticism of the performance options of modern plays in New York City, specifically the playwright’s goal of obtaining an audience, noting that today’s serious playwrights struggle with “reaching out,” or struggle to get their plays into Broadway performance venues. A playwright is seen as “selling out” when he or she writes, directs, and has a play performed on Broadway, as it can be seen as uninspired commercial entertainment for mainstream audiences (Smith 2005, 125). However, Smith also mentions that Miller himself viewed selling out and appealing to the commercial interests of the theater world as a necessary evil since it allowed his work to be seen by the widest audience. Smith further demonstrates the concept of the playwright selling out through the example of Eve Ensler, author of The Vagina Monologues. Ensler began as a “downtown” playwright, and upon the mainstream success of The Vagina Monologues, she used her newfound fame to bring women’s issues to mainstream attention alongside her published work. Smith points out that Ensler is derided by theater critics as being “a creature of the modern publicity system,” a sell-out who is “regrettably willing to work the media and to enlist pop celebrities to promote her causes” (Smith 2005, 124).

130

MUSIC VIDEO GAMES

In his discussion of the historical construction of the concept of “selling out” in Hollywood, Tom Dardis dubbed “selling out” a myth that perpetuated the belief that Hollywood was “truly a monstrous place, totally hostile to art” (Dardis 1984, 167), that corrupted writers by making them “barter their values for a weekly check” (Dardis 1984, 168), and when writers were not “writing down” to be understood by general audiences, selling out “[drained] writers of their creative talents to such a degree that they can no longer write at all once they leave California” (Dardis 1984, 170). However, Dardis offers counter evidence to disprove the myth, noting that writers like F. Scott Fitzgerald and William Faulkner would find some success post-screenwriting, and that Dashiell Hammett made the conscious decision to no longer write literary works after working as a screenwriter not because he was creatively drained but because he had tuberculosis and believed that he “saw no more reason to write when he not only had all the money he needed but was assured of all he would ever need the remainder of his life” (Dardis 1984, 170). While literature writers involved in Hollywood were categorized as sell-outs based on criticisms that could be accepted at face value, these criticisms were highly inaccurate. While selling out has been primarily applied to those working in all of the arts, the concept has been associated with music especially. In his examination of DIY Punk record labels, Kevin Dunn points out that while selling out by signing with a major record label goes against the “punk ethos,” selling out also allows the artists to “use the tools of the system” to spread their political messages to a wider audience through their association with the record label (Dunn 2012, 232). However, Dunn also notes that bands who sell out often become unsatisfied with their inability to disseminate their political messages by “raging against the machine from the inside” and instead revert to independent record labels for creative freedom. Greg Sevik posited that selling out occurs among modern musicians when the musician makes music not for artistic purposes but for “commercial venture,” with the ultimate goal being to become famous and wealthy (Sevik 2001, 87). For those artists who are already established in the mainstream music scene, selling out occurs again when they sell the rights to their songs to large corporations for use in commercials, preferring to sacrifice their artistic integrity for the promise of being

CONSUMERISM HERO

131

“back in the public eye … famous again and making money” (Sevik 2001, 88). Stom Gloor also noted that “selling out” is now related to product tie-ins and has become “something [artists] embrace” (Gloor 2008, 30) due to the extra exposure they are given. Artists who were once revered for their refusal to “sell out,” such as Bob Dylan and Led Zeppelin, have since gone on to not only have their music used in commercials but they often make personal appearances in commercials as well (Gloor 2008, 32). Additionally, Gloor states that while some artists may lose fans after being dubbed “sell outs” they also stand to gain new fans as a result of their association with brands and advertisers. The attempt to define authenticity in popular music has long been a topic of examination by scholars. Lawrence Grossberg offered a critique of the visual aspects of popular music, claiming that visual interpretations of popular music fostered inauthenticity, and he argues that authenticity of popular music, including rock, should instead be “measured by its sound” (Grossberg 1992, 207), allowing fans to determine authenticity by how well the musical selection fits in the overall genre. Thus, according to Grossberg, fans can dismiss the look of a rock band or the visuals of a rock music video as being inauthentic constructions, but they cannot deny that a particular song is a rock song if it contains instrumentation and stylistic elements of the genre. Allan Moore argued for the redefining of authenticity by seeking to connect authenticity to the individual rather than the music itself. In doing so, authenticity could be achieved from three points of view. According to Moore, first person authenticity is characterized by the actions of the individual to present himself or herself as a genuine representative of a culture or genre. Second person authenticity allows the listener to relate to the performance and gives them a sense that their “life experiences are validated.” Third person authenticity “arises when a performer succeeds in conveying the impression of accurately representing the ideas of another, embedded within a tradition of performance” (Moore 2002, 218–20). Within the world of hip hop, Kembrew McLeod found that authenticity could be determined through the concept of “keeping it real.” Using “dimensions of authenticity,” McLeod found that the more “real” characteristics an artist had, such as coming from

132

MUSIC VIDEO GAMES

the streets as opposed to the suburbs, or being an underground success as opposed to a commercial success, then the more authentic they were considered to be in the world of hip hop (McLeod 1999, 139). Thus, McLeod’s assessment was that authenticity could not be achieved solely through music, visuals, or artist performance individually, but rather a combination of the three. Leslie Meier, on the other hand, examined how authenticity is constructed in the context of music used for marketing purposes, which she termed “promotional ubiquitous music.” Meier determined that authenticity is achieved in commercial licensing of music when the music can “manufacture an affective and seemingly credible link between the brand’s ‘identity’ and ‘values’ and those of the brand’s target market” (Meier 2011, 409). As such an authenticity can be achieved by linking the artist’s ideals with those of the brand that has licensed their music, effectively allowing them to cash in without necessarily “selling out.” Academic literature on video games has looked at the political economy of the video game industry as well as the authenticity and realism of the medium. Dyer-Witheford & Sharman concluded in a political-economy assessment of Canada’s video game industry that while the industry had experienced a large amount of success, it was not immune to the problems that plagued the global video game industry, including the “stagnation associated with the increasing concentration of ownership in the hands of a handful,” and whether or not growth could be “maintained on the basis of formulaic game sequels and clones” (Dyer-Witheford and Sharman 2005, 205). Looking at how players defined the realism of Half-Life 2, Wannes Ribbens determined that perceived game realism was a “complex multidimensional construct” that could be determined according to a six-factor model that took simulational realism, freedom, perceptual pervasiveness, social realism, authenticity, and character involvement into account (Ribbens 2013, 34). Furthermore, Brian Rejack’s (2007) look at the historical authenticity of video games is broken down into two main determinants: fidelity of events (or historical accuracy) and adherence to the conventions of video games. Studies on rhythm music games, such as Guitar Hero and Rock Band, have focused primarily on two aspects of the genre: the social

CONSUMERISM HERO

133

aspects of playing rhythm music games and the aesthetics of the virtual performances through which games allow players to simulate music making. Kiri Miller’s look at both series of games found that players participate in “schizophrenic performance,” or the process of “stitching musical sound and performing body back together” by game players, audiences, game designers, and recording artists (Miller 2009, 424). Miller points out that in spite of critiques of performance authenticity directed toward them, these games offer players a new way to experience music by placing them in a performing body. Rather than simply listening to a song, the player is involved in its playback and the effectiveness of the player’s performance dictates how accurate the playback will be. David Roesner’s look at Guitar Hero led him to determine that players participated in performativity and “musicking” of rock through the use of “well-worn clichés of rock music’s paratexts,” such as sex, drugs, and rock & roll, and performing through the use of a persona (Roesner 2011, 283). As such, the players are not actually performing as the source of rock music but rather as a channel through which pre-existing concepts of rock are displayed. When attempting to determine whether performance of songs in Guitar Hero and Rock Band could be considered authentic performances, Derksen and Hick (2009) found that under certain conditions the performance of songs in the games can be considered true performances; namely, when songs have been re-recorded by the artist for inclusion in the game, or when the player manages to get a perfect score while playing the game at the expert difficulty setting, the player’s performance can be viewed as authentic. However, they also note that certain aspects of performance, such as whether the player’s intentions are to play the game for fun or play the game to “perform” can affect how gameplay is perceived. When considering authenticity, Roger Moseley also examined the concept of “genuine fake” among air guitarists and Guitar Hero and Rock Band players in which they admit that their actions are fake but the actions they mimic are genuine actions associated with playing an instrument (Moseley 2012, 33). Furthermore, live performance plays a large role in building authenticity, as Moseley points out that four players playing Metallica’s Death Magnetic album in Guitar Hero “is more authentic than Metallica’s own

134

MUSIC VIDEO GAMES

recording… recorded piecemeal and subjected to extensive digital manipulation,” while players perform “the entire songs in real time” (Moseley 2012, 34). While previous academic literature has looked at authenticity in music and authenticity in video games, there is room for Guitar Hero and Rock Band to be studied, not as sources of popular rock culture, but as video games that simulate and attempt to reproduce popular rock culture. Additionally, the literature on “selling out” lays a foundation for the definition that will be created and assigned to both the Guitar Hero and Rock Band franchises. This study will also attempt to add to the existing literature on rhythm games and authenticity by bridging determinants of authenticity in popular music and determinants of authenticity in video games to define inauthenticity, or the “sell out” label in Guitar Hero and Rock Band.

Method I include all Guitar Hero and Rock Band games and related spin-offs released on home video game consoles, specifically Sony’s PlayStation 2 and PlayStation 3 and Microsoft’s Xbox 360, beginning with the first Guitar Hero video game released in 2005, and ending with Rock Band Blitz (Harmonix 2012), released in 2012 as the last game in the Rock Band series.4 The home-console (PlayStation 2, PlayStation 3, Xbox 360, and Wii) versions of the games were selected due to their distinction as the core product for both the Guitar Hero and Rock Band video game series.5 Furthermore, both video games series have been analyzed diachronically for the changes that occurred with each new release in their respective series regarding gameplay experience, visual presentation in the game, consumer packaging, track lists, advertisements, celebrity tie-ins, game direction and purpose, and instruments. Several concepts have been used to determine specific instances in the timelines of both series of games relating to how each one may have “sold out.” Sirois and Wasko point out that recorded music “is both a text and a commodity” that can be analyzed for its cultural significance and the value that is placed upon it by the market (Sirois

CONSUMERISM HERO

135

and Wasko 2011, 332, loc. 8809). Since rhythm music games use popular music as a source of their content, they too can be viewed as both a text and a commodity, having value in the market as a product created by a content producer and value as a text that holds some significance within a culture. In an effort to categorize the franchises as sell-outs, I offer my definition of selling out for the purpose of this chapter as the absence or loss of ideals relating to the cultural and entertainment aspects of rock music and video games, in favor of the commoditized aspects of music and video games. Under this definition, maintaining and presenting the various aspects of rock music accurately in the games can be viewed as being authentic, while the overt-selling and commercial presentation of rock culture can be defined as selling out. Fürsich and Roushanzamir’s commodification model of communication has been used to examine how Guitar Hero and Rock Band sold out. According to Fürsich and Roushanzamir, the commodification model of communication “positions the audience as a consumer … public spaces of discourse become increasingly privatized and commercialized while seemingly maintaining a public/ democratic image and functions” (Fürsich and Roushanzamir 2001, 393). Guitar Hero and Rock Band have been examined using this model to find specific instances of how they sold out by commercializing the discursive space of the video game and represented themselves as authorities of culture. Thorstein Veblen’s concept of conspicuous consumption has also been applied to the texts to determine how both series sold out. Conspicuous consumption occurs when one person accumulates wealth and uses that wealth to display superiority over another by “the giving of valuable presents and expensive feasts and entertainments” (Veblen 1899, 56). In this study, conspicuous consumption applies to Guitar Hero and Rock Band’s use of brand exclusives, specifically each game’s use of artists who became exclusive to one series. Furthermore, these two concepts have been used to conduct an interpretive analysis of the franchises for recurring patterns and themes related to political-economy, including corporate acquisition, commercialism, the production and consumption of goods, and specific instances of authenticity related to music and video games.

136

MUSIC VIDEO GAMES

Dividing and conquering The first instance of Guitar Hero and Rock Band selling out can be traced all the way back to 2006 when Harmonix and RedOctane, partners in the creation of Guitar Hero, were each purchased by larger conglomerates in the video game world. Activision first acquired RedOctane for almost $100 million in early 2006, while Viacom’s subsidiary MTV Games purchased Harmonix for around $175 million in late 2006. With both companies being acquired by larger corporations in the video game sector, the argument can be made that RedOctane and Harmonix sold out by giving up their status as independent studios much in the same way an independent artist can be termed a sell out for signing with a major record label. Guitar Hero III: Legends of Rock, the first Guitar Hero by RedOctane released under the Activision banner, featured the first form of non-rock related in-game product placement on the PlayStation 3 and Xbox 360 game consoles. Unlike the PlayStation 2 game console, which did not support real-time advertising due to a lack of a constant high-speed Internet connection, PlayStation 3 and Xbox 360’s access to high-speed Internet allowed a revolving assortment of real-time advertisements in the form of in-game objects to be placed into the game. Suddenly, cans of Red Bull appeared on the amp of the player’s avatar, posters for the film Tropic Thunder were displayed in some of the game’s musical venues, packs of 5 Gum appeared on the side of the menu when players selected songs to play, and players’ avatars could jam out on the AXE Body Spray guitar. However, this is not a new phenomenon; Guitar Hero II featured real-life brands and in-game products, such as Gibson guitars, Ernie Ball strings, and concert tours and venues (such as the Vans Warped Tour), but all of these brands are seen by rock music aficionados as being authentic connections to rock music. Thus, while Guitar Hero III would continue to place authentic rock brands such as instrument retailer Guitar Center in the game, the appearance of non-rock related products, such as 5 Gum and Pontiac brand vehicles, were advertisements that could also influence the player to think that all products featured in the game were related to authentic rock music. While the in-game advertisements did not

CONSUMERISM HERO

137

affect the gameplay, they did become symbols of Activision’s new corporate influence over the series. Mass media conglomerate Viacom6 also demonstrated its new corporate influence over Harmonix in the first Rock Band video game released in 2007. Rock Band’s track list included the song “Timmy and the Lords of the Underworld” by the (fictional) eponymous band featured in the Comedy Central television show South Park. This was not the first time a song from a cartoon was included in a rhythm game; Guitar Hero II featured the song “Trogdor” by Internet animators Strong Bad as a bonus track but neither Harmonix nor RedOctane had a visible financial relationship with the Strong Bad cartoonists. Instead Viacom’s new corporate influence over the game developers was evidenced by the display of “corporate synergy” between Harmonix and Comedy Central with the inclusion of South Park content as a playable track within the game. Further evidence of corporate synergy would occur a year later in 2008 when the song “Charlene (I’m Right Behind You)” by Comedy Central’s political satirist Stephen Colbert’s fictional band Stephen and the Colberts became available as a free download exclusive to Rock Band players. Additionally, Fox News pointed out that further corporate synergy occurred when Colbert devoted a segment of his program to having the rock band Rush play their song “Tom Sawyer” on Rock Band.7 Interestingly, South Park spoofed the Guitar Hero series/fad in an episode titled “Guitar Queer-o” released on November 7, 2007; this was just ten days after the North American release of Guitar Hero III, and thirteen days before the North American release of Rock Band. The episode offered a critique of Guitar Hero as the producers chose to use parodies of songs that appeared in the Guitar Hero series, and the graphics used in the episode were nearly identical to those of Guitar Hero. Ultimately the show associated Guitar Hero with inauthentic rock performance that players considered to be authentic, much like Miller (2009) points out in her study. Incidentally, Rock Band also appeared in a South Park episode almost two years later, but the game received no criticism and instead served to promote interest from fans of the show and Rock Band for downloadable versions of the song “Poker Face” (the original sung by Lady Gaga and the other sung by South Park’s character Eric Cartman), which also became exclusive to the Rock Band franchise.

138

MUSIC VIDEO GAMES

Harmonix and RedOctane made Guitar Hero and Rock Band not only symbols of rock culture to mainstream audiences, but by allowing themselves to be purchased by larger conglomerates in the video game industry, they became further demonstrated that rock always sells out for the right price. An argument can be made that remaining independent co-producers of the Guitar Hero franchise would have led to outside competition in the band rhythm game market, rather than the competition developed between co-creators Harmonix and RedOctane. In turn, this could have potentially kept the original series free from outside corporate influence as it had been up until the point of their separation, but it might have also led audiences to take other band rhythm game competitors, like Rock Revolution much more seriously.

The polyphonic (spending) spree Overexposure seems to be another factor that dictates a “sell out.” If something is overexposed in the media then that generally means it has been accepted by mainstream audiences and has become commercialized, and this applies equally to Guitar Hero and Rock Band. Both were sold out by oversaturating not only the rhythm music game genre but the entire video game market with both games and instrument controllers. From 2005 until 2012, approximately 28 Guitar Hero and Rock Band sequels, spin-offs, and expansion packs were released on seventh-generation home consoles. Guitar Hero started as a series that released a new entry every year between September and December. However, beginning in 2008 both series began to change the release schedule and push for ways to capitalize on the popularity of the band rhythm game genre. The first indication of a potential change in release schedules was the release of Guitar Hero II for the Xbox 360 in April 2007 (Hatfield 2007), as Guitar Hero II had been released the previous November on the PlayStation 2. This would be followed by the release of Guitar Hero III for all major home console gaming systems in October 2007 (Activision.com, 2007). The schedule would be modified again with the release of

CONSUMERISM HERO

139

Guitar Hero: Aerosmith (Neversoft) in June 2008. In 2009, Guitar Hero and Rock Band further flooded the market with games, often released within two to three months of each other. The release schedule of Guitar Hero and Rock Band home console video games in 2009 was as follows: l

March 2009: Guitar Hero: Metallica (Neversoft)

l

April 2009: Rock Band Track Pack, Vol. 2 (Harmonix/Pi Studios)

l

May 2009: Rock Band Classic Rock Track Pack (Harmonix)

l

June 2009: Guitar Hero: Smash Hits (Beenox)

l

July 2009: Rock Band Country Track Pack (Harmonix/Demiurge Studios)

l

September 2009: Guitar Hero 5 (Neversoft), The Beatles Rock Band (Harmonix/Pi Studios), and Rock Band Metal Track Pack (Harmonix/Demiurge Studios)

l

November 2009: Band Hero (Activision) and Lego Rock Band (Harmonix)

l

December 2009: Guitar Hero: Van Halen (Neversoft)

It can be assumed that the oversaturation of both franchises, along with weekly updates to both the Guitar Hero and Rock Band downloadable song catalog, never properly allowed demand to build as consumers could never want more since they were already being offered more than enough.This oversaturation also led to further selling out as the games began to target niche consumer markets rather than the core player. Both Harmonix’s Lego Rock Band and Activision’s Band Hero, with their family friendly soundtracks consisting of songs like “ABC” by the Jackson 5 and “Kung Fu Fighting” by Carl Douglas, were created to appeal to younger gamers and families for whom the original series’ soundtracks may have been too rock-heavy. Bandspecific entries like Guitar Hero: Metallica and The Beatles Rock Band seemed to place a greater focus on gaining the interest of the casual music fan that may have never been interested in playing the games until the name of a well-known, established band was attached to the game.

140

MUSIC VIDEO GAMES

Ironically, the innovative controllers that helped Guitar Hero and Rock Band stand out from other rhythm games in the genre also indicated their willingness to sell out. From 2005 until 2011, over 20 different officially licensed Guitar Hero and Rock Band controllers were produced for use with their respective games. However, several issues arose from the oversaturation of instruments in both series, particularly in relation to gameplay needs. Guitar Hero and Rock Band made the purchase of at least one instrument peripheral a necessity in order to play the game correctly, because without it, the player was not getting the full rock star gameplay experience (although Guitar Hero could be played using the standard PlayStation 2 controller, doing so made progressing through the game extremely difficult). Subsequent guitar controllers for the PlayStation 2 took the original tethered guitar design and made it wireless, giving players the ability to explore their rock star fantasies even further by allowing them to be disconnected from their systems, move around and swing their plastic guitar controllers, and mimic other popular aspects of the rock star stage performance. While upgrading to the wireless controller was not a necessary addition if players had the original guitar, purchasing a new guitar became mandatory when upgrading consoles as the USB controller input slots for the PlayStation 3 console would not support PlayStation 2 guitars. Instead players would have to invest in new instruments made specifically for the new system. Rock Band would also make the purchase of new peripherals—the USB microphone, wireless drum kit, and the Rock Band guitar—necessary for full gameplay of its product right out of the box. The Rock Band guitar also offered a cosmetic difference from the Guitar Hero model by featuring ten fret buttons (five located at the top and five located at the bottom of the guitar neck) instead of the usual five buttons. While players usually paid around $80 for a guitar and game bundle or $100 if purchased separately (Costhelper.com 2007), Rock Band offered players a new software and controller bundle that included every new instrument for approximately $170 (Silwinski 2008). By 2008, both Guitar Hero and Rock Band offered consumers multiinstrument bundles. While there was no longer a need to produce controllers, per se, each side continued to offer new instruments with their games, but the new instruments offered nothing new in terms of

CONSUMERISM HERO

141

gameplay and instead were mostly superficial or cosmetic in nature. With the release of Guitar Hero: Aerosmith in 2008, consumers could purchase the bundle that gave them the Aerosmith guitar controller, which offered nothing new except for a guitar with an Aerosmith logo faceplate. In 2009, The Beatles: Rock Band was released and players were again given the choice of purchasing another game and peripheral bundle. The “Limited Edition” bundle gave players the game; a microphone; drumset controller with a pearl finish and Beatles bass drum cover, similar to Ringo Starr’s kit; and a Hofner bass controller, similar to the one Sir Paul McCartney played on the Ed Sullivan Show. Sold separately were controllers that were replicas of the guitars that John Lennon and George Harrison played on the Ed Sullivan Show as well. In the case of The Beatles: Rock Band, the “limited edition” label placed on John and George’s instruments seemed like an attempt by Rock Band producers MTV Games and Harmonix to appeal to fans who wanted the full Beatles experience, especially since The Beatles: Rock Band was designed to appeal to fans of the band who had probably never played a band rhythm music game. Additionally, The Beatles: Rock Band included a new gameplay feature that involved the use of two additional microphones to achieve three-part vocal harmonies. Attributing the new feature to Rock Band’s attempt to try to offer players the experience of singing in three-part vocal harmonies in the styles of the Beatles, it also increased the number of instruments needed to use a feature that was not an essential part of the genre’s original gameplay (Frushtick 2009). With this, Rock Band raised the count of total possible players and instruments to six: three singers, one bassist, one guitarist, and one drummer. Meanwhile Guitar Hero released two new guitars that differed from older models only in terms of visual aesthetics, but the more important peripheral came in the form of a turntable controller with 2009’s DJ Hero. Just as Guitar Hero had offered players the opportunity to enact their rock star fantasies, DJ Hero offered players the chance to act out their dreams of being a DJ by mashing up songs from different artists. However, DJ Hero’s controller could not be used with the existing Guitar Hero games, which added to gamers’ grief that the series was being oversaturated with games and controllers that made them “less desirable because of space,” and “[cluttered]

142

MUSIC VIDEO GAMES

up their front rooms.”8 Meanwhile the release of Rock Band 3 in 2010 brought two more instruments: the keytar (a 25-key electric piano that could be used to play the new piano parts as well as existing guitar parts in the game) and the Pro Guitar (a guitar controller that also functioned as a playable guitar with 6 strings and 102 touchsensitive fret bar buttons). While neither of these instruments was required for gameplay, they were additional controllers that could be used to enhance the experience of Rock Band 3 by attempting to make gameplay more realistic. Thus the oversaturation of games and controllers by Guitar Hero and Rock Band suggests that the focus for both companies was on profits and not on quality of gameplay. The introduction of new gameplay methods, such as the four-piece band, introduced new and exciting ways to enjoy the games but also led to the need for consumers to spend more money to acquire the instruments needed to play. Eventually, the changes to the original gameplay formula, along with the constant bombardment of new game and controller releases, led observers to believe that Guitar Hero and Rock Band are sell outs as they took the once-novel idea of interactive rhythm music games and milked it for every last drop of profit.

VIP treatment and the rocking dead One of the major ways in which both series demonstrated conspicuous consumption throughout the years was through the use of exclusivity contracts in order to secure music catalogs of iconic bands, making them exclusive to the franchise. These deals represented their selling out by limiting players’ gaming options to the franchise that had exclusive rights to the music they wanted to play. Of the two games, Guitar Hero has the highest rate of using exclusivity contracts as they signed The Eagles, Aerosmith, Van Halen, and Metallica to contracts that limited them from appearing in other band rhythm games. One example of the effect that this had on the competition can be seen in the option to export songs from the first Rock Band game. Players who had purchased both Rock Band 1 and Rock Band 2 could also purchase an export code that would allow them to export the songs from the Rock Band 1 game disc onto their

CONSUMERISM HERO

143

system’s hard drive, making them available to play on Rock Band 2. One of the songs on the original Rock Band game was Metallica’s “Enter Sandman,” which was not exportable due to Guitar Hero’s licensing agreements and the exclusivity contract that Metallica had signed post-Rock Band 1. Likewise, Rock Band had exclusivity contracts with Green Day, and most notably, the Beatles. Once Rock Band had secured the rights to both of these groups, their music would not appear in any series other than Rock Band. Each of these groups also got their own Rock Band title dedicated entirely to them. This was not the case with Guitar Hero, which based games on an artist and included music from other bands to pad out the setlist. Rock Band also parlayed the Beatles exclusivity partnership into a deal that led to the inclusion of some of John Lennon’s solo work in future Rock Band titles, which was a major coup for the Rock Band franchise. These exclusivity contracts limited players on which game they might be able to find their favorite artist. If someone were a Metallica fan, they would more than likely pick up a copy of Guitar Hero, while a person who is a fan of the Beatles would be more likely to purchase a copy of Rock Band. By securing the likenesses and song catalogs of these artists, Guitar Hero and Rock Band ensured that any fans of these groups would be purchasing the games to play these songs, selling out by turning band loyalty into brand loyalty and ignoring lesser-known bands in favor of the big-name acts that would attract the money of their fans. Guitar Hero featured the inclusion of the likeness of more rock stars, albeit in the form of avatars, than did the Rock Band franchise. While the Rock Band franchise featured the likenesses of the members of Green Day, The Beatles, and Lego versions of the rock band Queen, Guitar Hero featured over fifteen different playable rock star avatars over the course of the series. Guitar Hero III contained the first instance of playable characters based on real physically existing individuals, with avatars of guitarists Tom Morello, of Rage Against the Machine, and Slash, of Guns N’ Roses. Guitar Hero World Tour (Neversoft 2008) included a wider variety of musicians, such as Ozzy Osbourne, Sting, Ted Nugent, Travis Barker, and Hayley Williams, in order to emphasize the game’s new four-piece instrumentation.

144

MUSIC VIDEO GAMES

The inclusion of Jimi Hendrix in Guitar Hero World Tour became the first instance of a deceased rock star’s image being licensed for use in the band rhythm game genre. Guitar Hero 5 included the likenesses of living rock stars, like Shirley Manson of Garbage and Matt Bellamy of MUSE, but the inclusion of deceased Nirvana singerguitarist Kurt Cobain had the biggest name recognition, as evidenced by Activision’s press release claiming that Cobain’s appearance in the game continued his “enduring legacy” (Activision 2009). It can be argued that Guitar Hero sold out not only by licensing the likenesses of Cobain, Johnny Cash, and Hendrix in an attempt to establish a symbiotic connection in which they gained credibility due to the association with the artists’ personal legacies and authenticity within the music world, but they also sold out by framing the inclusion of these artists in Guitar Hero as a way of preserving and promoting the artists’ legacies in spite of the decision largely being one tied to generating the most profit. Thus consumers might have considered Guitar Hero to be the more rock authentic game in the genre because it contained performers like Cobain and Hendrix, while the Rock Band franchise did not. As shown earlier, the exclusives these games provided were demonstrative of Veblen’s concept of conspicuous consumption by framing the appearance of certain artists, in song and likeness form, as rewards for the consumer purchasing the game. As each franchise became more successful, they could afford to enter into deals with artists that allowed them to show players that one product was highly superior to the other in terms of content because they featured certain musicians (even musicians that have died) whereas the competition did not, and therefore those artists also preferred the game of one franchise over that of the competition. For example, the packaging of The Beatles Rock Band notes that the game was created with the involvement of the Beatles and provided gamers a “true-to-life gaming experience” (The Beatles Rock Band packaging). Thus, statements by a deceased musician’s estate, similar to that of Janie Hendrix who felt that Jimi Hendrix would’ve played Guitar Hero since he “was a kid at heart,” were released by game producers in an effort to tout their brand as the brand a deceased musician, such as Kurt Cobain, would have picked had they still been alive today (Hiatt 2008).

CONSUMERISM HERO

145

Authenticity becomes inauthentic Much in the same way McLeod coded authenticity in hip hop through the use of the term “keeping it real,” I believe that a term should be applied to both the Guitar Hero and Rock Band series of games in order to determine their authenticity within the world of video games and rock performance. As such, I am suggesting the term “counterfeit rock performance,” which I define as an inauthentic performance of rock and its culture through a surrogate, which in this case is the player. By applying this term to Guitar Hero and Rock Band, it can be seen that both series sold out by losing sight of the context within which their authenticity was determined and what they were designed to be: a leisure activity that used cultural performances of rock in order to allow the player to live out their rock star fantasies regardless of music proficiency level. When Guitar Hero debuted in 2005, the game was not marketed as a “music education” game. The game featured a loose approximation of how to play the guitar due to the guitar controller, but the game was never meant to teach players how to actually play the real instrument. However, once Guitar Hero became a mainstream success, the critique that the game was not teaching players how to play real instruments became a common complaint among established musicians like Jack White and Jimmy Page (NME.com). This demand by critics to “authenticate” Guitar Hero’s playing of music was something that could not be applied to other video games. In the past, no one had clamored about the inauthenticity of a pair of plumber brothers traveling through a series of pipes to rescue a princess, and no one had questioned the authenticity of aliens invading Earth while only moving in a side-to-side and down pattern. Even the more visually and technologically advanced war games, such as Medal of Honor, did not cause the public to question the game’s authenticity and demand that players use controllers that resemble firearms. However, for some reason, Guitar Hero raised these issues due to general public’s misconception that the game served as a source of music performance and education, as opposed to simply serving as a form of entertainment that mimicked real music making. Whereas Super Mario Bros. (Nintendo 1985) and Space Invaders (Tatio 1978)

146

MUSIC VIDEO GAMES

were most definitely marketed as entertainment video games, Guitar Hero and Rock Band were marketed as music performance simulators. Players were told they could “compose, record, edit, and share their music,”9 and “live out” their rock star fantasies and “master guitar, bass, drums, and vocals.”10 However, it’s important to note that these statements are made by the game producers within the context of video gaming and not within the context of actual technical music performance. Conversely, the packaging of The Beatles Rock Band does not make claims that players will become as technically proficient as the Beatles but rather that players will “[m]eet the Beatles” and “follow [their] legendary career.”11 Both Guitar Hero and Rock Band took turns trying to establish themselves as musically educational, and in doing so relegated their original role as video game entertainment to be the second purpose of the game. Guitar Hero established the creation of “GHTunes” in Guitar Hero World Tour, which allowed players to create their own songs through the use of an in-game recording studio and then share them with others. Rather than input notes directly on a staff, players had to hold down the frets on the guitar and strum in order to create a song. However, in spite of the fact that gamers may not be completely technically proficient as regard to music making, players did have to demonstrate some level of musical proficiency in order to craft a song. Rock Band stood to gain the most of both franchises by defending its position as strictly a video game designed to allow players to fulfill rock star fantasies (Rock Band packaging). Rock Band boasted the largest collection of downloadable songs in the rhythm game genre (Rock Band 2 packaging), the first instance of custom avatar creation in band rhythm games (Rock Band packaging), and unlike Guitar Hero, it lacked the in-game commercialism of non-music related product placement. However, Rock Band producers ultimately caved in to the mainstream criticism that band rhythm games had no educational value and did little to teach players to use a real instrument, and Rock Band producers’ response—attempting to teach players how to actually play instruments through the use of controllers that doubled as MIDI instruments outside of the game—was ultimately the final nail in the coffin of band rhythm games. The introduction of the “Pro Mode” in Rock Band 3 took the focus off the fun living out one’s rock star fantasies without being musically educated and instead echoed the criticism

CONSUMERISM HERO

147

to its own players that after years of the same old button mashing and click strumming gameplay, players really should “graduate to real instruments” (Rock Band 3 packaging). Essentially, Rock Band not only sold itself and its fans out by attempting to change the gameplay experience of the genre and by implying that players’ enjoyment of the game strictly for entertainment should take a backseat to answering critics who wanted a more educational product. As evidenced in the previous paragraphs, both franchises lost sight of their value as video games that promoted the counterfeit performance of rock and instead attempted to establish themselves as authentic pillars of rock music culture, much in the same way Fürsich and Roushanzamir’s commodification model of communication states that the goal of companies is not always to sell products but “to establish and normalize private-for-profit corporations as important cultural authorities” (Fürsich and Roushanzamir 2001, 393). Guitar Hero and Rock Band sold out by catering more to their critics, like guitarist Jimmy Page, who called for musical authenticity and real instruments, instead of catering to the core gamers who valued the games for the entertainment and the chance to live out their rock fantasies.

Conclusion As shown previously, Guitar Hero and Rock Band both shared common traits that could be proof of selling out. While those traits might have influenced the decline of the rhythm game genre’s popularity with consumers, the effects of selling out were not immediate. In fact, if selling out had been the main cause of the genre’s decline, then the genre would have begun to lose momentum in 2006 beginning with the announcement of the split between Harmonix and RedOctane, who were going their separate ways. Instead, both Guitar Hero and Rock Band franchises went on to achieve great critical and commercial success before losing momentum. What can be said from the evidence presented in this chapter is that Guitar Hero and Rock Band sold out by moving away from their status as video games designed strictly for the entertainment of acting out a rock star fantasy, toward corporately influenced franchises aiming to be the pillars of rock culture and authentic musicianship.

148

MUSIC VIDEO GAMES

Notes 1

Konami introduced a guitar controller peripheral with its arcade game Guitar Freaks in 1999 but the series was never produced for home gaming consoles.

2

Part of TorsoPants.com, “Band Geek Hero” t-shirt line that was retired after a trademark infringement lawsuit by Activision.

3

Star Power in Guitar Hero and Overdrive in Rock Band were features that in addition to multiplying a player’s score when engaged, could also be used to save other members of the band from “failing” the song.

4

Rock Band 4 and Guitar Hero Live have since been announced for release in late 2015 and are the newest entries in both franchises.

5

These games were the core products, while mobile and iOS, Nintendo DS, LCD key carabiner, and arcade could all be viewed as secondary products.

6

Viacom owns television networks such as MTV, BET, and Comedy Central; Paramount film studio; and Rock Band’s parent company MTV Games.

7

See Fox News, “The Week in Games: September 24, 2008.”

8

Comments by users “Billy Brush” and “Rusty James” on Keith Stuart’s 2011 article “Guitar Hero axed: five reasons why music games are dying.” The Guardian, February 10. Accessed October 18, 2015. http://www.theguardian.com/technology/gamesblog/2011/ feb/10/guitar-hero-axed.

9

Guitar Hero World Tour packaging.

10 Rock Band packaging. 11 The Beatles Rock Band packaging.

References Activision Publishing, Inc. 2007. “Activision Tears It Up On Store Shelves With The Launch of Guitar Hero (R) III: Legends of Rock.” Accessed October 28, 2007. http://investor.activision.com/releasedetail. cfm?ReleaseID=271598. Activision Publishing, Inc. 2007. 2009. “Grunge Rock Pioneer and Icon Kurt Cobain to Make His Video Game Debut this September in Guitar Hero 5.” Accessed August 28, 2009. http://investor.activision.com/ releasedetail.cfm?releaseid=405747

CONSUMERISM HERO

149

Costhelper.com. 2007. “Guitar Hero cost.” Accessed March 9, 2015. http://electronics.costhelper.com/guitar-hero.html Dardis, Tom. 1984. “Myth That Won’t Go Away: Selling Out in Hollywood.” Journal of Popular Film & Television 11 (4): 167–71. Derksen, Craig, and Darren Hudson Hick. 2009. “Performance Hero.” Contemporary Aesthetics 7. Accessed August 22, 2015. http://www. contempaesthetics.org/newvolume/pages/article.php?articleID=541. Dunn, Kevin. 2012“If It Ain’t Cheap, It Ain’t Punk’: Walter Benjamin’s Progressive Cultural Production and DIY Punk Record Labels.” Journal of Popular Music Studies 24 (2): 217–37. Dyer-Witheford, Nick, and Zena Sharman. 2005. “The Political Economy of Canada’s Video and Computer Game Industry.” Canadian Journal of Communication 30 (2): 187–210. Frushtick, Russ. 2009. “‘The Beatles: Rock Band’ Raises The Bar With Three-Part Harmony.” Accessed November 8, 2009. http://www.mtv. com/news/1620982/the-beatles-rock-band-raises-the-bar-with-threepart-harmony. Fürsich, Elfriede, and Elli P. Lester Roushanzamir. 2001. “Corporate Expansion, Textual Expansion: Commodification Model of Communication.” Journal of Communication Inquiry 285 (4): 375–95. Gloor, Storm. 2008. “‘Selling Out’ Taken to New Levels: The Evolving Relationship Between Brands, Artists, and Record Labels.” MEIEA Journal: Journal of The Music & Entertainment Industry Educators Association 8 (1). Accessed October 27, 2015. http://www.meiea.org/ Journal/html_ver/Vol08_No01/Gloor-2008-MEIEA-Journal-Vol-8-No1-p29.htm. Grossberg, Lawrence. 1992. We Gotta Get Out of This Place. London: Routledge. Hatfield, Daemon. 2007. “Guitar Hero Makes Heroic Return.” Accessed April 3, 2007. http://www.ign.com/articles/2007/04/03/guitar-heromakes-heroic-return. Hiatt, Brian. 2008. “Hendrix Comes to Guitar Hero.” Accessed July 17, 2008. http://www.rollingstone.com/music/news/hendrix-comes-toguitar-hero-20080717. Lenhart, Amanda, Joseph Kahne, Ellen Middaugh, Alexandra MacGill, Chris Evans, and Jessica Vitak. 2008. “Teens, Video Games, and Civics: Teens’ Gaming Experiences are Diverse and Include Significant Social Interaction and Civic Engagement.” Washington, D.C.: Pew Research Center. Accessed September 16, 2008. http:// www.pewinternet.org/2008/09/16/teens-video-games-and-civics. McLeod, Kembrew. 1999. “Authenticity Within Hip-Hop and Other Cultures Threatened with Assimilation.” Journal of Communication 49: 134–50.

150

MUSIC VIDEO GAMES

Meier, Leslie M. 2011. “Promotional Ubiquitous Musics: Recording Artists, Brands, and ‘Rendering Authenticity.’” Popular Music and Society 4: 399. Miller, Kiri. 2009. “Schizophrenic Performance: Guitar Hero, Rock Band, and Virtual Virtuosity.” Journal of the Society for American Music 3: 395–429. Moore, Allan. 2002. “Authenticity as Authentication.” Popular Music 21 (2): 209–23. Moseley, Roger. 2012. “Playing Games with Music (and Vice Versa): Ludomusicological Perspectives on Guitar Hero and Rock Band.” Accessed January 23, 2016. http://www.rogermoseley.com/ Music/musicology/Entries/2012/1/19_Playing_Games_with_ Music,_and_Vice_Versa_Performance_and_Recreation_in_Guitar_ Hero_and_Rock_Band_files/Playing%20Games%20with%20 Music.pdf. NME.com. 2009. “Jack White and Jimmy Page Diss ‘Guitar Hero’ Video Game.” Accessed June 22, 2009. http://www.nme.com/news/thewhite-stripes/45521. Rejack, Brian. 2007. “Towards a Virtual Reenactment of History: Video Games and the Recreation of the Past.” Rethinking History 11 (3): 411–25. Ribbens, Wannes. 2013. Perceived Game Realism: A Test of Three Alternative Models. Cyberpsychology, Behavior, And Social Networking 16 (1): 31–6. Rock Band. 2013. Facebook post. Accessed April 8, 2013. https://www. facebook.com/RockBand/posts/10151666622150209. Roesner, David. 2011. “The Guitar Hero’s Performance.” Contemporary Theatre Review 21 (3): 276–85. Sevik, Greg. 2001. “What Price, Fame?” Popular Music & Society 25 (3/4): 85–90. Silwinski, Alexander. 2008. “Original Rock Band Bundle Rolls Down Price.” Accessed November 13, 2008. http://www.joystiq. com/2008/11/13/original-rock-band-bundle-rolls-down-price. Sirois, André, and Janet Wasko. 2011. “The Political Economy of the Recorded Music Industry: Redefinitions and New Trajectories in the Digital Age.” In The Handbook of Political Economy of Communications, edited by Janet Wasko, Graham Murdock, and Helena Sousa, 331–357. Chichester: Wiley- Blackwell. Smith, Wendy. 2005. “Miller’s Tale: The Playwright Drew a Line Between Reaching Out and Selling Out.” American Scholar 74 (2): 121–5. Veblen, Thorstein. [1899] 2012. The Theory of the Leisure Class [Kindle edition]. Oxford and New York: Oxford University Press.

CONSUMERISM HERO

151

Games Beenox. Guitar Hero: Smash Hits. [Playstation 2, et al.]. Activision: Santa Monica, 2009. FreeStyleGames, et al. DJ Hero. [PlayStation 2]. Activision: Santa Monica, 2009. Harmonix. Guitar Hero. [PlayStation 2]. RedOctane: Mountain View, CA, 2005. Harmonix. Guitar Hero 2. [PlayStation 2/Xbox 360]. Activision: Santa Monica, CA, 2006. Harmonix. Rock Band Classic Rock Track Pack. [Xbox 360, et al.]. MTV Games. New York, 2009. Harmonix. Rock Band Blitz. [PlayStation 3, et al.]. Harmonix. Harmonix: Boston, 2012. Harmonix/Backbone Entertainment. Rock Band 3. [Xbox 360, et al.]. MTV Games/Mad Catz. New York, 2010. Harmonix/Demiurge Studios. Rock Band Country Track Pack. [Xbox 360, et al.]. MTV Games. New York, 2009a. Harmonix/Demiurge Studios. Rock Band Metal Track Pack. [Xbox 360, et al.]. MTV Games: New York: 2009b. Harmonix, et al. Rock Band. [Xbox 360, et al.]. MTV Games/Electronic Arts: New York, 2007. Harmonix, et al. Lego Rock Band. [Playstation 3, et al.] Warner Bros. Interactive Entertainment/MTV Games: New York, 2009. Harmonix/Pi Studios. Rock Band 2. [Xbox 360, et al.]. MTV Games. New York, 2008. Harmonix/Pi Studios. Rock Band Track Pack, Vol. 2. [Xbox 360, et al.]. MTV Games. New York, 2009a. Harmonix/Pi Studios. The Beatles Rock Band [PlayStation 3, et al.]. MTV Games: New York, 2009b. Konami. Dance Dance Revolution. [Arcade, et al.]. Konami: Tokyo, 1998. KMetro Graphics. Britney’s Dance Beat. [GBA, et al.]. THQ: Agoura Hills, CA, 2002. NanaOn-Sha. PaRappa the Rapper. [PlayStation/PlayStation Portable]. Sony Computer Entertainment: Tokyo, 1996. Neversoft, et al. Guitar Hero III: Legends of Rock. [PlayStation 2, et al.]. Activision, et al.: Santa Monica, 2007. Neversoft, et al. Guitar Hero: Aerosmith. [PlayStation 2, et al.]. Activision, et al.: Santa Monica, 2008a. Neversoft, et al. Guitar Hero World Tour. [PlayStation 2, et al.]. Activision, et al.: Santa Monica, 2008b. Neversoft, et al. Band Hero. [PlayStation 2, et al.]. Activision: Santa Monica, 2009a.

152

MUSIC VIDEO GAMES

Neversoft, et al. Guitar Hero 5. [PlayStation 2, et al.]. Activision, et al.: Santa Monica, 2009b. Neversoft, et al. Guitar Hero: Metallica. [PlayStation 2, et al.]. Activision, et al.: Santa Monica, 2009c. Neversoft, et al. Guitar Hero: Van Halen. [PlayStation 2, et al.]. Activision: Santa Monica, 2009d. Nintendo R&D Group 4. Super Mario Bros. [NES/Famicom]. Nintendo: Kyoto, 1985. Taito. Space Invaders. [Arcade, et al.]. Taito: Tokyo, 1978. TValve Corp. Half-Life 2. [Windows, et al.]. Valve Corp. Bellevue, WA, 2004. Zoë Mode/HB Studios. Rock Revolution. [Nintendo DS, et al.]. Konami: Tokyo, 2008.

6 Beat it!—Playing the “King of Pop” in Video Games Melanie Fritsch

T

hroughout video game history, many pop and rock stars composed songs or entire soundtracks for video games. Moreover, some musicians have lend their voices and/or likeness for a character in a game; for example, Ozzy Osbourne, Lemmy Kilmister, and Rob Halford were all featured in the Heavy Metal game Brütal Legend (Double Fine 2009).1 Other games focused on specific bands, such as hip hop musician 50 Cent in 50 Cent: Bulletproof (Genuine Games 2005) or the group Journey in Journey Escape (Data Age 1982).2 Unsurprisingly, the “King of Pop” Michael Jackson also figured prominently in several video games from 1989 until today.3 He made his first digital appearance in the game franchise Michael Jackson’s Moonwalker (Sega 1990), based on his movie Moonwalker, which hit cinemas in 1988. In her discussion of the game, Karen Collins remarks: Michael Jackson’s Moonwalker, released by Sega for the Genesis and the arcades in 1990, was another game attempting to exploit the success of a music artist. […] Unlike many musician-based games, Moonwalker was not released to coincide with an album—in fact, it relied on two older songs from Thriller (“Beat it”

154

MUSIC VIDEO GAMES

and “Billy Jean”) and three from 1987s Bad (“Smooth Criminal,” “Another Part of Me” and “Bad”). The game, then, […] became less about the songs and more about Jackson’s public persona. (Collins 2008, 37) Almost ten years later, the music game Michael Jackson—The Experience (Ubisoft 2010) was released for consoles and handhelds, trying a similar approach as the older game by turning on Jackson’s persona but of course with latest hardware. Produced and published by major game publisher and development company Ubisoft in 2010, one year after Jackson’s death, the game is promoted on the associated homepage as follows: Michael Jackson The Experience is the ultimate interactive game, allowing players to immerse themselves in Michael Jackson’s unique universe. […] Perform in authentic and soulful environments inspired by Michael Jackson’s groundbreaking music videos and stage performances. Feel the beat, master the rhythm, and step into his shoes … Don’t Stop ‘Till You Get Enough! (Ubisoft 2010) With his music exclusively licensed for the games and Michael Jackson himself as the main subject, this could have been a recipe for success, as he is one of the brightest stars in recent pop music history. His lifetime achievements include a strong influence on the development of the music video as an art form, popularizing several iconic dance moves, such as “the moonwalk,” and having a decisive stylistic influence on numerous pop artists both during and after his life. Holding multiple records, viz. for being the “Highest Paid Entertainer of All Time” (Jet Magazine 1989, 26), “First Entertainer to Sell More than 100 Million Albums outside the US” (CityNews Toronto 2006), and according to Forbes magazine still being the top earning dead musician in 2014,4 he was and still is also one of the most prosperous artists in terms of economic success. His album Thriller, released in 1982, has been listed as the best-selling album of all time,5 and he was showered with awards throughout his fourdecade career, making him the most celebrated recording artist in

BEAT IT!—PLAYING THE “KING OF POP” IN VIDEO GAMES

155

the history of popular music (Lewis 2007). Furthermore, he was highly engaged with charitable work and humanitarian aid, both in terms of financial aid and in his role as celebrity representative for charity organizations. When he died from cardiac arrest caused by an overdose of medication in 2009, he was grieved by millions of fans all over the world, and his public memorial service was broadcasted internationally. Despite these successes, there have also been stains on this colorful career. Jackson faced several personal scandals, such as accusations of child molestation and even sexual abuse, followed by criminal prosecution in 1993 and 2003. Also, his many cosmetic surgeries and presumed skin bleaching6 with the supposed aim to change his African-American features to appear more Caucasian were often subjected to harsh criticism. He was also criticized for his strange, betimes childlike, and even shocking behavior; for example, Jackson’s home was called the Neverland Ranch, named after the fantasy island in the Peter Pan story. When he visited Berlin, he held his son out of a hotel window thereby almost dropping the baby from the fifth floor (the so-called “baby-dangling incident”). Not only was his entire personal life tightly supervised by media and fans, but it was also harshly scrutinized and subject to rumors and conspiracy theories, such as his supposedly make-believe marriage to Lisa Marie Presley. As can be seen just in the brief sketch of his life, career, and legacy, he was a glitzy and highly controversial figure. Consequently, when skimming the existing academic and biographical literature about the “King of Pop,” one thing becomes increasingly clear: That there is no such thing as the one Michael Jackson, but rather a blurry cloud of ascriptions, discourses, and performances. This chapter will focus on the question of how and on which levels Michael Jackson as a “public persona” is presented in the games in order to put the player “in his shoes.” Moreover, what is this public persona? So as to find an approach for analysis, it is first necessary to pin down the concept of persona with regard to Michael Jackson. Secondly, the two Michael Jackson–related games mentioned previously will be compared in terms of how they staged his public persona in order to convey a “Michael Jackson experience.”

156

MUSIC VIDEO GAMES

“Behind the Mask”: Michael Jackson’s musical persona In his study “Dead Man in the Mirror: The Performative Aspects of Michael Jackson’s Posthumous Body,” Michael Mario Albrecht discusses Jackson as the passive object of media discourses about him rather than as “an active agent or a unified self who is consciously or subconsciously constructing himself through his media performances” (Albrecht 2013, 708). He sets this mediacreated chimera apart from the person by emphasizing “the difference between Michael Jackson as a unique individual (knowledge of which is impossible) and Jackson as a set of media discourses (the only way in which the star can be accessed)” (Albrecht 2013, 708). This differentiation is also supported in Jackson’s own autobiography, in which the singer himself claims: It always surprises me when people assume that something an artist has created is based on a true experience or reflects his or her own lifestyle. Often nothing could be farther from the truth. […] An artist’s imagination is his greatest tool. It can create a mood or feeling that people want to have, as well as transport you to a different place altogether. (Jackson 2009, 179–80) In his study of Michael Jackson’s vocal performances, in which he refers to this very statement, Mats Johansson comes to the same conclusion as Albrecht, stating that it is necessary to differentiate between the private person and the star. In contrast to Albrecht though, he contends that rather than being a passive agent in the creation of his own persona, “Jackson’s artistry appears as nothing less than a legacy of self-conscious insincerity. In no way did he try to obscure the fact that his public image was an act; a performance in its most profound sense. Even as a child star, he seemed aware that artistry is about acting out a role, about putting on a mask with the aim of creating a convincing representation of a character” (Johansson 2012, 272, original emphases). Both Albrecht and Johansson agree that studying the public persona of Michael Jackson as a performance is the appropriate approach. It is due to their different definition of the

BEAT IT!—PLAYING THE “KING OF POP” IN VIDEO GAMES

157

concept of performance that they come to the contrasting findings sketched in the previous paragraphs. Even though these different concepts would be interesting issues to discuss in their own right, I will instead suggest a third approach that usefully conflates both viewpoints in the light of performance. In his discussion of Nicholas Cook’s7 ideas about studying music as performance, Performance Studies scholar Philip Auslander agrees that from a traditional musicological standpoint, it is the musical work that has to be performed. Even though he favors Cook’s endeavor to break with this tradition by describing “a musical work as a set of parameters for a social interaction among musicians, rather than an ideal object to be reproduced through performance” (Auslander 2006, 101), he also argues that Cook maintains the underlying idea by just renaming the musical work as a script. He further observes that Cook discusses these social relationships only with regard to the musicians while performing.8 For that reason, Auslander agrees with Cook’s basic idea that “the verb to perform demands a direct object” (Auslander 2006, 101, original emphasis), but states that this object does not necessarily have to be “a text such as choreography, a dramatic script, or a musical work. Many other things can be understood as performative constructs: personal identity may be seen as something one performs” (Auslander 2006, 101). From this standpoint, and referring to David Graver’s concept of personage,9 Auslander argues: Musical performance may be defined, using Graver’s terms, as a person’s representation of self within a discursive domain of music. I posit that in musical performance, this representation of self is the direct object of the verb to perform. What musicians perform first and foremost is not music, but their own identities as musicians, their musical personae. (Auslander 2006, 102, original emphases) Building on sociologist Erving Goffman’s work10 Auslander further states that “music is a primary social frame. […] To perceive a sonic event as music is to understand it as intentionally produced by a human agent operating in relation to a given social group’s understanding of what music is” (Auslander 2006, 104, original emphasis). If we

158

MUSIC VIDEO GAMES

apply this assertion to the “King of Pop,” he clearly aligned himself into discourses of pop music as the primary social frame. By doing so, he dismissed other frames, such as ideas of rock authenticity, which were relevant during Jackson’s career.11 From this point of view, Johansson’s statement—that Michael Jackson has indeed created his musical persona intentionally to fit in a certain discursive domain of music—can be corroborated as we read Jackson’s own biography, wherein he himself characterizes his musical persona as a creation crafted with the aim “to create a mood or feeling that people want to have” (see earlier Jackson quote). This indicates that his musical persona responds to wishes and desires of his audience, thereby affirming Albrecht’s claim that “Michael JacksonTM” is also a result of discourses about him (Albrecht 2013, 708–9). Auslander addresses this very circumstance in his reasoning when he brings up examples, in which musicians have tried to change their musical personae unilaterally and these attempts have been dismissed by the audience, pointing toward two important issues: The first […] is the fact that the performer’s persona is precipitated by interaction with an audience and is, in that sense, a social construct, not simply an individual one. The other is the way these episodes indicate the investment audiences have in the performance personae they help to create. […] Audiences try to make performers into who they need them to be, to fulfill a social function. (Auslander 2006, 115, original emphasis) He defines “[t]he ‘front’ [as the] point at which performances intersect with larger social contexts” (Auslander 2006, 108). Again, he borrows his terminology from Goffman, who included in his description of the personal front “both ‘relatively fixed’ signs like race and gender and ‘relatively mobile’ signs ‘such as facial expression’” (Auslander 2006, 109). Regarding Michael Jackson, the “relatively fixed” must be heavily underlined because as Albrecht states at the very beginning of his study regarding questions of black or white, masculine or feminine, gay or straight, adult or child, genius or insane, and human or plastic, “Jackson cannot be easily pigeonholed into any single category across any of these binaries” (Albrecht 2013, 706). He explicitly links the fascination for Jackson’s musical persona

BEAT IT!—PLAYING THE “KING OF POP” IN VIDEO GAMES

159

to this indeterminate front. For Auslander, such an indeterminacy of a musical persona, strictly speaking, should have been problematic because “[t]he question of whether and how a performer justifies the claims made through the presentation of a front is a serious one that impinges on matters of repertoire, style, and persona. If all of these aspects of a musician’s performance are not perceived as consistent in one way or another with the claims the musicians has advanced by presenting a front, the performer risks being discredited” (Auslander 2006, 113). In this point, Albrecht and Johansson again agree that it was exactly the lack of a stable identity, which partly defined Jackson’s musical persona. In addition, Johansson takes Jackson’s vocal flexibility and musical variability into account, which further contributed to this overall indeterminate persona.12 He was not bound to one specific musical genre, but could demonstrate his skills within the wider frame of pop music in general.13 In a sense, then, the racial and gender indeterminacy of his appearance works together with the stylistic diversity of his music to produce a distinctively unique, yet ambiguous representation. […] Jackson’s vocal performativity embodies agency partly as a unique combination of styles and idioms (“voices”), and partly by the very fact that it is being addressed from the multifaceted and perplexing “site” that is the star image of Michael Jackson. (Johansson 2012, 275) As we can see, it was Jackson’s musical, bodily and personal indeterminate front that produced the unique musical persona of Michael Jackson, albeit not as a unified and clearly conceptualized identity but rather as a “tabula rasa” in the sense of being all and nothing at the same time.14 However, Jackson’s flexibility was not only his own doing but conceded to and at the same time demanded from him by his audience, who judged him in the context of pop music as a primary social frame.15 Furthermore, “[i]n such cases, the performer’s flexible persona serves as a bridge among institutionally and culturally distinct musical genres, audiences, and repertoires—and that very flexibility is part of the performer’s appeal to an audience” (Auslander 2006, 114). In addition to his talent and skills as singer, dancer, performer, and his overall showmanship, this risky approach toward the

160

MUSIC VIDEO GAMES

creation of Jackson’s musical persona was certainly one ingredient to his overwhelming success. His apparent transgressions of “correct” performances of gender, voice, racial, or social roles (such as mores regarding appropriate behavior as a heterosexual male adult), made him, on the one hand, suspicious and notorious, and gave rise to all sorts of speculations; on the other hand, this again contributed to making him more fascinating, because “the truth about Jackson’s alleged legal transgressions or his bizarre behavior is inconsequential insofar as those discourses continue to circulate as truth and people behave as if they are true” (Albrecht 2013, 708). Since the 1980s media coverage of his personal life has been continuous, and stories about Jackson’s eccentricities (like sleeping in an oxygen chamber or his desire to buy the skeleton of John Merrick, the nineteenth-century “Elephant Man”) were spread,16 which—true or not—earned him the nickname “Wacko Jacko”—a moniker of which he was never able to rid himself. His audience and the media responded to his openness by engaging in the co-creation of this flexible musical persona, even though this did not always go as Jackson wanted it. Nevertheless, he often did not deny or comment on some stories about him, further fueling the rumors and debates. This again resonates with Auslander’s concept, because “the audience, not the performer, plays the most decisive role in the process of identity formation, since it is the audience that produces the final construction of an identity from the impressions created by the performer. In some cases, this audience role can go well beyond the acceptance or rejection of the performer’s claim to a particular musical identity: an audience can actually impose an identity on the performer” (Auslander 2006, 114). The claim, and at the same time the promise, Jackson made through this front was that the audience could read their wishes as well as their fears into the gap left by a lack of a stable identity on any level. By being all and nothing at the same time, he laid the ground for the audience, in Auslander’s words, to make the performer into who they needed him to be: the genius musician who had to sacrifice his own childhood for his career but created his own sparkling world, or the insane child-molesting holey-faced monster everyone could get into a fuss about. At the same time, his musical persona was so open to interpretation from the audience that one could conclude that it should have predestinated him as a video game character

BEAT IT!—PLAYING THE “KING OF POP” IN VIDEO GAMES

161

that players easily could engage with by offering many opportunities for identification for different players. So in a next step it will be of interest how game designers implemented this character blueprint so-to-speak in the two games analyzed here.

“Music and Me”—Michael Jackson’s Moonwalker The Sega Genesis version of Michael Jackson’s Moonwalker was released in 1990 as a single-player beat’em up/platformer featuring 16-bit instrumental MIDI versions of songs like “Smooth Criminal,” “Billie Jean,” and “Beat It.”17 The game is based on the movie’s Smooth Criminal segment,18 and starts with a short cutscene showing the car, in which Michael had morphed in order to escape from Mr. Big’s (the video’s supervillain) henchman. Gameplay starts, after Michael flips a coin into the jukebox inside Club 30s (in homage to another iconic scene from Smooth Criminal). After the lights are switched on and the music has started, the player has to guide Michael, who is depicted as a pixelated thin figure wearing a white tuxedo and a white tilted hat, through five stages19 in order to rescue children. Each stage is subdivided into three levels, separated with short cutscenes, in which Michael appears and shouts some of his well-known catch phrases, such as “wooo!” or “who’s bad?” In the console version, the children are not individualized, but all depicted as a small blonde girl in a red dress, holding a teddy bear in her hand. The girl is modeled after the character Katie in the movie who had been abducted by Mr. Big. In the game, she sits on the floor or is hidden behind objects such as doors or windows or in car trunks, and cries. When Michael passes by, she jumps up, calling out a happy “Michael!” and runs off-screen accompanied by a blue magic spark under her feet.20 The number of children left to rescue is indicated at the bottom of the screen in the form of teddy bear heads besides a frontal picture of the kid. When all children in a level are rescued, a chimpanzee (Bubbles) is transported onto his shoulder, riding on a blue spark. He leads Michael toward the boss of the level. When the boss appears, depicted as a brawny figure

162

MUSIC VIDEO GAMES

in a grey coat, the chimpanzee flees. After shouting “Haha! You’ll never catch me!”21 the boss runs off, leaving Michael behind to face various enemies, such as mobsters, zombies, spiders, or dogs— all of which also appear in the movie or in other music videos. Similar to the children, the enemies are also depersonalized—only differentiated as being part of one of several groups. When the player hits the button for an attack, Michael performs one of his trademark dance moves, such as a high kick, shouting his typical sound: “Woo!”22 A group of stars (his “star power”) is thrown toward the enemy, causing them to fly diagonally out of the screen or fall to the floor and disappear. Also, Jackson’s signature move, the moonwalk, can be performed in order to overcome obstacles (such as conveyor belts); he also uses a jump attack that mimics another typical dance move. Furthermore, a special attack in the form of a rotation can also be executed. When performed, this special ability decreases Jackson’s health level, but after a certain amount of time, Michael will throw his hat, hitting multiple enemies. When the attack button is held longer, Michael will perform a group dance scene with the enemies, after which all of them are destroyed.23 If the health bar falls under a certain level, Jackson’s infamous crotch grab can be performed as a taunt. In some levels the player can trigger a shooting star by rescuing children in a specific order. This star transforms Michael into a robot, as in the movie. While transformed, Michael may not collect children, but he can fly around the level and attack enemies with an arsenal of weapons (such as shooting laser beams out of his eyes). Critics highlight several negative issues with this game, which in the end, lead to the rather mediocre to slightly positive ratings. For example: Moonwalker is a lazily produced game. The stages are entirely monotonous, dressed up with repeating backdrops. Even worse, the enemies are paint-by-number thugs with zero personality and the same tailor. […] Compound this with the single line of AI code that drives each and every one: Run left/right and collide with Jackson. Dealing with these morons while searching doors and windows for missing kids across a dozen stages is woefully dull. (Buchanan 2008)24

BEAT IT!—PLAYING THE “KING OF POP” IN VIDEO GAMES

163

Aside from this criticism regarding gameplay, the game was unanimously lauded for its well-implemented MIDI versions of Jackson’s songs, the nicely animated dance moves, and its “quirk factor.”25 Interestingly, when reading user reviews it is exactly the game’s quirky “Wacko-Jacko” flair that is highlighted as a positive aspect, because it matches Jackson’s musical persona at that time.26 As a reviewer, Janus, puts it: Sometimes the truth is more baffling and bizarre than anything dreamt up by the imagination of mere humans. Sega may sit back and revel in the creative genius behind Sonic and Nintendo may praise the day they invented Mario, but sometimes there lives a character so crazy that fictionalizing his persona for the purposes of a video game just isn’t necessary. […] You don’t have to hysterically flock to his hotel window waiting to be showered by money/memorabilia/babies to recognize, that Michael Jackson is, or was, a towering figure in music. His celebrity status may be fueled more by the lunacy of his actions these days but Michael Jackson’s Moonwalker stands as an unforgettable confirmation of both his weirdness and his coolness. (Janus 2005) Interestingly, the commercial that advertised the game highlighted exactly this fact that “Genesis does what Nintendon’t,”27 as scenes from the movie and the game seamlessly switch from one to another. This can be interpreted as Sega’s assertion that they have the guts to create a game that captures the flavor of Jackson’s musical persona and to transform him into a video game hero. Even though the game featured repetitive gameplay, it seems to have succeeded at least in this endeavor: Jackson is staged as a savior of children (which, in the late 1980s, did not yet have the negative connotation it would come to have after the controversies of the 1990s and 2000s) in the strange, “kooky-cool” (Buchanan 2008) and adventurous world of the movie, imagined and created by Jackson himself. The game contains all necessary ingredients, such as his iconic music, his signature dance moves and shouts, and his child-like fantasy or “Wacko Jacko weirdness” (depending on the perspective), represented by being magically transformed into a robot and shooting enemies with laser beams out of his eyes, creating magical sparks with his moves,

164

MUSIC VIDEO GAMES

incorporating the appearance of Bubbles, and his shocking behavior (represented, e.g., by the crotch grab). In short, the game succeeds in sketching Jackson’s musical persona in all its indeterminacy: the glitzy genius performer and musician substantiated via his dance moves, his unconventional shouts, and his music, and at the same time, the insane, childish weirdo with a blooming and egomaniacal fantasy that expresses itself in some kind of a Michael Jackson universe.28

“Ghost”: Michael Jackson—The Experience Eleven years after the Moonwalker games and one year after Jackson died, the game Michael Jackson—The Experience was released for several platforms.29 The version on which I will focus is for the Xbox 360 Kinect, which features full-body tracking. The game does not include any background story or career mode, but rather is a mere combination of a dance and karaoke game: Players are challenged to match the choreography given and sing along classical Jackson songs. Points and multipliers30 are awarded for matching the moves, rhythm and pitch as accurately as possible. Beyond that, by using the “Player Projection” feature (with the help of the Kinect camera), players are even put in the place of the “King of Pop,” taking the form of a sparkly reflection of themselves on screen. Jackson himself does not appear in the game in the form of an avatar;31 instead, he can only be seen on posters on the walls and can, of course, be heard. The game provides thirty songs, which may be played in the two playing modes: “solo” and “party.” In the solo mode, players can choose between “practice” and “perform.” Before playing an entire song, the choreography can be learned in the practice mode, in which the song is broken up into several sections. A dancer on screen shows the moves, and much like other dance games, such as Dance Central, little cue cards pop up in the lower right corner that indicate the next move. The “party” mode features two further options, “Co-op” and “Battle,” in which players compete in teams of up to four players. Each player is represented by one of the following symbols: Michael’s hat, his gloves, his shades, or one of his iconic jackets. In the “MJ School” mode, players may learn typical Jackson dance moves, which are not featured in the other modes or not accurately explained.

BEAT IT!—PLAYING THE “KING OF POP” IN VIDEO GAMES

165

Reviews of the game have been mixed, resulting in an overall mediocre score on Metacritic.com.32 Some critics lauded it for its inventive use of the Kinect (especially the way in which the game literally puts the player in Michael’s shoes on screen) and the shiny graphics, whereas others emphasized the lack of typical game features, as evidenced, for example, by the unimportance of scoring points, “because no matter how few you get, it’s impossible to fail out of a song. While this makes the game much more accessible, it removes some of the challenge. There’s no incentive to strive for higher scores; no bonus content to unlock, and no leaderboards to see how you measure up against other players” (Walton 2011). Furthermore, the game suffered some player feedback and latency issues; for example, the sparkly on-screen avatar and the actual player in the room are often out of sync, and except for the overall judgment players get at the end of each song, no precise feedback is offered to let the player know which dance move he or she executed incorrectly.33 Another point of criticism was the half-baked design. The cue cards only pop up after a short countdown, making it difficult to follow the choreography. Additionally, there is no pitch bar for the singing sections, even though the singing is rated based on accurate pitch and rhythm (Walton 2011). In particular, the multiplayer modes are criticized for poor design: Multiplayer could have been a saving grace for Michael Jackson: The Experience, but it’s just another source of disappointment. Co-op play, which tasks players which [sic] switching on and off mid-song, doesn’t allow for enough time in-between turns to let players situate themselves. Battle mode is even worse, because one team performs the entire song first and then the other team performs the same song … again. A good multiplayer game doesn’t make half the players wait for five minutes before letting them compete. (Clements 2011) Reviewers further claimed that players must both dance and sing to some songs, and it is not possible to switch one mode off, whereas other songs are for either singing or dancing only. Also, the difficult “Master Performance” mode is not provided for every song. Besides performing the songs, “there’s nothing else to do, which makes ‘The

166

MUSIC VIDEO GAMES

Experience’ part of the title something of a misnomer. There are no music videos, no behind-the-scenes footage, no interviews, and no download store at which to acquire more tracks. […] Because it lacks a Career mode, unlockables, or even any music videos, this isn’t so much an ‘experience’ as it is a simplistic dance offering” (Walton 2011). As in Moonwalker, Jackson’s musical persona is represented through the medium of his songs, his dance moves, and some typical accessories (the hat, the gloves, etc.), as well as alluding to his music videos through the use of some backdrops (such as a graveyard, which alludes to “Thriller”). Unlike its predecessor, Jackson’s music could now be included in its original audio version, thanks to better technology. Regarding the question of whether or not the game succeeds to convey the promised Michael Jackson experience, reviewers are quite divided. Whereas some really appreciated the idea to remove Jackson’s body out of sight, and instead put the player himself in his place, others harshly criticized the under-developed gameplay features. Furthermore, the game rather celebrates Jackson’s genius as a performer and musician, rather than the indistinct oddball, “Wacko Jacko.” Even though this second approach constituted the peculiar charm of the earlier Moonwalker game, pursuing the same strategy would have been risky, primarily due to Jackson’s overall development, his drastic changes in appearance, and of course his scandals during the ten years in between the two games, especially the child molestation accusations. Secondly, when Michael Jackson—The Experience was released, the “King of Pop” had tragically died just seventeen months prior. Emphasizing the positive and non-questionable or scandalous part of Jackson’s musical persona, which is celebrating him as genius musician and performer, therefore seems more reasonable, but lacks Jackson’s original flair.

“Will You Be There”: Character and the immersive fallacy Interestingly, both games have one thing in common: Even though the designers seem to have been able to capture and convey Jackson’s musical persona or at least some parts of it, they somehow failed in both cases to transform it into an overall satisfying game experience.

BEAT IT!—PLAYING THE “KING OF POP” IN VIDEO GAMES

167

In their seminal publication, Rules of Play, Katie Salen and Eric Zimmerman highlight a problem, which they came to call the immersive fallacy: The immersive fallacy is the idea that the pleasure of a media experience lies in its ability to sensually transport the participant into an illusory, simulated reality. According to the immersive fallacy, this reality is so complete that ideally the frame falls away so that the player truly believes that he or she is part of an imaginary world. (Salen and Zimmerman 2004, 450-1) According to Salen and Zimmerman, the ideal situation for both VR and game designers would be the Holodeck as presented in science fiction TV shows such as Star Trek. Referring inter alia to film studies scholar Elena Gorfinkel, the authors explain why, from their point of view, the concept is problematic: As Gorfinkel states, mistaken ideas about immersion can be framed as confusion between the intrinsic qualities of a media object and the effects that object produces. Gorfinkel argues that to understand the subtleties of “immersion,” we need to look not just at the attributes of games (such as how detailed the graphics are), but at the way games function in relation to the experience of the player. (Salen and Zimmerman 2004, 452) For that reason Salen and Zimmerman do not absolutely dismiss the idea of immersion but slightly alter the focus, writing: It is possible to say that players of a game are “immersed”— immersed in meaning. To play a game is to take part in a complex interplay of meaning. But this kind of immersion is quite different from the sensory transport promised by the immersive fallacy. (Salen and Zimmerman 2004, 452, original emphasis) This meaning is not just provided by the game designers and passively absorbed by players, it is also co-produced by players through the act of playing. Their contextual and cultural knowledge, their ideas and knowledge about the game’s world, characters and narrative with which

168

MUSIC VIDEO GAMES

they are presented, their gaming literacy,34 player skills, and many other circumstances strongly contribute to how they interpret the goings-on, and how they react, in short: how they experience the game. Therefore, when approaching a game in terms of its immersive qualities, that is, the meaning created and experienced during play, it is unquestionably necessary to take the players into account. In the case of the two Jackson games, the intended meaning was to create an experience of Jackson’s musical persona. In the first case, this was attempted by transforming Jackson’s own creation (the Moonwalker movie) as faithfully as possible into a game. In other words, Jackson himself created the Moonwalker experience, thereby having a say in the creation of his musical persona, and the game designers were merely responsible for the translation from one media form into another. In the second case, using the previously described “Player Projection” feature, the central approach was that the player should completely identify with the game’s main character, Michael Jackson, by becoming Jackson himself. Due to the indeterminacy and openness of Jackson’s musical persona described in the preceding paragraphs, this should have worked out, as such a character should appeal to a broad range of possible players. But interestingly, the designers fell into the trap of the immersive fallacy, because, as Salen and Zimmerman state regarding video game characters: The immersive fallacy would assert that a player has an “immersive” relationship with the character, that to play the character is to become the character. In the immersive fallacy’s ideal game, the player would identify completely with the character, the game’s frame would drop away, and the player would lose him or herself totally within the game character. (Salen and Zimmerman 2004, 453, original emphasis) In the Moonwalker game, players could engage in a dialogue with the Jackson musical persona presented to them, co-creating the game’s main character with the help of their view on and knowledge about the “real-world” Jackson musical persona. In the Kinect game, players would take the place of Jackson themselves, filling the gap on screen with their own bodies, and singing over Jackson’s voice as well. The result was seeing and hearing oneself as a mediocre dancer (even good dancers experienced sync problems) and singer, experiencing oneself rather as a “wannabe” Jackson. In other words,

BEAT IT!—PLAYING THE “KING OF POP” IN VIDEO GAMES

169

in Michael Jackson—the Experience, no (or at least little) Jackson was left to experience. Discussing the player/avatar relationship further, Salen and Zimmerman write: A player’s relationship to a game character he or she directly controls is not a simple matter of direct identification. Instead, a player relates to a game character through the double-consciousness of play. A protagonist character is a persona through a player exerts him or herself into an imaginary world: this relationship can be intense and emotionally “immersive.” However, at the very same time, the character is a tool, a puppet, an object for the player to manipulate according to the rules of the game. In this sense, the player is fully aware of the character as an artificial construct. (Salen and Zimmerman 2004, 453) For that reason, even gameplay could not help out, because the game provides neither a character nor a well-made game with which to play. Therefore, just relying on Jackson’s musical persona as the main attraction of a game cannot suffice as long as the gameplay is monotonous, inconsistent, or otherwise poorly designed. Besides, cutting away half of what made Jackson’s musical persona so interesting and glitzy, flattening and smoothing it to the “he was a genius” discourse about him is tinged with superficiality which furthermore easily smacks of a mere cash-in. However, in the case of the Moonwalker game, the quirky “Wacko-Jacko” effect may have been sufficient thanks to the brevity of playtime (ca. 2 hours), possibly preventing players from being bored by the repetitive gameplay. Furthermore, even if players could not (or did not want to) engage in the dialogue with the Jackson musical persona with which they were presented, they could at least play an acceptable beat’em up game by exerting effort in the gameworld via a maneuverable puppet on screen.

“Mind Is the Magic”: Conclusions In this chapter, I discussed how the concept of musical persona could be made fruitful when analyzing games that turn on musicians. As the case studies have shown, this is not only important for understanding

170

MUSIC VIDEO GAMES

player experience but is also significant from the perspective of game designers. In order to successfully convey the experience of a musical persona, it is neither necessary nor does it suffice to literally put the player in the shoes of the star, thereby trying to make him or her totally immersed in this persona. Instead, providing the player with only the basic frame of the respective musical persona in the form of trademark characteristics, important songs, dance moves, etc., and a basic avatar through which the player can access the artistic world of this musical persona seems to be more satisfying. In addition, it appears to be mandatory to think of the game in traditional gaming terms, providing players with opportunities for satisfactory, meaningful actions such as winning, losing, and/or competing with other players.

Notes 1

See, for example, Fritsch (2014).

2

See, for example, Collins (2008).

3

In 1999 he made a cameo appearance as “Space Michael” in Tetsuya Mizuguchi’s Space Channel 5 (United Game Artists 1999), followed by a more prominent role in the sequel Space Channel 5: Part 2 (United Game Artists 2002). In both games, “Space Michael” is modeled after and voiced by Jackson himself. Although not credited, he further seems to have been involved at some point in the development of Sonic the Hedgehog 3 (Sonic Team 1994), released by Sega in 1994. Due to his scandals in 1993, the collaboration ended. Another Michael Jackson–based game project, entitled Planet Michael (see Digital Studios AB, N.D.), which is currently planned as an MMORPG, has not yet been released.

4

See O’Malley Greenburg (2014).

5

See Anderson (2009).

6

Another explanation for his changing skin color has been vitiligo. This skin disease causes pigment cells to work insufficiently or even die, leaving the patient with white spots.

7

See Cook (2001).

8

This holds true for the specific example Auslander refers to, but in a later section of the text Cook does address the role of the audience.

BEAT IT!—PLAYING THE “KING OF POP” IN VIDEO GAMES 9

171

See Graver (1997).

10 See Goffman (1959), and Goffman (1974). 11 See, for example, Auslander (2008, 73ff) and Johansson (2012, 274–5). 12 Johansson elaborately describes and analyzes Jackson’s voice, sound-making and singing:



It should be noted that many of Jackson’s voice-produced sounds are not easily classifiable along the chest voice/head voice/ falsetto scale, which is partly due to his often seamless transitions between registers. Nor is it a straightforward matter to categorize his variously styled vocals in terms of voice color or timbre, not to mention that there is no commonly agreed upon terminology for describing timbral variation in singing. What remains undisputable, however, is that he used his knowledge of vocal technique to create a wide palette of sounds: intricate nuances and shades of timbral variety resulting from an explorative use of vocal tract configurations and different ways of blending registers (Johansson 2012, 268–9). He comes to the conclusion that Jackson’s indeterminacy and ambiguity is also reflected in his elusive singing style, which again was the unique signature Jackson sound.

13 “On the Dangerous album alone, Jackson moves from New Jack Swing to classical, hip hop to gospel, R&B to industrial, funk to rock. It was music without borders or barriers, and it resonated across the globe” (Vogel 2012). 14 “Michael Jackson is mass culture, not pop culture—he appeals to everybody,” said Charlie Kendall, program director of the New York rock radio station WNEW-FM. “No one can deny that he’s got a tremendous voice and plenty of style, and that he can dance like a demon. He appeals to all ages and he appeals to every kind of pop listener. This kind of performer comes once in a generation” (Pareles 1984). 15 “There is a continuum from types of musical performance in which the musicians’ personae are strongly mandated […] to types in which musicians have a great deal of freedom to construct personae. In no case, however, is the musician in a position to construct a persona autonomously—personae are always negotiated between musicians and their audiences within the constraints of genre framing” (Auslander 2006, 114). 16 See, for example, N.N. (2005). 17 U.S. Gold published the home computer versions in 1989, whereas Sega was in charge for the arcade and home console versions released in 1990s.

172

MUSIC VIDEO GAMES

18 The movie does not follow a consecutive narrative arch, but rather, is subdivided as eight separate music videos. 19 The stages are modeled after scenes from the aforementioned movie segment. The succession is as follows: Stage 1—Club 30s, Stage 2—Street, Stage 3—Woods, Stage 4—Cavern, and Stage 5—the Enemy Hideout. The final battle takes place in outer space, where Michael has to shoot down Mr. Big’s spaceship. During the credits, Michael dances with another child from the movie, Zeke “Baby Bad” Michael. 20 This could be an allusion to Michael’s “lucky star” that he uses in the movie in order to morph. 21 The shout is displayed with a comic-style speech bubble. On the Sega Genesis, including audible speech was not possible, except for short utterances in the form of samples. 22 Throughout the game several typical shouts are featured, such as “hee-hee,” “shamone,” “hoo!,” “ow!,” and “who’s bad?” 23 Even though in his adult career Jackson appeared as a solo entertainer, his music videos and stage performances of up-tempo songs usually featured highly energetic group dance scenes. Jackson himself features as the group leader, who engages the others in the choreography, or as the focal point the entire choreography pivots around. In any case, his background dancers were trained in his particular dancing style. 24 Kramer (2008) puts it similarly: “You take on enemies with a handful of classic MJ dance movies [sic], such as his snap kick, hat toss and the like, and, while these moves are animated quite well, it’s easy to get the feeling that there just aren’t enough of them in the game. And that pretty much sums up the Moonwalker experience—that there isn’t just enough.” He also specifies the lack of variety of the levels and enemies, as well as the monotony of gameplay as the main points of criticism. 25 See Mean Machines (1991), Buchanan (2008), and Kramer (2008). 26 See, for example, Zoolander (2001). 27 The original commercial is still accessible on video-sharing platforms, such as YouTube. See, for example, https://www. youtube.com/watch?v=i7GxI9aiiYE (accessed May 4, 2015). 28 It would have been interesting to see how the MMORPG Planet Michael would have approached this very concept. Unfortunately, the game, which was scheduled for release in 2011, is still on the hold (see Cowen 2010). 29 These include seventh-generation consoles, such as Nintendo’s Wii and Wii U, PlayStation 3, Xbox 360; handhelds, such as the

BEAT IT!—PLAYING THE “KING OF POP” IN VIDEO GAMES

173

Nintendo 3DS, Nintendo DS, PlayStation Vita, PlayStation Portable, iPhone; as well as Mac OS X; iPad; and iPad 2. 30 Players performances are rated with score points. The additional verbal ratings “perfect,” “good,” or “Ok” increase players score point multipliers up to 4x, a bad rating such as “miss” resets the multiplier. 31 In the Wii and PlayStation 3 version he is present on screen and the player tries to match his movements using the respective interface device, that is, the Wii Remote or PlayStation Move. 32 See http://www.metacritic.com/game/xbox-360/michael-jacksonthe-experience. 33 “There’s a significant delay between performing a move and seeing your onscreen self replicate it, which is incredibly distracting and makes keeping time difficult—not to mention that the way your movements don’t sync with those of the background dancers makes you look like a terrible dancer” (Walton 2011). 34 Regarding the concepts of gaming literacy and game music literacy see Zimmerman (2009), Zagal (2010), van Elferen (2016 in prep.), and Fritsch (2016, in prep). In short: Players interpret video games (and video game music) with respect to several contexts. If players are not literate in these contexts, for example, if they have no idea who Michael Jackson was he or she won’t be able to grasp the meaning of this pixilated white figure in the Moonwalker game. Furthermore, players do not always play the games as intended by the designers. They bend the rules, look for bugs or exploits, or even create their own games within the game. One example is the practice of stapling avatars in Quake III (id Software 1999): Players found out that it was possible to place one’s avatar on the head of another avatar. Soon, they started to compete in building avatar towers (see Kringiel 2005).

References Albrecht, Michael Mario. 2013. “Dead Man in the Mirror: The Performative Aspects of Michael Jackson’s Posthumous Body.” The Journal of Popular Culture 46 (4): 705–24. Anderson, Kyle. “Michael Jackson’s Thriller Set To Become Top-Selling Album of All Time.” MTV News, July 20, 2009. Accessed May 4, 2015. http://www.mtv.com/news/1616537/michael-jacksons-thrillerset-to-become-top-selling-album-of-all-time. Auslander, Phillip. 2006. “Musical Personae.” TDR: The Drama Review 50 (1): 100–19.

174

MUSIC VIDEO GAMES

Auslander, Phillip. 2008. Liveness. Performance in a Mediatized Culture, 2nd Edn. New York: Routledge. Buchanan, Levi. “Michael Jackson’s Moonwalker Review: Annie, are you OK?.” IGN, March 31, 2008. Accessed March 13, 2015. http://www. ign.com/articles/2008/03/31/michael-jacksons-moonwalker-review. Clements, Ryan. “Michael Jackson: The Experience Review: Tonight’s Kinect activities: singing, dancing, disappointment.” IGN, April 21, 2011. Accessed March 14, 2015. http://www.ign.com/ articles/2011/04/22/michael-jackson-the-experience-review-5. Collins, Karen. 2008. “Grand Theft Audio? Popular Music and Intellectual Property in Video Games.” Music and The Moving Image 1 (1): 35–48. Cook, Nicholas. 2001. “Between Process and Product: Music and/ as Performance.” Music Theory Online: The Online Journal of the Society for Music Theory 7 (2). Accessed October 27, 2015. http:// www.mtosmt.org/issues/mto.01.7.2/mto.01.7.2.cook.html. Cowen, Nick. “Planet Michael developer interview.” The Telegraph, September 28, 2010. Accessed March 1, 2015. http://www.telegraph. co.uk/technology/video-games/8028516/Planet-Michael-developerinterview.html. Fritsch, Melanie. 2014. “Worlds of Music: Strategies for Creating MusicBased Experiences in Video Games.” In The Oxford Handbook of Interactive Audio, edited by Karen Collins, Bill Kapralos, and Holly Tessler, 167–77. New York: Oxford University Press. Fritsch, Melanie. 2016 (In preparation). ‘“It’s a-me, Mario!’—Playing With Video Game Music.” In Ludomusicology—Approaches to Video Game Music, edited by Michiel Kamp, Tim Summers, and Mark Sweeney. Sheffield: Equinox Publishing. Goffman, Erving. 1959. The Presentation of Self in Everyday Life. New York: Doubleday. Goffman, Erving. 1974. Frame Analysis: An Essay on the Organization of Experience. New York: Harper & Row. Graver, David. 1997. “The Actor’s Bodies” Text and Performance Quarterly 17 (3): 221–35. Jackson, Michael. 2009. Moonwalk. New York: Crown Archetype. Janus. 2005. “Michael Jackson’s Moonwalker.” Sega-16: Genesis Reviews, February 2, 2005. Accessed March 11, 2015. http://www. sega-16.com/2005/02/Michael-jacksons-moonwalker. Johansson, Mats. 2012. “Michael Jackson and the Expressive Power of Voice-produced Sound.” Popular Music and Society 35 (2): 261–79. Kramer, Josh. “Michael Jackson’s Moonwalker.” Thunderbolt, November 20, 2008. Accessed March 13, 2015. http://www.thunderboltgames. com/review/michael-jacksons-moonwalker. Kringiel, Danny. “Spielen gegen jede Regel: Wahnsinn mit Methode.” Spiegel Online—Netzwelt. Accessed September 30, 2005. http://

BEAT IT!—PLAYING THE “KING OF POP” IN VIDEO GAMES

175

www.spiegel.de/netzwelt/web/spielen-gegen-jede-regel-wahnsinnmit-methode-a-377417.html. Lewis, Monica. “20 People Who Changed Black Music: Michael Jackson, the Child Star-Turned-Adult Enigma.” Miami Herald, June 14, 2007. Accessed March 11, 2015. http://www.miamiherald.com/ incoming/article1928146.html. Mean Machines. 1991. “Moonwalker.” Mean Machines Magazine, 4 (January), 34–5. Accessed March 14, 2015. http://www. meanmachinesmag.co.uk/issue/4/mean_machines_issue_4.php. O’Malley Greenburg, Zach. 2014. “The Top-Earning Dead Musicians of 2014,” Forbes Magazine, October 31, 2014. Accessed May 4, 2015. http://www.forbes.com/sites/zackomalleygreenburg/2014/10/31/thetop-earning-dead-musicians-of-2014. Pareles, Jon. “Michael Jackson at 25: A Musical Phenomenon.” The New York Times, January 14, 1984. Accessed March 14, 2015. http:// www.nytimes.com/1984/01/14/arts/michael-jackson-at-25-a-musicalphenomenon.html. Salen, Katie, and Eric Zimmerman. 2004. Rules of Play. Game Design Fundamentals. Cambridge, MA: The MIT Press. Ubisoft. 2010. [No title]. Accessed March 14, 2015. http://theexperiencethegame.ubi.com/michael-jackson/en-GB/game-info/index.aspx. van Elferen, Isabella. 2016 (In preparation). “The ALI Model: Towards a Theory of Game Musical Immersion.” In Ludomusicology—Approaches to Video Game Music, edited by Michiel Kamp, Tim Summers, and Mark Sweeney. Sheffield: Equinox Publishing. Vogel, Joseph. “The Misunderstood Power of Michael Jackson’s Music.” The Atlantic, February 8, 2012. Accessed March 14, 2015. http://www.theatlantic.com/entertainment/archive/2012/02/themisunderstood-power-of-michael-jacksons-music/252751. Walton, Mark. 2011. “Michael Jackson: The Experience Review.” GameSpot, April 14, 2011. Accessed March 9, 2015. http:// www.gamespot.com/reviews/michael-jackson-the-experiencereview/1900-6308558. Zagal, José P. 2010. Ludoliteracy: Defining, Understanding, and Supporting Games Education. Pittsburgh: ETC Press. Zimmerman, Eric. 2009. “Gaming Literacy: Game Design as a Model for Literacy in the Twenty-First Century.” In The Video Game Theory Reader 2, edited by Bernard Perron, and Mark J. P. Wolf, 23–31. New York: Routledge. Zoolander, Derek (pseudonym). “Michael Jackson’s Moonwalker.” GameFaqs, August 1, 2001 (Updated July 31, 2003). Accessed March 12, 2015. http://www.gamefaqs.com/genesis/586315-michaeljacksons-moonwalker/reviews/review-21357.

176

MUSIC VIDEO GAMES

N.N. “Best selling record.” Guinness World Records. Accessed March 14, 2015. http://www.guinnessworldrecords.com/worldrecords/70133-best-selling-album. N.N. “Michael Jackson Earns $126 Million And Remains Highest Paid Entertainer.” Jet, October 2, 1989, 26–8. N.N. “Michael Jackson Named Most Successful Entertainer of All Time.” CityNews Toronto, November 15, 2006. Accessed March 12, 2015. http://www.citynews.ca/2006/11/15/michael-jackson-named-mostsuccessful-entertainer-of-all-time. N.N. “Michael Jackson: The Experience.” Metacritic. Accessed March 14, 2015. http://www.metacritic.com/game/xbox-360/michael-jacksonthe-experience. N.N. “Music’s misunderstood superstar.” BBC News, June 13, 2005. Accessed March 12, 2015. http://news.bbc.co.uk/2/hi/ entertainment/4584367.stm.

Games Data Age. Journey Escape. [Atari 2600]. Data Age: Campbell, CA, 1982. Double Fine. Brütal Legend. [PlayStation 3, et al.]. Electronic Arts/ Double Fine: Redwood City, CA, 2009. Genuine Games, et al. 50 Cent: Bulletproof. [PlayStation 2, et al.]. Sierra Entertainment/Vivendi Games: Fresno, 2005. Harmonix. Dance Central. [Xbox 360]. MTV Games, et al.: New York, 2010. id Software. Quake III Arena. [Microsoft Windows, et al.]. Activision: Santa Monica, CA, 1999. SEE Digital Studios AB. Planet Michael [Microsoft Windows]. SEE Virtual Worlds, LLC.: Los Angeles, N.D. Sega/Triumph. Michael Jackson’s Moonwalker. [Sega Genesis]. Sega: Tokyo, 1990. Sonic Team. Sonic the Hedgehog 3. [Sega Genesis, et al.]. Sega: Tokyo, 1994. Ubisoft Montpellier, et al. Michael Jackson—The Experience. [Nintendo DS, et al.]. Ubisoft/Triumph International: Montreuil, 2010. United Game Artists, et al. Space Channel 5. [Dreamcast, et al.]. Sega, et al.: Tokyo, 1999. United Game Artists. Space Channel 5: Part 2. [Dreamcast, et al.]. Sega: Tokyo, 2002.

7 Virtual Jam: A Critical Analysis of Virtual Music Game Environments David Arditi

W

hen Digidesign released Pro Tools in 1991, the new digital audio workstation (DAW) transformed the contemporary recording studio. Since that time, DAWs have changed the fundamental system of composing music. DAWs are digital software applications that allow musicians and producers to record and edit music on a computer. Pro Tools was different from earlier DAWs because it graphically represented music in a manner similar to analog recording equipment. A DAW “is a visually oriented, random-access form of technology that allows engineers to record not only ‘tracks,’ in the traditional sense, but to operate at the sub-track level, freely editing, processing and moving bits of digital audio data around in ways that would be impossible in a linear, analogue system” (Théberge 2006, 82). Sound engineers transitioned to DAWs smoothly because Pro Tools spoke their language. However, DAWs continue to operate in a political economy where selling recorded music is the dominant system of musical production. Video games, on the other hand, present the opportunity to disturb the political economy of music using the technology first developed for DAWs.

178

MUSIC VIDEO GAMES

Online multiplayer gaming and digital media delivery services (i.e., Xbox Live, PlayStation Network, and Nintendo Network) present the opportunity to break the structures of music commodification. In Jacques Attali’s seminal book, Noise: The Political Economy of Music, Attali postulates a new political economy of music which he calls “composition,” in which the structures of capitalism are broken down and replaced with an idea for the creation of music for itself. Beyond the rupture of the economic conditions of music, composition is revealed as the demand for a truly different system of organization, a network within which a different kind of music and different social relations can arise. A music produced by each individual for himself [sic], for pleasure outside of meaning, usage and exchange. (Attali 1985, 137) To that end, I take up Attali’s call for researchers to conceptualize “the coming order on the basis of the designation of the fundamental noise” (Attali 1985, 133). For Attali, noise can disturb the power structure and music is organized noise. To explore fundamental noise is to examine the system as it is presented and how it could be. Technology offers the opportunity to create a cacophony of sound that destabilizes the political economy of music. Music video games could enable a musical performance system where music is no longer commodified, but composed/performed/recorded for its own sake. However, the technology and ideology upon which these spaces are built always-already commodify music. As people develop new technologies to circumvent alienation, they demonstrate the strength of capitalism as corporations attempt to colonize, commodify, and monetize these developments. Creating an online game environment for musicians to jam has the potential to alter the basic notions of music composition, production and consumption. While DAWs, such as Pro Tools and Reason, ushered in the era of self-composition and musical performance. However, there has yet to be a significant break in the system that places the commodity of recorded music at the center of the political economy of music. Online music environments that allow people at distant locations to play music together enable a break from previous modes of music production and reproduction.

VIRTUAL JAM: VIRTUAL MUSIC GAME ENVIRONMENTS

179

Composition In Noise, Jacques Attali delineates four different political economies of music: sacrificing, representing, repeating, and composing. During each period of political economy, the organization of music reflected the system of power in place at the time. Each political economic organization foreshadows what is to come. Sacrificing represented the power of the king; representation demonstrated the power of the bourgeoisie; repetition reified the commodity form of industrial capitalism.1 Composition will be the next period, but Attali saw the seeds of its arrival when he was writing in the 1970s. For the next political economy of music, Attali insists that the people will be the source of power and music will be decentralized, decommodified, and democratized. Under composition, people will make music for oneself. Composition is creating “our own relation with the world and try[ing] to tie other people into the meaning we thus create” (Attali 1985, 134). We refashion the codes of music and musicality in a way that enables communication between people. Participating in composition does not require a musical background; Attali proclaims everyone can compose. Everyone can compose because Attali declares that the codes of music are no longer important to the creation of music. He claims that people do not need to be trained formally to compose (Attali 1985, 135). Since everyone can compose, Attali asserts that everyone can communicate. The code of music is broken down and given back to the people. While this is a hopeful mode of production, it is predicated on the development of a technological system that allows such codes to be shared. Fundamentally, Attali claims that composition is opposed to the capitalist system. Rather than revising the system as the shift from representation to repetition, composition disturbs the logic of capitalism. Attali explicates: Composition, then, beyond the realm of music, calls into question the distinction between worker and consumer, between doing and destroying, a fundamental division of roles in all societies which usage is defined by a code; to compose is to take pleasure in the instruments, the tools of communication, in use-time and exchangetime as lived and no longer as stockpiled. (Attali 1985, 135)

180

MUSIC VIDEO GAMES

Because music is produced for oneself, composition destabilizes music’s commodity form. By analyzing the role of the worker and consumer, Attali envisions the breakdown of one of the tenants of capitalism. As production and consumption merge, the worker becomes less alienated. Karl Marx contends that worker alienation from the product, other workers, and other consumers are key elements to capitalist exploitation (Marx 2000, first published in 1844). The division of labor and organization of labor under capitalism not only allows for labor exploitation, but separates the worker from his or her human essence. In today’s industrialized recording studio, this is clear as different musicians record their individual parts and have no control over the final product. Record executives manage the final sound and those involved in the process are disconnected from each other, the final product and the audiences who ultimately listen to the recordings. Composition aims to intervene in this process of alienation. Creating one’s own music gives the composer access to the product of their labor; therefore, they feel connected with their labor. They have ultimate say over the final product. Finally, the composer understands the labor that other composers put into creating music. Furthermore, Attali’s discussion of use-time and exchange-time exemplifies a destabilization of capitalism because one uses music as one creates it; there is no stockpiling. Without stockpiling, music cannot exist outside of the composer as an alienated object (Ibid.) Composition abolishes the fundamental division of labor upon which capitalism is sustained. Predating Alvin Toffler’s “prosumer” (Toffler 1980), Attali highlights the blurring of producer and consumer from a Marxian perspective. Attali saw the beginning of this blurring in composition. Where other perspectives of the prosumer, prosumption, and the audience commodity (Smythe 1981) focus on the way that consumers add value to commodities, Attali saw the disintegration of consumption altogether. Composition eliminates the commodity because music is unrecorded and un-monetized. There are no longer separate fields of producer and consumer because the commodity ceases to exist. People make music for themselves without a concern about whether or not other people will listen. Under the political economic regime of consumption, music will be made for the moment—for the pleasure of creating music. To compose under this new system

VIRTUAL JAM: VIRTUAL MUSIC GAME ENVIRONMENTS

181

is not to compose in the traditional sense where music is created to be replayed by people reading sheet music or replaying recorded compositions. Rather, composition is a singular act meant to satisfy the human need to create. It allows for collaboration between musicians in the moment, but the compositions do not need to be recorded and replayed for others. By disrupting the commodification of music, Attali sees the possibility for a political economy that does not operate on the production, sale, and consumption of commodities. Attali’s hope is that the oppositional seeds contained within composition foreshadow a massive change to the political economic system. However, this is not to argue that composition eliminates the other systems, but rather that it stands against them. Composition does not eliminate sacrifice; music will always purvey myth and ideology. Composition does not eliminate repetition; people will still consume recorded music. Composition does not eliminate representation; people continue to attend concerts. In fact, the bourgeoisie have developed new means to represent their power and wealth at concerts and festivals as VIP packages become more common (Knopper 2014). Attali contends that the power of composition lies in its refusal to be defined by previous political economic systems of music. Since history has gone through political economic stages that Attali claims are preceded by shifts in the political economy of music, he sees the coming of the next shift.

Music Rhythm Games Even though computers were in their infancy when Attali published Noise in 1975, the Internet & Communication Technologies (ICTs) are the ideal communication media upon which to build composition. ICTs encompass computers, networks, gaming systems, cell phones, etc. ICTs allow for telematic music, which are distributed musical performances that occur at a distance through telecommunication systems.2 Gaming systems were transformed in 2007 with the production of Harmonix’s Rock Band, a music rhythm game stemming from the technology of Dance Dance Revolution (DDR) and Guitar Hero. Following the release of Guitar Hero in 2005, the Rock Band franchise went beyond plastic guitar-shaped controllers by adding

182

MUSIC VIDEO GAMES

drums, vocals, bass, and later keyboards. The game is played by triggering control pads (or singing into a mic) when a scrolling guitar tab indicates to trigger the pad; whether that is striking a drum pad or pressing fret buttons. Rock Band allows gamers to perform their favorite pop songs on their home gaming system. “The Guitar Hero and Rock Band games are designed to put players in the virtual shoes of live rock-concert performers” (Miller 2009, 396). Rock Band and Guitar Hero allow users to perform music in their homes, and ICTs help create a platform on which composition can be realized. In Guitar Hero and Rock Band, gamers are immersed in a concertlike virtual environment. The crowd cheers and boos depending on the performance of the gamer. When the player reaches a high streak of notes, they receive “star power,” which in turn can be used to receive bonus points (Miller 2009, 412). The focus of the game moves beyond going through the motions of playing a guitar and embraces the overall performance. At their core, music rhythm games have some ethnomusicologists positing that musical performance has changed. Kiri Miller describes “Guitar Hero and Rock Band gameplay as a developing genre of collaborative, participatory rock music performance that is generating debate about the nature of musical and performative authenticity” (Ibid., 401). Further, Miller contends that these games signal “the changing nature of amateur musicianship in an increasingly technologically mediated world” (Ibid.). In other words, playing music rhythm games is a form of musical performance and production, which has fundamentally transformed the way people play music. Alongside the shift in what counts as performance is the change in what counts as a musical instrument. What is a musical instrument? When do we decide if an object becomes a musical instrument? Marsha Siefert explores “the way in which the texts that defined and described sound-recording technology aided in legitimating sound recording as a ‘musical instrument’ within the music industry” (Siefert 1995, 418). This demonstrates that recording devices have always been conceived of as musical instruments, but this conception has always been shrouded by conflict over whether this is an accurate perception. Jonas Braasch takes up the question of what counts as a musical instrument in “The Telematic Music System” (2009) and decides that telematic music technology must be thought of

VIRTUAL JAM: VIRTUAL MUSIC GAME ENVIRONMENTS

183

as an instrument. By claiming that telematic systems are musical instruments, Braasch creates space for Attali’s composition in telematic environments. Online Rock Band and Guitar Hero would count as telematic musical instruments, but we must consider the context and function of game controllers as musical instruments. The guitar controller interface that comes with both Rock Band and Guitar Hero looks like a real guitar, but it is a piece of plastic with five buttons. Dominic Arsenault asks “Can a plastic controller shaped like a guitar, with five buttons and a crude ‘strum bar,’ manage to represent the act of playing a guitar, if not perfectly, at least accurately enough for us to recognize it as such?” (Arsenault 2008, 1). To answer this question, Arsenault conducts a close reading of the differences between playing a real guitar and a guitar controller. He finds that Guitar Hero provides a decent representation of playing a guitar based on the ways in which the progression of the notes and their arrangement on the fretboard differ from a real guitar. However, if Rock Band can be thought of as a musical instrument, then gamers would have the potential to compose. Even though gamers lack the autonomy to play the notes of their choosing, this is not much different from a musician playing a score in an orchestra; they are both limited to the notes dictated by the form. Miller wrestles with the notion of whether or not Rock Band is similar to sight reading musical notation. “The player has no choice as to what comes next, but neither does anyone who is trying to read from sheet music” (Miller 2009, 408). A comparison of Rock Band to composition hinges on the opportunity to create music, not just play back music. While Miller astutely analyzes the way that music rhythm games can be a musical instrument performance, there is still a lack of autonomy. As with all recorded media, what can be done with the medium is always-already limited by the medium itself. Theodor Adorno once quipped the “only thing that can characterize gramophone music is the inevitable brevity dictated by the size of the shellac plate” (Adorno 2002, 278 first published in 1934). Today, the only thing that can characterize music video games is the limitations dictated by game programmers. Without the autonomy to choose one’s notes, playing a musical instrument is not the same as composing. Whether someone plays second cello in the National Symphony Orchestra or Guitar Hero, they are playing someone else’s music. But are they composing music?

184

MUSIC VIDEO GAMES

While Miller does not draw the parallel between performance and composition herself, her description is precisely in line with composition. Attali’s own description of composition focuses on the potentials of Music Minus One, a company founded in the 1950s that produced recordings of music without one instrumental track. This allowed a guitarist to purchase a Music Minus One recording of their favorite band minus the guitar part. Then this guitarist could jam along with the recording creating their own guitar track. These recordings developed as both a way to practice and a leisure activity. They are a way to practice because Music Minus One allows musicians to mimic their music idols. But mimicry is never just a reflection. In the process of learning to play like their music idols, musicians begin to embody the technique. By embodying techniques learned through Music Minus One, these musicians develop new skills and new sounds. On the other hand, playing along to music is like jamming with a musician’s favorite bands (even if only in one’s imagination). There is a degree of leisure since they would likely be listening to this music if they were not playing along; but it is important to remember that leisure-time in capitalism is always already commodified (Horkheimer and Adorno 1972 first published in 1944; Marcuse 1991 first published in 1964). Playing along to Music Minus One is generally not recorded, so the performance is fleeting, and tends to be for oneself. Rock Band and Guitar Hero are influenced by Music Minus One because playing these games involves playing along to a recording, leaving the lead track to be played with the game controller. But Miller herself establishes that for Rock Band and Guitar Hero “Performers go through the motions of playing instruments, but they have no creative dominion over the song that comes out; it was originally produced by some other guitarist in a recording studio” (Miller 2009, 404). This is the difference between Rock Band and Music Minus One—Rock Band does not allow the gamer to create, whereas Music Minus One allows for the possibility of composition. An interesting phenomenon paralleling (and making use of) Music Minus One is a new genre of video on YouTube. Musicians record themselves performing along to cover music to demonstrate their musical talent for others. However, these YouTube Minus One-ers move from the composition political economy back to the system of repetition because the music is recorded and generates revenue

VIRTUAL JAM: VIRTUAL MUSIC GAME ENVIRONMENTS

185

(Nelson 2013). While many YouTube musicians hope to be discovered through their videos, others do it for fun to share with their friends. However, my aim here is not to develop ethnography of YouTube musicians, but rather point to a potential site of composition that has much in common with video games. While not explicit, the potential for Rock Band and Guitar Hero to create composition exists in their online interface. Some versions of the games allow gamers to connect using Xbox Live, PlayStation Network, and Nintendo Network. Players can join games with friends or with random people online. In this networked environment, gamers can play from their home with others in different parts of the world. These performances are fleeting because they are not recorded. People playing the game only perform in the moment for the satisfaction of playing along. Again, this highlights the leisure of the game and the pleasure that people obtain from playing, but arguably they are still not composing. Rock Band 3 went even further by allowing players to plug in real guitars, electronic drum kits and MIDI keyboards. And in Rock Band 3, the seeds were sown to push music rhythm games to composition because the game introduced a mode that allows the drums to play freely outside of the notes that appear on the screen. Drummers can play along and compose their own drum parts within the game. This moves beyond the rigidity created by the game where a missed note results in a missed part of the recording. It allows the player to develop their own beats within the existing structure of the recorded song; this is Music Minus One for a gaming system. To play in the free play mode gives the gamer autonomy to create his or her own part. The key is to go further by allowing entire game environments that embrace the logic of free play. All MIDI instruments could conceivably plug in through their game console and play with other people on the same system. Building an online environment using these gaming consoles would create an immersive musical environment that contributes to the potential of composition. People would meet online, jam with random people or friends, and create unrecorded music. Jamming through the game environment could lead to other collaborations or it could serve to be a form of leisure and pleasure. As David Hesmondhalgh argues “music can combine

186

MUSIC VIDEO GAMES

a healthy integration of different aspects of our being, combing reflection and self-awareness with kinetic pleasure” (Hesmondhalgh 2013, 53). Playing music online can stimulate this type of pleasure. Joining or leaving these jams could be as easy as joining or leaving a game of online poker. The technological capacity exists to create this kind of environment, but the environment does not exist. All potential for new systems is solidly formed under the technological and economic system under which they are developed. And this is a capitalist system; therefore, the potential only exists to the extent that companies can generate surplus value. If the motivation to create a virtual jam environment through video games existed, it would be developed with the profit motive at the fore of the game development, so composition is not likely through gaming networks. Guitar Hero and Rock Band do not create space for composition because they are video games made by large video game conglomerates, which are played on gaming systems by larger consumer electronics conglomerates. The political economy behind these games does not allow for composition. Whereas Attali argues that the seeds of new political economies are embedded within the existing political economic system, the reverse is true; any new system would contain the residue of previous systems (Hall 1993). Culture is never produced in isolation and the dominant culture always aims to appropriate that of oppositional cultures. Just as Attali’s previous political economic systems are contained within the repetition phase, composition will not only include repetition, but the very logic of composition would be derivative of repetition. Attali’s position lends itself to technological determinism because his position lacks “a sense of contradictions [about] the dialectics of technology and society” (Fuchs 2013, 202). While Attali uses a dialectic method to think about the political economy of music, he does not adequately address the relationship between technology and society. A better approach described by Christian Fuchs acknowledges that “society conditions the invention, design and engineering of technology and technology shapes society in a complex ways” (Fuchs 2013, 202–3). By critically engaging the relationship between technology and society, we can begin to see just how difficult it would be to develop a political economy based on composition.3

VIRTUAL JAM: VIRTUAL MUSIC GAME ENVIRONMENTS

187

Building Community While music rhythm games may not be able to fully develop an oppositional political economy similar to Attali’s theorization of composition, telematic video games do hold other promise in their capacity to bring people together across distances. In Why Music Matters (2013), David Hesmondhalgh demonstrates that one of the most valuable aspects of music is its capacity to develop a sense of sociability and place. Specifically, in the “domain of shared musical activity which necessarily requires a somewhat higher level of skill and commitment—playing musical instruments together in groups” (Hesmondhalgh 2013, 112). Musicians, both professional and amateur, play together for many reasons beyond making money by playing gigs. In fact, actually making money playing music is the exception to the rule; the structure of record contracts are such that even touring groups rarely have the resources to pay for fuel and a place to sleep, as they end up spending more money than they make (Arditi 2014c). Hesmondhalgh questions the logic of the division of labor for professional musicians. Rather than focusing on the political economy of playing music, Hesmondhalgh demonstrates that sociality is a key to musical performance. Furthermore, the “desire to create something together through joint cultural interaction is certainly one of the key elements of current work in the field” (Braasch 2009, 429). People want to interact because they are human. Music rhythm games could develop a system where musicians come together across distant places and begin to develop a sociality unavailable without media networks. Connecting with other people to be social is one of the main needs of being human, and music allows a space for people to make that connection. Perhaps this is where music rhythm games could most effectively be considered as a source of Attali’s composition. When musicians play together, there is pleasure in the performance; playing alone in one’s room is not as fun as playing with other musicians and/ or an audience. A Rock Band platform performed either at home with friends or over the Internet with strangers offers ways for people to connect with each other.

188

MUSIC VIDEO GAMES

Creating music is fundamentally a social phenomenon, but the recording industry tries to make it individualistic through the exploitation of copyrights. Hesmondhalgh describes: The accumulation of profit in the cultural industries, including the music industries, is based on the romantic ideology of authorship, as expressed in copyright law, which relies on the romantic idea that expressions are the property of heroic creators, thus denying the social nature of creativity—and also obscuring the fact that ownership of rights usually belongs to the corporations that finance, distribute, and market music rather than creators. (Hesmondhalgh 2013, 128) The laws that are supposed to protect musicians are used to exploit them by enforcing a nostalgia for a romantic period (that never existed) in which the solitary author/artist/genius spends their time toiling alone to create music (Marshall 2005; Arditi 2014a; Arditi 2014b). As Lee Marshall identifies in Bootlegging (2005), the nineteenth century Romantic Era of art is identified with both the autonomist musician and the need to close access to music by using copyrights to protect the product of their labor. “If Romanticism is understood to have emerged as a result of the changing material circumstances of artists themselves,” Marshall argues, “one of the most significant of the changes was the possibility of them being the owners of cultural property through copyright” (2005, p. 54). However, even though musicians own their “cultural property,” this is a means the industry’s ends whereby labels purchase these rights from musicians. After musicians sign away their copyrights, they become laborers who are alienated from their labor. Telematic jamming has the capacity to overthrow this property regime. Telematic music environments create the potential for de-alienated labor as people regain their sociality in three ways. First, in a telematic musical environment, musicians experience the product of their labor. Music is created in its final product at the time of performance; there is no remixing, quantizing, or splicing. Second, they are connected with the other workers. Everyone performing hears the music as they play it. Third, they are connected with the consumers of their labor. Since the consumers are the producers, there is no mediation aside

VIRTUAL JAM: VIRTUAL MUSIC GAME ENVIRONMENTS

189

from network through which they are connected. A random person with no connection to the performance of the music has no ability to consume it. The collective labor that produces this music does not create surplus value, so every performer produces the music as a whole human being. At the moment of performance, telematic music also creates the type of social setting that all musical performance creates. In this way, Hesmondhalgh claims that the key to music performance is its “sociable publicness” and its ability to create collective deliberation (Hesmondhalgh 2013, 146). What he means is that music creates a public sphere, but not a public sphere focused on political struggle. Rather, this is a public sphere that makes people feel part of something. “Music, especially when combined with other forms of communication…” Hesmondhalgh claims, “can be very powerful in forging, fostering, solidifying, and challenging values and attachments, for better or for worse” (Hesmondhalgh 2013, 146). The strength of musical performance remains within the bonds that it creates between those involved. Music telematics creates a mediated space to bring together musicians and form these bonds despite physical separateness.

Conclusion The technology exists to create a telematic music system that facilitates virtual jamming, but that system is only conceivable through the political economy in which we live. Since Attali’s composition requires a fundamental break from the previous political economy, the existing platforms for telematic music would only reinforce the power of capital in the political economy of music. There is a contradiction using gaming system to develop virtual jamming because they commodify composition. While composition may decommodify music, it does not decommodify the system. In fact, the system becomes commodified in terms of the technologies and networks used to create music. When people compose using networks, they are consuming a service; so composition resembles the service economy in networked society.

190

MUSIC VIDEO GAMES

There are opportunities to develop an open source free telematic gaming system. Some systems, such as JackTrip, are already available to download free. These systems seemingly de-alienate musicians at the same time that they decommodify music. However, even these systems are developed within a political economy that alienates labor because the game designer continues to be alienated, along with everyone who maintains the network and electricity. Attali’s theorization of composition creates room for an important critical thinking exercise about the way we can develop a new political economy. However, since the new political economy has to be embedded in the current political economy, there are disparities between what could be developed and the reality of the systems that would be developed. Rock Band and Guitar Hero present an opportunity to rethink musical commodification, but their commoditization always already enables the commodification of music.

Notes 1 Jacques Attali demonstrates that these four periods of music’s political economy reflect and foreshadow changes in the political economy of society. Sacrificing was a period of music where music was positioned in relation to power as opposed to money. The court Jester or Jongleur, Attali argues, is the quintessential musician in this time period. Music operates as a system of distribution for “orders, myths, and religious, social, or economic relations of symbolic societies” (Attali 1985, 31). And yet music changes as courts begin to hire musicians (e.g., Beethoven) to write and perform music. This leads to and foreshadows the period of representing. In essence, representing is the way that the bourgeois demonstrate their power. Music during representing is performed in orchestras and it costs a lot of money to go to a concert. The bourgeois represent their new power by paying to go see performances. Repeating, the next period, is an effect of “the age of mechanical reproduction” (Benjamin 1936). Attali contends that with mechanical reproduction (i.e., recordings), music could be listened to by the masses. However, music is only available to the masses through the logic of consumer capitalism. Finally, repeating gives way to composing. And composition, for Attali, is the site of restoration of the power of music to the people.

VIRTUAL JAM: VIRTUAL MUSIC GAME ENVIRONMENTS

191

2 While telematic music systems hold the potential to connect musicians over great distances, they are not without their problems. When musical performances are transmitted over the Internet, they theoretically travel at the speed of light, which would make it possible for two people on opposite ends of the United States to perform with each other as if they were in the same room. However, friction within the Internet cables slows the transmission down, and the routing of data slows down the transmission even further. As a result, the largest barrier to effective telematic musical performances is audio latency: the time interval between the articulation of a sound and the hearing of a sound. 3 In order to play in a hypothetical virtual jam environment by Rock Band through PlayStation 4 would require the PS4 (approximately $400), the game itself (estimated around $50) and access to the online PlayStation Network ($9.99 per month). In addition to these costs, it is likely that Rock Band would develop a way to charge for access or in-game applications. This is the basic political economy of digital capitalism where users pay for a service rather than a commodity. In effect, a digital network gaming platform that allows musicians/gamers to jam online would advance the expansion of the means consumption (Aglietta 2001) in the form of services under digital capitalism.

References Adorno, Theodor W. 2002. “The Form of the Phonograph Record.” In Essays on Music/Theodor W. Adorno, edited by Theodor W. Adorno, Richard D. Leppert, and Susan H. Gillespie, 277–280. Berkeley, CA: University of California Press. Aglietta, Michel. 2001. A Theory of Capitalist Regulation: The US Experience. Translated by David Fernbach. New Edition. London and New York: Verso. Arditi, David. 2014a. “Downloading Is Killing Music: The Recording Industry’s Piracy Panic Narrative.” In The State of the Music Industry, edited by Victor Sarafian and Rosemary Findley, 13–32. Toulouse: Presses de l’Universeité Toulouse. Arditi, David.. 2014b. “Digital Downsizing: The Effects of Digital Music Production on Labor.” Journal of Popular Music Studies 26 (4): 503–20. doi:10.1111/jpms.12095. Arditi, David. 2014c. iTake-Over: The Recording Industry in the Digital Era. Lanham, Maryland: Rowman & Littlefield Publishers.

192

MUSIC VIDEO GAMES

Arsenault, Dominic. 2008. “Guitar Hero: ‘Not Like Playing Guitar At All’?.” Loading… 2 (2). Accessed October 27, 2015. http://journals.sfu. ca/loading/index.php/loading/article/view/32. Attali, Jacques. 1985. Noise: The Political Economy of Music. Theory and History of Literature. Minneapolis: University of Minnesota Press. Benjamin, Walter. 1936. “The Work of Art in the Age of Mechanical Reproduction.” Accessed October 27, 2015. http://www.marxists.org/ reference/subject/philosophy/works/ge/benjamin.htm. Braasch, Jonas. 2009. “The Telematic Music System: Affordances for a New Instrument to Shape the Music of Tomorrow.” Contemporary Music Review 28 (4/5): 421– 32. doi:10.1080/07494460903422404. Fuchs, Christian. 2013. Social Media: A Critical Introduction. Thousand Oaks, CA: SAGE Publications Ltd. Hall, Stuart. 1993. “Encoding, Decoding.” In The Cultural Studies Reader, edited by Simon During, 507–17. London and New York: Routledge. Hesmondhalgh, David. 2013. Why Music Matters. Malden, MA: WileyBlackwell. Horkheimer, Max, and Theodor W. Adorno. 1972. “The Culture Industry: Enlightenment as Mass Deception.” In Dialectic of Enlightenment. Translated by John Cumming, 94–136. New York: Herder and Herder. Knopper, Steve. “Why VIP Packages Are Ruining Rock Festivals.” Rolling Stone, June 5, 2014. Accessed October 27, 2015. http://www. rollingstone.com/music/news/why-vip-packages-are-ruining-rockfestivals-20140522. Marcuse, Herbert. 1991. One-Dimensional Man: Studies in the Ideology of Advanced Industrial Society. Boston: Beacon Press. Marshall, Lee. 2005. Bootlegging: Romanticism and Copyright in the Music Industry. 1st Ed. California: SAGE Publications Ltd. Marx, Karl. 2000. “Economic and Philosophical Manuscripts.” In Selected Writings, edited by David McLellan, 83–121. Oxford and New York: Oxford University Press. Miller, Kiri. 2009. “Schizophonic Performance: ‘Guitar Hero,’ ‘Rock Band,’ and Virtual Virtuosity.” Journal of the Society for American Music 3 (4): 395–429. Nelson, Noah. “Covering Pop Hits On YouTube Is Starting To Pay.” NPR. org. May 13, 2013. Accessed October 27, 2015. http://www.npr.org/ blogs/therecord/2013/05/13/182880665/covering-pop-hits-on-youtubeis-starting-to-pay. Siefert, Marsha. 1995. “Aesthetics, Technology, and the Capitalization of Culture: How the Talking Machine Became a Musical Instrument.” Science in Context 8 (2): 417–49. Smythe, Dallas Walker. 1981. “On the Audience Commodity and Its Work.” In Dependency Road: Communications, Capitalism,

VIRTUAL JAM: VIRTUAL MUSIC GAME ENVIRONMENTS

193

Consciousness, and Canada, edited by Dallas Walker Smythe, 22–51. Norwood, N.J.: Ablex Pub. Corp. Théberge, Paul. 2006. “Music/Technology/Practice: Musical Knowledge in Action.” In The Popular Music Studies Reader, edited by Andy Bennett, Barry Shank, and Jason Toynbee, 283–91. London and New York: Routledge. Toffler, Alvin. 1980. The Third Wave. New York: Bantam Books.

Games Harmonix. Guitar Hero. [Playstation 2]. RedOctane: Mountain View, CA, 2005. Harmonix/Backbone Entertainment. Rock Band 3. [Xbox 360, et al.]. MTV Games/Mad Catz: New York, 2010. Harmonix, et al. Rock Band. [Xbox 360, et al.]. MTV Games/Electronic Arts: New York, 2007. Konami. Dance Dance Revolution. [Arcade, et al.]. Konami: Tokyo, 1998.

PART THREE

Concerts, Collaboration, and Creativity

8 Guitar Heroes in the Classroom: The Creative Potential of Music Games David Roesner, Anna Paisley, and Gianna Cassidy

T

his chapter presents initial findings from the first phase of the Arts and Humanities Research Council–funded Research Network, “Guitar Heroes in Music Education?: Music games and their potential for musical and performative creativity” (January 2014– December 2015). The network brings together international experts from academia (including music psychology, music education, game theory, and theater studies), the video game and music industry and practitioner stakeholders, to explore both the notion of musical and performative creativity in music-based games (rhythm-action,

While we make every effort to credit individual members of the network for their thoughts and contributions, this won’t always be possible: We should emphasize therefore that the following actively shaped our thinking during the live and virtual encounters through the network up until the point of writing: Karen Collins, Frank Fitzpatrick, Melanie Fritsch, Jennifer Groff, Graham McAllister, Kiri Miller, Anna Paisley, Yann Seznec, Jennifer Snow, Tim Summers, and Isabella van Elferen. A first intensive meeting and public event took place in Canterbury, UK in July 2014 (see http://musicgames-creativity-network.blogspot.co.uk [23.01.2015])

198

MUSIC VIDEO GAMES

free-form, performance, and serious), and opportunities for positive impact of music games in music education. In recent academic discourse, music is perceived and discussed frequently in connection with its performance and embodiment, which have—not only in popular music—become an integral part of music production and perception (see Auslander 1999; deNora 2000; Small 1998). This shift from understanding music as a “piece of work” toward seeing it as a complex process with many ingredients (not all of which are directly involved in sound production) has further been highlighted by music-based video games: While in some games the production of sound can be quite limited or highly regulated, the games often offer creative license within or with the game to embody, perform, and play (Roesner 2011). At the same time, music games are increasingly pervasive to the formal and informal musical activity and creativity of learners (see Cassidy and Paisley 2013; Gower and McDowall 2012; Miller 2012; Peppler et al. 2011). While learners are more and more engaged with such digital music participation outside the classroom, evidence indicates that learners are increasingly disengaged with formal music education (e.g., Abril and Gault 2008; Dillon 2003; Lamont and Maton 2008). The challenge for music educators is to capitalize on evident learner motivation for music participation through music games as a tool to create new opportunities to inspire and engage learners with music in educational or wider creative contexts (see Collins 2013; Hargreaves et al. 2003; Miller 2012), promoting exposure to the intellectual, personal, and social benefits of music participation. In this chapter we will attempt a first outline of the field and its challenges and opportunities based on the—still limited—existing literature and thematic/discourse analysis of the discussions of the AHRC network. We will address the following questions: l

How do current notions of “creativity” in music and musical performance need to be revisited in the light of the different kinds of interactions, experiences, and expressions that music-based games facilitate?

l

What consequences does this have for educational contexts? How existing games might be utilized to foster a creative artistic engagement for gamers?

GUITAR HEROES IN THE CLASSROOM l

How can music game research be successfully utilized to inform the future study and development of music games and collaborative work among educators, academics, and industry?

l

What existing creative engagements with music game software and hardware (“playing with games” rather than “playing games”) are there, and what emerging forms of music and music participation have these brought forth?

199

How can music-based games foster creativity? For many parents (and arguably a significant number of educators, too), video games are seen as a potential threat to the creative development of their child and a drain on their imaginative energies. Seeking to locate and define the creative potential in certain groups or genres of video games thus needs some clear foundational definitions and differentiations, in order not to throw the proverbial creative child out of the murky bathwaters of video games suspected of being mere entertainment or worse: time wasting. Firstly, it is worth acknowledging the sheer amount of time, energy, and persistence that games are demonstrably able to demand from their users: Gamers are known to devote extraordinary time to playing, show high levels of intrinsic motivation, and have admirably high thresholds for frustration. It would be very difficult to name another arena in which this perfect cocktail for potential learning can be found in such high numbers and spread widely across borders of class, race, ability, or gender. Next we need to unpack what we mean by “creativity”—a contested and often-woolly idea. We suggest the following distinctions, which by no means exhaust the list of definitions and options, but are particularly pertinent for our context: 1. Creativity, according to Ken Robinson, “is the process of having new ideas that have value.”1 At the risk of complicating this neatly to-the-point definition, we should

200

MUSIC VIDEO GAMES

add and emphasize the processual nature of creativity, the idea of novelty, the notions that ideas do not have to be discursive, but could also come in the form of music, movement, images, engineering, design, etc. Further, the notion of creativity seems to depend upon an entity, peer group, institution, etc., that attributes this value; Mihaly Csikszentmihalyi calls these the “gatekeepers” of the creative domain in question,2 whereas Margret Boden uses the notion of “conceptual spaces”: “The dimensions of a conceptual space are the organizing principles that unify and give structure to a given domain of thinking. In other words, it is the generative system that underlies that domain and defines a certain range of possibilities: chess moves, or molecular structures, or jazz melodies” (Boden 1996, 79). 2. According to Steven M. Smith, we can distinguish paradigmatic versus revolutionary creative thinking (or we would add “creative doing”): “Paradigmatic creative thinking generates new ideas in small, incremental steps; revolutionary creative thinking opens bold new vistas and perspectives” (Smith 2003, 15). 3. According to Boden, we should differentiate between a psychological sense of creativity (P-creativity) and a historical sense (H-creativity) (Boden, 76): “A valuable idea is P-creative if the person in whose mind it arises could not have had it before; it does not matter how many times other people have already had a the same idea. By contrast, a valuable idea is H-creative if it is P-creative and no one else, in all human history, has ever had it before” (Boden 1996, 76). 4. In relation to music (and other arts that are—at least in part— interpretive arts), it is worth stating that creativity can be a generative but also an interpretive activity:3 Glenn Gould, for example, created new and original artistic insight and value by revolutionizing the way we hear Bach’s Goldberg Variations, to name but the most famous piece from his repertoire. It would go against our common sense to deny him his creativity just because he did not actually compose the piece.

GUITAR HEROES IN THE CLASSROOM

201

We should also briefly acknowledge a significant shift in musicology, which impacts the way in which we may want to assess the potential for musical creativity in music-based games: Namely, the increasing (even if still reluctant) acceptance that music is not a “thing,” a work or a score, but an activity, a cultural practice,4 that deserves studying as a wide range of processes and articulations: From a mother’s lullaby to her child to the latest rendition of the aria of the Queen of Night, from a ritualistic chant in an African tribe to a multi-Wattpowered rock concert in Wembley Stadium. Christopher Small has captured this notion of music as a meaningful set of relationships that involve making music in his neologism “musicking” (Small 1998). Musicking, in his coinage, means “to take part, in any capacity, in a musical performance” (Small, 9), and while he does not mention music-based games explicitly,5 it is most certainly covered by his definition. Not only do we need to differentiate what we call “creative,” then, but also, where and when we situate this process in relation to different types of games and different types of relations between “gaming” and “musicking.” Typical interplays include: 1. Gamers select music to accompany their otherwise unrelated gameplay (racing games, for example). 2. Musicking is the actual purpose of the game, that is, players can play, DJ, sing along, or even compose their own music, such as in Guitar Hero World Tour, SingStar, Rocksmith6 (see Collins 2013, 13) or even The Lord of the Rings Online (Turbine 2007).7 3. The game play generates the music, while seemingly being about something else (jump & run, first person shooter, tower defense, etc.) (Sound Shapes, Child of Eden [Q Entertainment 2011], Bad Hotel [Lucky Frame 2012], Pugs Luv Beats [Lucky Frame 2012]); 4. The adaptation and performance of game music outside of the gameplay, such as performing the Super Mario Bros. theme tune on a guitar, acappella, or on carefully arranged tuned bottles (see, e.g., Collins 2013; 105, Fritsch 2012).

202

MUSIC VIDEO GAMES

5. What Collins calls “creating music from the game,” which means that “the software and hardware of games have also become content and context from musical production” (2013, 111), for example, by modifying “game engines into musical instruments” (2013, 121).8 Creativity articulates itself in a range of ways in these different interplays and its validation varies. Following Boden’s reminder about “the importance of a new idea being positively valued by a certain social group if it is to be generally recognized as creative” (Boden, 2) We would argue that there are three main points of reference for establishing a creative interaction with music-based games (as opposed to seeing them purely as an entertaining pastime and/or virtuosic display of skill, which are both dominant and perfectly valid appreciations): inbuilt feedback in the game design, opportunities to create content within the game, and customization of hardware and other aspects of performance. The game design itself, with its inbuilt systems of reward (points, unlocked features or levels, percentage of accuracy etc.) tends not to actually feedback on creativity. For example, playing an interesting drumfill in Band Hero will lose you points and not be recognized by the game, and even those games that allow for the creation of one’s own content (such as in the music editor mode in Guitar Hero World Tour, where players can record their own songs based on preset sounds, samples, and loops), tend to keep this separate from the reward-based gameplay.9 Other games, such as Sound Shapes or Musyck (FastFan 2012) do not validate creativity, per se, either, but through the relative lack of a competitive, point-based reward systems, they clearly advocate and facilitate being playfully inventive with the pre-set sounds and shapes, which create soundscapes and rhythmic and melodic patterns according to the player’s input. In December 2014, for example, playstationtrophies.org announced a “Sound Shapes Level Design Contest,” calling for the submission of user-created levels of the players’ own “original design”10 and offering a reward after a jury pre-selection and a public vote on the best level. Here, features intrinsic to a game are taken up already by the communities and “gatekeepers” outside the game itself, who determine and attribute creative merit. This warrants a more detailed

GUITAR HEROES IN THE CLASSROOM

203

look. Most music-based games imply some kind of audience. This can be the immediate presence of friends and family in the living room who witness and provide feedback on a player’s performance or the various online communities on gaming sites and popular videosharing platforms such as YouTube. Many games now have online features built in, allowing players to share content they have created within the game. Alternatively, players sometimes film themselves and/or their screens when playing music-based games. Creative leaps can hence be made by the player through creating new content: not only songs and levels, but also new outfits and/or avatars or other design features that are more about the paratexts of musical performance. In those games that allow for and highlight the players physical performance as an integral part of the game play (i.e., by singing, dancing, and/or playing via instrument-shaped controllers), it is precisely the double nature of “performance” as both achieving merit by the measuring standards of the game, as well as activating their performative presence (see Giannachi et al. 2012),11 their stage persona (see Auslander 2006), and their imagination regarding the paratexts of performance that becomes the creative domain.12 To support our argument that the creative domain in music games may shift somewhat toward the “performative,” we would like to address this slippery term in more detail: The linguist Ekkehard König has recently attempted to reign in the proliferation of the many—and at times contradictory—notions of performativity, on the one hand, by bringing the definition of the term back to its roots in language philosophy and, on the other hand, by highlighting those aspects of the term that are truly transferable and not media-specific. König highlights two core aspects from his in-depth reading of J.L. Austin’s famous speech-act theory and its further developments in the writings of Judith Butler, Michel Foucault, and Jacques Derrida. Performativity then entails: (a) The power of symbolic practices to constitute reality (b) The inversion of the traditional view on the relationship between implementing, performing, and realizing in relation to a schema, a script, a work, a rule, and thus a “rehabilitation of surface” […] against a supposedly essentialist deep structure (König 2011, 63–4).

204

MUSIC VIDEO GAMES

In relation to the first statement, Austin describes that language has not only constative properties to make statements about the world (“The sky is blue today”), but also performative properties that can directly create or transform reality (“I hereby declare you husband and wife,” “I sentence you to two year in prison”). While Austin still separates real speech acts from so-called parasitic speech acts, which are made invalid on a theatrical stage, for example, by the “as if” nature of the context, this strict separation has increasingly being called into question. Another aspect of this idea has been developed by Judith Butler in her theories on gender identity and performance, and she emphasizes the continuous processual nature of these acts. Gender, for example, then becomes neither a mere biological fact nor a single speech act (“It’s a boy!”), but a continuous iterative performance that brings forth or disguises, stabilizes or destabilizes, gender identity of the individual and the underlying notions shared by members of a social group or society. This brings us to König’s second statement. Much of our understanding of language and of culture is based on the idea of an underlying script, set of rules or works, which are individually realized and performed. These schemata can be vocabulary and grammar (or what Saussure calls “langue” and Chomsky “competence”) in language, scores and rules of harmony, rhythm, and melody in music, or dramatic texts and dramaturgies in theater. Each utterance, each performance of a piece of music, each staging of a text is an individual—often ephemeral and subordinate—instantiation of this higher order. In linguistics, the Chomskian model has been widely disputed, and so called usage-based language theories have taken its place (see Bybee and Hopper 2001). Langue and parole (or competence and performance) are much less clearly separable and have an inverted relationship: Grammar does not determine but follows our usage of language. Similar thoughts have recently been put forward in relation to music (Cook 2013; Small 1998). Music reveals itself in the observable cultural practices of musicking, not in a series of masterworks and related theoretical models. And in theater recent developments toward post-dramatic and devised performances and performance art, live art, or body art have equally put the “surface”

GUITAR HEROES IN THE CLASSROOM

205

of the individual fleeting moment center stage as the artistic event, rather than seeing it as a mere realization of a pre-existing master piece. This also suggests that in both music and theater we deal with a shift of emphasis toward the performative production of materials and materiality and an emergence of meaning in the act of performing that is not pre-encoded. The notion of emergence highlights that meaning and/or form of the performance are (a) not entirely controllable and (b) dependent on the dialogic relationship between musicians and listeners or actors and spectators. If we apply these concepts and the aforementioned idea of a “rehabilitation of surface” (Krämer cit. in König 2011, 64) to the gaming context, they support the assertion that our evaluation of the meaning, validity, and creative potential of playing these games needs re-examining. Critics have often followed the well-trodden path of pointing out the fake, inferior, and “parasitic” nature of this form of musicking and perpetuated the perception that performers of rhythmaction games “only function as the gatekeepers for a stream of previously recorded music” (Miller 2009, 397). The acknowledgment of the performativity, which these games facilitate, inverts this hierarchy. Music games bring to the fore a notion of music as an activity for which its performance and embodiment are integral—both in music production and perception (see Auslander 1999; deNora 2000; Small 1998).13 While the actual agency in the production of sound can indeed be quite limited or highly regulated, these games offer creative license with or within the game to embody, perform, and play (Roesner 2011). They give the player opportunity to invent and experiment with his or her “musical persona” (Auslander 2006).14 A lot of this performative creativity will inevitably be more in the area of P-creative behavior; adopting an outgoing “rock star” persona with idiosyncratic outfits, gesture, and unique playing techniques may signify a personal creative leap, but by no means a historical breakthrough. It is also likely, by the aforementioned definitions, that the creative thinking in play here is paradigmatic rather than revolutionary, but it can definitely constitute a process of original ideas that have value in that they surprise and excite the players themselves, boost their confidence and imagination, create social cohesion in a group, entertain and inspire others, etc.

206

MUSIC VIDEO GAMES

In Kiri Miller’s seminal study of Guitar Hero players, this personal aspect is particularly highlighted by a number of responses players gave on this very topic, emphasizing the fact that even just within the prescribed and highly regulated gameplay itself in which the player’s capacity to modify the sonic outcome is rather marginal, players still feel creative, even though being aware that by conventional standards they probably are not (see Miller 2012, 103, 118).15 Even in the domain of rhythm-action games, which allow little room for expressing new ideas of value, the imagination is activated and the emotional experience of how it might feel to be creative can be achieved. We would argue that this has value in and of itself, but also that, as studies demonstrate,16 it often acts as a trigger for a desire and curiosity to pursue further creative endeavors, with or without the game.17 The paradox of increasing learner disengagement with formal music education and concurrent rise in informal musical participation (c.f. Lamont and Maton 2008) underscores the need to translate such findings to the realm of music education. Further, this reinforces the need to authenticate the “fit” between the musical lives of learners and formal music participation in consideration of new outcomes and opportunities facilitated by digital music technologies (Dillon 2003; Folkestad et al. 1998; Green 2002; Lamont et al. 2003; Miell and MacDonald 2002; North and Hargreaves 2008). In this light, the following section reviews key findings from a body of work that sought to address this gap.

Music games and education: Research and practice Given the potential for music games to spur broader musical participation (Missingham 2007), it is perhaps unsurprising to learn that in recent times, music curriculum has grown increasingly supportive of the use of music games, particularly rhythmic-action games, such as Guitar Hero and Rock Band (Dillon 2003, 2004). With specific regard to the former, a handful of UK-based projects have spawned innovative approaches, utilizing games such as Guitar Hero as a catalyst for teacher-driven exploration of the educationally

GUITAR HEROES IN THE CLASSROOM

207

appropriate use of music games, and more broadly, as a tool for augmenting learner engagement with music (User “bmclaren,” 2010; Robertson 2011). While this work has been efficacious in pioneering the integration of music games within the classroom, with recourse to the power of music games to facilitate an intrinsic appreciation of musical creativity among learners (Miller 2012), work has neglected to explore the antecedents of the music game experience that may give rise to this particular phenomenon. Likewise, in the absence of empirical investigation and contextual implementation in music education, a need to investigate the potential of music games to scaffold and enrich conventional music curriculum objectives prevails. To this end, Cassidy and Paisley (2011) sought to “better understand the potential of music games to inspire and engage learners with music” by way of elucidating “music-game processes, experiences and features that support authentic and inclusive music-making opportunities” (Cassidy 2011/13). This was achieved through an EPSRC-funded (Engineering and Physical Sciences Research Council) project (Principle Investigator Gianna Cassidy, Co-Investigator Anna Paisley, April 2011-April 2013) entitled: “Music Games: Supporting New Opportunities in Music Education”: a twenty-four-month research program investigating the nature of interaction with music games and potential opportunities and outcomes for intellectual, social, and personal development. The work aimed to explore a multidisciplinary perspective encompassing not only music education and psychology of music, but also game-based learning, informed by growing literature addressing the use of digital games in educational contexts (c.f. Gee 2007), and growing recognition of the unique ability of games to empower learners through personalized, differentiated, and self-directed learning (Groff et al. 2010; Sandford and Williamson 2005). The potential of digital game participation to harness learner autonomy further cemented the need to coalesce research findings across these disciplines, given that the agency of the individual and fundamental principles of self-directed learning are also considered to run central to the efficacy of contemporary music pedagogy (Hargreaves et al. 2003). Review of the collective social, emotional, and scholastic benefits of both digital games and music subsequently alluded to the

208

MUSIC VIDEO GAMES

application of the concept of “flow” as a particularly fruitful theoretical stance uniting these disciplines. Indeed, scholars assert that the manifestation of optimal experiences and learning environments is contingent upon the fit between an individual’s perceived level of skill and perception of the challenge at hand, which when balanced, gives rise to an increase in self-esteem, motivation, and cognitive efficiency, thus resulting in a state of “flow” (Csikszentmihalyi 1992). Crucially, while albeit independently, both musical participation and digital games have been shown to represent optimal flow experiences (Cowley et al. 2008; Whalen 1997). The use of flow in game-based research has been successfully employed as a tool to investigate user experience, allowing researchers to identify key features and supporting mechanics of gameplay that engender a flow experience, including: altered perception of time, clear goals, immediate feedback, sense of identity, immersion, and challenge (Cowley et al. 2008; Jones 1998; Sweetser and Wyeth 2005). Likewise, the concept of flow has been adopted as a means to investigate the cognitive music-making processes of young learners, proffering deeper insight into the construction of musical meaning and emotional experiences among learners, culminating in the development of the “Flow Indicators in Musical Activities” (FIMA) instrument for future evaluative purposes (Custodero 1998, 1999). We assert that such research presents a novel theoretical framework with which to elucidate the antecedents of musical creativity in particular. Further, we suggest this should be explored in recognition of optimal flow conditions as conducive to creativity, particularly when one considers that the conceptual origins of “flow” rest upon a series of empirical investigations into the determinants of creative experiences among artists, musicians, writers, and so forth (Csikszentmihalyi 1997). Indeed, initial research within this area highlights correlation between optimum levels of flow and higherquality musical compositions and, perhaps most importantly, greater musical creativity (Byrne et al. 2003; MacDonald et al. 2006). In this light, we argue that combined knowledge of the application of flow theory in both music and games provides a unique and timely epistemological structure with which to empirically investigate the relationship between flow and the processes and outcomes of music game participation. Further, we suggest that exposition of music

GUITAR HEROES IN THE CLASSROOM

209

game features, which give rise to an optimum level of flow should, in turn, help identifying features of music game participation that imbue wider, or the necessary prerequisites for further creative experiences in music. Armed with this methodological approach, preliminary findings from Cassidy and Paisley’s (2011) body of work yielded a greater theoretical understanding of the capacity of music games to engage learners with music. In turn, this systematically exposed features of the music game context that embodied fundamental elements of flow (see Cassidy and Paisley 2013). Indeed, via a series of indepth usability sessions (n = 95) with Rock Band 318 across learners of different ages and with varying degrees of musical training and video game experience, the authors were able to eke out those latent aspects of the “optimal” music game experience (indicative of a peak flow experience of engagement and enjoyment), and to subsequently map these to existing models of flow in both games (Sweetser and Wyeth 2005) and in music (Custodero 1998). In doing so, we gained tools to help demystify connections between positive experiences in games and music, and in music games and real-world music participation, helping bridge informal and formal music-making practices.19 For example, in regard to the overarching premise of flow, we established a number of central, experiential features and mechanics of the music game that appeared to give rise to congruent levels of perceived challenge and skill (see Cassidy and Paisley 2013). Learners frequently alluded to the ability to select a particular peripheral (instrument) and concordant difficulty level as being particularly efficacious, resulting in subsequently higher levels of intrinsic motivation and subjective value of music game participation. This in turn gave rise to a more personally meaningful experience in that it fostered a deeper sense of agency across learners, thus in keeping with the central tenets of a self-directed approach in music education (Hallam and Creech 2010). Similarly, with regard to the fundamental mechanics of the game, learners routinely vocalized the benefits of obtaining immediate feedback (e.g., scores in real-time, “audience” reaction, and onscreen visualization of performance via scrolling notation) as pivotal to the development of meaningful goals as a benchmark with which

210

MUSIC VIDEO GAMES

to gauge personally relevant achievements within the music game context. To echo existing game-based research, it is important to note that such factors are considered hallmark features of the experiential nature of flow (Sweetser and Wyeth 2005), and further, that the attainability of such goals, in essence, relies upon the accessibility and inclusivity of that activity (Custodero 2002). Accordingly, learners routinely cited the core mechanic of the game as critical to the inclusivity of the activity, for example, the benefits of utilizing color to denote the scrolling notation. This latter finding is perhaps somewhat unsurprising given that the development of the Rock Band (and Guitar Hero) franchise stems from a wealth of MIT research exploring intuitive interfaces for musical participation, irrespective of prior experience and ability (Machover 2009). Less obvious, however, was the stark realization that, while learners were keen to assert that they did not view the music game activity as a viable alternate to real-world musical participation, accumulation of wider musical skills, such as rhythm, pitch, dexterity, hand-eye coordination, and acquisition of chords, and subsequent acknowledgment of the relationship between aural and notated musical representation, engendered a deeper appreciation of wider musical participation. Additionally, recognition of the incremental mastery of these skills appeared to precipitate concordant levels of self-esteem, motivation, and intrinsic value of the activity, which, crucially, were not solely attributed to that of the music game context, but rather to the experience of performing, appreciating, and creating music in a much more general sense. This was catalyzed by the ostensible blurring of music game participation and formal music activities that permeated the subjective accounts of learners (Cassidy and Paisley 2013, 2014). The next logical step was to capitalize on these findings as a means of exploring contextual implementation of music games within the classroom. It was important to explore this in line with existing curriculum goals to harness, not compromise, these positive commonalities between music game and wider music-making that make for “optimal” creative environments (flow), as a vehicle to connect informal and formal musical participation in the twentyfirst century (Green 2002, 2006; Green et al. 2005). To this end, we generated a co-created (researchers and educator) scenario to allow

GUITAR HEROES IN THE CLASSROOM

211

the classroom teacher to investigate and ensure the educationally appropriate fit20 of Rock Band 3 across one Primary 6 class (aged 10–11) of thirty-three pupils, while retaining previously identified elements of the music game experience thought crucial to the experience of flow (Paisley and Cassidy 2013).21 The chief outcome of this process was the development of a pupil-led approach that saw the music game embedded within the term topic of “Great Britain,”22 proffering a range of interrelated “Rock Band Activities” that transfused all areas of the curriculum framework23 as a means of ensuring an integrated context (combining creation, performance, and appreciation) that sought to highlight the relevance of musical participation to wider education. To further enhance the likelihood of this integration, pupils were subsequently asked to complete a “learning journal” encompassing elements of both peer- and selfassessment that required personal reflection upon one’s participation in all activities, to include: the creation of a band name, logo, and band member identity and image; design of album artwork and merchandise; composition of a “debut single” and album track list; a written biography of the band; and an “imaginary” tour of Great Britain. With specific regard to performance (in accordance with their existing “structured play” activities), children were divided into working groups of four or five for the purposes of performing “Space Oddity” by British recording artist David Bowie. Additionally, an accompanying set of guidelines was agreed upon by the teacher and researchers, in consideration of the foregoing research (Cassidy and Paisley 2013, 2014) and wider recognition of “best practice” in music education (Hargreaves et al. 2003). In particular, the teacher elected to adopt the role of facilitator (rather than instructor), within which pupils were afforded the freedom to complete the activities in any order and, crucially, with minimal instruction as a means of cultivating pupil autonomy and self-directed learning, thus exemplifying the essence of a pupil-driven approach to education (Education Scotland 2014). Likewise, within the context of gameplay, by way of acknowledging the heterogeneity of pupils with regards to musical knowledge and experience, and with a view to increasing learner agency and personalization, no formal assessment criteria (e.g., teacher evaluation of musical performance) applied. Rather, the teacher capitalized upon the existing performance

212

MUSIC VIDEO GAMES

evaluation within the game (e.g., resultant scores), thus adapting learning outcomes in light of each child’s prior ability by encouraging the children to select their own peripheral and difficulty level within the game, and most importantly, allowing the children to take ownership of their own aspirations and outcomes via the setting of their own idiosyncratic goals and achievements. This particular mode of execution gave rise to two overarching outcomes, both of which were empirically vindicated by the cumulative research evaluations of the pupils throughout the study and subsequent reflections of the teacher at post-test.24 The nature of the music game provided an immediate gateway for further exploration into the subjective rewards of standard musicianship. This appeared to bypass the many confines of musical aptitude and skill that can often present a barrier in formal musical participation, and often outside of one’s perceived musical repertoire. In view of this, research suggests that such technologies provide a platform for self-development by proffering a range of musical environments and other elements as a backdrop for identity construction (Cerulo 1997). Accordingly, both the children and teacher repeatedly referenced the benefits of being able to adopt a variety of musical personas within the game (for example, via instrument choice) as a means with which to explore and negotiate their own musical identities. Perhaps unsurprisingly, there now exists an abundance of research within the domain of digital games highlighting the benefits of being able to enter a virtual world through the adoption of an avatar or player-character to scaffold and support the development of self and the creation of a “virtual self” (Murphy 2004). It was further apparent that this ability to reconstruct and negotiate an internal representation of what it means to be “musical” was subsequently bolstered via direct involvement in related Rock Band activities, often thought privy to that of accomplished musicians. Indeed, rather than simply perform a song, children were equipped with the freedom to explore what it means to be a musician via direct involvement in and ownership of tasks, such as creating a band member identity and image, devising a tour, and designing merchandise, and so forth. Perhaps most importantly, however, the children were free to do so in the absence of the constraints that typically accompany formal modes of musical participation, such as the

GUITAR HEROES IN THE CLASSROOM

213

presence of performance evaluations, consequences, and influential others that can often present a subjective threat to one’s identity. It appears the music game context thus epitomizes what Hargreaves et al. (2003) conceptualized as the “third environment”—an inclusive and autonomous informal setting that appears not to predicate a priori ability and is marked by the lack of assessment and instructor. Additionally, this may go some way toward explaining the higher levels of perceived musicality across the learners at post-test. Indeed, when asked to rate the extent to which they perceived themselves to be musical25 prior to and following completion of all Rock Band 3 activities, we found a dramatic increase across all learners. This particular outcome is of crucial significance here, particularly when one considers that the extent to which children perceive themselves to be “good at music” and concordant perceptions of what it means to be a musician can have a direct bearing upon the likelihood of them developing as such (Hargreaves et al., 2003; Hargreaves et al. 2007). Furthermore, it is with reference to this latter finding in particular where we encounter a number of implications and avenues for future directions in music games and indeed, for the network.26

Future directions: Music game research and development Moving forward, our collective suggestions in this chapter assert a framework to inform future study and development of music games through three major implications bridging educators, academics, and industry interests. First, we advocate the application of those positive aspects of the music game context as identified by learners, to the educational arena, that is, those beneficial features of the music game experience that can be implemented within the classroom, irrespective of the physical presence of games, per se, and in consideration of Hargreaves’ et al. (2003) models of “opportunities” and “potential outcomes of music education.” This is particularly timely given that, of the research conducted to date, findings suggest that learners appear to be acknowledging a number of connections between virtual

214

MUSIC VIDEO GAMES

musical participation and formal music education practice (Cassidy and Paisley 2013; Paisley and Cassidy 2014; Peppler et al. 2011). In particular, we assert the need to extend the parameters of conventional music education, with its often narrow focus on performative aspects of “specialist” music education (such as reading notation, ear training, and performance skills)27 to capture the wider experiences and benefits of both being and becoming a musician, especially though, features of informal musical participation that provide an integrated (performance, appreciation, and creation) platform for identity exploration and reconstruction in music. These features are encapsulated in many of the off-the-shelf, commercial music games, such as Guitar Hero and Rock Band, through playercharacter customization and in-game choice.28 In doing so, we further promote the efficacy of utilizing a self-directed learning approach, combining elements of self and peer-assessment and increased personalization within these activities to preserve and promote learner agency and autonomy, as evidenced by our recent classroom investigation (Paisley and Cassidy 2014). This extends to the recognition of individual prior knowledge, experience, and ability, and perhaps most importantly, a definitive ideological shift in “focus from viewing musical learners from within a deficit versus talent/ expertise framework” to “the idea that all musical learners in all contexts of development have musical strengths and competencies” (O’Neill 2012, 166). Second, we posit a number of departure points for the use of existing games in educational contexts and parallel streams of research. Primarily, we argue for a radical change to the way music game participation is viewed—from a concrete, “low-grade simulation” of musical performance (Roesner 2011) to a stimulus for broader music education practice and wider engagement with other curriculum areas. With credit once more to the pedagogical stance adopted by the teacher and resultant outcomes in our classroombased study specifically (Paisley and Cassidy 2014), we commend the efficacy of embedding music games within education contexts to connect musical learning with areas of mathematics, languages, geography, etc. as a vehicle for enhancing the inherent value of musical participation for wider learning. In addition, and fundamental to this process, we echo the sentiments of contemporary music

GUITAR HEROES IN THE CLASSROOM

215

education practice (Hargreaves et al. 2003; O’Neill 2012; North and Hargreaves 2008) and current nationwide pedagogical guidelines (c.f. Education Scotland 2014) in that we encourage, in particular, learner-generated experiences rather than teacher-driven sequences of work. To do so, we invite both educators and researchers alike to draw upon and expand our own recommendations and guidelines borne out of our collective strands of investigation (Cassidy and Paisley 2013, 2014; Paisley and Cassidy 2014; Roesner 2011). Furthermore and by way of extension, beyond that of scholastic gains, we point to the potential for music games as an agent for social and emotional development. It is well-attested that both formal music and digital game participation often command and enhance the development of interpersonal skills, collaboration, and responsibility (Gee 2007; Hallam and Creech 2010; Hamalainen 2008, Hargreaves et al. 2003; O’Neill 2012). This was especially evident throughout the classroom study with recourse to the organic nature of peer-assisted learning and social responsibility witnessed by the researchers (and subsequently buttressed by the teacher’s recommendations), with particular regard to the future use of music games as a means of increasing social cohesion (Paisley and Cassidy 2014). However, given that this study was the first till date, we further declare a clear and pressing need to extend such research avenues to the exploration of other existing and emerging forms of music games, outside that of the rhythmic-action and entertainment music games to include, for example, music training games that claim to instill “actual” musical skill, such as Rocksmith by Ubisoft (see Ubisoft 2012).29 Our final recommendation pertains specifically to the future design and development of music-based digital games. We assert the need for design to consider educational fit and wider musical participation, drawing upon the merits of using a co-created scenario to explore uses of music games within our own research (Paisley and Cassidy 2014), and in recognition of the palpable gains from doing so. Specifically, we strongly advocate the use of a “participatory design” approach (c.f. Simonsen and Robertson 2012) in the co-design and development of future music games. We suggest employing such methods to incorporate early perspectives and requirements with

216

MUSIC VIDEO GAMES

music games among learners, educators, and games industry experts by way of aligning music game outcomes with that of conventional music education goals and, most importantly, to bridge the gap between informal musical participation with contemporary music education. In the same vein, we would further argue for the use of music game “flow” here as the theoretical building blocks, not only for evaluative purposes but also as a tool for the generation of positive musical experiences in future games (Sweetser and Wyeth 2005). Indeed, while the bulk of our studies have focused on the implementation of “flow” as a theoretical construct for assessing the subsequent social, emotional, and cognitive efficacy of music games post-production, we call for a broadening of such usage to extend to the use of “music game flow” (Cassidy and Paisley 2013, 2014; Paisley and Cassidy 2014) as a framework for producing optimum environments for musical creativity in music games. While such research remains in its infancy, the efficacy of this body of work thus far is underscored by a need to recognize and combine: (1) efforts of scholars across disciplines of games and music, (2) recent developments in both music education and “serious games” industries, and crucially, (3) the social and cultural perspectives of young musical learners in light of recent technological advancements and, indeed, (4) a penchant for playing with games.

Playing with games: A conclusion? As mentioned earlier, computer or platform games in general, and music games in particular, have spurned on players’ creativity and imagination as a material to play with: modifying hardware, cheating with the software, taking creative license with the rules of the game, taking musical or other ingredients from games and transforming them playfully into “real life” events—all of these avenues have been tested and explored by gamers, musicians, and performers. When music is the object of these playful acts of disobedience, re-contextualization, or development, the creative and educative potential becomes even more evident than already discussed. We suggest distinguishing four different directions this may take—these

GUITAR HEROES IN THE CLASSROOM

217

are of course not always neatly discreet and may well overlap or merge. Play with games may consist of: 1. Creative arrangements of video game music for traditional or non-traditional instruments30—Collins calls this “performing game music” (2013, 105). 2. Creating new original music or arrangements with the sounds, the software and/or hardware of specific games31— this includes musical genres such as Chiptunes,32 Bitpop, or 8-bit music, but also attempts to modify game controllers or consoles into musical instruments.33 These first two cases would constitute what Isabella van Elferen calls “superdiegetic music”34—music that starts in the game and then moves outside of it entirely.35 3. Cheating the gameplay to modify its original purpose toward musical entertainment, such as choreographing fighting avatars from a “Beat’em up” game to dance to an external music track.36 This constitutes a reversal of van Elferen’s “super-diegetic”: Here the music does not move outside of the game, but moves inside of the game. 4. Using the game to develop transferable musical skills and employing these in musical activities unrelated to game settings, aesthetics, and equipment. People have, for example, learned to play the bass or guitar via Rocksmith and now play in bands, they have acquired new dance moves through Dance Central (Harmonix 2010) and show them off at parties and clubs, and they have sophisticated their attentive listening skills via Band Hero or Sound Shapes and now enjoy getting more out of everyday music consumption.37 Clearly, all of these activities constitute creative behavior at least in the sense of coming up with original and unforeseen solutions to problems and/or imagining and developing new ways of “coloring outside the lines”—using the materials, parameters, and/or frameworks provided by the games and their hardware to invent new ways of playing and of musicking.

218

MUSIC VIDEO GAMES

Finally, the potential of music-based games for musical creativity should also be considered by noting the impact they may have on academic disciplines, which govern and influence our understanding of music, particularly musicology and music education. Van Elferen suggests38 that music games and game music challenge—and therefore potentially liberate—the way we think of and attend to music. With references to the already mentioned concept of “musicking” and Carolyn Abbate’s notion of the “drastic” in musicology (2004), she reiterates how games force us to encounter music not as text but rather as an event, not as a series of masterpieces but as a cultural practice. This, then, calls into question the methods of selecting, preserving, and analyzing that have dominated musicology for many years and opens up avenues of developing new frameworks and modes of analysis for musical practices that do not fall within the established patterns and hegemonies of Western art music.

Notes 1

Other writers on creativity use slightly different wordings, but as a working definition, Robinson’s phrase is pithy and plausible. See: Csikszentmihalyi 1996.

2

In many types of games, “gatekeeping” is, of course, a core aspect of the game design: by restricting content or player features which first have to be earned. Playing creatively with player outfits, for example, in certain games requires previous mastery of introductory levels of the gameplay. In addition, there is a keen online community, which will further reward and appreciate skill and creativity in making music with/in games. The function of the “gatekeepers” has thus been somewhat democratized: not the conservatoire professor, the recital jury or the Rolling Stone critic judge your musical creativity, but the crowd on YouTube or similar platforms, who vote with clicks, comments, likes, and re-tweets.

3

Kiri Miller and Jennifer Snow expanded on this point at our network meeting in Canterbury (see Endnote 1).

4

See Rebstock (2008).

5

The earliest so-called rhythm-action video games such as PaRappa the Rapper (Sony 1996), were only just launched at the time of publications of Small’s book.

GUITAR HEROES IN THE CLASSROOM

219

6

This game is an interesting point, as it is designed in such a way, that an actual guitar or bass becomes the controller of the game.

7

William Cheng writes: “The Lord of the Rings Online is a multiplayer role-playing game with a music system that allows players to perform tunes with their avatars in the virtual world” (2012, 25).

8

We will come back to this category in the section titled “Playing with games.”

9

See Michael Austin and Peter Schultz’s chapters in this same volume for further aspects of level design.

10 See: “Sound Shapes Level Design Contest,” Website Entry, posted by ShadowAFC, December 03, 2014, http://www. playstationtrophies.org/news-3459-Sound-Shapes-Level-DesignContest.html [12.12.2014]. 11 Cf. particularly the chapters of Rebecca Schneider and Philipp Zarrilli. 12 Kiri Miller accordingly calls the Guitar Hero franchise “not a guitar simulator but more a ‘rock performance simulator’” (2012, 121). 13 The rock musician Sting, who participated in creating himself as a musical avatar for the Guitar Hero World Tour game, discusses this idea in a “making of” video: “What’s fascinating about it to me is that I’d never really thought about the idea that the way you move is as much a signature as the way you sing or the way you play.” See: “GHWT Sting BTS,” YouTube Video, 1:15, posted by “GuitarHeroGames,” October 24, 2008, https://youtu.be/OxUn23eYFv0. 14 See also Melanie Fritsch’s thoughts on performativity and persona in her chapter “Beat It! —Playing the ‘King of Pop’ in Video Games” in this same volume. 15 See Mario Dozal’s chapter on Guitar Hero and consumerism in this same volume for a different take on the franchise. 16 See Cassidy and Paisley (2013), Miller (2012), and Peppler et al. (2011). 17 We will come back to this point in the section titled “Playing with games.” 18 Rock Band 3 was purposively selected as the most technologically challenging yet socially inclusive example of the music game at that time, evidenced by the initiation of the “pro-instruments” to simulate real instrumental facility and transferable musical skill(s) (Harmonix 2010). 19 An exhaustive account of these studies is beyond the scope of this chapter. However, for a comprehensive overview of the initial phases of research from this program of work, see Cassidy and Paisley (2013, 2014).

220

MUSIC VIDEO GAMES

20 The presented study took place in a non-denominational, coeducation school in Scotland, governed by the Curriculum for Excellence (CfE; Education Scotland, 2014). 21 See Paisley and Cassidy (2014) for a detailed account of the classroom study. Nathan Fleshner offers further perspectives and examples on the implementation of music games in the classroom in this same volume. 22 A ten-week project that sought to advance cultural and historical knowledge of Great Britain. 23 Expressive Arts, Languages, Health & Wellbeing, Numeracy & Mathematics, Social Studies, Sciences, Religious and Moral Education and Technologies. 24 Following completion of the classroom study with the children, a one-to-one interview was conducted with the teacher for the purposes of evaluating the efficacy of the project. 25 We asked: “How musical do you think you are?” with responses recorded using a Visual Analog Scale from “Not at All” to “Very.” 26 Results from the classroom study were presented at the first public event of the network on July 12th 2014 at the University of Kent: http://music-games-creativity-network.blogspot.co.uk/ 27 See North and Hargreaves (2008). 28 Indeed, a number of recent “best practice” guidelines have since emerged, championing the need to foster a more complete understanding of musicianship that consolidates authentic and active music-participation akin to that of real-world music-making and thus, a paradigmatic switch to focus on those aspects of musical participation relevant to the learner (Furlong and Davies 2012). 29 See Daniel O’Meara’s chapter on Rocksmith in this same volume for a more in-depth discussion of this game. 30 E.g., the Mario Theme arranged for two electric guitars, played simultaneously by one person (“Super mario theme on 2 guitars at the same time,” YouTube Video, 1:05, posted by “StupidPr0ducti0n,” June 20, 2008, https://www.youtube.com/watch?v=syqbxkizCaI [20.01.2015]) or for tuned bottles, set at distinct distances and struck by a passing remote controlled toy car with a beater (“Mario Theme song with bottles,” YouTube Video, 1:15, posted by “volcom8967,” May 28, 2008, www.youtube.com/watch?v=3ZtBeiGO5MA [20.01.2015]). 31 See, e.g., the 8-bit remix of Daft Punk’s Get Lucky at “Daft Punk—Get Lucky (8-Bit NES Remix),” YouTube Video 10:00,

GUITAR HEROES IN THE CLASSROOM

221

posted by “rakohus,” April 21, 2013, https://www.youtube.com/ watch?v=G7HVrOkhuLw [20.01.2015]). 32 See also Collins 2012, 112–18. 33 See, e.g., Yann Seznek’s presentation of a loop-machine using the Wii-controller (“Wii Loop Machine 2.0 preview,” Blog entry with Video, posted by Yann Seznek, January 14, 2008, http://www. yannseznec.com/wii-loop-machine-20-preview/ [26.01.2015]) or Julian Corrie’s performance on obsolete consoles and other hardware (video directed by James Houston): “‘Everything you hear is coming from a vintage console or computer peripheral,’ says Houston. A cheap guitar-to-MIDI converter allowed Corrie to use a traditional guitar as an input for a Rube Goldberg-esque rig that consisted of a SEGA Mega Drive, two Commodore 64s, a few floppy drives, and an assortment of GameBoys and CRT televisions” (“Watch: Obsolete Game Consoles Become Instruments for a Rock Song,” Online magazine article, posted by Joseph Flaherty, September 13, 2013, http://www.wired. com/2013/09/an-orchestra-made-with-old-video-games). 34 She introduced this notion at the first AHRC network meeting (July 11, 2014, Canterbury). See also here: http://joebennett. net/2014/03/29/ludomusicology-and-the-new-drastic-crassh3cisabella-van-elferen. 35 Another interesting example is Luke Plunkett music track, which is compiled from console sounds: “not the sounds from games, the sounds of actual consoles. Start-ups, menus, that sort of stuff.” http://kotaku.com/catchy-tune-made-entirely-using-game-consolesounds-1281370524 [26.01.2015]. 36 Melanie Fritsch pointed us to this example: Chris Brandt and Jesse Reklaw playing “against the grain” of the SoulCaliber (Project Soul 1998) game (see “Dance, Voldo, Dance—THE ORIGINAL,” YouTube Video, 3:42, posted by “Bain Street Productions,” October 2, 2010, https://www.youtube.com/watch?v=k08gjJbxG-A. 37 Miller expands on this last point, by listing things to be potentially learned: “New modes of musical listening, a sense of the physical relationship musicians developed with their instruments, an intimate knowledge of a particular selection of songs, and assorted elements of rock history and ideology” (2012, 120). 38 This is based on her yet unpublished talk she gave at the AHRC network meeting in Canterbury, 12 July 2014. See also David Roesner, “Videogames—Music—Creativity—Education: Public event 2014: Documentation,” Blog entry, September 30, 2014. http://music-games-creativity-network.blogspot.co.uk/2014/09/ public-event-2014-documentation.html.

222

MUSIC VIDEO GAMES

References Abbate, Carolyn. 2004. “Music—Drastic or Gnostic.” Critical Inquiry 30 (Spring 2004): 505–36. Abril, Carlos R., and Brent M. Gault. 2008. “The State of Music in Secondary Schools: The Principal’s Perspective.” Journal of Research in Music Education 56 (1): 68–81. Arsenault, Dominic. 2008. “Guitar Hero: ‘Not Like Playing Guitar At All?’” Loading … 2 (2). Accessed October 28, 2015. http://journals. sfu.ca/loading/index.php/loading/article/view/32. Auslander, Philip, 1999. Liveness: Performance in a mediated culture. New York: Routledge. Auslander, Philip. 2006. “Musical Personae.” TDR: The Drama Review 50 (1): 100–19. Boden, Margaret A., ed. 1996. Dimensions of Creativity. Cambridge, MA: The MIT Press. Bohm, David. 1998. On Creativity. Edited by Lee Nichol. London and New York: Routledge. Bröckling, Ulrich. 2006. “On Creativity: A Brainstorming Session.” Educational Philosophy and Theory 38: 513–21. Bybee, Joan L., and Paul J. Hopper, eds. 2001. Frequency and the Emergence of Linguistic Structure. Amsterdam: John Benjamins. Byrne, Charles, Raymond MacDonald, and Lana Carlton. 2003. “Assessing Creativity in Musical Compositions: Flow as an Assessment Tool.” British Journal of Music Education 20 (3): 277–90. Cassidy, Gianna G., and Anna M. Paisley. 2011. Music Health and Wellbeing Conference, Sydney de Hann Centre, Folkstone, United Kingdom, September 9–11, 2011. Cassidy, Gianna G., and Anna M. Paisley. 2013. “Music-games: A Case Study of Their Impact.” Research Studies in Music Education 35 (1): 119–38. Cassidy, Gianna G., and Anna M. Paisley. In Press. “Music Games and Music Identities.” In Oxford Handbook of Musical Identities, edited by Raymond MacDonald, Dorothy Miell, and David Hargreaves. Oxford: Oxford University Press. Cerulo, Karen A. 1997. “Identity Construction: New Issues, New Directions.” Annual Review of Sociology 23: 385–409. Collins, Karen. 2008. Game Sound: An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design. Cambridge, MA: The MIT Press. Collins, Karen. 2013. Playing with Sound: A Theory of Interacting with Sound and Music in Video Games. Cambridge, MA: The MIT Press. Collins, Karen, Bill Kapralos, and Holly Tessler. 2014. Oxford Handbook of Interactive Audio. Oxford: Oxford University Press.

GUITAR HEROES IN THE CLASSROOM

223

Cook, Nicholas. 2013. Beyond the Score: Music as Performance. New York: Oxford University Press. Cowley, Ben, Darryl Charles, Michaela Black, and Ray Hickey. 2008. “Towards an Understanding of Flow in Video Games.” ACM Computers in Entertainment (CIE) 6 (2): Article 20. Csikszentmihalyi, Mihaly. 1992. Flow: The Psychology of Happiness. London, UK: Random House. Csikszentmihalyi, Mihaly. 1997. Flow and the Psychology of Discovery and Invention. New York: Harper Perennial. Custodero, Lori A. 1998. “Observing Flow in Young Children’s Music Learning.” General Music Today 12 (1): 21–7. Custodero, Lori A. 1999. “Constructing Musical Understandings: The Flow-Cognition Interface.” Bulletin for the Council of Research in Music Education 142: 79–80. Custodero, Lori A. 2002. “Seeking Challenge, Finding Skill: Flow Experience in Music Education.” Arts Education and Policy Review 103 (3): 3–9. DeNora, Tia, 2000. Music in Everyday Life. Cambridge, MA: Cambridge University Press. Dillon, Teresa. 2003. “Collaborating and Creating on Music Technologies.” International Journal of Educational Research 39 (8): 893–97. Dillon, Teresa. 2004. “It’s in the Mix Baby: Exploring How Meaning is Created within Music Technology Collaborations.” In Collaborative Creativity: Contemporary Perspectives, edited by Dorothy Miell, and Karen Littleton, 144–57. London: Free Association Books. Education Scotland. 2014. “Principles for Curriculum Design.” Accessed October 27, 2015. http://www.educationscotland.gov.uk/ learningandteaching/thecurriculum/howisthecurriculumorganised/ principles. Folkestad, Göran, David J. Hargreaves, and Berner Lindström. 1998. “Compositional Strategies in Computer-based Music-making.” British Journal of Music Education 15 (1): 83–7. Fritsch, Melanie, and Stefan Strötgen. 2012. “Relatively Live: How to Identify Live Music Performance.” Music and the Moving Image 5: 1–16. Furlong, John, and Chris Davies. 2012. “Young People, New Technologies and Learning at Home: Taking Context Seriously.” Oxford Review of Education 38 (1): 45–62. Gee, James Paul. 2007. What Video Games Have to Teach Us About Learning and Literacy. New York: Palgrave Macmillan Trade. Giannachi, Gabriella, Nick Kaye, and Michael Shanks. 2012. Archaeologies of Presence. Art, Performance, and the Insistence of Being. London: Routledge.

224

MUSIC VIDEO GAMES

Gower, Lily, and Janet McDowall. 2012. “Interactive Music Video Games and Children’s Musical Development.” British Journal of Music Education 29 (1): 91–105. Green, Hannah, Keri Facer, Tim Rudd, Patrick Dillon, and Peter Humphreys. 2005. Personalisation and Digital Technologies. Bristol, UK: Futurelab. Green, Lucy. 2002. How Popular Musicians Learn: A Way Ahead for Music Education. Hampshire, UK: Ashgate. Green, Lucy. 2006. “Popular Music Education in and for Itself, and for “Other” Music: Current Research in the Classroom.” International Journal of Music Education 24 (2): 101–18. Groff, Jen S., Cathrin Howells, and Sue Cranmer. 2010. “The Impact of Console Games in the Classroom: Evidence from Schools in Scotland.” Accessed October 27, 2015. http://www2.futurelab.org.uk/ resources/documents/project_reports/ConsoleGames_report.pdf. Hallam, Susan, and Andrea Creech, eds. 2010. Music Education in the 21st Century in the United Kingdom: Achievements, Analysis and Aspirations. London, UK: Institute of Education. Hamalainen, Raija. 2008. “Designing and Evaluating Collaboration in a Virtual Game Environment for Vocational Learning.” Computers & Education 50 (1): 98–109. Hargreaves, David J., Nigel A. Marshall, and Adrian C. North. 2003. “Music Education in the Twenty-First Century: A Psychological Perspective.” British Journal of Music Education 20 (2): 147–63. Hargreaves, David J., Ross M. Purves, Graham F. Welch, and Nigel A. Marshall. 2007. “Developing Identities and Attitudes in Musicians and Classroom Music Teachers.” British Journal of Educational Psychology 77 (3): 665–82. Harmonix. “Rock Band 3 features, set list and more!” June 11, 2010. Accessed April 25, 2015. http://www.rockband.com/blog/rock-band-3announcements. König, Ekkehard. 2011. “Baussteine Einer Allgemeinen Theorie Des Performativen Aus Linguistischer Perspektive” In Theorien des Performativen Sprache—Wissen—Praxis. Eine kritische Bestandsaufnahme, edited by Klaus Hempfer, and Jörg Volbers, 43–68. Bielefeld: Transcript. Lamont, Alexandra, David J. Hargreaves, Nigel A. Marshall, and Mark Tarrant. 2003. “Young People’s Music in and out of School.” British Journal of Music Education 20 (3): 229–41. Lamont, Alexandra, and Karl Maton. 2008. “Choosing Music: Exploratory Studies into the Low Uptake of Music GCSE.” British Journal of Music Education 25 (3): 267–82. MacDonald, Raymond A.R., Charles Byrne, and Lana Carlton. 2006. “Creativity and Flow in Musical Composition: An Empirical Investigation.” The Psychology of Music 34 (3): 292–307.

GUITAR HEROES IN THE CLASSROOM

225

Machover, Tod. 2009. “Beyond Guitar Hero—Towards a New Musical Ecology.” RSA Journal (London), January–March 2009. Miell, Dorothy E., and Raymond A.R. MacDonald. 2002. “Social Processes in Musical Communication: A Study of Children’s Collaborative Compositions.” European Journal for Semiotic Studies 14 (1–2): 321–38. Miller, Kiri. 2009. “Schizophonic Performance: Guitar Hero, Rock Band, and Virtual Virtuosity.” Journal of the Society for American Music 3 (4): 321–429. Miller, Kiri. 2012. Playing Along: Digital Games, Youtube, and Virtual Performance. Oxford: Oxford University Press. Missingham, Andrew. 2007. “Why Console-games are bigger than rock ‘n’ roll: What the Music Sector Needs to Know and How it can Get a Piece of the Action.” Report Commissioned by Youth Music. London, UK: Youth Music. Accessed April 25, 2015. http://dev1.youthmusic. org.uk/assets/files/Console%20games%20and%20music_1207.pdf. Murphy, Sheila C. 2004. “’Live in Your World, Play in Ours’: The Spaces of Video Game Identity.” Journal of Visual Culture 3 (2): 223–38. North, Adrian C., and David J. Hargreaves. 2008. The Social and Applied Psychology of Music. Oxford: Oxford University Press. O’Neill, Susan A. 2012. “Becoming a Music Learner: Towards a Theory of Transformative Music Engagement.” In The Oxford Handbook of Music Education, vol. 1, edited by Gary E. McPherson, and Graham F. Welch, 163–86. Oxford: Oxford University Press. Paisley, Anna M., and Gianna G. Cassidy. In Press. “Music-Games: New Opportunities for Music Education.” In Critical Perspectives: Music, Technology and Education, edited by Andrew King, and Evangelos Himonides, Thousand Oaks and London: SAGE. Peppler, Kylie, Michael Downton, Eric Lindsay, and Kenneth Hay. 2011. “The Nirvana Effect: Tapping Video Games to Mediate Music Learning and Interest.” International Journal of Learning and Media 3 (1): 41–59. Purser, Ronald E., and Alfonso Montuori. 1997. “Social Creativity: The Challenge of Complexity.” Pluriverso 1 (2): 78–88. Accessed October 17, 2007. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.20 0.9164&rep=rep1&type=pdf. Rebstock, Matthias. 2008. “Theorie der Praxis, Praxis als Theorie. Überlegungen zu einer ‘praktischen Musik-Theater-Wissenschaft.’” In Theorie und Praxis der Künste: Jahrbuch für Kulturwissenschaften und ästhetische Praxis 2008, edited by Stephan Porombka, Wolfgang Schneider, and Volker Wortmann, 61–80. Tübingen: Francke. Robertson, Derek. “Gavinsburn PS—The REAL School of Rock.” Consolarium Blog. April 25, 2011. Accessed October 28, 2015. http:// blogs.educationscotland.gov.uk/consolarium/2011/04/25/gavinburnps-the-real-school-of-rock.

226

MUSIC VIDEO GAMES

Roesner, David. 2011. “The Guitar Hero’s Performance.” Contemporary Theatre Review 21 (3): 276–85. Sandford, Richard, and Ben Williamson. 2005. Games and Learning. Bristol, England: Nesta Futurelab. Simonsen, Jesper, and Toni Robertson, eds 2012. Routledge International Handbook of Participatory Design. New York: Routledge. Small, Christopher. 1998. Musicking: The Meanings of Performing and Listening. Middletown, CT: Wesleyan University Press. Smith, Steven M.. 2003. “The Constraining Effects of Initial Ideas.” In Group Creativity: Innovation Through Collaboration, edited by Paul B. Paulus, and Berbard A. Nijstad. New York: Oxford University Press. Summers, Tim. 2011. “Playing the Tune: Video Game Music, Gamers, and Genre.” Act: Zeitschrift für Musik & Performance (2). Accessed January 19, 2015. http://www.act.uni-bayreuth.de/en/ archiv/2011/02/04_Summers_Playing_the_Tune/index.html. Sweetser, Penelope, and Peta Wyeth. 2005. “GameFlow: A Model for Evaluating Player Enjoyment in Games.” Computers in Entertainment 3 (3): 1–24. Ubisoft. 2012. “Rocksmith 2014: The Fastest Way to Learn Guitar.” Accessed April 25, 2015. http://rocksmith.ubi.com/rocksmith/en-GB/ home/index.aspx. User “bmclaren.” “Falkirk Guitar Heroes.” Consolarium Blog. June 16, 2010. Accessed October 28, 2015. http://blogs.educationscotland. gov.uk/consolarium/2010/06/16/falkirk-guitar-heroes. Whalen, Samuel. 1997. “Assessing Flow Experiences in Highly Able Adolescent Learners.” Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL, March 24–28, 1997. Whalen, Zachary. 2004. “Play Along: An Approach to Video Game Music.” Game Studies 4/1, November 2004. Accessed January 15, 2015. www.gamestudies.org/0401/whalen.

Games FastFan, Inc. Musyck. [Microsoft Windows/OS X]. FastFan, Inc.: Austin, 2012. Harmonix. Guitar Hero. [PlayStation 2]. RedOctane: Mountain View, CA, 2005. Harmonix/Backbone Entertainment. Rock Band 3. [Xbox 360, et al.]. MTV Games/Mad Catz: New York, 2010. Harmonix, et al. Rock Band. [Xbox 360, et al.]. MTV Games/Electronic Arts: New York, 2007.

GUITAR HEROES IN THE CLASSROOM

227

Lucky Frame. Bad Hotel. [iOS, et al.]. Lucky Frame: Edinburgh, 2012a. Lucky Frame. Pugs Luv Beats. [iOS]. Lucky Frame: Edinburgh, 2012b. NanaOn-Sha. PaRappa the Rapper. [PlayStation/PlayStation Portable]. Sony Computer Entertainment: Tokyo, 1996. Neversoft, et al. Guitar Hero World Tour. [PlayStation 2, et al.]. Activision, et al.: Santa Monica, 2008. Neversoft, et al. Band Hero. [PlayStation 2, et al.]. Activision: Santa Monica, 2009. Nintendo R&D Group 4. Super Mario Bros. [NES/Famicom]. Nintendo: Kyoto, 1985. Project Soul. SoulCaliber. [Arcade, et al.]. Namco: Tokyo, 1998. Q Entertainment. Child of Eden. [Xbox 360/PlayStation 3]. Ubisoft: Montreuil, 2011. Queasy Games/SCE Santa Monica Studios. Sound Shapes. [PlayStation 3, et al.]. Sony Computer Entertainment. Tokyo, 2012. SCE London Studio. SingStar. [PlayStation 3/ PlayStation 4]. Sony Computer Entertainment: Tokyo, 2007. Turbine, Inc. The Lord of the Rings Online. [Microsoft Windows/OS X]. Turbine, Inc. et al.: Needham, MA, 2007. Ubisoft San Francisco. Rocksmith. [Xbox 360, et al.]. Ubisoft: Montreuil, 2012.

9 Rocksmith and the Shaping of Player Experience Daniel O’Meara

A

t the peak of its franchise, Guitar Hero III sold over 16 million copies.1 Since 2008, however, sales of music games like Guitar Hero have plummeted. In 2008, music games raked in $1.4 billion and the following year they only made about half of that—despite the release of the very popular The Beatles: Rock Band (Kreps 2009). For some, this failure emerged from an implied comparison to “real” musicianship. As Kiri Miller has observed, media coverage has frequently emphasized the non-musical nature of music games like Guitar Hero—and even within the community of Guitar Hero enthusiasts, players sometimes make suggestions like, “Dude, just learn how to play the instrument” (Miller 2009, 407). And yet in her surveys of Guitar Hero and Rock Band players, Miller also notes that many of these individuals actually have experience playing a musical instrument (Miller 2009, 407). Out of this apparent gap in a flagging market, recent games like Rocksmith (Ubisoft 2012) and BandFuse: Rock Legends (Realta 2013) aim to provide a Guitar Hero-style game in which players use real guitars and basses. Crucial to their success seems to be this emphasis on authenticity.2 Rocksmith bills its series as “authentic guitar games,” and highlights the fact that players “plug in any real guitar.”3 The makers

230

MUSIC VIDEO GAMES

of BandFuse similarly stress the system’s difference from the plastic Guitar Hero and Rock Band controllers of the past by describing the use of “REAL” instruments, with the word “real” capitalized on their website.4 Rocksmith 2014 (Ubisoft 2013), the second in the Rocksmith series, adds to this trajectory with an explicitly pedagogical aim “to learn with the Rocksmith method.”5 Unlike Guitar Hero’s cartoonish third-person representations of the player via characters like Johnny Napalm, Judy Nails, and Axel Steel, Rocksmith remains largely in first-person perspective.6 Similarly, the game’s internal menus depict photorealistic spaces mimicking the everyday surroundings of a rock musician: couches, coffee tables, amplifiers, and tour cases. As game developers embrace real-life elements and attempt to discard the trappings of virtuality that pervade music games like Guitar Hero, an irony emerges. Rocksmith presents a version of the music game “grown up,” but game elements persist; amid traces of realism, features like the “Guitarcade” remind the player that Rocksmith exists as a virtual entity. Rocksmith blurs the boundaries between the real and the virtual, and this blurring occurs not only in the cultures that surround the game but also emerges from the game itself.7 Embedded in the game’s difficulty settings are implicit musictheoretical underpinnings that fundamentally shape the player’s listening and performing experiences—even outside of the game. Rocksmith’s difficulty system presents a series of different versions of the same song based on the player’s proficiency. This system reinforces and generates musical structure; Peter Shultz has suggested that the difficulty levels in music games like Guitar Hero encourage players to adopt a “hierarchical order of musical awareness”. According to Shultz, the easy, medium, hard, and expert levels in Guitar Hero support a framework in which players “conceive of difficult patterns as elaborated versions of simpler ones” in a manner akin to reductive or generative forms of music analysis (Shultz 2008, 187).8 For music games like Guitar Hero or Rocksmith, the dispersal of musical materials among various difficulty levels encourages a general hierarchical awareness, as Shultz says. But difficulty levels also embed specific, analytical choices that impact the way the listener hears each individual song. Rocksmith encourages both novice and advanced players to repeat the song at harder and harder difficulties in order to build up proficiency. As the difficulty gradually ramps up,

ROCKSMITH AND THE SHAPING OF PLAYER EXPERIENCE

231

either section-by-section using the game’s “riff repeater” or with sequential playthroughs, the player encounters the musical events of the easier levels more frequently than those of the harder levels. The developers’ choices about structure and ornament are thereby reinforced in the player’s ears, eyes, and hands. In this process, what is deemed easier becomes what is deemed essential, and musical structure emerges not merely at a conceptual level, but also as a physically and aurally embodied reality. In this chapter, I explore how Rocksmith cultivates the player’s experience both within the game and outside of it. First, I investigate the consumer context surrounding the game; examining Rocksmith’s relationship to its predecessors in the Guitar Hero and Rock Band franchises, I unpack an underlying ontological tension between Rocksmith as a game and Rocksmith as a teaching tool. In the second section, I explore how the game developers’ focus on guitar pedagogy impacts the stratification of musical materials in the game’s difficulty system. In particular, I explore one musical ramification of this difficulty system: how rhythmic fragments from lower difficulty levels can radically reshape the way a player perceives a song. Throughout the section, I examine repetitive guitar riffs transcribed from the game, in which the riffs’ cyclical repetitions accentuate the game developers’ divisions between musical structure and ornament.

“Learn a Song” and the “Guitarcade”: Pedagogy and game in Rocksmith’s consumer context Rocksmith seems poised in an awkward position, both rebelling against Guitar Hero by billing itself as a teaching tool, and yet simultaneously acting act as an extension of its predecessor with a continuation and expansion of its “gamier” elements.9 In order to accommodate these two aims, the developers annex the “game” component (including minigames, a player leaderboard, and a “Score Attack” mode) into a separate virtual space called the Guitarcade. In addition to its allusions to video games of the 1980s and 1990s, this mode’s design elements contrast with Rocksmith 2014’s “Learn a

232

MUSIC VIDEO GAMES

Song” mode, which presents players with a realism-infused interface oriented toward the pedagogical side of the video game. This synthesis of music game and guitar pedagogy relies on the game’s hybrid notational system, which combines the rhythm-based game mechanics of Guitar Hero with traditional guitar tablature. In Guitar Hero or Rock Band, rhythmic events approach the player along a perspectival plane. The horizontal span across the screen signifies pitch, and this orientation follows the layout of a right-handed guitar; pitches to the right side of the screen move up the neck of the guitar and are therefore higher than those to the left side.10 Rocksmith expands the rhythm and pitch notation from Guitar Hero’s dual axes to a three-dimensional system. While in Guitar Hero, there is only one “string” (comprising five buttons, or “frets”) and thus only one dimension to represent pitch, Rocksmith includes all the strings of the instrument (six for guitar, four for bass) as well as a full 22-fret guitar layout. This transforms the one-dimensional, single “string” of Guitar Hero into a two-dimensional pitch “plane” that conveys both fret (horizontally) and string (vertically). Because the small changes in the vertical axis are difficult to distinguish from afar, each string is also color coded: the strings E, A, D, G, B, and E correspond to the colors red, yellow, blue, orange, green, and purple, respectively.11 Just as in Guitar Hero, rhythmic events—this time in the form of colored rectangular boxes—approach the foreground. Players are expected to strike the correct note as they pass through a simulated, transparent guitar neck, and their success is judged with respect to the accuracy of the attack in terms of pitch and rhythm.12 In Rocksmith 2014’s “Learn a Song” mode, the gameplay attempts to cast off as many signifiers of its virtual existence as possible. The game’s main menu shows a realistic-looking room (Figure 9.1a): a stool and a guitar case stand in front of a poster, and a version of the Rocksmith logo, made up of individual guitar effects pedals, adorns the wall next to a bright window. In this mode, no score is kept and the gameplay is streamlined. No pop-up messages appear, except for discreet indications when a note or chord is played late or missed, and instead of a score, players receive a “mastery” percentage, which appears only after the song finishes. Within Learn a Song, players encounter in-game emulations of “real” guitar and bass amplifiers, including models by sponsored brands such as Marshall, Eden, and Orange.

ROCKSMITH AND THE SHAPING OF PLAYER EXPERIENCE

233

FIGURE 9.1a  Rocksmith 2014 startup menu.

FIGURE 9.1b  Marketing image for Rocksmith 2014; courtesy Ubisoft.

This emphasis on realism—or, at least, emphasis on minimizing virtuality—even appears in the marketing materials for Rocksmith 2014, which try to bridge the gap between the virtual and real worlds. On the Ubisoft website and on the packaging for Rocksmith 2014, images depict a player holding a guitar and looking toward a television and an

234

MUSIC VIDEO GAMES

Xbox 360 game console, with the game’s design bleeding over into the room surrounding it (Figure 9.1b). The virtual space shown in the game’s menu—the hardwood floors, windows, furnishings, and even the lighting—extends into the player’s “reality”: an identically decorated room.13 In this way, the Learn a Song mode attempts to minimize its presence as a virtual entity by suggesting the physical space that players should occupy when interacting with the game. In fitting with the pedagogical side of Rocksmith, the priority is not showing what the game can do, but instead emphasizing what you, the player, can do. Other marketing approaches similarly blur the boundaries between real and virtual by injecting the game’s virtual elements into the real world with physical products. Along with the game disc, Rocksmith 2014 includes stickers to attach to the player’s real-world instruments; these mark the guitars frets, but they also correspond aesthetically to the game’s typographical appearance. Similarly, several companies offer color-coordinated guitar strings “designed specifically to enhance your Rocksmith™ game play” by matching the string colors in Rocksmith’s notation system.14 Rocksmith-branded sweepstakes by musical instrument companies Epiphone and Ernie Ball further highlight the realism evoked in Rocksmith’s Learn a Song mode.15 In its multifaceted relationship with its players, Rocksmith exists not only as a game but also as a musician-oriented brand. In contrast to the Learn a Song mode, the Guitarcade mode selfconsciously reflects the nostalgic “unreal.”16 Taking on the trappings of an earlier era of video gaming, the typographical features of Learn a Song stiffen and pixelate in the Guitarcade. The lighting shifts from white light to bright colors, and an arcade cabinet appears onscreen. These features evoke an 8- or 16-bit aesthetic strongly associated with the 1980s and 1990s, which—especially in the 2010s—draws attention to its own artifice. Each of the Guitarcade’s technique-building minigames references a different genre of old-school arcade game and mimetically recreates visual and sonic aspects of the arcade video game experience. Before loading each game, the player encounters a “boot screen” akin to those of a 1980s arcade cabinet. One minigame, Scale Warriors (from Rocksmith 2014), requires players to control two fighters by correctly playing various patterns in a musical scale. Even in its opening (which includes a tongue-in-cheek copyright indication of 1991), the minigame appears to be modeled on 1980s and 1990s

ROCKSMITH AND THE SHAPING OF PLAYER EXPERIENCE

235

street-brawling games like Capcom’s Final Fight (Capcom 1989); both Scale Warriors and Final Fight begin with a startup screen showing the games’ titles skewed toward the upper right-hand corner of the frame, both include an orange and yellow explosive corona above the letter “i,” and both are in front of a brick wall background. In the Guitarcade song performance mode, called Score Attack, the player receives a numerical score as he or she plays. Constant pop-up messages appear, commenting on the player’s accuracy, phrase completion, streaks of correct notes, as well as exclamations like “Amazing!” and “Godlike!” The semi-realistic amplifier of Learn a Song becomes a consciously fake simulacrum as the circular speaker crudely pixelates and the gameplay chirps with 8-bit-style sonic feedback. Unlike the unadorned sonic landscape of Learn a Song, the game mode’s intermittent chimes constantly remind the player of Score Attack’s ludic imperative. The tension between Rocksmith’s role as a game and as a teaching tool structures not only the design and mechanics of the game, but also shapes the player’s experience outside of it. Rocksmith obscures the line between the real and virtual worlds, gently suggesting the physical spaces that a player should occupy, the equipment he or she should use (perhaps a Gibson guitar with Ernie Ball strings and a Marshall amp), and, more subtly, how the music in the game should be parsed and interpreted. At a basic level, the act of calling one section a chorus and another section a verse provides the listener with labels—epistemological categories—that redefine how they sound. Rocksmith’s difficulty system is more impactful still. It sculpts the ways in which players hear, perform, and think about the game’s music.

Dynamic difficulty and embedded rhythms While the divisions between difficulty levels are clearly apparent in Guitar Hero or Rock Band, the seams between levels become far less visible in Rocksmith.17 In each of the songs in Rocksmith and Rocksmith 2014, the game developers attempt a smooth curve of difficulty that is able to accommodate players of all skill levels. Rocksmith’s dynamic difficulty system constantly adjusts the musical texture based on the player’s accuracy, continually moving the player upward as he or she

236

MUSIC VIDEO GAMES

successfully hits the notes in a phrase. Instead of the four difficulty levels of Guitar Hero (easy, medium, hard, and expert), Rocksmith’s Learn a Song mode displays a “mastery” percentage that scales from 0 to 100 for each section of the song. According to this metric, an easier song in Rocksmith, such as The Ramones’ “Blitzkrieg Bop,” comprises thirteen levels of difficulty, while a more challenging track, like Iron Maiden’s “The Trooper,” occupies several hundred levels.18 This pedagogical system allows players to perform a range of simplified arrangements of each song before gradually approaching the final, recorded version.19 Because many of these levels have only one or two notes altered from the neighboring version, players of different abilities are able to gradually gain familiarity with even a complicated guitar solo at an individualized pace.20 On a basic level, the difficulty system often suggests a metric hierarchy for each song. Consider the simple 1-bar excerpt shown in Figure 9.2. Taken from the chorus of Blur’s “Song 2,” Figure 9.2

FIGURE 9.2  Blur, “Song 2,” excerpt from chorus.

ROCKSMITH AND THE SHAPING OF PLAYER EXPERIENCE

237

shows the same passage at seven different levels of difficulty, with the number to the left of the staff designating the player’s “mastery” percentage.21 For the purposes of clarity, I have only included one measure of the transcription, but the eighth-note pulse of A♭ octaves repeats throughout the entire chorus and occupies much of the song. Examining the seven levels shown here reveals an expected metric hierarchy, in which each leap in difficulty further subdivides the measure’s rhythmic pulse—first in whole notes (level 0), then half notes (level 16), then quarter notes (level 50), and then eighth notes (level 83). (For now, let us ignore levels 33 and 66, which will be discussed later.) In the final level, the player expands the A♭ from single notes into octaves. For the player, this metric hierarchy is reinforced through repeated playing. The meter’s strong and weak beats are statistically reinforced; a player going through each difficulty level would play the measure’s downbeat 280 times, beat three 240 times, beat two 200 times, and beat four 160 times, with each subsequent added note appearing less and less frequently. Dividing a part into more and more levels, however, yields inadvertent musical consequences. If a phrase’s harmonic progression is deemed the feature most important to preserve at all difficulty levels, the repetition of a melodic sequence might take on lesser significance. If game designers prioritize the kinesthetic considerations of how a player is physically navigating his or her hand along the fretboard, the passage’s metric relationships may become obscured. Rocksmith, with its focus on guitar and bass pedagogy, prioritizes economy of motion, especially at the less-challenging difficulty levels. Both Rocksmith and Rocksmith 2014 feature in-game video lessons, in which players are frequently reminded to not push themselves too hard and to stop playing if they experience pain. In one of the first lessons in Rocksmith 2014, the video instructor warns that “you can do real damage to yourself” with improper picking technique. These in-game reminders, however, cannot prevent all player injury, and discussions in online Rocksmith forums frequently suggest that players supplement Rocksmith’s instruction with in-person lessons with an experienced teacher, in order to develop and maintain proper technique.22 With this pedagogical orientation, Rocksmith’s difficulty system is largely oriented toward minimizing players’ physical motions as much as possible. This priority sometimes overpowers other potent

238

MUSIC VIDEO GAMES

FIGURE 9.3  Muse, “Plug in Baby,” excerpt from introduction.

ROCKSMITH AND THE SHAPING OF PLAYER EXPERIENCE

239

musical features. In a lead guitar riff from Muse’s “Plug In Baby,” for example (Figure 9.3), a cursory comparison of the hardest difficulty (level 100) with the easiest (level 0) reveals a strange disparity.23 The 6-bar phrase consists of three 2-bar units, reinforced by a melodic sequence and harmonic shifts; I have bracketed these groupings above the staff on level 100. Most notably, measures three and four repeat the preceding 2 bars raised up by a diatonic step. From the standpoint of the melodic phrasing in the passage, one expects that the easiest level would reflect this underlying phrase structure. A two-note reduction of the passage, therefore, would likely include the downbeat of bar 1 and the downbeat of bar 3. But instead, the phrase is broken up in two 3-bar chunks, with a strange mid-phrase note appearing on the downbeat of measure four (this segmentation appears bracketed below the staff on level 0). Although it seems odd from the standpoint of other musical parameters, this reduction fits with the aim of minimizing physical movement. Even within the same domain, however, focusing on easing the difficulty of one parameter can raise the difficulty of another. In the case of “Song 2,” for example (Figure 9.2), reinforcing the kinesthetic efficiency of the picking hand—gradually adding notes with as much time as possible in-between—comes at the expense of the physical differences between playing in single notes and playing in octaves. After gradually filling in the eighth notes in the texture in levels 0 through 83, the player must suddenly transition to playing the passage in octaves, which requires a different picking style and fretting technique. By prioritizing one type of technical ease, Rocksmith forces players to work through a different type of technical difficulty on their own. In all of these examples, Rocksmith’s attention on easing physical movement correlates centrally to how the game operates. Rocksmith’s game mechanics are built around evaluating the attacks of notes and, accordingly, economy of motion is interpreted largely in terms of the amount of time intervening between notes. In these and other cases, game developers’ choices about how to segment and arrange musical materials into a strict hierarchy produce inadvertent consequences. One such consequence is the emergence of what I call “embedded rhythms.” As previously mentioned, game developers usually add notes to the musical texture one or two at a time in order to generate a smooth curve of difficulty that is able to accommodate players of all

240

MUSIC VIDEO GAMES

levels. When notes are gradually added to a repetitive guitar or bass riff, however, the middle levels sometimes exhibit unique rhythmic features drastically different than the final version of the song. In the outro of the XX’s “Islands” (Figure 9.4a), for example, the developers provide each difficulty level with as much time as possible between the added notes.24 A first note materializes, and then a second, and third, steadily filling in the riff’s rhythmic palette while leaving the player as much time as possible between each attack. In this process of gradually adding each note, certain levels become rhythmically distinctive. Levels 77 and 81, in particular, take on a bourrée-like dance rhythm that would be completely absent if the player had simply heard the recorded song in its final version.25 First appearing in level 50, this quarter-eighth-eighth rhythm has been reinforced further by its repetition at earlier difficulty levels, so that by the time the player makes the leap to the passage’s final version (level 86 through 100), the rhythm has been thoroughly sewn into the fabric of the piece. By prioritizing gestural ease for novice players, the game developers embed an unexpected rhythmic framework into the passage. In many of the songs in Rocksmith and Rocksmith 2014, the difficulty system devalues the last note in each rhythmic grouping.26 In a set of four quarter notes, the fourth is usually the last to surface as the player progresses from easier difficulties and, likewise, the final eighth note in a measure often appears only at the harder levels. In Rocksmith’s songs (the majority of which are in quadruple meter), this results in a basic short-short-long rhythm resurfacing at various levels. In “Song 2” (Figure 9.2), this devaluation of the last note in a grouping arises at two different stages: the quarterquarter-half rhythm of level 33 and the eighth-eighth-quarter rhythm of level 66. Why is the final note in rhythmic groupings so often the last to appear in the difficulty system? In “Islands,” we see an alternative, with the second note suppressed in various rhythmic groupings. Instead of a recurrent short-short-long rhythm, therefore, a long-short-short rhythm appears at several levels; this relates to the musical context surrounding the passage, in which developers gradually introduce a challenging skip between non-adjacent strings.27 In general, however, Rocksmith’s songs emphasize the short-short-long rhythm for two interrelated reasons. First, the game prioritizes the beginnings of

ROCKSMITH AND THE SHAPING OF PLAYER EXPERIENCE

241

phrases rather than their endings. Second, harmonic changes— and correlating position changes—often occur on the downbeats of measures, and so preserving economy of motion requires Rocksmith to leave space at the ends of groupings (Figure 9.4a). In the outro to the Arctic Monkeys’ “R U Mine?,” for example (Figure 9.4b), devaluing the fourth note in groupings allows the player a free moment to shift his or her hand to the changed position on the fretboard.28 The earlier levels’ short-short-long rhythmic variants eventually prepare the sixteenth-note pulse that alternates between F# (2nd fret), A (5th fret), and B (7th fret).29 Rematerializing at each subdivision of the measure, “R U Mine?” contains four different shortshort-long accent patterns superimposed onto one another: half-halfwhole (levels 25 and 29), quarter-quarter-half (levels 41 and 45), eighth-

FIGURE 9.4a  The XX, “Islands,” excerpt from outro.

242

MUSIC VIDEO GAMES

FIGURE 9.4b  The Arctic Monkeys, “RU Mine,” excerpt from outro.

eighth-quarter (levels 58 and 62), and sixteenth-sixteenth-eighth (levels 75 and 79). Once again, by prioritizing physical movement, Rocksmith inadvertently embeds a recurrent short-short-long rhythmic pattern at various stages. Rhythmic figures like these appearing in lower levels of difficulty necessarily ingrain themselves in the listener’s ear. As I played through “Islands” and “R U Mine?” in Rocksmith, I found myself unable to “un-hear” the rhythms of the easier levels, and I would subconsciously accent a phrase with the notes I had played over and over at earlier stages.30 Similar experiences recurred throughout my time playing Rocksmith. In working out the main arpeggiated guitar riff in The Police’s “Every Breath You Take,” the consistent eighth-note pulse of the recording seemed mechanically altered by my incremental practice at lower levels. Some iterations evoke a 3+3+2 rhythm (dotted quarter-dotted quarter-quarter) almost akin to a tango, and this pattern found its way into not only how I performed the song within the game, but even how I heard the

ROCKSMITH AND THE SHAPING OF PLAYER EXPERIENCE

243

song in non-gaming contexts.31 An informal study of performances posted to YouTube reveals that my experience is not unique; players who have practiced with easier levels will sometimes drift between difficulties, at times evoking a rhythmic pattern that occurred several levels earlier. Especially, in repeating passages with even rhythms of consistent eighth- or sixteenth-note subdivisions, the rhythmic characters of earlier levels can fundamentally shape how a later level is performed.

Conclusion As I have suggested, the stark disparity between Rocksmith 2014’s two modes of gameplay—Learn a Song and the Guitarcade— correlates to the dual aims of Rocksmith. The game’s success has depended on its ability to succeed Guitar Hero (as a game) and yet simultaneously undermine it (as a pedagogical tool). Because of this tension, Rocksmith must fundamentally consider what it means to be a video game. At times, it accentuates this role, satirically enacting overfamiliar video game tropes.32 At other moments, Rocksmith does its best to make its presence as a video game recede (or even disappear entirely33) by aesthetically evoking the banal realities of a musician’s daily life. As it circles around issues of reality and virtuality, Rocksmith blurs this line further with its in-game difficulty settings, which determine not only how the game is structured but also how players hear the music they encounter. Distributing notes in Rocksmith’s difficulty system requires conscious and thoughtful consideration by the game’s developers. In an email to me commenting on an earlier draft of this chapter, game developer Brian McCune reflected the multilayered decisions that go into how each song progresses in difficulty, and the ramifications of these choices: Sometimes we have to make level design decisions not based on musical information, but on ergonomic decisions … and there are consequences of this approach … But in all cases, these decisions are well-considered and reviewed; it’s just a byproduct of the

244

MUSIC VIDEO GAMES

learning process and a testament to the retention of musical information presented in Rocksmith.34 McCune’s comments reveal the conscious intentions that undergird each song’s musical presentation within the game. When music game developers make design choices they are embedding implicit ways to think about music. Even small choices—what features the game mechanics choose to encourage versus what they choose to discourage, or what distinguishes one difficulty level from another— impact how music is conceptualized, interpreted, and experienced.

Notes 1

“VG Chartz” (2014).

2

The importance of authenticity within discourses of rock music has been widely discussed. See, for instance, Hugh Barker and Yuval Taylor (2007). Mario A. Dozal’s contribution to this volume thoroughly investigates issues of authenticity in the context of Guitar Hero and Rock Band. For further discussion of authenticity with respect to rock-oriented music video games, see Roger Moseley (2013).

3

Quotations are from the exterior of the game box for the Xbox 360 version of Rocksmith and Rocksmith 2014.

4

“BandFuse” (2014). Since accessing this page, the company that developed BandFuse has shut down, and the website no longer functions. Several archived versions of this site, however, remain available on Archive.org. “BandFuse,” accessed January 23, 2015, https://web.archive.org/web/20140208012708/http://home. bandfuse.com/.

5

Quotations are from the exterior of the game box for the Xbox 360 version of Rocksmith 2014.

6

The only exceptions to the game’s first-person perspective appear in the Guitarcade mode’s technique-building minigames, to be discussed later.

7

Just as Kiri Miller and others have shown for players Guitar Hero and Rock Band, a lively online community of Rocksmith and Rocksmith 2014 players post performances to YouTube. See Miller (2009).

ROCKSMITH AND THE SHAPING OF PLAYER EXPERIENCE

245

8

Citing Shultz’s observations, Nicole Biamonte (2010) has suggested games like Guitar Hero and Rock Band might be used as pedagogical tools for introducing students to ideas of musical reduction.

9

In the present chapter, I discuss both Rocksmith and Rocksmith 2014 under the general rubric of “Rocksmith.” Although the two games have some significant differences between them, the difficulty system seems to function similarly in both, and the two games are integrated to such an extent that, for my purposes, they can be discussed together.

10 This right-handed orientation can be switched in the game’s settings; for clarity, however, I am describing Rocksmith’s game mechanics using the default settings as a guide. 11 Rocksmith builds upon the color schema of Guitar Hero and Rock Band, both of which use the same colors to label the buttons on the controller (green, red, yellow, blue, orange). Rocksmith’s preservation of the red-yellow-blue-orange color sequence walks a fine line; it retains a Guitar Hero player’s ability to quickly distinguish the colors and order them from low to high, while at the same time recasting the domain in which this identification is operating (transforming it from the controller’s “fret” buttons to the guitar’s strings). 12 Further gestural cues are provided by changes to the note’s appearance: for example, a triangle (point-side down) in the rectangle signifies a “hammer-on,” in which the player uses their fretting hand to strike the string and sound a note. 13 A similarly designed space appears in Rocksmith’s video lessons, in which the environment that surrounds the in-game instructor subtly suggests the player’s physical setup outside of the game. 14 “Strings by Aurora” (2014). Another company’s description of their colored strings does not make specific reference to Rocksmith on their website, but Rocksmith players offer reviews on Amazon. com encouraging others to use them in conjunction with the game. “DR Strings” (2015). 15 Although these contests had ended prior to the writing this chapter, they are advertised via inserts in the Xbox 360 game case. 16 Nostalgia’s role in the contemporary culture of video games has been widely discussed, both in scholarship and gaming media. See, for instance, the contributions in Whalen and Taylor, eds. (2008). For a broader critique of nostalgia’s role in popular culture as a whole, see Reynolds (2011).

246

MUSIC VIDEO GAMES

17 This is not true of the “Score Attack” mode in Rocksmith 2014’s Guitarcade, which self-consciously presents an approximation of Guitar Hero’s appearance and gameplay, with only four levels of difficulty (easy, medium, expert, and master). 18 The Ramones, Ramones, Sire 7520, 1976, 33 1/3 rpm; Iron Maiden, Piece of Mind, EMI LC 0542, 1983, 33 1/3 rpm. Note that the examples referenced in this chapter were designed at different times, corresponding to the releases of Rocksmith and Rocksmith 2014 as well as the games’ significant additions available via DLC (downloadable content). As pointed out by Rocksmith developer Brian McCune via email correspondence, the difficulty system has been refined over time, so recently added songs may exhibit different treatment than earlier examples. Brian McCune, email message to author, April 28, 2015. 19 Especially with some of the most difficult tracks, players may not ever reach the final, recorded version. With this method, however, players at various levels of ability can master a song and play through it fully using simplified arrangements. 20 The idea of a system that continually ramps up challenges as a player improves his or her skill resonates with psychologist Mihaly Csikszentmihalyi’s theory of flow, which has been repeatedly applied to video games by scholars and game designers. In the traditional interpretation of Csikszentmihalyi’s theory, video game players want to exist in an optimal “flow channel” between the game being too hard (causing anxiety) and the game being too easy (causing boredom). See, for example, Shultz (2008, 178–80), and Juul (2009). See also the discussion of flow in the contribution by David Roesner, Anna Paisley, and Gianna Cassidy to this volume. 21 Blur, Blur, Virgin 42876, 1997, compact disc. 22 See, for instance, the blog review by Cano (2011). 23 Muse, The Origin of Symmetry, Mushroom MUSH93CD, 2001, compact disc. 24 The XX, xx, Young Turks YT031CD, 2009, compact disc. 25 A bourrée is a dance form in 4/4 time that often follows a basic repeating rhythmic framework of a quarter note followed by two eighth notes. 26 Email communication with the Rocksmith team at Ubisoft confirms the prevalence of this trend in the difficulty levels for various songs in Rocksmith and Rocksmith 2014. Brian McCune, email message to author, April 28, 2015. 27 In this chapter, I delimit the scope of my musical examples for clarity’s sake, but Rocksmith’s developers consider the dispersal of

ROCKSMITH AND THE SHAPING OF PLAYER EXPERIENCE

247

notes in the game’s difficulty system with respect to the entirety of the musical context. Where the player is coming from and where he or she is going impacts how notes are introduced. I would like to thank one of the members of the Rocksmith development team for Ubisoft for pointing this out to me in our e-mail correspondence. Brian McCune, email message to author, April 28, 2015. 28 Arctic Monkeys, AM, Domino WIGCD317S, 2013, compact disc. 29 There are, of course, other ways to play this passage that require no position switching, but this one-string approach on the low E string is the fingering that Rocksmith recommends. 30 The accents, in part, could relate to the physical choices (e.g., picking patterns) made at earlier levels that influence later levels. 31 The Police, Synchronicity, A&M UDCD 511, 1983, compact disc. The emergence of this 3+3+2 rhythm relates centrally to how Rocksmith’s difficulty system eases players into the riff’s arpeggiated pattern. The riff requires the guitarist’s fretting hand to stretch considerably; the easier levels, therefore, limit the number of notes included in the arpeggio, rather than introducing notes according to their metric position. In some versions, this involves playing only the root and fifth of the arpeggio, which inadvertently emphasizes a 3+3+2 pattern. 32 One particularly striking example is the Guitarcade minigame “Return to Castle Chordead,” in which a player strums chords to fend off attacking zombies. With its title, this minigame from Rocksmith 2014 alludes to Activision’s Return to Castle Wolfenstein (Activision 2001), while simultaneously lampooning the cheesy dialogue and overwrought voice acting that sometimes characterize video games (particularly those from the era that first integrated pre-recorded rather than synthesized audio). In a distinctive outburst from the game’s opening, the minigame’s villain, a flying, caped rocker in a black skull T-shirt—who is not dissimilar looking to the playable characters from Guitar Hero—wildly proclaims: “My rock zombies will bring you doom!” This over-the-top speech reminds the player that although Rocksmith presents a minigame filled with trite video game clichés, it simultaneously parodies the genre itself. 33 In the game’s “master mode,” for instance, Rocksmith’s notation fades and then disappears for players who have memorized the song’s guitar or bass part. 34 Brian McCune, email message to author, April 28, 2015.

248

MUSIC VIDEO GAMES

References BandFuse. Accessed March 21, 2014. http://home.bandfuse.com Barker, Hugh, and Yuval Taylor, eds. 2007. Faking It: The Quest for Authenticity in Popular Music. New York: W.W. Norton. Cano, Miguel. “Video Game Review—Rocksmith: Finally a Gaming Experience for Guitarists.” The Contrapuntist (blog), November 28, 2011. Accessed February 25, 2015. http://www.thecontrapuntist. com/2011/11/28/video-game-review-%E2%80%93-rocksmith-finallya-gaming-experience-for-guitarists. “DR Strings.” Accessed February 25, 2015. http://www.drstrings. com/#!neon-multi-color-list/c8us. Juul, Jesper. 2009. “Fear of Failing? The Many Meanings of Difficulty in Video Games.” In The Video Game Theory Reader 2, edited by Mark J. P. Wolf, and Bernard Perron, 237–52. New York: Routledge. Kreps, Daniel. “Despite The Beatles: Rock Band, Music Video Game Sales Slip in ’09.” Rolling Stone (online edition) December 2009. http://www.rollingstone.com/music/news/despite-the-beatles-rockband-music-video-game-sales-slip-in-09-20091222. Miller, Kiri. 2009. “Schizophonic Performance: Guitar Hero, Rock Band, and Virtual Virtuosity.” Journal of the Society for American Music 3 (4): 395–429. Moseley, Roger. 2013. “Playing Games with Music (and Vice Versa): Ludomusicological Perspectives on Guitar Hero and Rock Band.” In Taking It to the Bridge: Music as Performance, edited by Nicholas Cook, and Richard Pettengill, 279–318. Ann Arbor: University of Michigan Press. Nicole Biamonte, “Musical Representation in the Video Games Guitar Hero and Rock Band,” in Pop-Culture Pedagogy in the Music Classroom, ed. Nicole Biamonte (Lanham, MD: Scarecrow Press, 2010), 133–7. Reynolds, Simon. 2011. Retromania: Pop Culture’s Addiction to Its Own Past. London: Faber & Faber. Shultz, Peter. 2008. “Music Theory in Music Games.” In From Pac-Man to Pop Music: Interactive Audio in Games and New Media, edited by Karen Collins, 177–88. Farnham, Surrey: Ashgate. “Strings by Aurora.” Accessed March 24, 2014. http://www. stringsbyaurora.com/our-strings/rockpack-guitar-strings. VG Chartz. “VG Chartz, Game Database.” Accessed March 15, 2014. http://www.vgchartz.com Whalen, Zachary, and Laurie N. Taylor, eds 2008. Playing the Past: History and Nostalgia in Video Games. Nashville: Vanderbilt University Press.

ROCKSMITH AND THE SHAPING OF PLAYER EXPERIENCE

249

Games Capcom. Final Fight [Arcade, et al.] Capcom: Osaka, 1989. Harmonix. Guitar Hero. [PlayStation 2]. RedOctane: Mountain View, CA, 2005. Harmonix, et al. Rock Band. [Xbox 360, et al.]. MTV Games/Electronic Arts: New York, 2007. Harmonix/Pi Studios. The Beatles Rock Band [PlayStation 3, et al.]. MTV Games: New York, 2009. Neversoft, et al. Guitar Hero III: Legends of Rock. [PlayStation 2, et al.]. Activision, et al.: Santa Monica, 2007. Realta Entertainment Group. BandFuse: Rock Legends. [Xbox 360/ PlayStation 3]. Mastiff: Tokyo and San Francisco, 2013. Ubisoft San Francisco. Rocksmith. [Xbox 360, et al.]. Ubisoft: Montreuil, 2012. Ubisoft San Francisco. Rocksmith 2014. [Microsoft Windows, et al.]. Ubisoft: Montreuil, 2013.

10 Rhythm Sense: Modality and Enactive Perception in Rhythm Heaven Peter Shultz

David Sudnow’s rhythm sense

I

n 1979, David Sudnow won a Guggenheim Fellowship for his book Ways of the Hand, a body-first, anti-cognitivist account of learning to play jazz piano (Sudnow 1993). In Sudnow’s account, it is not the mind that learns—nor even the “I,” which stands for himself as a subject, a person—but the hand. His next project would be a similar account of learning to play Breakout (Atari 1976) on the family’s Atari. Over the course of the resulting book, Pilgrim in the Microworld, Sudnow describes his attempts to develop the bodily habits of a master Breakout player.1 In a chapter entitled “Practice,” he tries to construct a repeatable “opening” sequence of moves. This requires extremely precise timing, so he decides to put his jazz-piano training to work by learning to hear the game’s sound effects as a “melody” and employ his musical sense of rhythm. Unfortunately this attempt fails, for which Sudnow offers two reasons. First, the ball moves unrealistically. It moves at a “constant speed” in a “nongravitational field,” which

252

MUSIC VIDEO GAMES

frustrates any attempt to “grasp it as a human gesture.” Second, the actions themselves must be performed at irregular time intervals, which undermines his “attempt to establish an unfolding rhythm” (Sudnow 1983, 136): I reach with an established pulse toward the next node of the melody. The ball comes down and I’m moving to bring that paddle to the right location in time with that pulse. But the ball won’t be at the paddle when any next body beat would occur. So as it comes down toward its next point of contact, I’m thoroughly dependent upon the eye’s guidance to remind me of the steadiness of the ball’s movement and to neutralize any surge toward a next pulse that would in fact produce a mistake. From each shot to the next I must latch on to the ball anew to give my hand a next time of arrival completely independent of what happened before, so the attempt to establish an unfolding rhythm to link up one shot to the next is forever undermined and altogether pointless. (Sudnow 1983, 95) For Sudnow, this is in part a failure of action: He finds it difficult to make the right preparatory movements to set up his shot. But it is also a failure of perception: When his sense of rhythm (“body beat”) is blocked, he is forced to rely on his sense of sight (“the eye’s guidance”). He wants to “link up one shot to the next” by means of the “established pulse,” but its disruption forces him to “latch on to the ball anew.” Whereas the “pulse” and “body beats” would have let him manage time as intervals, the “eye’s guidance” shows only moments in isolation. This passage evokes a musical sense of time, experienced not through the eye or ear, but kinesthetically through the body and hand. This time-sense functions analogously to vision, as part of a sensory field: it offers Sudnow a way of organizing his movement in the world. Moreover, it offers sensory information that is commensurate with visual information: when the pulse fails, the eye takes over. And even when this sense fails, as described in this passage, it does so in a manner analogous to sight: just as he would not be able to see without light, he cannot feel “body beats” without an “established pulse.”

RHYTHM SENSE

253

These perceptions of beats and pulses are part of Sudnow’s phenomenological world, at least when that world is organized as music. They offer a way of working with rhythmical time that is more precise and integrated than “the eye’s guidance” alone, but in Breakout, where actions unfold irregularly and arrhythmically, they are useless.

Rhythm Heaven’s rhythm sense Each installment of the video game series Rhythm Heaven (developed and published by Nintendo, starting in 2006) consists of several dozen levels that require the player to perform simple, repetitive tasks, often involving a single button, in precise synchrony with a preset musical track. While some of these levels are overtly musical performance or dance, most of them are presented as non-musical activities: sports (such as badminton or golf), repetitive labor (such as working an assembly line or harvesting vegetables), or military drills (such as marching). Like Breakout, Rhythm Heaven requires the player to perform these activities with precise timing, but it choreographs them to a musical soundtrack, so that each action is synchronized to a particular musical instrument or gesture (this will be explained in more detail later in the chapter). This musical organization allows, and indeed encourages, players to play in precisely the way that Breakout refuses, as coherent gestures planned and felt as bodily pulses. The Rhythm Heaven games even introduce a term for this, exhorting players to depend on their “rhythm sense.” While it is common to speak of a “sense of rhythm” in the same way as a “sense of humor” or a “sense of propriety”—as a general awareness of rhythm, or a capacity to appreciate it—the Rhythm Heaven games use the term “rhythm sense” to indicate something much more specific and farreaching. In their textual instructions and in the course of play, they encourage the player to experience bodily timekeeping—that is, physical entrainment—as a perceptual sense, of essentially the same type as sight, sound, smell, touch, and taste. Their text adumbrates “rhythm sense” with surprising thoroughness, and the levels require the player to rely on entrainment to coordinate timing, by organizing gestures into repeating grooves and by obscuring audiovisual cues.

254

MUSIC VIDEO GAMES

In doing so, they implicitly raise significant questions about the role of training and skill in sensory perception, which this chapter will take up briefly at its close.

Blocking sight, blocking sound The official origin story of the Rhythm Heaven series, as told by the developers to Nintendo president Satoru Iwata, begins when the celebrated music producer Tsunku♂2 notices that music games tend to prioritize visual cues over a truly musical experience. To remedy this situation, he decides to make a game in which “nothing like a music score appears on the screen,” in order “to improve Japanese people’s sense of rhythm.”3 The Rhythm Heaven games offer a world in which actions may be timed according to “body beats” and “pulses,” the musical time-sense that Breakout denies. Whereas the irregular patterns of Breakout forced Sudnow to rely on “the eye’s guidance,” Rhythm Heaven aims to wean players from it: Many of the micro-games obscure the visual cues or even block them entirely. But, perhaps surprisingly, it does not simply replace visual cues with audio cues. Some of the micro-games, notably the introductory “rhythm test” levels from Rhythm Tengoku (Nintendo SPD 2006) and Minna no Rhythm Tengoku (Nintendo SPD 2012), remove both aural and visual cues, forcing the player to keep time solely by bodily means: counting out loud, nodding their head, or tapping their fingers or feet— “body beats,” in Sudnow’s terms. These levels frame the game’s concept of “rhythm sense” as a sensory channel in its own right, separable from, and not reducible to, vision or hearing.4 The “rhythm test” at the beginning of Rhythm Tengoku is presented as a medical screening, similar to a vision or hearing exam. It consists of three exercises, each designed to test a different aspect of rhythm sense. The first asks the player to tap a button along with a moderate pulse (120 bpm) indicated by a beep and a flashing light. Each tap is marked relative to its corresponding target pulse, showing whether the player was early or late. The results are displayed on a sheet of paper somewhat resembling an electrocardiogram, but they take several seconds to be ejected from

RHYTHM SENSE

255

the machine and become visible to the player. The second exercise shows a numeric display counting down from seven to zero in time with the pulse: The player is meant to tap on a button just as the count reaches zero. As the exercise goes on, it removes visual and auditory cues starting from the end of the sequence: 7–6–5–4–3– 2–1–0 becomes 7–6–5–4–3–2 … 0, then 7–6–5–4–3 … 0, and so on until the game gives only the first pulse, “7,” and the player must count down to “0” on their own steam, testing the player’s ability to maintain an internal pulse without the aid of visual or aural cues. The third exercise tasks the player with maintaining the same steady pulse as in the first exercise, but this time to do it while hearing a drum track that moves in and out of metric alignment: At first the drums’ downbeats align with the player’s pulse, but after a few bars they shift to offbeats and syncopations that could throw the player off balance. As in the first two tests, the results of the exam are hidden from the player for several seconds, so he or she can succeed only by learning to recognize mistakes and recover from them by reorienting to the pulse of the drums. These rhythm tests not only serve as an assessment of the player’s ability to press buttons in time with music but also function as a tutorial to the game’s repertoire of actions: pressing buttons in time with beats. The next installment in the series, Rhythm Tengoku Gold (Nintendo 2008, released in English as Rhythm Heaven in 2009) was developed for the Nintendo DS’s touchscreen and stylus. Accordingly, it replaces the rhythm test with a tutorial for the characteristic “flicking” action required by many of its levels. But with the third installment, Minna no Rhythm Tengoku (2012, released in English as Rhythm Heaven Fever) for the Wii, the series returned to its original button-based gameplay, along with the rhythm test from the original Rhythm Tengoku. The test in this version consists of just the first and second trials described earlier, with the results displayed instantly on a slightly aged-looking, green-and-black display reminiscent of an oscilloscope. This time, the instructions are delivered by a trio of friendly cartoon characters, who nevertheless disturb the second test by holding up signs blocking the display readout until the test has finished. The effect is similar to the EKG-style results in Rhythm Tengoku: Deprived of visual cues, the player is forced to rely on other sensory modes.

256

MUSIC VIDEO GAMES

These tests require a set of distinct but interrelated musical skills involving entrainment, the ability to perform an action in synchrony with a steady pulse—essentially a technical term for Sudnow’s “body beats.” The player must be able not only to entrain their tapping to an audiovisual pulse but also to continue the pulse in the absence of audiovisual stimulus, whether by tapping, other bodily movement, or silent counting. These skills underlie most known kinds of musicmaking and indeed are often thought to be universal in human musical culture, and more or less peculiar to humans.5 In particular, the rhythm tests in the Rhythm Heaven games suggest a distinction between, on the one hand, the ability to match pulses that are perceived through hearing, vision, or touch, and on the other, the ability to generate steady pulses in the absence of perceptual input. This aligns well with a recent “ecological model of entrainment,” which identifies three requisite abilities, or “building blocks” (PhillipsSilver et al. 2010, 6): (1) the ability to detect rhythmic signals in the environment; (2) the ability to produce rhythmic signals (including rhythmic signals that are byproducts of other functions, such as locomotion or feeding behavior); and (3) the ability to integrate sensory information and motor production that enables adjustment of motor output based on rhythmic input. By temporarily removing the visual and aural indicators of a pulse, the “rhythm test” levels force the player to rely on their own rhythmically entrained pulse production, which may take a variety of forms. The games’ creator, Tsunku♂, urges players to “[p]lay […] this game standing up … keep the rhythm with your body, with your chest … and your knees and feet,” while the developers admit to remaining seated but tapping their feet.6 These “rhythm test” levels portray this kind of bodily timekeeping as a sense that can fill in when other senses are obstructed or disabled—a motif that reappears in some of the later games as well, often to comic effect. Several games in Minna no Rhythm Tengoku depend on the obstruction of vision. In “Badminton” (known in English as “Air Rally”), the player and a cartoon dog each pilot a small propeller plane while volleying a shuttlecock back and forth. Toward the end of the level, they fly into

RHYTHM SENSE

257

a cloud bank that renders the planes and the shuttlecock effectively invisible. At this point the volleys are cued only by sound: the musical beat, the whacking sound of the rackets, and the opponent’s voice occasionally warning the player of a lob. In “Akuryou Taisan” (Samurai Slice) the player controls a samurai defending a town from bat-like demons who appear from a portal, announcing their presence with rhythmic cackles. The music is cinematic and grandiose, ostensibly to glorify the samurai’s heroism, but the game turns this celebration into an impediment, when partway through the level, a series of commemorative paintings completely blocks the player’s view. At these moments, the player can fulfill the role of “heroic samurai” only by ignoring the visual distractions and focusing on the music and sound effects. Other games remove sound cues as well as vision: “Batting Show” (Exhibition Match) requires the player to hit a series of baseballs, each exactly four beats after it is tossed into the air. The balls are off-screen while airborne, so there are no visual cues to prepare the player, and most of the time the musical accompaniment during these four beats is simply a sustained note or chord, which offers no rhythmic cues either. This situation approximates the second of Rhythm Tengoku’s rhythm tests by having the player fill in the missing beats themselves. A few times, the accompaniment adds a syncopated drum fill that could throw off an inattentive player, approximating the conditions of the third rhythm test. Similarly, “Shiroi Obake” (“Sneaky Spirits,” in Rhythm Tengoku) tasks the player with stopping ghosts from escaping a haunted house by firing arrows from a bow on a precise beat. The music changes tempo every few bars, and many of the phrases fade out to silence several beats before the required shot, requiring the player to fill in the gap by counting. Some levels preserve the audiovisual cues but render them distracting and unhelpful. “Ura Omote” (“Lock Step,” in Rhythm Tengoku Gold) and “Tori noTaigun” (“Flock Step,” in Minna no Rhythm Tengoku) the player controls a person and a bird respectively, who attempt to march in synchrony with a crowd, which gets larger and smaller over the course of the song. At their largest, these crowds hide the player-controlled character amid a moiré-like profusion of identical copies, so the player-controlled character is visually distinct only when player error causes it to fall out of step. In these

258

MUSIC VIDEO GAMES

levels, the visual cues are never completely obscured, but they are buried amid visual clutter and thus not especially helpful. The audio cues are obscured less often than the visual ones, although several levels contain musical or sonic elements that threaten to upstage them, which players must learn to “hear past.” In “Bossa Receive” (“Bossa Nova,” Minna No Rhythm Tengoku), for example, volleyballs fly in from both sides of the screen. The player controls a cartoon baby who must return the balls from one side by tapping a button in rhythm with their arrivals, while a computer-controlled partner returns the balls from the other side. (Together, the two patterns form a bossa nova beat, which gives the game its title.) Every bar or two, the players switch sides, signaled by a rapid pattern of four soft handclaps. The visual cues are occasionally obscured as in “Samurai Slice,” and the audio cues are often in danger of being upstaged by the voices in the music: Not sung vocals, but grunts and moans that grow louder and by the end are unmistakably sexual, putting the cartoon baby characters in a new context.7 On one level, these voices serve a similar function to the syncopated drum beat in the “Rhythm Test” levels, as an auditory distraction that the player must learn to ignore. But their transgressive absurdity adds an additional challenge: In order for the hand claps to remain audible, the player (and any spectators) must keep quiet and refrain from laughing or commenting. Thus, while the signal for switching sides has both auditory and visual components, neither of them is sufficiently reliable that players can afford to ignore the other: They must attend to both the visual and the auditory cues, while regulating their own bodily pulse in time with the music. When the games deny audiovisual cues, both in the rhythm tests and in the ordinary levels, they implicitly present the player’s entrained actions as a substitute source of perceptual information. Just as players can get timing cues from sight and sound, they can also produce and respond to timing cues by counting, tapping, and otherwise “feeling” the beat. When the game and its creators speak of rhythm sense, they refer to something more concrete than the general ability to recognize and appreciate rhythm. Rhythm sense in these games is a kind of perception.

RHYTHM SENSE

259

But rhythm sense in these games is not purely informational; it has an affective dimension as well. Even when the audiovisual cues remain available, the levels tend to emphasize the “feeling” of rhythm through their musical organization.

Answers, imitations, and grooves To date, the series includes a total of approximately 80–120 levels, depending on how “sequels” (reappearances of a level with a higher difficulty level and lightly reworked music) and other activities (rhythm tests, bonus levels, and free-play modes) are counted. Although they represent a wildly diverse range of musical styles, characters, and scenarios, their requisite activities tend to fall loosely into three genres:

Answer a cue The player responds in rhythm (i.e., after a given duration) to a cue given in the song and/or visual scenario (usually both).

Imitate a pattern The player alternates with another character, imitating the rhythm pattern it provides.

Groove, interrupted The player maintains a steady rhythmic pattern, occasionally altered or interrupted. Many of the levels could be assigned to more than one of these categories. The “Answer” levels sometimes present cues in rapid succession, so the player must respond by imitating a short rhythmic pattern. Conversely, every “Imitate” level can be recast as an “answer” exercise where the answers are delayed by the length of the pattern itself.8 Many “Answer” levels are structured around repeating sequences of cues that can shade into the “Groove”

260

MUSIC VIDEO GAMES

category as well. But although they overlap, these categories make different demands on the player’s rhythmic attention and imagination. These differences become crucial in the “remix” levels, which are perhaps the games’ most inventive feature. Each game presents its levels in sets of five: four ordinary levels followed by a remix level that incorporates all of their musical figures and challenges. At the end of each game, all the levels are combined into a final “remix” that flits from one to the next, rarely dwelling on any for more than a few beats. This demands not only intense concentration from the player, and a command of the constituent levels, but also the mental agility to change from one mode of attention to another on short notice. For all their differences, though, the levels broadly share a strategy of building repetitive grooves that allow the player to feel the rhythm as a process, instead of simply reacting to each event as it happens. “Hole in One,” the first level in Minna no Rhythm Tengoku. The player controls a golfer, who has to time his swings to hit balls tossed by a cheerful monkey standing nearby. At first, the game presents this as an exercise in hand-eye coordination: The player must watch the ball as it flies from the monkey to the golfer, and swing the club when it reaches the right part of its arc. This sort of task is, of course, standard fare in video games, but the timing requirements are unusually stringent, especially for an introductory level. After the player swings at a few balls, the game adds a simple drum track. Now the player can measure the optimal moment to swing by paying attention to the musical rhythm: The monkey grunts while tossing the ball, and the optimal moment to swing is precisely one beat later. Then the game adds a mandrill who hurls the ball at the player so quickly that the ball’s flight is effectively instantaneous: In order to hit it, the player must swing at the very moment the ball is released. Fortunately the mandrill has a wind-up sequence lasting three beats (accompanied by a distinctive pattern of grunts), so the player can watch and listen for their cues and react appropriately. “Hole in One,” like many of the game’s levels, is prefaced by a short training exercise designed to familiarize the player with its repertoire of actions and musical cues. In the training exercise, the

RHYTHM SENSE

261

throws are several seconds apart, so they seem more like individual actions than like any kind of sustained pulse or groove. This makes it a clear example of the “answer a cue” type of game, consisting of a series of discrete cues and responses: When the monkey grunts, the player should swing after one beat; when the mandrill starts his sequence, the player should swing in three beats. But once the training exercise finishes and the song begins, the balls come more frequently and predictably, and they start to feel more like a groove. For most of the song, the monkey provides a steady stream of balls on the downbeats, which the player can simply pick off on the upbeats. After a few repetitions this pattern becomes comfortable, and the player no longer needs to react to each event individually but can begin to feel the pattern (i.e., entrain to it) as a continuous groove. This organizational strategy of building up grooves from simple gestures appears in other “answer a cue” levels. While the groove in “Hole in One” is a simple offbeat pattern, some of them are more complicated. For example, in “Packing Pests” (originally titled shiwake, or “sorting,” in the Japanese release), the player controls a worker in a candy-packing plant who must grab pieces of candy— tossed from offscreen, presumably by a coworker, and signaled by a drum beat—and throw them into boxes, while swatting away an alarming number of spiders, which announce their presence with a squeak. The cue for a piece of candy (which sounds like a drum hit) prompts the player to “grab” (pinch the A and B buttons together) after one beat. The cue for a spider (a squeak) prompts the player to “swat” (press the A button) after one beat. When two spiders are about to appear in quick succession, a voice offers advance notice (Figure 10.1). As with “Hole in One,” the training episode presents these as discrete actions that may appear in any combination. For all the player can tell, the song may well consist of unpredictable blasts of spiders and candy. And yet, like “Hole in One,” it proceeds largely according to repetitive, predictable grooves. The song’s basic pattern consists of 3 bars of alternating grabs and swats on offbeats, followed by 1 bar with a double spider warning.9 After the player has mastered the song, this level returns in a more difficult version, with the swats

262

MUSIC VIDEO GAMES

FIGURE 10.1  Cues to grab a piece of candy or swat a spider, and the twospider warning.

shifted up by a half beat and the grabs pushed back by half a beat at the end (Figure 10.2). This is typical of the “answer”-type levels in the Rhythm Heaven series. Other levels (those in the “groove” category) take this principle as a starting point, asking the player to maintain a consistent activity until instructed otherwise. Many of them cast the player as a runner, a marcher, or a member of some sort of performing troupe, in order to justify the need for relatively uninterrupted action. For example,

RHYTHM SENSE

263

FIGURE 10.2  The basic pattern in “Shiwake”/“Packing Pests,” and the basic pattern in its sequel, “Shiwake 2.”

“Night Walk” (which appears in Rhythm Tengoku and as the credit sequence for Minna no Rhythm Tengoku) has the player control a tiny person cheerfully strolling across an abstract structure, set against a field of stars. Each step corresponds to a beat in the music, and the character hops in time.10 Similarly, “Dosun Nouen” (“Crop Stomp,” Rhythm Tengoku Gold) has the player march through a field and at each step either harvest a turnip or pull up a mole and drop-kick it to the horizon. These levels are essentially dressed-up versions of the first rhythm test, in which the player simply taps along with a pulse—though they also have elements of the third: The musical accompaniment in “Night Walk” is rife with cross-rhythms that could seduce a careless player into rushing ahead of the beat. The imitation levels rely less on direct, continuous entrainment, as the computer and player trade off bar by bar. Each bar has a different pattern, and the player has only one chance to perform it in the course of a playthrough, so there is less opportunity to feel out repetitive grooves. Each unit of the song (1- or 2-bars long, depending on the level) is played twice: once for the player to listen and learn, and the second time for them to play along. While the game sometimes adjusts the melodic or harmonic materials on the second time (most consistently at the end of the song, where the harmonic and melodic resolution tends to be deferred to the player’s turn, so the song doesn’t seem to end twice), the rhythms are generally homogeneous: All the instrumental parts

264

MUSIC VIDEO GAMES

tend to perform the same figure and rarely get in the player’s way. But even though groove plays a diminished role, the player can still navigate the songs in these levels by means of entrainment and felt pulses: The bars often build on each other by small additions or changes that amount to a kind of progressive elaboration, so an attentive player can understand a bar as, for example, “like the previous one, but with two extra taps at the end.” Beyond these three main categories (and the remix levels mentioned earlier), there are some odd levels that seem to defy categorization, especially in the first game, Rhythm Tengoku. “Quiz,” uniquely among the entire Rhythm Heaven series, does not even require the player to perform in any particular rhythm, asking the player to answer a drum pattern with another pattern containing the same number of hits.11 But overwhelmingly—and increasingly as the series has progressed—the levels closely resemble the three overlapping types described in the preceding text, all of which are founded on repetition, and most of which ask the player to work in and around predictable grooves. This consistency owes partly to the repetition inherent in the popmusic styles on which the game is built, but it also reflects on the game’s concept of “rhythm sense.” It is not purely about being able to measure time accurately, but also about feeling repetition in the body and learning to respond to the kinesthetic timing cues of entrained motion. While there is a strong element of surprise in the game, and an emphasis on quick reactions, the game is always concerned with affording repetitive grooves to which the player can entrain.

“Rhythm,” “sense,” “feeling,” and “groove” So far this chapter has discussed two tendencies in the games that show how they construct rhythm as a sense: they block audiovisual cues to encourage players to orient themselves by means of entrained pulses, and they organize their levels into grooves that players can learn to “feel.” Based on these tendencies, it seems the games’ idea of “rhythm sense” encompasses two roles: it helps players interpret and produce precisely timed gestures, and it has something to do with the feeling of moving rhythmically.

RHYTHM SENSE

265

The games also offer textual explanations of these concepts, which amount to a short treatise on the relationships between rhythm, sense, feeling, and groove. The Japanese games refer frequently to “rizumu-kan,” rendered in English as “rhythm sense.” Most obviously, the games treat rhythm sense as a prerequisite to success: Players who complete a level successfully are congratulated on their rhythm sense, and those who fall short are encouraged to work on it. But other parts of the game offer more insight. In an unlockable bonus text, Rhythm Tengoku distinguishes rhythm sense from rhythm: “Rhythm ≠ Rhythm sense. Rhythm is the ability to count time intervals; rhythm sense is what you express, feel, and spontaneously engender through groove.”12 This distinction seems to capture two aspects of “sense”: Whereas rhythm is a mere skill, rhythm sense is a kind of feeling, including expressive and affective aspects. The game elaborates “rhythm sense” further in a set of five haiku, each followed by an explanatory gloss: Let’s forge that which everyone has: rhythm sense. “It is possible to develop latent rhythm sense with training/ practice. Daily repetition is even more effective.” Daily movement: is this rhythmical? “It’s good to feel and raise your rhythm sense during everyday life. While walking, brushing your teeth, while cooking, etc … you should move with consciousness of rhythm.” Rhythm sense: if your groove is good, it’s even cooler. “However, rhythm sense and groove are separate things. While honing your rhythm consciousness, let’s simultaneously obtain good groove.”

266

MUSIC VIDEO GAMES

Break: that girl who can’t wait during it is a blockhead. “Counting a break accurately is difficult, and is easy to feel tempted to jump in early. The ability to stay composed and wait or not largely affects the coolness.” Even for adults that which grows visibly: rhythm sense. “From simply trying to be conscious of rhythm, rhythm sense will grow exceptionally even in adults.” While these haiku stop short of offering a full definition of “rhythm sense,” they add significantly to the ideas developed in the game. Unlike sight or hearing, rhythm sense is latent (senzaiteki) and needs to be developed through daily practice. It has to do with counting during rests, but also with coolness and groove. And although it is primarily associated with music, it can also be developed through non-musical activities by moving with consciousness of rhythm (rizumu wo ishiki shite ugoku). The Japanese morpheme “kan” (as in “Rizumu-kan,” “rhythm sense”) covers a similar range of meanings to the English “sense,” encompassing perceptions and feelings: a kanji is a feeling, gokan are the five senses, anshinkan is a sense of safety, and so on. It appears elsewhere in the game as well, notably at the end of the training segment of each level. When the player masters the basic movements of each micro-game, the music gives way to a burst of canned applause and some text acknowledging their success. The messages usually say something like “Doesn’t it feel good?” (Ii kanji desu ne) or “That’s the feeling!” (Sono kanji da).13 This emphasis on “feeling” suggests that the player should cultivate a kind of rhythm sense that is not just effective at completing the levels, but also enjoyable. Then there is “groove” (nori). According to the third haiku, groove is not necessary for rhythm, but it is part of a fully developed rhythm sense. The unlockable bonus text in Rhythm Tengoku refers to

RHYTHM SENSE

267

groove as the “spontaneous,” “expressive” aspect of rhythm sense, as distinguished from mere counting. At one point the developers identified their goal for the games as “groove sense” (norikan), an evocative neologism that does not appear in the games themselves (Iwata 2008).14 Like the English word “groovy,” the Japanese word nori can be applied to personalities as well as music: A person with “good nori” is easy to get along with.15 These uses suggest that groove should be thought of as the affective dimension of rhythm sense.16 “Rhythm sense” has several features that might seem to cut against the perceptual status that this chapter has advocated. Perceptual senses are not normally thought to require any special skill or exercise—let alone the practiced consciousness prescribed by the third haiku. They are thought to be innate and basically universal (except in marginal cases of disability), their acuity governed by innate biological limits like nearsightedness or special sensitivity to aromatic compounds. But according to the game, rhythm sense is latent until cultivated by training, and it improves with practice. So on this reading, the games’ text might seem to cast rhythm sense as something more like a “sense of humor” after all. If it is a perceptual sense, it is a very strange one indeed. Nevertheless, the game shows that rhythm sense can provide information about the world that is commensurate with that provided by the standard perceptual apparatus: It treats rhythm sense as a direct substitute for sight and sound. And the importance of learning and cultivation is not necessarily as foreign to perception as it might seem: They are fundamental to a growing, though still controversial, group of perception theories that treat learning, skill, and body movement as essential to sense perception. A full account of these “enactive” theories of perception far exceeds the scope of this chapter, but the last section can offer a brief sketch, and show some implications of taking rhythm seriously as a sense.

Sense, skill, and action To take rhythm sense seriously as a kind of perception is to abandon the traditional Cartesian, modular, “information-processing” model of perception, in which the sense organs receive information, and

268

MUSIC VIDEO GAMES

the mind processes it, then directs the body to respond accordingly. Some prominent alternatives to this view come from “enactive” and “extended” theories of embodied cognition, which generally emphasize the role of the environment in shaping the cognitive and perceptual capabilities of organisms.17 In these accounts, human senses depend on more learning and skill than is usually recognized. Even something as seemingly automatic as the sense of smell can be improved with training (Porter et al. 2007). Other studies show that they can be profoundly disabled by cognitive impairment, even when the sensory organs themselves are intact. Alva Noë uses this phenomenon, which he calls “cognitive blindness,” to support an account of enactive perception based on skilled action and learned “sensorimotor profiles” of experience (Noë 2004). In his account, sense perception depends not only on the sensory organs and cognitive processing, but also on body movement, integrated at a fundamental level. In short, he says, our model for sense perception should be less like sight and more like touch. The ecological theory of entrainment cited earlier draws on many of the same basic concerns as enactive theories like Noë’s, but it stops short of treating entrainment as an act of perception, as the idea of “rhythm sense” would require (Phillips-Silver et al. 2010). Whereas the ecological theory identifies three requisite abilities for entrainment—the ability to detect a pulse, to produce a pulse, and to “integrat[e] sensory information and motor production”—the enactive version would deny the distinction between the first two and eliminate the third. In an enactive account of perception, there can be no sensory information without motor production, and they are always already “integrated.” The Rhythm Heaven games thus propose a surprisingly radical idea. Their idea of “rhythm sense” requires a thorough overhaul of the standard picture of sense perception, probably along enactivist lines. In this account, it is not a problem that rhythm sense has no particular sensory organ corresponding to the eye, ear, tongue, nose, or skin, since even sight, hearing, taste, smell, and touch depend on sensorimotor integration that transcends any individual organ. It might be thought of as a temporal analog to the physiological sense of balance, which recruits many sources of information, including vision, proprioception, and movement, in order to designate a spatial

RHYTHM SENSE

269

orientation relative to the world. Rhythm sense does the same, but designates a temporal orientation instead. In an enactive view, there is no particular reason to deny the status of “sense” to balance or rhythm, as they combine feelings and skilled movement to navigate and manipulate the world.

Conclusion The games present rhythm sense as something that can be measured, cultivated, and used in daily life, as simultaneously a channel of information and a way of feeling. An enactive account of sense perception offers a way to interpret it as a fully perceptual sense. But even granting this approach, rhythm sense would seem to have limited applicability: It requires an environment in which actions happen somewhat predictably, at regular intervals. Even actions that are predictable by vision, like the motions of the ball and paddle in David Sudnow’s game of Breakout, may be opaque to rhythm sense. Since rhythm sense depends so heavily on a particular environment, it may be tempting even at this stage to take a step back and stop short of claiming it as a true sense. But this is precisely how ordinary senses work, too: Every sense depends on some environmental affordances, such as the presence of light or air. By constructing a world governed by musical logic and repetition, the Rhythm Heaven games can be said to create a new sense, which allows humans to interact smoothly and pleasurably within its world.

Notes 1

Breakout is a cousin to Pong, in which the player slides a paddle from side to side in order to bounce a ball to the other side of the screen. But whereas Pong puts two players against each other, the opponent in Breakout is a field of rectangular blocks, which break when the ball hits them. The object of the game is to break all the blocks without letting the ball past the paddle.

2

Born Mitsuo Terada, he is known professionally as “Tsunku♂,” including the symbol for masculinity, ♂.

270

MUSIC VIDEO GAMES

3

The story of Rhythm Heaven’s creation is told in Harris (2009), Iwata (2008), and Kohler (2009).

4

Several reviewers claim that the cues are entirely aural, and that the game could be played “with your eyes closed.” While that may be true for some levels, the visual elements are still prominent: Most cues consist of both sound and animation, and the training segments rely heavily on explanatory text. This distinguishes it from truly audio-only music games, such as Blind Hero (Yuan and Folmer 2008).

5

The literature on entrainment is extensive, with ongoing debates over its significance for humans and its prevalence in non-human species; for a broad overview, see Clayton (2005). In particular, it is frequently cited as a “universal” prerequisite for musical activity: see Brown (2013). A currently influential explanation for why it seems to appear in only humans and a few other species is the “vocal learning hypothesis” proposed in Patel (2006). But it is interesting to note that the Rhythm Heaven games do not present “rhythm sense” as confined to humans: Throughout the games the player is joined in musical activity by a variety of animals, robots, and cartoon characters of various degrees of anthropomorphism.

6

See part 5, “Keeping Rhythm with Your Chest,” in Iwata (2011).

7

The sexual aspect is intentional, as described by the developers in part 2, “Everyone Decides Together,” of Iwata (2011).

8

That is to say, instead of seeing the player’s response in an “Imitation” game as a coherent pattern that matches the target pattern, every individual action can be mapped to the corresponding event in the original pattern. The game thus becomes an “Answer the cue” game with a long delay between cue and answer.

9

As in the previous examples, the lower staff uses solid noteheads to indicate drum hits and crosses to indicate squeaks; the upper uses solid noteheads to indicate “grabs” and crosses to indicate “swats.”

10 The Wii version augments it slightly by using marks on the ground to indicate whether each step is an ordinary hop or a two-stage rolling jump. 11 Despite its laxity with regard to the player’s rhythm, “Quiz” also makes a strong argument for the usefulness of rhythm sense. The host performs memorable rhythmic sequences, so rather than trying to count the taps numerically—the longest pattern consists of forty-three taps—the player may find it easier to imitate their rhythm by ear. In this case, rhythm sense takes the place not of sight or sound, but of mathematical calculation.

RHYTHM SENSE

271

12 “Rizumu wa jikan wo kizamu mono, rizumukan wa nori de hyougen shitari, kanjitari, shizen ni kamoshidashitari suru mono nanode.” Rhythm Tengoku was never officially released in English, so the translations in this section are adapted from the unofficial fan translation Rhythm Heaven Silver (Hat et al. 2015). The only significant change is the translation of the word nori, which the fan translation renders as “flow,” but which in musical contexts is more commonly translated as “groove.” 13 The English versions of the games render these loosely as “Nice!” or “Great!,” which is more idiomatic in English but loses the connotation of “feeling.” 14 The official English translation renders the term as “groove sense,” but it could also be rendered as “groovy feeling.” 15 Satoshi Kawase and Kei Eguchi allude to the importance of the term nori within Noh theater, where it indicates a rhythmic style of vocal performance (Kawase and Eguchi 2010). The consistency with which these terms apply to music and people might suggest that it may not simply be the case that one usage is a metaphorical extension of the other, but rather that both usages may reflect an underlying sociality of ensemble performance. See Monson (1996). 16 Some music scholars might find Rhythm Heaven’s insistence on conformity—its requirement that players precisely match the metronomic timing of the game’s tracks—to be fundamentally antithetical to “groove.” Charles Keil, for instance, considers groove to be dependent on “participatory discrepancies,” or “measurable differences or discrepancies in attack points and release points along a time continuum” (Keil 2010). 17 Philosophers, psychologists, and cognitive scientists distinguish among several related approaches to embodied cognition: ecological, enactive, embedded, and extended. Each of these would give a slightly different account of “rhythm sense” as a sensory mode, though they all give similar objections to the conventional Cartesian picture. See Wilson and Foglia (2011).

References Brown, Steven, and Joseph Jordania. 2013. “Universals in the World’s Musics.” Psychology of Music 41 (2): 229–48. doi:10.1177/0305735611425896. Clayton, Martine, Rebecca Sager, and Udo Will. 2005. “In Time with the Music: The Concept of Entrainment and Its Significance for

272

MUSIC VIDEO GAMES

Ethnomusicology.” European Meetings in Ethnomusicology 11 (ESEM Counterpoint 1). 1–82. Harris, Craig. 2009. “Tsunku♂ on Rhythm Heaven.” IGN. April 8. http:// www.ign.com/articles/2009/04/08/tsunku-on-rhythm-heaven. Iwata, Satoru. 2008. “Iwata Asks—Rhythm Heaven.” Official website. RhythmHeaven.com. July 11. http://rhythmheaven.com/iwata1.html. Iwata, Satoru. 2011. “Iwata Asks: Rhythm Heaven Fever: Wouldn’t Just Buttons Be Perfectly Fine?” IwataAsks.Nintendo.com. July 20. http:// iwataasks.nintendo.com/interviews/#/wii/rhythmheavenfever/0/0 Japanese version: http://www.nintendo.co.jp/ds/interview/ylzj/vol1/ index.html. Kawase, Satoshi, and Kei Eguchi. 2010. “The Concepts and Acoustical Characteristics of ‘Groove’ in Japan.” PopScriptum 11, The Groove Issue. http://www2.hu-berlin.de/fpm/popscrip/themen/pst11/pst11_ kawase-eguchi.html. Keil, Charles. 2010. “Defining ‘Groove.’” PopScriptum 11, The Groove Issue. http://www2.hu-berlin.de/fpm/popscrip/themen/pst11/pst11_ keil02.html. Kohler, Chris. 2009a. “Review: Rhythm Heaven Is Portable Musical Brilliance.” Wired. April 3. http://www.wired.com/2009/04/reviewrhythm-h/. Kohler, Chris. 2009b. “J-Pop Producer Tsunku Perfects Music Games With Rhythm Heaven.” Wired. April 10. http://www.wired. com/2009/04/qa-japans-pop-i/. London, Justin. 2004. Hearing in Time: Psychological Aspects of Musical Meter. New York: Oxford University Press. http://proxy. uchicago.edu/login?url=http://dx.doi.org/10.1093/acprof:oso/978019 5160819.001.0001. Monson, Ingrid T. 1996. Saying Something: Jazz Improvisation and Interaction. Chicago: University of Chicago Press. Noë, Alva. 2004. Action in Perception. Cambridge, MA: MIT Press. Patel, Aniruddh D. 2006. “Musical Rhythm, Linguistic Rhythm, and Human Evolution.” Music Perception: An Interdisciplinary Journal 24 (1): 99–104. Phillips-Silver, Jessica, C. Athena Aktipis, and Gregory A. Bryant. 2010. “The Ecology of Entrainment: Foundations of Coordinated Rhythmic Movement.” Music Perception 28 (1): 3–14. doi:10.1525/ mp.2010.28.1.3. Porter, Jess, Brent Craven, Rehan M. Khan, Shao-Ju Chang, Irene Kang, Benjamin Judkewitz, Jason Volpe, Gary Settles, and Noam Sobel. 2007. “Mechanisms of Scent-Tracking in Humans.” Nature Neuroscience 10 (1): 27–9. doi:10.1038/nn1819. Sudnow, David. 1983. Pilgrim in the Microworld. Hardcover, New York: Warner Books.

RHYTHM SENSE

273

Sudnow, David. 1993. Ways of the Hand. Cambridge, MA: MIT Press. W Hat, Sir Niko, Dirtie, S3v3nx3, BRPXQZME, Iq132, Ryan A., et al. 2015. “Rhythm Tengoku Translation.” GBATemp. Accessed March 28. http://wiki.gbatemp.net/wiki/Rhythm_Tengoku_Translation. Wilson, Robert A., and Lucia Foglia. 2011. “Embodied Cognition.” In The Stanford Encyclopedia of Philosophy, edited by Edward N. Zalta, Fall 2011. Accessed October 28, 2015. http://plato.stanford.edu/archives/ fall2011/entries/embodied-cognition. Yuan, Bei, and Eelke Folmer. 2008. “Blind Hero: Enabling Guitar Hero for the Visually Impaired.” In Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility, 169–76. Assets ’08. New York, NY, USA: ACM. doi:10.1145/1414471.1414503.

Games Atari, Inc. Pong [Arcade]. Atari Inc.: Sunnyvale, CA, 1972. Atari, Inc. Breakout. [Arcade, et al.]. Atari Inc.: Sunnyvale, CA, 1976. Nintendo SPD. Rhythm Tengoku. [Game Boy Advance/Arcade]. Nintendo: Kyoto, 2006. Nintendo SPD/TNX Music Recordings. Rhythm Heaven. [Nintendo DS]. Nintendo: Kyoto, 2008a. Nintendo SPD/TNX Music Recordings. Rhythm Tengoku Gold. [Nintendo DS]. Nintendo: Kyoto, 2008b. Nintendo SPD. Minna no Rhythm Tengoku. [Wii]. Nintendo: Kyoto, 2012. Yuan, Bei and Eelke Folmer. Blind Hero. Yuan and Folmer: Reno, 2008.

11 Pitching the Rhythm: Music Games for the iPad Nathan Fleshner

Rhythm games are perhaps the most prominent within the genre of music-related video games. Two particularly well-known games in this genre are Dance, Dance Revolution (DDR) and Guitar Hero. These games incorporate players actively responding to both visual and auditory cues in a quest to accurately mimic various rhythmic patterns. While the music in these games certainly involves both melody and harmony, any true control or interaction the user has with pitch in both DDR and Guitar Hero is superficial at best. As pitch is such an important component of musical structure, it is curious that more games do not involve pitch relationships in a more active role. This chapter explores the apparent lack of a focus on pitch in these games and the pedagogical significance of this absence. It then turns to two music games for the iPad, Soundrop (Develoe 2010) and Circadia (Simple Machine 2013), and discusses how those games bring pitch into a more prominent role.1 The inclusion of pitch in these iPad games adds a cognitive element that greatly enhances their possibilities as music training tools.

276

MUSIC VIDEO GAMES

Rhythm games of the past Both Guitar Hero and DDR fall within a genre of games known as Bemani, a term derived from “Beat Mania” (Auerbach 2010, 4).2 These games all involve a rhythmic mimicry that evokes a sense of “karaoke for the feet, hands, or body, depending on whether interaction with the game occurs via a handheld controller, dance pad, or mock instrument” (Auerbach 2010, 4). In this chapter, we encounter all three types of controllers in the mock instrument of Guitar Hero, the dance pad of DDR, and the handheld controller of the iPad itself. Much like Guitar Hero, DDR is a rhythmic-based game in which players respond to visual and auditory stimuli by performing dance-like steps on a floor-based controller. The pedagogical implications of DDR on the player’s musical cognitive abilities has been studied by Brett Auerbach in a laboratory setting at the University of MassachusettsAmherst. Auerbach posited that DDR’s “emphasis on accurate rhythmic performance and the presence of an objective, real-time scoring mechanism” (Auerbach 2010, 1) create substantial pedagogical benefits in the music theory classroom, the strongest of those being improved sight reading skills, such as chunking and reading ahead (Auerbach et al. 2010, 155–7). Peter Shulz has noted the additional pedagogical benefit of hierarchical thinking skills that result from playing the same song at increasingly more difficult and more complex levels (Shultz 2008). In addition to a pilot study of such benefits, Auerbach notes three points of interaction between DDR and common issues in the aural skills classroom: computer-assisted instruction (CAI), eurhythmics, and general sight-singing pedagogy (Auerbach 2010, 9). As a CAI device, DDR is a strong pedagogical tool due to its orientation as a game and relative simplicity (Auerbach 2010, 10–12). Through its use of the feet as the primary means of data delivery, DDR engages the entire body, a concept encouraged in eurhythmics. Auerbach notes, however, that DDR uses choreography to provide greater specificity and direction to the user’s rhythmic motions than typically found in eurhythmics (Auerbach 2010, 13–14). Finally, Auerbach notes that DDR provides just the right kind of skill practice that enhances sight reading skills, including chunking of patterns, planning ahead as the screen scrolls ahead, and emphasizing fluency by preventing a user to backtrack for error correction (Auerbach 2010, 15). These strengths certainly also

PITCHING THE RHYTHM: MUSIC GAMES FOR THE IPAD

277

transfer to many of the other Bemani games found across multiple platforms, such as the two for the iPad discussed later. Auerbach notes four specific pedagogical areas in which DDR can improve students’ aural skills (Auerbach 2010, 25): 1.  Improved ability to synchronize and maintain (conserve) a consistent beat. 2.  Improved reproduction and performance of sounded and notated rhythms. 3.  Improved sight-reading ability through practice with basic rhythmic shapes. 4.  Improved transcription ability. Auerbach highlights that these skills are progressive in nature, each building on the previous skill. After expanding on each of these four pedagogical areas, he concludes that: The design and game play of Dance, Dance Revolution… is sufficiently musical and flexible to merit its consideration as a CAI device suitable for aural skills instruction. Among DDR’s advantages is that it drills accurate rhythm performance and fosters good sight-reading skills. It further offers the opportunity for structured, meaningful practice outside of class. The feedback mechanism of DDR is nuanced and effective, occurring in real time. The “screen + dance pad + soundtrack interface” is redundant, serving visual, tactile, and aural learners. (Auerbach 2010, 46) While all of these pedagogical points are certainly important for the music classroom, it is still striking that neither DDR nor Guitar Hero involve pitch in a prominent role, a problem that raises certain cognitive issues discussed next.3

Rhythm games for the iPad A large corpus of Bemani games has also been developed for the iPad. The majority of these games mimic Guitar Hero and DDR with little innovation, the only difference being the use of a touch

278

MUSIC VIDEO GAMES

screen rather than a handheld controller as with Guitar Hero or a foot-based controller as with DDR. Examples of these include the apps Groove Coaster (Matrix Software 2011) and Tone Sphere (Sta Kousin 2014), both of which are not so veiled doppelgängers of Guitar Hero and DDR. Groove Coaster is much like Guitar Hero. Much as players encounter targets coming toward them on the screen’s virtual fretboard in Guitar Hero, Groove Coaster players travel along a virtual roller-coaster track that twists and turns much like a real roller coaster. Along this path, players encounter goals placed rhythmically in time with the music that must be struck in time with the music, exactly as in Guitar Hero. Tasks shown by these goals include striking the screen on a single goal, holding the finger to the screen to create a sustained tone, swiping toward the side of the screen in specific directions, rapid striking of the screen (almost like a trill), and rapid rubbing of the screen back and forth, much like vibrato. Tone Sphere follows quite closely with the basic principles of DDR. However, instead of stepping in the correct direction on a footbased controller, users simply strike the screen. In Tone Sphere, players strike the screen rhythmically in time as goals appear rhythmically across the screen. Tone Sphere differs slightly from Groove Coaster in that where one strikes the screen matters slightly more. There is no virtual coaster track on which the rhythmic stimuli appear. Rather, the goals appear at different points on the screen, requiring the player to strike different places much as they have to step on different places in DDR. As with other traditional Bemani games, Groove Coaster and Tone Sphere are purely rhythm games that leave pitch and pitch relationships noticeably absent from the gaming experience.

Pitching the rhythm For all of their pedagogical benefits, the exclusion of pitch in these Bemani games limits their potential benefits from both a musical and cognitive perspective. While DDR, Guitar Hero, and other rhythm games certainly include pitch in their delivery of rhythmic content, the user is not necessarily held accountable for pitch content beyond

PITCHING THE RHYTHM: MUSIC GAMES FOR THE IPAD

279

a very general level. Rather, color-coded or directionally coded notation directs the manner in which correct rhythms are delivered. Likewise, they lack a creative approach to music production. The chapters by Dana M. Plank, David Arditi, and David Roesner, et al. in this volume also discuss the use of creativity within the gaming experience. They are primarily reactive in nature, with users being scored on accuracy of rhythmic matching. The heavy reliance on a visual component in Guitar Hero, DDR, and similar games also potentially weakens their pedagogical efficacy. Users can match a shape’s or color’s arrival at a given visual location without any reference to auditory clues. The importance of a combined pitch and rhythm approach is well documented in the pedagogical and cognitive literature. Gary Karpinski notes: Whereas there is no inherent pedagogical problem per se with such isolated rhythmic dictation, one must consider its efficiency as a means toward developing comprehensive listening skills. One variety of rhythmic dictation asks listeners to notate from a purely rhythmic stimulus. This will have little direct application for all but unpitched music. Another asks listeners to notate only the rhythms of melodies. In such instances, the stimulus is now contextual, but the response is still somewhat acontextual. But as students develop their listening skills, they should be striving to remember whatever portion of a musical passage they are working out at any given time, including both its rhythms and pitches. This means that in taking purely rhythmic dictation they will discard whatever pitches they have remembered after each listening. Because of this, it seems more musical and more responsive to the innate behavior of musical memory to contextualize rhythmic listening, understanding, and notation as part of the broader study of melodic dictation. Metric and rhythmic cognition play an integral role as part of this more comprehensive process—particularly as practitioners begin to grapple with each remembered section of music. (Karpinski 2000, 32; original emphasis) He emphasizes that “… isolated rhythmic drills, if practiced at all, should be integrated as soon as possible into the world of

280

MUSIC VIDEO GAMES

pitches. The interaction between rhythmic and metric features on the one hand and tonal function on the other is so important that it should be instilled continually from the earliest possible point” (Karpinski, 115). Similarly, Michael Rogers notes that in rhythmic performance, “singing rather than clapping relates more naturally to pitch singing too” (Rogers 1984, 144). He adds that “rhythmicdictation melodies should use pitch content also for ease of memory” (Rogers 1984, 144). Important work in cognitive psychology reinforces these pedagogical observations. Fred Lerdahl and Ray Jackendoff’s A Generative Theory of Tonal Music proposes a hierarchical theory of musical structure based on the linguistic work of Noam Chomsky. Based in perception, their theory determines to create “a formal description of the musical intuitions of a listener who is experienced in a musical idiom” (Lerdahl and Jackendoff 1983, 1). They note that “the listener hears pitch-events in the context of rhythmic units composed of a combination of metrical and grouping time-spans” (Lerdahl and Jackendoff 1983, 284). They create an analytical system called “prolongational reduction,” consisting of a network of branch-like structures that represent a hierarchical organization of the pitches and rhythmic structures in a given musical composition. “Prolongational reduction expresses one of the most basic rhythmic intuitions: the breathing in and out, the tensing and relaxing, inherent in the motion of pitch-events. The component places pitches in a dynamic relationship” (Lerdahl and Jackendoff 1983, 285). Thus, Lerdahl and Jackendoff’s theory explicitly integrates pitch and rhythm. There are also neurological reasons for our cognitive need for rhythm and pitch to commingle. Robert Jourdain notes that “melodies are better perceived when presented to the left ear, which means that they are predominantly channeled to the right brain. The converse is true for rhythmic patterns, which are perceived more accurately when fed to the right ear, and hence to the left hemisphere. Experimental data of several kinds support the notion that rhythmic perception and harmonic perception are favored by different sides of the brain” (Jourdain 1997, 149). However, a wellrounded musical experience that coordinates the full brain’s function requires both hemispheres working together. Indeed, Jourdain

PITCHING THE RHYTHM: MUSIC GAMES FOR THE IPAD

281

provides additional evidence that, while the primary processing of rhythm and pitch is in opposite hemispheres of the brain, we process at least some portion of our rhythmic perception along with pitch. “Damage to left-brain secondary auditory cortex can greatly interfere with the ability to replicate metrical patterns. But left-brain damage does not wipe out rhythmic skill to the degree that rightbrain damage can wipe out harmonic skill. In fact, basic rhythmic abilities may be preserved even when the entire left hemisphere is momentarily disabled by feeding it an anesthetic (sodium amytal) through an artery in the neck” (Jourdain 1997, 150–1). In other words, our processing of rhythm and pitch may be more intertwined than we originally thought. Following this all-encompassing notion of rhythm and pitch interaction, Grosvenor Cooper and Leonard Meyer open their monumental book on rhythm stating, “To study rhythm is to study all of music. Rhythm both organizes, and is itself organized by, all the elements which create and shape musical processes. Just as a melody is more than simply a series of pitches, so rhythm is more than a mere sequence of durational proportions. To experience rhythm is to group separate sounds into structured patterns. Such grouping is the result of the interaction among the various aspects of the materials of music: pitch, intensity, timbre, texture, and harmony—as well as duration” (Cooper and Meyer 1960, 1).4 They conclude that “it is the intimate and intricate interaction of temporal organization with all the other shaping forces of music which makes the study of rhythm both a rewarding task and, at times, a difficult and perplexing one…. By adding a new dimension to our understanding of related fields such as melody, harmony, counterpoint, and orchestration, it makes possible a more precise and penetrating analysis of those processes” (Cooper and Meyer 1960, 1). All of these pedagogical and cognitive reasons for combining pitch and rhythm come into play in the two iPad games explored in this chapter. As they initially seem so strongly centered around pitch relationships, Soundrop and Circadia appear to be polar opposites to Bemani games such as Guitar Hero and DDR discussed earlier. However, rhythm and pitch are indeed engaged in a more intimate manner in both games, explicitly in Soundrop and in a more veiled manner in Circadia.

282

MUSIC VIDEO GAMES

Soundrop Soundrop for iOS is a substantial departure from the score-driven quests for accuracy of games in the Bemani genre. While pitch is passively encountered in previous rhythmic games, the user’s control of pitch relationships is largely absent. Soundrop includes pitch and rhythm on equal footing and gives the user increased control over both parameters. Rather than scoring users on an accurate duplication of rhythmic patterns, Soundrop provides a blank canvas on which users compose a unique work of art. The user’s role is reversed from reaction to a stimulus to the creation of the stimulus itself. This immediate control over both rhythm and pitch is not something commonly found in many other iPad games. As such, Soundrop is really a vehicle for music composition. Indeed, it is simultaneously both a compositional score and a musical instrument. The only other apps or games that incorporate this type of compositional freedom are generally virtual instruments, such as keyboard- or guitar-playing apps, a seemingly endless array of which exists for Apple’s iOS-based iPhones and iPads. This section discusses the basic components of Soundrop and its reformulation into a pedagogical game for the music theory classroom and thus an important addition to the genre of music games. In Soundrop, tiny balls drop from the top of the screen (see Figure 11.1). The player draws lines that are then struck by the balls as they fall. Like virtual xylophone or marimba bars, each drawn line produces a pitch when struck by a falling ball. Sounding bars can be drawn at different angles and at various locations across the screen. The placement of the bars alters the rhythm of the dropped balls, and the height of each line, relative to its dropping distance, determines the pitch sounding as it is struck. Balls can then ricochet in various directions toward other bars creating increasingly complex soundscapes. Including options available in “Menu,” Soundrop provides the user with control of eight parameters that affect the musical composition: number of nodes, pitch, rhythm, timbre, textural complexity (chosen from eight timbres with a maximum of five sounding simultaneously), gravitational strength, air friction, and bounce.

PITCHING THE RHYTHM: MUSIC GAMES FOR THE IPAD

283

FIGURE 11.1  Soundrop with two sounding bars drawn (screenshot from iPad). Arrows added to highlight the trajectory of the balls. Source: Develoe LLC. Soundrop, Apple App Store, Version 1.2.1 (2010).

Soundrop begins at the top-left corner of the screen with a single node from which balls drop at regular intervals. The remainder of the screen is initially blank. The user then draws straight lines or sounding bars of any length or angle, one at a time. These bars sound

284

MUSIC VIDEO GAMES

when struck by the falling balls. Pitch is affected by the location of the ball’s contact with the bar along the screen’s y-axis as it relates to the dropping node. The angle of the bar itself does not alter the pitch, but, as we will see later, a single bar can differ in pitch depending on where a ball hits it. The angle of the bar does alter the direction of the ball’s bounce, but it only alters the pitch if changing the angle raises or lowers the ball’s contact point along the screen’s y-axis. As an originating node can be moved anywhere within the Soundrop screen, the pitch is determined by the distance a sound bar is placed from the node rather than by a fixed location on the screen. The closer a bar is drawn to the dropping node, the lower it sounds. Likewise, the further a bar is drawn from the dropping node, the higher it sounds. Perhaps in order to provide a more harmonious compositional outcome to the average user, Soundrop uses pitches from a pentatonic collection to the exclusion of a complete major or minor collection. Pentatonic scales are often seen as favorable due to their ease of singing as well as their common usage in folk songs and popular music. The major pentatonic scale, as used in Soundrop, consists of the solfege syllables, DO-RE-MI-SOL-LA, or the pitches C-D-E-G-A, when oriented with C as a tonal center. This can also be reoriented to a minor sound as LA-DO-RE-MI-SOL, or the pitches A-C-D-E-G, when oriented with A as the tonal center. There are other formations within the pentatonic collection as well, such as its vertical reorientation as a major triad with an added 9th and 13th, a chord that has a particularly pleasing (one might even say “jazzy”) sound. All of these factors contribute to expediting a pleasing and somewhat consonant compositional structure, even in complex compositions, while playing Soundrop. In other words, these limitations set up composers to be pleased with the musical results of their composition. It sets them up to create a product they are conditioned to enjoy. Rhythm is also an active component of Soundrop and is controlled by the angle of each bar and its distance from the dropping node or from the previously sounded bar. Balls drop at regular rhythmic intervals from each node. However, the speed of each ball can be adjusted in the settings menu. The settings menu also allows the user to alter the power of the gravitational pull toward the

PITCHING THE RHYTHM: MUSIC GAMES FOR THE IPAD

285

bottom of the screen, the density of the friction in the air, and the intensity of the bounce as each ball strikes a bar. Gravity can be set from 0.01 to 2.00. The higher the gravitational setting, the faster a ball drops toward the bottom of the screen. Air friction can be set from 0.00 to 0.10. The higher the air friction is set, the denser the air through which a ball travels and the slower that ball travels. The intensity of the bounce can be set from –2.00 to 2.00. The higher the intensity setting, the higher the ball bounces, and viceversa. With a final setting that affects the speed, and therefore the rhythm of each ball’s journey, a user can turn “real gravity” on or off. The default setting of Soundrop is “real gravity” off. With “real gravity” switched off, balls drop toward the bottom of the screen, as though the perspective of the user is as standing on earth. In other words, the user is looking at the sounding node placed above them in the sky and sounding bars are placed below the node at a lower altitude. If a user tilts the iPad slightly while holding it, balls consistently drop unaffected toward the bottom of the screen. This provides the user with a consistency in the rhythmic dropping of the balls toward the sounding bars. When a user switches “real gravity” on, the balls drop according to the actual gravitational pull of the Earth and the angle of the iPad affects the direction of a ball’s drop. This is possible due to the iPad’s virtual gyroscope, or MEMS (micro-electro-mechanical systems) gyroscope (Philippides 2011). In this setting, the game reacts to the angle at which the iPad is held. Balls no longer drop consistently toward the bottom of the screen; they now drop toward the actual external ground. If the user alters the angle at which he or she is holding the iPad, the direction of a ball can be altered, even mid-flight. If a ball is dropping toward a sounding bar when the iPad is held at its normal vertical orientation, it will drop away from that same sounding bar if the iPad is turned upside down. Amazingly, if the iPad is placed with its back down, flat on a level table, the balls appear to travel virtually weightless, without a gravitational pull toward any side of the iPad’s screen. “Real gravity” allows the user to alter the manner in which they play the dropping balls against the grid of sounding bars in real time. It allows Soundrop to be used as a virtual instrument in more subtle and sensitive ways, resulting in a greater sense of compositional control.

286

MUSIC VIDEO GAMES

At a first glance, Soundrop appears to be just a music composition app and not a game at all. However, classroom usage of Soundrop transforms it into a game with minimal effort. There are many mainstream games that involve creative activities. Notable examples include Pictionary (Angel Games, Inc. 1985), involving drawing, and Charades (and its marketed forms, such as Guesstures [Hasbro 1990]), involving acting, both of which turn these creative activities into games. In a similar manner, Soundrop can be used to turn music composition into a game. The general rules to a well-known game, Apples-to-Apples (Out Of The Box Publishing 1999), lend themselves well to an adaptation through Soundrop, resulting in tremendous pedagogical potential. In a classroom equipped with iPads,5 students sit in a circle. One student is named the judge for that round. Using the variables allowed within Soundrop, the judge provides the players with a list of parameters to be used for that round. Such parameters include: 1) number of sounding bars; 2) number, frequency, and bounce properties of the balls; 3) preset sounds to be used; and 4) some type of aesthetic goal. An aesthetic goal is perhaps the most fun part of the game. Here, students can pick between any programmatic representation, such as “a composition depicting a basketball game” or “a knight rescuing a princess from a tower,” or they can choose a more abstract goal, such as “green” or “fluffy.” The remaining students create compositions using Soundrop based on those parameters. The students then pass their iPads anonymously to the judge. The students listen to each composition. The judge leads a discussion of each composition, discussing the pros and cons, their likes and dislikes of each composition in its representation of their preset rules. Discussions should include specific aspects of rhythm and pitch, and their interaction. Finally, the judge declares one composition as the winner, and the game continues until each student has been the judge for a round. The student with the most wins at the end of the session is the winner. There are many pedagogical benefits to using Soundrop as a game. Students are given ownership of discussions regarding pitch and rhythm relationships, reinforcing concepts learned in class. As a variety of answers will be created during each round, students can compare and contrast different musical representations of each

PITCHING THE RHYTHM: MUSIC GAMES FOR THE IPAD

287

judge’s original prompt. More than just what a given rhythm or melodic structure involves, students can discuss exactly how such musical structures benefit an aural representation of that prompt, providing insight into the nature and characteristics created using different rhythmic and melodic combinations. Additionally, Soundrop provides hands on, visual, and aural components to the composition process, simultaneously benefitting different learning styles. The use of games in class also disguises the “work” of learning as a fun activity. In this manner, Soundrop is more than just a game; it is also a powerful pedagogical device. It provides the user vastly greater control over both rhythm and pitch, and it allows the user to explore pitch and rhythmic relationships with a greater freedom than games in the Bemani genre. As a result, all facets of the brain are involved as the user explores the compositional space. The exploratory freedom in Soundrop makes it particularly useful as a game within a classroom context. For beginners, the preset parameters invite instantly pleasing sonorities, encouraging students to explore further. The multiplicity of setting options allows more advanced students to create compositions of greater complexity. In addition to the game, assignments could be made with specific instructions regarding the number of parameters to be used and specific settings required for a composition. Soundrop is immediately beneficial for curricula with a greater emphasis on improvisation. Finally, the inclusion of a visual and tactile approach of Soundrop gives learners of various types a different approach to exploring composition than the traditional means of musical notation.

Circadia Circadia tightens some of the compositional freedom encountered in Soundrop. However, it does not fully return to the “Simon Says” stimulus/response protocol found in other games of the Bemani genre. Rather than keeping score based on the accuracy of a user’s mimicking of rhythmic patterns as in DDR and Guitar Hero, Circadia is a puzzle game that appears at first glance exclusively to involve pitch relationships and rewards players for correct understanding

288

MUSIC VIDEO GAMES

of these relationships. In each musical puzzle, Circadia contains colored dots that serve as virtual bells. When pressed, a bell sounds a particular pitch and releases a virtual sound wave (or “bursts” as Circadia’s on-screen prompt refers to them) in the form of a circle emanating away from the bell. As with actual acoustics, lowerpitched sound waves travel slower than higher-pitched sound waves. Early instructional levels of Circadia contain just one sounding bell, but most levels have two bells, usually pitched differently, which must be struck in a manner that coordinates the motion of their sound waves. The early levels of Circadia do an excellent job of teaching the player these basics of acoustics, stating through onscreen instructions that “lower tones release slower bursts, higher tones release faster bursts.”6 Indeed, all relationships are separated in a pedagogically sound manner, slowly instructing users one principle at a time. The game of Circadia also has a small white dot that is located away from the sounding bell, or set of bells, and serves as the goal for each sound wave. A level is completed when the sound wave from a bell arrives at the goal. As most levels involve more than one bell, the sound waves “must touch the dot at the same time.”7 The first time this is introduced in Circadia, the bells are identical in both pitch and color, sharing the same shade of light blue. As the goal is placed equidistant from the two bells and the sound waves proceed at the same speed, the bells must be pressed at the exact same time to reach the goal simultaneously. However, the game does not instruct the player that the bells must be struck simultaneously; the user discovers this through trial and error. The various ways in which a player can solve each puzzle highlights the uniqueness of Circadia. A player can certainly solve a puzzle through trial and error, as mentioned earlier, eventually moving on to the next level. In this way, Circadia is very much a rhythm game in the tradition of DDR and Guitar Hero. Players can simply rely on observation of the speed of the sound waves, the location where the sound waves cross, and adjustments made to the calculation of precisely when a given sequence of bells must be struck to achieve a coordinated and simultaneous goal. All that is essentially required is the recognition of higher and lower pitches, resulting in slower- and faster-moving sound waves, respectively.

PITCHING THE RHYTHM: MUSIC GAMES FOR THE IPAD

289

A user is not required to recognize the exact pitch relationships for basic puzzle solving. Rhythm and meter are essentially the division of time into attack patterns with specific relationships. Milton Babbitt notes that “the precise placement of time points and their associated durations, though easily and exactly specifiable, takes one into the area of rhythm” (Babbitt 1972, 150). While rhythm is not an explicit component of Circadia’s puzzles, a player can divide the time that a sound wave travels into equal divisions, Babbitt’s “time points,” thus projecting a rhythmic structure on the sound wave’s traveling space. Indeed, because the sound waves travel at specific speeds and cross specific distances from sounding bell to goal node, players can use these rhythmic calculations to make better-informed decisions regarding when to sound the bells. If a player counts the time between sounding bell and goal node, thus projecting an artificial pulse on that time span, they establish a tempo for that sound wave, composed of regularly occurring rhythmic pulses. Drawing from principles of Gestalt psychology, Fred Lerdahl and Ray Jackendoff note that listeners prefer to divide such time points into subdivisions that are equidistant from each other (Lerdahl and Jackendoff 1983, 49–50). Depending on the speed of each user’s counting, the tempo of this projected rhythmic division of the sound wave’s path will vary. Some users may count three pulses at a slower tempo, while other users count a faster four pulses for the same sound wave. Such calculations can remove some of the guesswork out of the trial-anderror method. Users can use these rhythmic projections to calculate the time it takes for a sound wave to reach the goal, giving them a more precise point at which to press sounding bell with a slowertraveling sound wave. While the puzzles can be solved without knowledge of specific pitch relationships as described earlier, specific intervallic relationships can enhance these rhythmic divisions. The intervallic relationships between the two sounding bells give a greater level of precision to the projected rhythmic structure of each puzzle. Indeed, knowledge of these relationships greatly increases the speed at which each puzzle can be solved. In doing so, users combine the speed-based perceptions with more precise calculations of the speed and distance between sounding bells and the goal node. At

290

MUSIC VIDEO GAMES

the beginning instructional levels, a player could easily observe that the bells are the same color, assume that they are the same pitch and their sound waves move at the same rate of speed, and solve the puzzle on their first try. While the simplicity of like-colored, likesounding bells with precisely the same speed of sound waves may seem obvious, this relationship grows more complex as the game advances. As Circadia uses equal-tempered tuning, users can calculate the correct moment to press the respective sounding bells by knowing the intervallic relationship between the two pitches. In equaltempered tuning, the twelve chromatic pitches are equidistant from one another by the acoustical measure of 100 cents. As shown in Figure 11.2, two pitches, an octave apart, are 1,200 cents apart. This creates a great ease for the user in calculating the relationships between the different sounding pitches in Circadia. One simply needs to think of how an interval divides an octave. Many intervals divide the octave equally. A semitone divides an octave into twelve equal parts, 100 cents apart, as does a whole tone, into eight equal parts, 200 cents apart, as with C-D and D-E. A minor third, such as C-E♭, divides an octave into four equal parts, 300 cents apart, as does a major third, C-E, into three equal parts, 400 cents apart. As a result, their inversions, the sixth and seventh, can also be gleaned from combinations of these equal divisions. A minor sixth is 800 cents apart, which can be calculated from combining eight semitones, four whole tones, or two major thirds. A major sixth is 900 cents apart, which can be deduced from combining nine semitones or three minor thirds. A minor seventh is 1,000 cents apart, which can be discovered from combining ten semitones or five whole tones, and a major seventh, 1,100 cents apart, consists of eleven semitones, or just 100 cents away from a perfect octave. Additionally, both versions of the tritone, the augmented fourth and diminished fifth, divide the octave equally in half (600 cents apart) or in combinations of six semitones, three whole tones, or two minor thirds. The final two

FIGURE 11.2 The octave divided into semitones. The space between each pitch represents the distance of 100 cents.

PITCHING THE RHYTHM: MUSIC GAMES FOR THE IPAD

291

remaining intervals, a perfect fourth and perfect fifth, are less helpful in this manner. Neither divides the octave equally and neither has an equal division of the octave as a common denominator as do other intervals such as the minor seventh, which is a combination of whole tones. This lack of symmetry is one of the primary reasons that perfect fifths are so important in the hierarchical system of functional tonality. This knowledge of intervallic distances aides the Circadia player in the following manner: In Circadia, players are tasked with coordinating two sound waves, traveling at different speeds, to reach a single goal simultaneously. Users with trained ears who are able to identify the intervallic relationship between the two pitches have an advantage in that they can pre-calculate exactly when each bell needs to be pressed without reference to trial and error or projecting a rhythmic metric onto the path of the bell via the process of discovery. Indeed, these properties can be used to transform the game into a powerful pedagogical tool. Level 6 presents the interval of a minor third with both sounding bells placed equidistant from the goal. This is the first level in which a player encounters balls of different pitches. Here, the minor third interval created by the balls divides the octave into four equal parts. Provided the player is able to aurally identify the interval between the two sounding bells as a minor third, this intervallic division of the octave can be used to solve the puzzle in Level 6 and thus reveal an implicit rhythmic perspective within a seemingly pitch-centric game. As discussed earlier, the sound wave of the lower-sounding bell travels slower than the higher-sounding bell. To solve the acoustic puzzle using their knowledge of a minor third, the player must divide the path between the lowest-sounding bell and the goal into four equal parts. This gives a greater precision to the projected rhythmic patterns discussed previously. If the player counts too quickly, the sound wave will reach the goal too late. If the player counts too slowly, the sound wave will reach the goal too early. Having established a proper tempo for the four divisions of the lower-sounding bell, the player simply strikes the highersounding bell after the lower sound wave reaches the first of the four divisions, following the principle of dividing the octave via the minor third (see Figure 11.3).

292

MUSIC VIDEO GAMES

FIGURE 11.3  Circadia, Level 6 (screenshot from iPad). Source: Simple Machine, LLC. Circadia, Apple App Store, Version 1.3 (2013).

Level 7 is similar to Level 6, but involves the interval of a major sixth, equaling three minor thirds and moving the striking of the higher-sounding bell two lines to the right, as shown in Figure 11.3. Levels 8 and 9 of Circadia involve the interval of a tritone, which divides the octave equally. The process of calculation necessary for the minor third in Level 6 and the major sixth in Level 7 also carries over to Levels 8 and 9. In Level 8, the higher-sounding bell is twice as far from goal node as the lower-sounding bell. As the two bells are related by a tritone, dividing the octave into two equal halves, the sound wave of the higher-sounding bell travels twice as fast as the lower-sounding wave. As a result, both bells in Level 8 must be pressed simultaneously for the wave to reach the goal node together. Level 9 also involves a tritone, but the order of the bells is reversed from Level 8. The lower-sounding bell with the slower sound wave is twice as far away from the goal than the higher-sounding bell with the faster sound wave. As a result, the lower-sounding bell must be struck first with the faster sounding bell struck after the slower sound

PITCHING THE RHYTHM: MUSIC GAMES FOR THE IPAD

293

wave has passed it and travelled half of the distance from the highersounding bell to the goal. Upper levels of Circadia continue building on these and other intervallic relationships in a similar manner. These precise intervallic calculations certainly aid in solving each problem. However, the broader principle that chance is not the only way to determine when to strike the bells is even more important. Indeed, this principle highlights the pedagogical and cognitive importance of Circadia. Any object that travels over a span of time inherently has both a rhythmic pulse and a tempo attached to that rhythmic pulse. This facet of the game can be experienced, perhaps equally successfully, without reference to specific intervallic content and their distance from the necessary goal. Indeed, Circadia players can explore the speed at which each sound wave travels from sounding bell to goal in their own manner, establishing subdivisions other than the divisions suggested in the preceding paragraphs. They can do this for each sounding bell, noticing that the sound wave emanating from one sounding bell travels at a different rate of speed from the other and, therefore, has slightly different metric and rhythmic properties. The player can then calculate their own intervallic relationships between the two sounding bells and where the placement of the higher-sounding bell’s strike needs to be in relation to that of the lower-sounding bell. Perhaps, this process of selfdiscovery makes Circadia’s pedagogical potential even more valuable. In either case, the resulting lesson is that pitch relationships share a rhythmic and metric organization to one another. Circadia has tremendous potential for the music theory classroom. As intervallic identification is a strong component, it provides immediate benefits for aural skills classes. The puzzle configuration adds a level of enjoyment not often found in computerassisted aural skills software. The placement of the sounding bells in different relationships also adds a level of complexity beyond mere intervallic identification. The acoustic principles at play also make it a valuable game for physics and acoustic classes, providing interesting discussion points for intervallic calculations. Soundrop and Circadia add significant pedagogical components to the previous genre of rhythmically centric Bemani games. Indeed, just as discussed throughout the pedagogical and cognitive science literature, the inclusion of pitch with rhythm greatly enhances both the user’s musical and cognitive benefit from playing such games.

294

MUSIC VIDEO GAMES

For Soundrop, this benefit involves a creative freedom not often introduced in other music-based games for the iOS. The innovation in the control of the sounds themselves, veering away from the ubiquitous virtual keyboards and guitars available for the iPad and iPhone, gives the user both an aural and visual soundscape in which to play with pitch and rhythm alike. Circadia returns some of the strictures of the Bemani genre, but benefits the user through the incorporation of the puzzle-game genre in fortifying pitch relationships with underlying rhythmic divisions of time. In doing so, it also reinforces the pedagogical and cognitive principles of pitch and rhythm discussed in this chapter. Both Soundrop and Circadia serve as powerful responses to Karpinski’s charge that “the interaction between rhythmic and metric features on the one hand and tonal function on the other is so important that it should be instilled continually from the earliest possible point” (Karpinski, 115). These games help define a new direction in the possibilities of music games for tablets and other handheld devices.

Notes 1 For more on the use of smartphone apps for music theory pedagogy, see Fleshner (2013). 2 Nicole Biamonte describes the pitch relationships in Guitar Hero as dependent on contour, requiring users to simply discern between high- and low-pitch relationships. This contour-based approach diminishes any great pedagogical benefit to pitch formations beyond a fairly basic level. Biamonte notes the common disruption of high and low distinctions in the many songs that encounter “more than five notes in a single direction,” a problem as there are only five fret positions available on the guitar controller (Biamonte 2010). 3 Karaoke Hero is a game in line with Guitar Hero and Rock Band that incorporates a greater dependence on pitch. As the focus of this article is on games that appear, at least at first glance, to focus on rhythm, Karaoke Hero has been left for future study. 4 Duration is an important component of rhythmic construction as varying durations alter the placement of attack points, and attack points are the very essence of what creates a rhythmic pattern.

PITCHING THE RHYTHM: MUSIC GAMES FOR THE IPAD

295

5 This is a realistic possibility in the very near future. In pilot tests and plans for the future, many schools have supplied all incoming students with iPads or other tablet devices. 6 Circadia, Levels 3 and 4, on-screen instructions. 7 Circadia, Level 5, on-screen instructions.

References Auerbach, Brent. 2010. “Pedagogical Applications of the Video Game Dance Dance Revolution to Aural Skills Instruction.” Music Theory Online 16 (1). Accessed October 28, 2015. http://www.mtosmt.org/ issues/mto.10.16.1/mto.10.16.1.auerbach.html. Auerbach, Brent, Bret Aarden, and Mathonwy Bostock. 2010. “DDR at the Crossroads: A Report on a Pilot Study to Integrate Music VideoGame Technology into the Aural- Skills Classroom.” In Pop-Culture Pedagogy in the Music Classroom: Teaching Tools from American Idol to YouTube, edited by Nicole Biamonte, 149–72. Lanham, MD: Scarecrow Press. Babbitt, Milton. 1972. “Twelve-Tone Rhythmic Structure and the Electronic Medium.” In Perspectives on Contemporary Music Theory, edited by Benjamin Boretz and Edward T. Cone, 148–79. New York: W. W. Norton and Co. Biamonte, Nicole. 2010. “Musical Representation in the Video Games Guitar Hero and Rock Band.” In Pop-Culture Pedagogy in the Music Classroom: Teaching Tools from American Idol to YouTube, edited by Nicole Biamonte, 133–48. Lanham, MD: Scarecrow Press. Cooper, Grosvenor and Leonard Meyer. 1960. The Rhythmic Structure of Music. Chicago: The University of Chicago Press. Fleshner, Nathan. 2013. “There’s An App for That: Music Theory on the iPad, iPhone, and iPod.” Journal of Music Theory Pedagogy 27: 269–79. Jourdain, Robert. 1997. Music, the Brain, and Ecstasy: How Music Captures Our Imagination. New York: Avon Books. Karpinski, Gary S. 2000. Aural Skills Acquisition: The Development of Listening, Reading, and Performing Skills in College-Level Musicians. Oxford: Oxford University Press. Lerdahl, Fred and Ray Jackendoff. 1983. A Generative Theory of Tonal Music. Cambridge, MA: The MIT Press. Philippides, Alexis. 2011. “What is the Gyroscope Inside the iPad and What’s in It for You.” Stuff Review. Accessed January 11, 2015. http:// www.stuff-review.com/2011-03/what-is-the-gyroscope-inside-theipad-2-and-whats-in-it-for-you.

296

MUSIC VIDEO GAMES

Rogers, Michael R. 1984. Teaching Approaches in Music Theory: An Overview of Pedagogical Philosophies. Carbondale: Southern Illinois University Press. Schultz, Peter. 2008. “Music Theory in Music Games.” In From Pac-Man to Pop Music: Interactive Audio in Games and New Media, edited by Karen Collins. Aldershot, Hampshidre, England: Ashgate, eBook Collection (EBSCOhost), EBSCOhost (accessed May 21, 2015). Smith, Jacob. 2004. “I Can See Tomorrow in Your Dance: A Study of Dance Dance Revolution and Music Video Games.” Journal of Popular Music Studies 16 (1): 58–84.

Games Angel Games, Inc. Pictionary. [Board Game]. Seattle, WA, 1985. Develoe, LLC. Soundrop. [iOS]. Develoe: Tucson, 2010. Freestyle Games, et al. DJ Hero. [PlayStation 2]. Activision: Santa Monica, 2009. Harmonix. Guitar Hero. [PlayStation 2]. RedOctane: Mountain View, CA, 2005. Harmonix, et al. Rock Band. [Xbox 360, et al.]. MTV Games/Electronic Arts: New York, 2007. Hasbro. Guesstures. [Board Game]. Pawtucket, RI, 1990. Konami. Dance Dance Revolution. [Arcade, et al.]. Konami: Tokyo, 1998. Kousin, Sta. Tone Sphere. [iOS/Android]. Bit192 Labs: Tokyo, 2014. Matrix Software. Groove Coaster. [iOS]. Taito Corp: Tokyo, 2011. Out Of The Box Publishing. Apples-to-Apples. [Board Game]. Out Of The Box Publishing: Madison, WI, 1999. Simple Machine. Circadia. [iOS/Android]. Simple Machine: New York, 2013.

Afterword—Toadofsky’s Music Lessons William Cheng

I

devoured this entire volume in a single, nearly uninterrupted sitting. Clicks and bloops rang in my ears, while colors and patterns popped into my mind’s eye. Vivid writing sucks you into a sensorial vortex; or, to use game-related vocabulary, such prose warps you (à la Mario’s magic whistle or Link’s ocarina) to other lands, other times. Striking essays in these pages abound with efforts to capture the sights, sounds, and feelings of music video games—an umbrella genre distinguished by a vital emphasis on rhythmic engagement and audiovisual coordination. Authors tackle broad themes, everything from agency and accessibility to community and pedagogy. Inquiries revolve around play’s musicality and music’s playfulness. In historical, technological, and cultural perspectives, much of ludomusicology’s literature to date upholds music and play as a match made in heaven, insofar as both activities echo with creativity, virtuosity, and the making and breaking of rules. In this brief afterword, I offer some modest musings on the ethical possibilities of music video games. By lending an ear to the affective affordances of this genre, I tease out the larger social stakes at work in day-to-day debates about artistry and recreation. With reference to a musical moment in the game Super Mario RPG: Legend of the Seven Stars (Square 1996), I use the metaphor of the safety net to weave together insights about what musicality entails, how it feels, and why anyone would seek to deny such feelings either to themselves or to others.

298

MUSIC VIDEO GAMES

Midway through Super Mario RPG, the player arrives at Melody Bay and encounters an angsty composer named Toadofsky (plausibly related to Pyotr Ilyich Tchaikovsky). Wearing a flashy ushanka, Toadofsky wishes to write a symphony, but he lacks inspiration. To invigorate Toadofsky, the player must collect a trio of melodies across the gameworld and sound them out for the composer by jumping on musical tadpoles (Figure A.1). Each melody consists of eight notes. Upon successful reproduction of the three phrases, the player receives the privilege of composing a final phrase—a tune comprising any eight pitches— to round out the symphony. The quest culminates in a cutscene that replays all four melodies in succession, replete with swelling harmonies pumped out by the audio capacities of the 16-bit Super Nintendo (Figure A.2). Playing this game at the age of eleven, I was fascinated by the numerous compositional possibilities in the fourth phrase (a musical afterword, if you will). Here was an opportunity to personalize the end of the symphony with any eight notes I wished; the fact that these final notes marched in double-time (half the rhythmic value of the notes in the previous three phrases) made the cadential push all the more exciting. My brother and I played the Toadofsky sequence over and over again, each time inputting different pitches. A magical realization dawned on us: We began to notice that, no matter what melody we composed for the fourth phrase (Figure A.3), it always came out… eh, not too bad. With only minimal knowledge of music theory, we couldn’t figure out how the game kept managing to make

FIGURE A.1 (Left) Tadpole tablature at Melody Bay and (right) endless opportunities to recompose final melody.

AFTERWORD—TOADOFSKY’S MUSIC LESSONS

299

FIGURE A.2  Toadofsky’s symphony, mm. 1–6 (my transcription).

FIGURE A.3  Three examples of how players might complete the melody of mm. 7–8.

even our deliberately crude pitch sequences (such as E-E-E-E-E-EE-E) sound good enough. In retrospect, the musical magic was child’s play. Using a harmonic sleight of hand, the game imposes a ii–IV–V7 (perhaps oddly, instead of the more typical IV–ii–V7) chord progression in the fourth phrase, rounded out by a perfect authentic cadence (ending on a root-position tonic chord with a tonic in the highest voice). With this fixed–bass line qua cadential anchor, any superimposed melody of a player’s devising can’t run terribly afoul. At the worst

300

MUSIC VIDEO GAMES

(read: straying farthest from tonal conventions), the player-created melody may sound like an overabundance of non-chord tones, a fleeting compositional lapse. By the end of the piece, however, the tonic resounds clearly, and all is forgiven, maybe even forgotten. The predetermined harmonies and capstone chord rein in the player’s musical invention, preventing it from going too far adrift, aesthetically speaking. The result is that players will tend to feel fairly musical upon hearing the symphony’s playback, regardless of whether they know why this is the case. With eight little notes, Toadofsky’s quest elicits the sort of wonder and performative delight granted by music video games. As with classical pieces based on theme and variations, harmony is the principal control, the foundational element that undergirds melodic, rhythmic, and textural variegation. In the case of Toadofsky, the preprogrammed harmonic progression in the fourth phrase serves as an artistic safety net. It is there to catch players even when they fall—that is, when they fail to compose a tonally sensible melody. Such uplifting and corroborative design informs gaming experiences more generally: Sometimes we feel like we’re fighting against a game (when glitches and lag take hold in survival-horror, or when an outrageously challenging Dark Souls boss keeps knocking us down); other times, we feel like a game is carrying us, offering leeway and enabling virtual gains even when our timing is a little delayed or when our aim is slightly off (via generous enemy hitboxes or activated aimassist). Granted, gamers malign auto-aim for the same reasons people deride musical aides such as Auto-Tune. The argument is that we should succeed based on abilities alone (digital reflexes and singing capabilities) without the crutches of ludic or pitch quantization. In an online first-person shooter, someone using Aimbot claims a foul advantage and upsets the playing field. In the music industry, someone who relies sneakily on Auto-Tune or lip-synching likewise makes off with supposedly unmerited earnings and praise. With both examples, critics harp on people who, by employing such assistive devices, do better—and also presumably feel better—than they deserve. This concern jibes with the moral panic that has shrouded Guitar Hero and Rock Band, as detractors fussed over how players of these games shouldn’t be entitled to call themselves real musicians.

AFTERWORD—TOADOFSKY’S MUSIC LESSONS

301

No, cheating isn’t cool: Using Aimbot to nab twenty headshots in an online first-person shooter is unfair to other players. But does jamming on plastic guitar peripherals with friends hurt anyone? Nothing in the Guitar Hero literature has reported adverse effects on the music or gaming industries, on youths’ artistic proclivities, or on social mores. Virtual musicianship in The Lord of the Rings Online (Turbine, Inc. 2007) has not endangered conventional musicianship, just as gay marriage has not destroyed traditional marriage. If this analogy sounds like a stretch, think of it in these terms: Why would anyone deny virtual musicians the feelings of musicality? Do we really believe we live in a zero-sum society, in which someone’s fantasies and love’s labors must come at the cost of others’ sense of accomplishment? Are dreams and happiness in short supply? Should people have the inalienable right to feel musical, even if such musicality falls outside accepted norms and rubrics? Concepts and vocabularies of authenticity take us only so far. In debates about whether Guitar Heroes do or do not deserve the badge of musicianship, the primary affect, I reckon, is not resentment or self-righteousness or pride. It’s envy. We envy stargazers, people who cling easily to reveries and walk just a couple of inches above the hard earth. Popping these fantasies yields the same kind of sharp emotion as popping a balloon: There’s the gratified exhilaration of the pow! and the instantaneous deflation of the once-airy object (or allegedly air-headed person); but hesitation and regret respectively precede and follow such gratification—the cringe of… should I? and the glum reflection of… should I have? Put another way: Although one can derive powerful satisfaction from prompting others’ rude awakenings, I think we know deep down that it’s not always the right thing to do, so long as these imaginations don’t injure us or make our own lives any less pleasant. If players of Mario Paint or Ocarina of Time or Rock Band tell you that they feel musical in certain moments, why not just let them be and extend congratulations? Finding musical pleasures in these games is unlikely to deter a player from pursuing training on traditional instruments; if anything, the opposite holds true. Surely sales of physical ocarinas (once frequently advertised in the magazine Nintendo Power) shot up after the Zelda namesake, and surely iOS music-making apps have inspired some players to learn instruments and take composition lessons.

302

MUSIC VIDEO GAMES

Video games, at large, can provide veritable lifelines when life gets hard. They offer a temporary means of escaping bullies at school, squabbling parents at home, and internal angst. The way I hear it, music video games grant a similar kind of sanctuary. If you’re getting low grades, receiving rejections from job applications, or having an overall bad day, a music game can turn things around, making you feel adequate and even stellar. At the risk of sounding cheesy, my point is that we all need a win sometimes—even if the win is just colorful scores on a screen and animated fans cheering us on. We depend on safety nets, whether it’s health insurance or a rainy-day fund or a reliable friend. If you’ve taken music lessons, perhaps you can recall how you felt cradled in the moments when your piano teacher played along with you (some octaves higher or lower, or on a separate piano)—the sense that even if you fumbled a few keys, there was someone else to keep the music going. Maybe these uplifting memories can inspire us to do well (and do good) by ensuring that others land safely too. Instead of policing musicality and musicianly identifications, what if we emphasized celebratory modes of imaginative play, even at the risk of sounding naïve, touchy-feely, or uncritical? In teaching courses on music video games, I find that students tend to ask the same sets of questions on a loop. Is Guitar Hero musical? Can the music be considered authentic and live? Are the skills transferable? Are the players delusional? Are they trying to pass as musicians (and failing to do so)? An inquiry that rarely gets air-time has to do with why any of these questions matter, and what might be gained by a momentary suspension of judgment (not judgment as in scholarly faculties, but judgment as in our habituated impulse to evaluate others according to their overt abilities or deficiencies). If real life is precarious and safety nets are scarce, we could do worse than to permit feelings of musicality to percolate, diversify, and transform with the times. Trolls and flame wars aside, strife won’t break out from an overabundance of musicians in the world. Concert halls won’t shut down. Given that affect has become trendy in academia, the first order of business is to rethink our priorities. Affectively and effectively, perhaps this means pushing past our instincts to invalidate and patrol others’ senses of artistry, and to refocus our energies on issues of care and support. In our race to sound better

AFTERWORD—TOADOFSKY’S MUSIC LESSONS

303

than one another (rhetorically, musically), we may leave too much in the dust—the values that make us good, if not great. Even when our melodies flounder, Toadofsky’s symphony carries us through. Its harmonies drive the tune forward and keep the player’s spirits high. Having spent seven years researching video game music, I admit I increasingly find myself wondering these days whether a different kind of ludomusicology is possible—a ludomusicology that bounces along feelings of musicality, pleasure, and imagination, rather than critical missions that get mired in agonism, definitional boundaries, and high scores. I can no longer recall the precise notes my brother and I chose each time we recomposed the fourth phrase for Toadofsky. I do recall the other sounds of these joyous moments: holding our breaths as the symphony played for the first time, and chortling whenever the game’s chord progression made our funky melodic creations into something… well, salvageable. In this gleeful hush and laughter, something musical came to pass.

Glossary of Music and Gaming Terms affordance A term coined by perceptual psychologist James J. Gibson to describe characteristics of an object that facilitates a specific use. For example, a doorknob is round and a size that fits into most hands, and is mounted on doors at a level that is easily reachable by most adults. These perceived design elements make its proper use apparent to the user—doorknobs are designed to be grabbed and turned when used to open a door. axiology A branch of philosophy concerned with value and is closely connected to ethics and aesthetics. cadence A series of chords (usually two) that occur at the end of a musical phrase. Particular chords are used in a cadence to give the phrase a sense of completion (authentic, plagal), to a make the phrase sound “unfinished” (half), or trick the listener with an unexpected ending (deceptive). concertmaster Usually the first violinist in an orchestra, the concertmaster is responsible for leading the tuning of the orchestra before performances and rehearsals, make decisions on the ways in which the violin parts should be bowed, and usually plays any violin solos within a piece played by the orchestra. concerto A musical work for soloist (or group of soloists) and orchestral accompaniment. core mechanics The basic actions a player performs while playing a game or how the player interacts with the rules of a game. For example, the core mechanic of a trivia game is to answer questions. The core mechanics of Street Fighter (Capcom 1987) are move (forward, backward, upstage, and downstage), punch, and kick. See also: mechanics. DAW An abbreviation for digital audio workstation; DAWs are hardware devices (such as digital mixing consoles) or software (such as Garage Band, Pro Tools) that facilitate the editing of audio files. diegetic/non-diegetic Diegetic refers to actions or sounds that are part of the on-screen story world. For example, sound effects caused

GLOSSARY OF MUSIC AND GAMING TERMS

305

by on-screen actions and music played on an on-screen radio are diegetic sounds. Non-diegetic (also called extra-diegetic) refers to actions or sounds that are not part of the on-screen story world and are experienced only by the audience. Voiceover narration and theme/ mood music (that is not played by instruments seen on-screen), and a game’s musical soundtrack are examples of non-diegetic sound. dynamics (games) The ways in which a player interacts with the interface of a game or the behaviors associated with playing a game. dynamics (music) Volume or changes in volume over time within a piece of music. endless runner A game wherein the character continuously “runs” to the end of the level or until the objective of the level has been completed. Endless runners are usually platformers and the player usually does not have control over the speed at which the avatar runs or the ability to make it stop running, leaving the player to avoid obstacles and enemies and to collect items. ludomusicology The academic study of music associated with video games and other forms of “play” (such as sports, live-action role playing games, etc.). Academic conferences on music and video games are held annually in the UK (and Western Europe) and North America. mechanics (game) A term that describes the “rules” by which a game operates. A game’s mechanics includes the actions a player can take during gameplay and the effect the actions have on the game overall. Mechanics also demarcate the way the game is “won” or “lost” (or how a player measures success in gameplay) and when the game is over. meter A term that describes the regular pattern of beats or pulses found in most music. MIDI An abbreviation for Musical Instrument Digital Interface —a protocol and file format that was standardized in 1983 to facilitate the transfer of digital musical information between computers, synthesizers, and other devices. A .midi file is the smallest audio file format. motif/leitmotif A musical idea used to symbolically represent a person, an emotion, a thing, or idea. music theory The academic study of the theoretical elements of music that is particularly concerned with the structural elements of music (such as form, pitch, rhythm, harmony, etc.) and aesthetics. musicology The academic study of music from a humanistic perspective. Historical musicology (the study of music history), ethnomusicology (the study of music as a cultural/social phenomenon), and systematic musicology (music psychology, music computing, etc.) are subdisciplines of musicology. non-player character (or non-playable character, or NPC) A character within a video game that is not controlled or controllable by the player.

306

MUSIC VIDEO GAMES

ocarina A small, potato-shaped vessel flute with a fipple and finger holes. octave The interval between two notes whose frequency is double or half that of the other note. The term also describes the set of 8 notes between these two notes that together comprise a major or minor scale. organology The academic study of musical instruments, their construction, their classification, and their use in various historical and ethnological contexts. pitch The relative highness or lowness of a tone. Pitch is measured in Hz and expressed physically as frequency; in music, pitch is usually assigned a letter name, A-G, and can be modified a half-step higher (with a sharp) or half-step lower (with a flat) when necessary. platform game (platformer) A video game genre wherein the player controls an avatar whose goal is to jump from one suspended platform to another, and/or to avoid a number of other obstacles in order to reach the end of a level. procedural rhetoric A term coined by Ian Bogost in “Persuasive Games: The Expressive Power of Video Games” (MIT Press, 2007) to describe the ways in which video games and other computer processes embody a set of social values and have a unique persuasive power that can be used to affect social change. quantization Part of the process of converting a continuous set of values into discrete values or integers. This process is especially important in the conversion of analog audio signals into binary code so that it can be processed digitally. See also: sample/sampling. rhythm The duration of tones in a piece of music. sample/sampling The part of the process of reducing an analog audio signal into discrete binary numbers for use in a digital system. A number of “recordings” (or samples) taken of a sound wave every second (44,100 is the standard number of recordings taken per second for CD-quality audio). Sampling can also refer to the compositional process of appropriating a portion from one recording of music for use in another. shredding A virtuosic style of playing for lead electric guitar, usually in rock or heavy metal music. telematic An adjective used to describe the use of telecommunication technology to facilitate live performances involving performers in more than one location. Second Life and other video games have been used for this purpose. tempo The speed of music. Tempo can be expressed in relative terms (fast, slow, walking pace, etc.) or in beats per minute (or bpm). theme (music) A term used to describe a prominent, recognizable melodic “idea” or “subject” in a piece of music.

GLOSSARY OF MUSIC AND GAMING TERMS

307

timbre The tone color of a sound. Timbre is a physical property of sound that results from the prominence of various overtones. velocity A MIDI term that describes how fast or forceful a key is pressed. Changes in velocity usually translates into varying degrees of volume; for example, the higher the velocity with which a key on a MIDI keyboard is pressed, the louder the resulting sound will be.

About the Contributors David Arditi (PhD, George Mason University) is Assistant Professor of Sociology at the University of Texas at Arlington. His research explores the relationship between music and technology, and the way that relationship affects music, culture, and society. He is the author of iTake-Over: The Recording Industry in the Digital Era (Rowman and Littlefield, 2015). Michael Austin (PhD, The University of Texas at Dallas) is Assistant Professor of Media, Journalism, and Film and Coordinator of the Interdisciplinary Studies Program in the School of Communications at Howard University, in Washington, D.C., where he teaches courses in audio engineering, music production, and sound design for TV, film, and video games. In addition to editing this volume, he is currently working on a monograph on representations of subalterity in music videos (under contract with Oxford University Press). Gianna Cassidy (PhD, Glasgow Caledonian University) is a music psychologist and Senior Lecturer in Psychology of Creative Technologies at Glasgow Caledonian University. Co-Investigator on the AHRC network, “Guitar Heroes in Music Education,” Gianna was the principle investigator on the European Physical Sciences Research Council grant, entitled: “Music Games: Supporting New Opportunities for Music Education.” Recent publications include “Music-Games: A case study of their impact” (Research Studies in Music Education) and “Music Games and Music Identities” (Handbook of Musical Identities).

ABOUT THE CONTRIBUTORS

309

William Cheng (PhD, Harvard University) is Assistant Professor at Dartmouth College. His books include Sound Play: Video Games and the Musical Imagination (Oxford University Press, 2014), Just Vibrations: The Purpose of Sounding Good (University of Michigan Press, 2016), Meritopia: Listening for Beauty and Injury in 21stCentury Life (Oxford University Press, forthcoming), and Queering the Field (co-edited with Gregory Barz, Oxford University Press, forthcoming). Mario A. Dozal (MA, The University of Texas at El Paso) is a PhD student in Communication at the University of New Mexico. His research focuses on media, pop culture, and professional wrestling. Nathan Fleshner (PhD, University of Rochester, Eastman School of Music) is Assistant Professor and Coordinator of Music Theory at Stephen F. Austin State University. Before coming to SFA, Dr. Fleshner taught at Ithaca College and was a member of the summer faculty at the Eastman School of Music. He has presented papers at regional, national, and international conferences, including the Society for Music Theory and the International Association for Analytical Psychology in Zurich, Switzerland, and has published in the Journal of Music Theory Pedagogy. His research interests include Schenkerian theory, the application of psychoanalysis to the study of music, connections between analysis and performance in cello music, and popular music. He is particularly interested in the portrayal of dreams and mental illness in music. Melanie Fritsch (MA, Freie Universität Berlin) has worked as research assistant at the Research Institute for Music Theatre, and has taught in the Music Theatre Studies department, both of which are located at the University of Bayreuth (BA/MA). Currently she is finishing her PhD thesis, Performing bytes. Musikperformances der Computerspielkultur (Performing bytes: Music Performances in Video Game Culture) in the research area of video games, performance, and music at the University of Bayreuth. She is also a member of the AHRC research network, “Guitar Heroes in Music Education? Musicbased video-games and their potential for musical and performative creativity.”

310

ABOUT THE CONTRIBUTORS

William Knoblauch (PhD, Ohio University) is an assistant professor of History at Finlandia University in Hancock, MI. He received his PhD in American History and was a fellow of the Contemporary History Institute in Athens, Ohio. His research examines the interplay between twentieth-century pop culture, politics, and foreign policy. He is currently working on a manuscript, Selling the Second Cold War, for the University of Massachusetts Press’ “Culture, Politics, and Cold War” series. Stephanie Lind (PhD, The University of British Columbia) is Assistant Professor of Music Theory and Analysis at Queen’s University School of Drama and Music in Kingston, Ontario. Her research focuses on Transformational Theory, Contemporary Canadian Art Music, Music Theory Pedagogy, and Video Game Music. She is particularly interested in harmonic process and formal structure in game music. Daniel O’Meara (MA, Princeton University) is a PhD student in Musicology at Princeton University. His research focuses on analytical approaches to listening and musical style in twentieth- and twentyfirst-century popular music. Anna Paisley (MRes, Glasgow Caledonian University) is a PhD researcher at Glasgow Caledonian University, having received an MRes in psychology and research methods from The University of Strathclyde. Anna was research assistant on the European Physical Sciences Research Council Grant, entitled: “Music Games: Supporting New Opportunities for Music Education”, and her doctoral work focuses on evaluating music interventions and the facilitative role of music technologies for health and well-being. Dana M. Plank (MM, Cleveland State University) is a PhD candidate in Historical Musicology at The Ohio State University. Her research interests include minimalist opera, the sacred music of Carlo Gesualdo, video game music studies, and music and disability studies. Her dissertation focuses on the aural representations of injury, disease, and mental illness in 8- and 16-bit soundscapes. David Roesner (PhD, University of Hildesheim) is Professor for Theater and Music Theater at LMU Munich. His research focuses on the broad spectrum of interplays between performance and music.

ABOUT THE CONTRIBUTORS

311

Recent publications include Composed Theatre (Intellect, 2012) and Musicality in Theatre (Ashgate, 2014). Since January 2014, he has been principal investigator of the research network “Guitar Heroes in Education?” funded by the AHRC. Peter Shultz (AB, Princeton University) is a PhD student in Music History and Theory at the University of Chicago and an instructor at Whitman College. His research focuses on the communicative roles of music in games, particularly in relation to senses of motion.

Author Index Aarden, Bret 295 Abbate, Carolyn 218 Abril, Carlos R. 198 Adorno, Theodor W. 183–4 Aglietta, Michel 191n. 3 Aktipis, C. Athena 172 Albrecht, Michael Mario 156, 158–60 Anderson, Kyle 170n. 5 Arar, Raphael 108 Arditi, David 18, 187–8, 297 Arsenault, Dominic 183 Attali, Jacques 178–81, 183–4, 186–90, 190n. 1 Auerbach, Brent 276–77 Auslander, Phillip 157–60, 170n. 8, 171n. 11, 198, 203, 205 Austin, Michael 219 Baer, Ralph 25–9, 31–3, 36–8 Babbitt, Milton 289 Barker, Hugh 143–44 Barton, Matt 37, 39n. 1 Baughman, James 28 Benjamin, Walter 190n. 1 Bennett, Lucy 60, 78n. 26 Berry, Wallace 102–3n. 14 Biamonte, Nicole 245n. 8, 294n. 2 Biancorosso, Giorgio 97–8 Black, Michaela 223 Blake, Harriet L. 33 Boden, Margaret A. 200, 202 Bogost, Ian 3, 6, 38 Bohm, David 222

Bostock, Mathonwy 295 Braasch, Jonas 182–3, 187 Bröckling, Ulrich 222 Brown, Mary 72–3, 81n. 47, 81n. 48 Brown, Steven 270n. 5 Bruske, Ed. 32, 39n. 6 Bryant, Gregory A. 272 Buchanan, Levi 162–3, 172n. 25 Burgess, Jean 58, 63, 77–8 Burnham, Van 26–7, 39n. 3 Butler, Mark J. 119–20 Byrne, Charles 208 Caillois, Roger 4–5 Call, Josh 21 Cano, Miguel 246n. 22 Caplin, William E. 94 Carlton, Lana 222, 224 Cassidy, Gianna G. 18, 198, 207, 209–11, 214–16, 219 Cerulo, Karen A. 212 Chang, Shao-Ju 272 Charles, Darryl 223 Clayton, Martine 270n. 5 Clements, Ryan 165 Collins, Karen 10, 12, 15–16, 84–5, 92–3, 98–9, 102n. 12 154, 170n. 2, 197–8, 201–2, 217, 221n. 32 Cook, Nicholas 157, 170n. 7, 204 Cooper, Grosvenor 281 Coppa, Francesca 53, 56, 76n. 15 Cowen, Nick 172n. 28 Cowley, Ben 208 Cranmer, Sue 224

AUTHOR INDEX Craven, Brent 272 Craven, Joan 35 Creech, Andrea 209, 215 Csikszentmihalyi, Mihaly 200, 208, 218n. 1, 246n. 20 Custodero, Lori 208–10 Dardis, Tom 130 Davies, Chris 220n. 28 DeNora, Tia 198, 205 Derksen, Craig 149 De Valck, Kristine 57 Dillon, Patrick 224 Dillon, Teresa 198, 206 Donnelly, K.J. 97 Downton, Michael 225 Dunn, Kevin 130 Dyer-Witheford, Nick 132 Edwards, Owen 38 Eguchi, Kei 271n. 15 Evans, Chris 149 Fabricatore, Carlo 111 Facer, Keri 224 Fenty, Sean 72, 80–1n. 46 Fernández-Vara, Clara 9 Fleshner, Nathan 19, 220n. 21, 294n. 1 Foglia, Lucia 271n. 17 Folkestad, Göran 206 Folmer, Eelke 270n. 4 Fowler, Charles B. 107 Fritsch, Melanie 13, 18, 170n. 1, 173n. 34, 197, 201, 219n. 14, 221n. 36 Frushtick, Russ 141 Fuchs, Christian 186 Furlong, John 220n. 28 Fürsich, Elfriede 129, 135, 147 Gadamer, Hans-Georg 3 Gault, Brent M. 198 Gee, James Paul 39n. 14, 207, 215 Gendle, Matthew H. 38

313

Gershman, Jacob 29, 31 Giannachi, Gabriella 203 Gibbons, William 20, 82 Gloor, Storm 131 Goffman, Erving 157–8, 171n. 10 Goldberg, Daniel 25 Goldberg, Marty 39n. 14 Gower, Lily 198 Graver, David 157, 171n. 9 Green, Hannah 210 Green, Lucy 206, 210 Greening, Chris 93 Groff, Jen S. 197, 207 Grossberg, Lawrence 131 Hall, Stuart 186 Hallam, Susan 209, 215 Halter, Ed 39n. 1f Hamalainen, Raija 215 Hargreaves, David J. 198, 206–7, 211, 213, 215, 220n. 27 Harmetz, Aljean 31–2, 39n. 7 Harris, Craig 270n. 3 Hatfield, Daemon 138 Hay, Kenneth 225 Hesmondhalgh, David 185–9 Hewett, Ivan 2 Hiatt, Brian 144 Hick, Darren Hudson 133 Hickey, Ray 223 Horkheimer, Max 184 Howells, Catherin 224 Huizinga, Johan 3–4 Humphreys, Peter 224 Hunicke, Robin 110, 112 Iwata, Satoru 254, 267, 270n. 3, 270n. 6, 270n. 7 Jackendoff, Ray 280, 289 Jackson, Michael 156 Jenkins, Henry 59–62, 77n. 24, 77–78n. 25 Johansson, Mats 156, 158–9, 171n. 11, 171n. 12

314

AUTHOR INDEX

Jordania, Joseph 271 Jourdain, Robert 280–1 Judkewitz, Benjamin 272 Juul, Jasper 5–6, 85, 100, 246n. 20 Kahne, Joseph 149 Kang, Irene 272 Kapralos, Bill 174, 222 Kapur, Ajay 107 Karpinski, Gary S. 279–80, 294 Kawase, Satoshi 271n. 15 Kaye, Nick 223 Keil, Charles 271 Khan, Rehan M. 272 Knight, Jerry 32 Knopper, Steve 181 Kohler, Chris 270n. 3 König, Ekkehard 203–5 Kozinets, Robert V. 54–5, 57–8, 76n. 18 Kramer, Josh 172n. 24, 172n. 25 Kreps, Daniel 229 Kringiel, Danny 173n. 34 Lamont, Alexandra 198, 206 LeBlanc, Mark 110, 112 Lenhart, Amanda 128 Lester Roushanzamir, Elli P. 149 Lerdahl, Fred 280, 289 Lerner, Neil 20, 82 Lévi-Strauss, Claude 8 Lewis, Monica 155 Liebe, Michael 85–6, 90, 102n. 7 Lindsay, Eric 225 Lindström, Berner 223 London, Justin 272 MacDonald, Raymond A. R. 206, 208 MacGill, Alexandra 149 MacHover, Tod 210 Malloch, Joseph 11 Manovich, Lev 110–11 Marcuse, Herbert 184

Marshall, Lee 188 Marshall, Nigel A. 224 Martine, Douglas 39n. 3 Marx, Karl 180 Maton, Karl 198, 206 McDowall, Janet 198 McFerran, Damien 41 McGonigal, Jane 38, 39n. 14 McLeod, Kembrew 131–2, 145 Meier, Leslie M. 132 Meyer, Leonard 281 Meyer, Stephen C. 96, 103n. 17 Middaugh, Ellen 149 Miell, Dorothy E. 206 Miller, Kiri 15–16, 133, 137, 182–4, 197–8, 205–7, 219n. 12, 219n. 16, 221n. 37, 229, 244n. 7 Missingham, Andrew 206 Monson, Ingrid T. 271n. 15 Montuori, Alfonso 225 Moore, Allan 131 Moormann, Peter 99, 103n. 17 Moseley, Roger 45, 133–4, 224n. 2 Munday, Rod 84, 95, 97, 99–100, 103n. 17 Mundhenke, Florian 92 Murphy, Sheila C. 212 Myers, David 7 Nelson, Noah 185 Neumeyer, David 84–5 Noë, Alva 268 O’Malley Greenburg, Zach 170n. 4 O’Neill, Susan A. 214–15 Paisley, Anna M. 18, 197–8, 207, 209–11, 214–16, 219n. 19, 220n. 21 Pareles, Jon 171n. 14 Patel, Aniruddh D. 270n. 5 Paul, Leonard J. 84, 93 Payne, Matthew Thomas 73 Peppler, Kylie 198, 214, 219n. 16

AUTHOR INDEX

315

Philippides, Alexis 285 Phillips-Silver, Jessica 256, 268 Porter, Jess 268 Purser, Ronald E. 225 Purves, Ross M. 224

Stam, Robert 8–9 Strötgen, Stefan 223 Sudnow, David 251–4, 256, 269 Summers, Tim 197, 226 Sweetser, Penelope 208–10, 216

Ransom, Michael R. 38 Reale, Steven Beverburg 10 Rebstock, Matthais 218n. 4 Rejack, Brian 132 Reynolds, Simon 245n. 16 Ribbens, Wannes 132 Robertson, Derek 207, 215 Robertson, Toni 226 Roesner, David 133, 198, 205, 214–15, 221n. 38, 246n. 20, 279 Rogers, Michael R. 280 Rudd, Tim 224

Tarrant, Mark 224 Taylor, Laurie 81–2, 248 Taylor, Yuval 244n. 2 Théberge, Paul 112, 177 Toffler, Alvin 180

Sager, Rebecca 271 Saiki, Aya 45 Salen, Katie 167–9 Salmans, Sandra 34 Saltz, David 10 Sandford, Richard 207 Settles, Gary 272 Sevik, Greg 130–1 Shanks, Michael 223 Sharman, Zena 132 Shultz, Peter 19, 230, 245n. 8, 246n. 20, 276 Siefert, Marsha 182 Silwinski, Alexander 140 Simonsen, Jesper 215 Sirois, André 134 Small, Christopher 85, 101n. 4, 198, 201, 204–5, 218n. 5 Smith, Fiona 72–73, 81n. 47, 81n. 48 Smith, Jacob 296 Smith, Steven 200n. 2 Smith, Wendy 129 Smythe, Dallas Walker 180 Sobel, Noam 272

van Elferen, Isabella 92–3, 98, 101, 101n. 5, 102n. 10, 103n. 18, 103n. 20, 173n. 34, 197, 217–18, 221n. 34 Veblen, Thorstein 129, 135, 144 Vitak, Jessica 149 Vogel, Joseph 171 Volpe, Jason 272 Voorhees, Gerald A. 7 Walton, Mark 165 Wasko, Janet 134–5 Watson, Mary Ann 28 Welch, Graham F. 224–5 Whalen, Samuel 208 Whalen, Zachary 81, 2, 84, 95, 99, 103n. 19, 248 Whitlock, Katie 21 Will, Udo 271 Williamson, Ben 207 Wilson, Robert A. 217n. 17 Wolf, Mark J.P. 7 Wright, Chapin 32 Wyeth, Peta 208–10, 216 Yuan, Bei 270 Zagal, José 173n. 34 Zimmerman, Eric 167–9, 173n. 34 Zito, Tom 32–3 Zoolander, Derek (pseudonym) 172n. 26 Zubek, Robert 11

Game Index Apples-to-Apples (Out Of The Box Publishing) 286 Bad Hotel (Lucky Frame) 201n. 3 Band Hero (Neversoft, et al.) 139, 202, 217 BandFuse: Rock Legends (Realta Entertainment Group) 9, 229 The Beatles Rock Band (Harmonix/Pi Studios) 139, 144, 146, 148n. 11 Blind Hero (Yuan and Folmer) 270n. 4 Bop-It (Hasboro) 37 Breakout (Atari, Inc.) 251, 253–4, 269, 269n. 1 Britney’s Dance Beat (KMetro Graphics)128 Brütal Legend (Double Fine) 153 Castlevania (Konami) 68, 79n. 37 Chiptune Runner (Evil Indie Games) 9, 13, 109, 114, 115–17, 120–1 Circadia (Simple Machine) 275, 281, 287–94, 295n. 6, 296n. 7 Computer Perfection (Lakeside Toy Co.) 35 Copycat (Tiger Electronics) 32 Cosmic DJ (GL33k) 114 Dance Central (Harmonix) 164, 217

Dance Dance Revolution [or DDR] (Konami) 9, 11–12, 14–15, 25, 36–7, 86, 109, 115, 120, 128, 181, 275–81, 287–8, 219n. 15 Dark Souls 300 Dataman (Texas Instruments) 35 DJ Hero (Activision) 128, 141 Duck Hunt (Nintendo) 74n. 2 50 Cent: Bulletproof (Genuine Games) 154 Final Fantasy (Square) 2 Final Fantasy VII (Square, et al.) 65, 68 Final Fight (Capcom) 235 Flagman (Nintendo) 39n. 8 Follow Me (Sears) 32, 39n. 8 Grand Theft Auto (DMA Design, et al.) 37 Groove Coaster (Matrix Software) 278 Guesstures (Hasbro) 286 Guitar Hero (Harmonix) 2, 10, 12, 15–16, 18, 25, 36–7, 66, 86, 100, 109, 112, 115, 120, 127–9, 1 32–47, 148n. 3, 181–6, 190, 206, 210, 214, 219n. 12, 229–32, 235–6, 243, 244n. 2, 244n. 7, 245n. 8, 245n. 11, 246n. 17, 247n. 32, 275–9, 281, 287–8, 294n. 2, 294n. 3, 300–2 Guitar Hero 2 (Harmonix) 2, 136–8

GAME INDEX Guitar Hero III: Legends of Rock (Neversoft, et al.) 136–8, 229 Guitar Hero 5 (Neversoft, et al.) 139, 144 Guitar Hero: Aerosmith (Neversoft, et al.) 141 Guitar Hero Live (Activision) 148n. 4 Guitar Hero: Metallica (Neversoft, et al.) 139 Guitar Hero: Smash Hits (Beenox) 139 Guitar Hero: Van Halen (Neversoft, et al.) 139 Guitar Hero World Tour (Neversoft, et al.) 143, 148n. 11, 201–2, 219n. 13 Half-Life 2 (TValve Corp.) 132 Incredibox (So Far So Good) 13 Isle of Tune (Happylander Ltd.) 114 Journey Escape (Data Age) 153 Jurassic Park (Ocean Software) 74n. 1 Karaoke Revolution (Blitz Games) 12, 25 Legend of Zelda: A Link to the Past (Nintendo EAD) 92, 101n. 1 Legend of Zelda: Ocarina of Time (Nintendo EAD) 13, 17, 83–105, 301 Legend of Zelda: Skyward Sword (Nintendo EAD Group 3) 97, 101, 102n. 6 Legend of Zelda: The Wind Waker (Nintendo EAD) 96, 102n. 6, Lego Rock Band (Harmonix, et al.) 139, 143

317

Little Professor (Texas Instruments) 135 Loopz (Mindscape) 37 The Lord of the Rings Online [or LOTR Online] (Turbine) 201, 219, 301 Maniac (Ideal Toy Company) 32–3, 39n. 9 Mario Paint (Nintendo R&D) 43–50, 52, 61–2, 72–3, 74n. 1, 74n. 4, 74n. 5, 74n. 6, 81n. 47, 301 Mario Paint Composer (Unfungames.com) 13, 44, 49–82 Merlin (Parker Brothers) 32, 35 Michael Jackson—The Experience (Ubisoft) 154, 164–166, 169 Michael Jackson’s Moonwalker (Sega) 153–4, 161–4, 166, 168–9, 172n. 24, 173n. 34 Minna no Rhythm Tengoku (Nintendo SPD) 254, 256–8, 260, 263 The Miracle Piano Teaching System (Software Toolworks) 26 Music Inc. (UK Music) 13 Musyck (FastFan) 202 My Singing Monsters (Big Blue Bubble, Inc.) 109, 114, 117–20 Nintendogs (Nintendo EAD Group 1) 114 PaRappa the Rapper (Nana On-Sha) 109, 128, 218n. 5 Pictionary (Angel Games, Inc.) 286 Planet Michael (Digital Studios AB) 170n. 3

318

GAME INDEX

Pocket SIMON (Milton Bradley) 33–4, 36 Pong (Atari Inc.) 27–8, 269n. 1 Pugs Luv Beats (Lucky Frame) 201 Quake III Arena (id Software) 173n. 34 Rayman Legends (Ubisoft Montpellier) 7, 114 Rayman Orgins (Ubisoft Montpellier) 114 Rhythm Heaven (Nintendo SPD/ TNX Music Recordings) 19, 253–271 Rhythm Tengoku (Nintendo SPD) 254–5, 258, 260, 263–64, 265–6, 271n. 12 Rhythm Tengoku Gold (Nintendo SPD/TNX Music Recordings) 255 Rock Band (Harmonix, et al.) 2, 11–12, 15–16, 18, 25, 37, 109, 115, 120, 128–9, 132–47, 148n. 3, 148n. 10, 181–7, 190, 191n. 3, 206, 212, 214, 229–32, 235, 244n. 2, 244n. 7, 245n. 8, 245n. 11, 294n. 3, 300–1 Rock Band 2 (Harmonix/Pi Studios) 142, 146 Rock Band 3 (Harmonix/ Backbone Entertainment) 142, 146–7, 185, 209–11, 213, 219n. 18 Rock Band 4 (Harmonix/Mad Catz) 148n. 4 Rock Band Blitz (Harmonix) 134, Rock Band Classic Rock Track Pack (Harmonix) 139 Rock Band Country Track Pack (Harmonix/Demiurge Studios) 139

Rock Band Metal Track Pack (Harmonix/Demiurge Studios) 139 Rock Band Track Pack, Vol. 2 (Harmonix/Pi Studios) 139 Rock Revolution (Zöe Mode/HB Studios)138 Rocksmith (Ubisoft San Francisco) 9, 19, 201, 215, 217, 220n. 29, 229–247 Rocksmith 2014 (Ubisoft) 230–5, 237, 240, 243, 244n. 3, 244n. 5, 244n. 7, 245n. 9, 246n. 17, 245n. 18, 245n. 26, 247n. 32 SIMON (Milton Bradley) 17, 25–6, 28–39 SIMON Flash (Hasboro) 36 SIMON Swipe (Hasboro) 36 SIMON Trickster (Milton Bradley) 36 SingStar (SCE London Studio) 12, 201 Sentris (Timbre Interactive) 114 SongPop (FreshPlanet) 13 Sonic the Hedgehog (Sega) 2, 65 Sonic the Hedgehog 3 (Sega Team) 170n. 3 SoulCaliber (Project Soul) 221n. 36 Sound Shapes (Queasy Games/ SCE Santa Monica Studios) 12, 201–2, 217, 219n. 10 Soundrop (Develoe, LLC) 275, 281–7, 293–4 Space Channel 5 (United Game Artists) 170n. 3 Space Channel 5: Part 2 (United Game Artists) 170n. 3 Space Invaders (Tatio) 145 Spacewar! (Steve Russell, et al.) 27 Street Fighter (Capcom) 15 Super Mario Bros. (Nintendo R&D Group 4) 37, 93, 112, 145, 201

GAME INDEX Super Mario RPG: Legend of the Seven Stars (Nintendo/ Square) 297–303 Tamagotchi (Bandai) 114, 117 Tennis for Two (William Higinbotham) 27 Tetris (Alexey Pajitnov, Nintendo Research & Development 1, et al.) 2, 111, 119–20 Tone Sphere (Sta Kousin) 278 TouchMix FX (GAMEVIL Inc.) 12

319

Wave Trip (Lucky Frame) 114 Wii Music (Nintendo EAD Group 2) 25, 45 Wii Tennis (Nintendo) 12 Wild Gunman (Nintendo R&G Group 1) 75n. 2 World Class Track Meet (Bandai Co., Ltd.) 74n. 2 World of Warcraft (Blizzard Entertainment) 2

General Index “ABC” (song) 139 acoustics 288 Aerosmith (band) 139, 141–2 aesthetics 110, 112–15, 133, 141, 217 affordances 77n. 23, 109, 112, 117, 121, 269, 297 “Africa” (song) 56, 77n. 22 Aimbot 300–1 Apple Computer 34 Arctic Monkeys (band) 241–2 Arts and Humanities Research Council (UK) 197 “Asphalt Cocktail” (song) 69–70 Atari 27–8, 32–33, 36–7, 109, 251 audience 1, 8–10, 14–15, 18, 55, 59, 61, 63, 66, 77n. 23, 84, 109, 119, 129–30, 133, 135, 138, 158–60, 170n. 8, 171n. 15, 180, 187, 203, 209 authenticity 16, 18, 20n. 2, 52, 128–129, 131–5, 144–5, 147, 158, 182, 229, 244n. 2, 301 “dimensions of authenticity” 131–2 first person authenticity 131 second person authenticity 131 third person authenticity 131 Auto-Tune 79n. 38, 300 avatar 7, 16, 80–1n. 46, 85, 92–3, 99–100, 114, 136, 143, 146, 164–5, 169–70, 173n. 34, 203, 212, 217, 219n. 7, 219n. 13

Baer, Ralph 25–29, 31–8, 39n. 1, 39n. 2, 39n. 3, 39n. 4 Barker, Travis 143 “Battle Theme from Paper Mario” (song) 67 Beach Boys (band) 68 “Beat It” (song) 153, 161 Beethoven, Ludwig von 190n. 1 Bellamy, Matt 144 Bemani games 276–77, 281–2, 277, 287, 293–4 “Billie Jean” (song) 161 biologism 9 Black Entertainment Television (BET) 148n. 6 “Blitzkrieg Bop” (song) 236 Blue Öyster Cult (band) 16 Blur (band) 236 “Bohemian Rhapsody” (song) 56, 64 Bowie, David 211 cadence 94, 103n. 15, 299 “Call Me Maybe” (song) 68, 79–80n. 39 carillon 107 Cash, Johnny 144 cell phones 181 “Charlene (I’m Right Behind You)” (song) 137 cheating 216–17, 301 Chomsky, Noam 204, 280 Close Encounters of the Third Kind (film) 31 Cobain, Kurt 144

GENERAL INDEX cognitive blindness 268 cognitive psychology 271n. 17, 280 Coleco 27, 33 Comedy Central 137, 148n. 6 see also South Park; Stephen and the Colberts (band) commodification 129, 135, 147, 178, 181, 190 composition 13, 17, 44, 46, 49, 51, 52–3, 55, 57, 59, 62, 66, 68–9, 75n. 8, 78n. 27, 95, 107, 109, 114, 119, 178, 179–87, 189, 190, 190n. 1, 208, 211, 280, 282, 284–7, 298, 300–1 conceptual spaces 200 console 11–12, 25, 27, 29, 32–4, 36–8, 39n. 2, 44, 109, 111, 128–9, 134, 136, 138–40, 148n. 1, 154, 161, 171n. 17, 172–3n. 29, 185, 217, 221n. 33, 221n. 35, 234 arcade consoles 109, 148n. 1, 148n. 5 Atari “Touch Me” cabinet 28–9, 32, 36 Atari Video Computer System 28 BBC Micro 33 “Brown Box” 27 ColecoVision 33 Commodore 64 33, 70, 221n. 33 Fairchild Channel F 33 Game Boy 46, 48, 66–7, 68, 221n. 33 “game box” 27 Mattel Intellivision 33 Microsoft Xbox 360 102n. 12, 134, 136, 138, 172–3n. 29, 234, 244n. 3, 244n. 5, 245n. 15 Nintendo Entertainment System

321

(NES) 37, 43, 74n. 2, 109 Nintendo 64 83, 86 Odyssey 27–8, 36, 39n. 2 Odyssey 2 28 PlayStation 37, 178 PlayStation 2 134, 136, 138, 140 PlayStation 3 134, 136, 140, 172–3n. 29, 173n. 31 PlayStation 4 191n. 3 Sega Genesis System 37 Sega Saturn 37 Sinclare ZX81 33 Super Nintendo Entertainment System (SNES) 37, 43 Texas Instruments TI/99 33 Xbox 360 Kinect 102n. 12, 164–5, 168 Wii 75, 102n. 12, 134, 172–3n. 29, 173n. 31, 255, 270n. 10 console sounds 29, 221n. 35 conspicuous consumption 34, 129, 135, 142, 144 controller 11–12, 44, 49, 74n. 1, 85–6, 92–3, 98–9, 102n. 6, 102n. 12, 119–20, 138, 140–2, 145–6, 183–4, 203, 217, 219, 276, 278 Aerosmith guitar controller 141 and “kinesthetic actions” 102n. 12 AXE Body Spray guitar 136 The Beatles: Rock Band Limited Edition Bundle 141 dance pad 12, 36, 276–7 firearms 6, 27, 74n. 2, 145 guitar controller 1–2, 11–12, 36–7, 86, 112, 127–8, 138, 140–2, 145–6, 148n. 1, 181, 183–4, 203, 219n. 6, 221n. 33, 230, 245n. 11, 276, 278, 294n. 2 gyroscope 285 Hofner bass controller 141

322

GENERAL INDEX

keytar 142 light gun 27, 74n. 2 liquid crystal displays (LCDs) 33, 148n 5 NES Power Glove 74n. 2 NES Power Pad 36, 74n. 2 NES Zapper 74n. 2 Pro Guitar SNES Mouse 45–6, 49, 74n. 1, 74n. 2, 74n. 4, 75n. 9 “Step Zone” 14 strum bar 1, 127–8, 183 touchscreen 36, 255 turntable control peripheral 128, 141 microphone 12, 119, 140–1 Wii controller 12, 85, 102n. 12, 221n. 33 wireless drum kit 140 copyright 28, 188, 234 counterpoint 281 creativity 18, 58, 188, 197–221, 279 Cruz, Taio 67 Daft Punk (band) 63–4, 67, 79n. 31, 220n. 31 “Dangerous” (album) 171n. 13 Death Magnetic (album) 133 Dickspeed 51, 53–4, 60, 69–70, 75n. 14, 78n. 30 diegesis 98, diegetic music 83–4, 86, 88, 91–5, 97–100, 101, 101n. 5 half-diegetic 101n. 5 linear non-diegetic game music 86 non-diegetic music 7, 83–4, 86, 90–5, 97, 100–1, 101n. 5 non-self-produced sounds 85 “over world” music 11 proactive diegetic music 86, 90, 99 reactive diegetic music 86, 90–1, 99

self-produced sounds 85 super-diegetic 217 supradiegetic 101 Digidesign 177 digital audio workstations (DAWs) 120, 177–8 Logic Pro 108 Garage Band 108 Ableton Live 109 disc jockey (DJ) 119, 121n. 2, 141, 201 Dolby, Thomas 34 “(Don’t Fear) The Reaper” (song) 16 Douglas, Carl 139 downloadable songs 137, 139, 146, 246n. 18 Dragonforce (band) 64, 66 Dylan, Bob 131 dynamics (game) 2, 110–15 dynamics (music) 44, 50, 67, 70 “Dynamite” (song) 67 Eagles, The (band) 142 ecological theory [model] of entertainment 256, 268 The Ed Sullivan Show 141 Edgar Winter Group (band) 127 engagement 56, 61, 71, 77n. 23, 84, 92, 95, 98, 100, 103n. 20, 198–9, 206–7, 209, 214, 297 Ensler, Eve 129 equal temperament 290 Ernie Ball strings 136, 234–5 “Every Breath You Take” (song) 242 Facebook 19n. 1 fandom 44, 52–3, 58–63 fan labor 17, 20, 44, 53, 58–9, 61–3, 71, 77n. 24 FAQs 66, 75n. 7 Faulkner, William 130 Feshareki, Shiva 2

GENERAL INDEX 50 Cent 153 filk songs 15, 20n. 2 film studies 8, 167 Fisher Price 43–4 Fitzgerald, F. Scott 130 flow 93, 95, 208–11, 216, 246n. 20, 271n. 12 form (music) 44, 50, 53, 94–5, 103n. 15, 183, 205, 246n. 25 “Frankenstein” (song) 127 freeware 44, 49, 51 “Funky Town” (song) 68 game theory 197 Game & Watch series 33, 45, game-based learning 207–8, 214–15, 255, 264, 275–95 Gamer Symphony Orchestra 2 Garbage (band) 144 Garrido, Nestor 26 gatekeepers 200, 202, 205, 218n. 2 gender 159, 160, 199, 204 Generative Theory of Tonal Music, A 280 prolongational reduction 280 genre (game) 2, 7–14, 100, 102n. 9, 102n. 13, 109, 114, 182, 199, 234, 247n. 32, 259, 275, 282 artist-matching games 13 brain games 25, 35, 38 endless runners 2, 9, 114, 116 learning games 4, 35, 100, 199, 207, 214–15, 287 metonymic music games 12–13 mixing games 12–13 pitch-matching games 12, 164, 279 puzzle games 2, 34, 115–16, 128, 134, 140, 142, 146, 287–91, 294 racing games 2, 201, 219n. 7

323

rhythm games 2, 5, 11, 127–8, 137–8, 144, 146–7, 181–7, 251–271, 275–281 rhythm-matching games 12 role-playing games 15–16 sandbox games 13, 37, 45 serious games 4, 9, 13, 18, 198, 216, simulation games 9, 114, 117, 132, 114 social games 19n. 1 title-matching games 13 see also Bemani games genre (music) 131, 159, 171n. 15, 217 bitpop 217 chiptunes 15, 217 classical 1, 44, 61, 94, 171n. 13, 300 8-bit music 217, 220n. 31 electro-pop 34 funk 171n. 13 gospel 171n. 13 hip hop 131–2, 145, 153, 171n. 13 industrial 171n. 13 jazz 5, 200, 251, 284 metal 66, 128, 153 New Jack Swing 171n. 13 pop 13, 61, 63, 67–8, 153–5, 158–9, 171n. 14, 182, 264 punk 130 R&B 171n. 13 rock 16, 61, 127–8, 131, 133, 134–40, 143–7, 153, 158, 171n. 13, 171n. 14, 182, 201, 219n. 12, 219n. 13, 221n. 33, 230, 244n. 2, 259n. 32 Gestalt psychology 289 “Get Lucky” (song) 63–4, 67, 77n. 25, 71n. 39, 220n. 31 GHTunes 146 Gibson Guitar Corp. 36, 136 Green Day (band) 143

324

GENERAL INDEX

groove 107, 253, 259–67, 271n. 12, 271n. 14, 271n. 16 Guitarcade 230–1, 234–5, 243, 244n. 6, 246n. 17, 247n. 32 Guns N’ Roses (band) 143

Internet & Communication Technologies (ICTs) 181–2 iPad 36, 172–3n. 29, 275–95 Iron Maiden (band) 236 “Islands” (song) 240–2

Halford, Rob 153 Hammett, Dashiell 130 handheld games 17, 25, 28–35, 37–8, 117, 154, 172–3n. 29, 294 harmony 4, 67, 74n. 3, 118, 204, 275, 281, 300 chord progression 299, 303 harmonic progression 300 Harrison, George 141 Hendrix, Jimi 144 Higinbotham, William 27 Hornbostel-Sachs system 7, 12 hydraulis 107

Jackson 5 (band) 135 Jackson, Michael 13, 18, 56, 64, 67, 76n. 21, 77n. 25, 153–73, and controversy 155 death 154 moonwalk 154, 162 musical persona 18, 156–61 Neverland Ranch 155 JackTrip 190 jamming 184–5, 188–9, 301 Jephsen, Carly Rae 68 Journey (band) 153

“I Get Around” (song) 68 “I Wanna Be Sedated” (song) 127 immersion 3, 17, 83–5, 93–5, 98–100, 116, 167, 208 see also kinesthetic immersion improvisation 120, 287 instrument 1, 5–12, 14–18, 36, 46–50, 66–70, 75n. 8, 75n. 13, 83–5, 93, 99–100, 107–9, 112, 117–20, 128, 133–4, 136, 140–3, 145–7, 179, 182–5, 187, 202, 209, 212, 217, 219n. 18, 221n. 33, 221n. 37, 229–30, 232, 234, 253, 276, 282, 285, 301 interactivity 7, 10, 12, 107 interface 10–12, 14, 26–7, 37, 43, 45, 84–6, 93, 95, 97, 99, 107, 111–14, 120, 183, 185, 210, 232, 277

karaoke 164, 276 see also singing; voice Kilmister, Lemmy 153 kinesonic congruence 99, 102n. 12 kinesonic synchresis 11 kinesthetic actions 92–3, 102n. 12, 252, 264 kinesthetic immersion 100 KISS (band) 128 Korg (Keio Organ Company) 108 “Kung Fu Fighting” (song) 139 Lady Gaga 67, 79n. 35, 137 Lakeside Toy Company 35 learner-generated experiences 215 Led Zeppelin (band) 131 leitmotif 84, 95–98 Lennon, John 141, 143 Link (game character) 86–8, 90–2, 96–8, 297 Lipps Inc. (band) 68 Little Caesar’s Brain Teaser 35

GENERAL INDEX loop-machine 221n. 33 Low A Glitch 51, 60, 75n. 13 ludomusicology 297, 303 ludus 5 Mackey, John 69–70, 80n. 44, 80n. 45 McCartney, Paul 141 Magnavox 27–8, 39n. 2 magnetic tape 108 Manson, Shirley 144 marimba 282 Mario (game character) 4, 11, 46, 48, 67, 297 Mario Sequencer 74–5n. 6 “Mario Theme” (song) 220 Marshall Amplification 232, 235 Marvin Glass & Associates 27, 29 Massachusetts Institute of Technology (MIT) 26–7, 34, 210 Mattel 31, 33 mechanics (game) 7, 9, 13, 17–19, 35, 51, 71, 109–16, 120, 208–9, 232, 235, 239, 244, 245n. 10 Mechanics, Dynamics, Aesthetics (MDA) 109–119 melody 3, 13, 32, 47, 49, 68, 86, 90–92, 94, 96–9, 102n. 11, 102n. 14, 103n. 15, 103n. 16, 118, 204, 251–2, 275, 281, 298–300 Metallica (band) 65, 133, 142–3 meter 47, 49–50, 90, 103n. 15, 237, 240, 289 metric alignment 255 Microsoft 34 Microsoft Paint 43, 46 Miller, Arthur 129 Milton Bradley 25, 29–33, 35–6, 38, 39n. 5 Miyamoto, Shigeru 45 mode (music) 97, 102n. 11

325

Gregorian chant 90, 96, 102n. 11 pentatonic scale 284 scales 47, 236, 284 modular synthesizer 108 Moog 960 108 Moog, Robert 108 Morello, Tom 143 movement 5, 10, 111, 115, 117, 173n. 31, 173n. 33, 200, 239, 242, 252, 256, 265–9 Muse (band) 144, 238–9, 246n. 23 music education 145, 197–221 Music Minus One 184–5 music psychology 197, 207, 280 Music Television (MTV) 136, 141, 148n. 6 music theory 19, 276, 282, 293, 294n. 1, 298 music-notation software 46, 49 Musical Instrument Digital Interface (MIDI) 37, 75n. 10, 108, 113, 146, 161, 163, 185, 221n. 33 musicality 2, 69, 179, 213, 297, 301–3 musicianship 147, 182, 212, 220n. 28, 229, 301 musicking 85, 133, 201, 205, 217–18 musicology 201, 218 see also ludomusicology Nails, Judy (game character) 230 Napalm, Jonny (game character) 230 narrative 4, 7, 45, 60, 73, 84, 95, 101, 112, 114, 167, 172n. 18 ludology vs. narratology debate 6 National Medal of Technology (US) 38 National Symphony Orchestra (US) 2

326

GENERAL INDEX

Nintendo 33, 36–7, 39n. 8, 43, 45, 62, 74n. 2, 83, 93, 163, 253–5 Nintendo Network 178, 185 Nirvana (band) 144 normativism 8, 60, 102–3n. 14 non-player character (NPC) 86, 88, 102n. 8 nostalgia 44, 49, 56, 61, 71–3, 80n. 46, 188, 245n. 16 notation 46, 51, 57, 71, 73, 89, 100, 183, 214, 232, 234, 247n. 33, 279, 287 color-coded notation 279 directionally-coded notation 279 scrolling notation 209–10 sheet music 57, 100, 181, 183 Nugent, Ted 143 ocarina 13, 83–103, 297 online music environments 178, 183, 185, 191n. 2, 191n. 3 organology 6–8 orchestration 70, 281 Osbourne, Ozzy 143, 153 Page, Jimmy 145, 147 paidia 5 Paramount Pictures 148n. 6 Parker Brothers 32 participation 50, 55–7, 59–60, 62, 72, 77n. 23, 77–8n. 25, 78n. 27, 92, 99, 198–99, 206–12, 214–16, 220n. 28 devotees 52, 57 haters 66 insiders 57 lurkers 54, 57, 77n. 23 participatory culture 17, 44, 53, 58–81 participatory design 215 “site of participation” 99 trolls 66, 302 see also fandom, fan labor

pedagogy 18–19, 207, 231–5, 276, 294n. 1, 297 peer-assisted learning 215 performance 1–2, 10–11, 14–18, 101n. 4, 107, 109, 112, 119–121, 129, 131–3, 137, 140, 145–7, 154–60, 165, 171n. 15, 172n. 23, 173n. 30, 178, 181–5, 187–9, 190n. 1, 191n. 2, 198, 201–5, 209, 211, 213–14, 219n. 12, 221n. 33, 235, 243, 244n. 7, 253, 271n. 15, 276–7, 280 performance studies 157 performativity schizophonic performance schizophrenic performance telematic performance 181 perspective 133, 159, 203, 205, 219n. 14 pianola 107–8 Pilgrim in the Microworld 251 pinball 15, 109 pitch 12–13, 43, 46–51, 71, 80n. 43, 85, 109, 112, 114, 118, 164–5, 210, 232, 275, 277–95, 298–300 pitch-events 280 interval 102n. 11, 289–93 intervallic identification 293 play 2–5 player piano 108 PlayStation Network 178, 185, 191n. 3 “Plug In Baby” (song) 238–9 “Poker Face” (song) 67, 79n. 35, 137 Police (band) 242, 247n. 31 Presley, Lisa Marie 155 procedural rhetoric 18 proprioception 268 prosumer 180 puzzles, see puzzle games

GENERAL INDEX quantization 113, 300 Queen (band) 56, 143, 201

327

“R U Mine?” (song) 241–2 race 158, 199 Rage Against the Machine (band) 143 Ramones (band) 127, 236 razz tone 31 remixes 52 Requiem (W.A. Mozart) 68, 80n. 41 rhythm 2–3, 7, 12, 19, 32, 44, 47, 50, 60, 66–7, 71, 118, 154, 164–5, 202, 204, 210, 231–2, 235–243, 246n. 25, 247n. 31, 251–71, 275–94, 297–300 back beats 69 body beats 252, 254, 256 cross-rhythms 69, 263 duration 112, 259, 281, 289, 294n. 4 “rhythm sense” 19, 253–71 rhythmic pulse 237, 293 rhythmic subdivisions 44, 50, 243, 289, 293 rickrolling 64, 79n. 38 Rolling Stone Magazine 218n. 2 rules 2, 4–7, 9, 13, 55, 72, 76n. 17, 100, 111, 121, 169, 173n. 34, 204, 216, 286, 297 see also mechanics

singing 118, 141, 165, 168, 171n. 12, 182, 203, 280, 284, 300 see also karaoke; voice Slash 143 smartphone apps 19n. 1, 294 “Smooth Criminal” (song) 154, 161 solfège 284 “Song 2” (song) 239–40 South Park (animated series) 137 Cartman, Eric 137 “Guitar Queer-o” (episode) 137 “Timmy and the Lords of the Underworld” (song) 137 “Space Oddity” (song) 211 Star Trek 167 Starr, Ringo 141 Steel, Axel (game character) 230 Stephen and the Colberts (band) 137 “Still Alive” (song) 64, 77n. 25 Sting 143, 219n. 13 subculture 51–7, 62–3, 88 Mario Paint Hangout 51–5, 57–59, 73, 75n. 7, 75n. 11, 76n. 16 MPC community 44, 50–63, 67–69, 71–73, 77n. 23, 77n. 25, 78n. 27, 81n. 47 virtual communities 53–4, 63 see also filk songs Sudnow, David 251–4, 256 . 269

sampling 17, 113 Scott, Raymond 108 sequencer 44, 74–5n. 6, 107–21 Faderfox Micromodul LV1 119 step sequencer 9, 12 video games as 13, 107–21 Shafer, R. Murray 16 sightreading 183, 276, 277 Simmons, Gene 128 SIMON 17, 25–39, 109, “Simon Says” 29, 31, 287

talent 69, 71, 130, 159, 184, 214 Tamagotchi 114, 117 Tchaikovsky, Pyotr Ilyich 298 telematic music, see performance tempo 14, 44, 47–51, 60, 66–7, 69–70, 74n. 4, 75n. 8, 75–6n. 14, 78n. 30, 80n. 43, 108, 113–14, 117–18, 172, 257, 289, 291, 293 tetriminos 111

328

GENERAL INDEX

Texas Instruments (TI) 29, 33, 35 “Those Chosen by the Planet” (song) 65, 67 “Thriller” (album) 153–4 “Thriller” (film) 166 “Thriller” (song) 56, 64, 67, 77n 25 “Through the Fire and Flames” (song) 64, 66, 78n. 29 Tiger Electronics 32 timbre 46, 49–50, 59–60, 67–68, 70, 109, 113–15, 171, 281–2 Toadofsky (game character) 297–303 Toto (band) 56, 77n. 22 “Trogdor” (song) 149 “The Trooper” (song) 236 Tropic Thunder (film) 136 Tsunku ♂ 254, 256, 269n. 2 Ubisoft 154, 215, 233, 246n. 26, 246–7n. 27 UnFun Games 49 The Vagina Monologues 129 “Vampire Killer” (song) 68, 79n. 37 Van Halen (band) 142 Vans Warped Tour 136 velocity 12, 75n. 10, 113 Viacom 136–7, 148n. 6

Video Games Live 2 virtual musical participation see telematic performance virtual self 212 virtuosity 15, 57, 66, 71, 297 voice 54, 60, 72, 90, 102n. 11, 153, 159–60, 168, 170n. 3, 171n. 12, 247n. 32, 257–8, 261, 299 see also karaoke; singing Votey, E.S. 107 Wall-of-Sound 108 Ways of the Hand 251 “What Does the Fox Say” (song) 68, 92n. 42 White, Jack 145 Williams, Hayley 143 Wong, Freddie 2 Wurlitzer Sideman 108 Xbox Live 178, 185 XX (band) 240–1, Xylophone 43–4, 46, 48, 67, 282 Ylvis (band) 68, 80n. 42 Yokoi, Gunpei 33, 36 YouTube 2, 13, 17, 44–45, 52–61, 63–71, 77n. 25, 184–5, 203, 218n. 2, 243, 244n. 7