Mobile Storytelling in an Age of Smartphones 3030872467, 9783030872465

Table of contents :
Contents
List of Contributors
List of Figures
List of Tables
1 Mobile Storytelling in an Age of Smartphones
References
Part I Mobile Storytelling: Imperative Concepts and Innovative Cases
2 Mobile Storytelling: Changes, Challenges and Chances
Mobile Screen Stories: Encompassing Transmedia and Cinema Screens
Mobile Social Media Stories
Mobile Montage
Mobile Stories
Towards Mobile Storytelling
References
3 Mobile Storytelling: Elements, Evolution, and Experience
Integrating Storytelling Elements
Traditional and Innovative
Reasons and Approaches
Leveraging Storytelling Evolution
From Multimedia to Omnimedia
From Following Stories to Being Followed by Stories
From Being Told to Being Experienced
Maximizing Storytelling Experience
Components of Mobile Experience
Factors Shaping Mobile Experience
Maximizing Mobile Experience
Conclusion
References
4 Mapping Through Mobile Filming: An iPhone Mapper’s Flirtation with Google’s Artificial Intelligence (AI)
Mapping “The Self” with Google AI
Mapping as Everyday Public Activity
Methodology
Mapping with Mobile Phone (iPhone SE)
Google Maps and AI Art
International Dimensions of Territory
Conclusion
References
5 Shoes and Taxis as Mobile Storytelling Tools: Stories from the Frontline About Immigration and Integration in Canada
Taxi Stories
The Shoe Project
The Shoe Project Shorts
Conclusion
Part II Mobile News Storytelling
6 Multimodal Framing COVID-19 Stories: The Case of Douyin
News Framing COVID-19
Multimodal News Framing Devices
Research Questions
Method
Results
Contributors of Popular Short Videos
Attractive News Frames
User Preferred Storytelling Patterns
Public Figures as Narrators
Frequent Framing Devices (Technique Ones)
Conclusion
References
7 Twittering China’s Image: The Case of Xinhua News Agency
Framing China Images on Twitter
Research Methods
Tweets Collected Via Mozdeh
Sentiment Research on Twitter
Textual Analysis and Thematic Analysis
Findings and Analysis
Framing China Image on Twitter
Nation Branding in Hashtag Analysis
Impression Management in Retweet Analysis
Sentiment Analysis Results
Strong Emotions
Discussion and Conclusion
References
8 Citizen News Going Mobile: Opportunities and Challenges
Citizen News and Citizen Journalism
Citizen News and Media Convergence
Citizen News Going Mobile
1818 Golden Eye
Ningbo Connection News APP
Conclusion
References
Part III Vulnerable Communities and Social Innovation
9 Mobile Storytelling with Seniors: Co-designing Smartphone Productions and Mobile Stories with Vulnerable Communities
Co-designing Mobile Stories
Mobile Storytelling with Older People: Reflective Case Studies
The Opera Project: Digital Storytelling for Social Impact
Conclusion: A Storytelling Framework for Community Engagement
References
10 Engagement with Mhealth: Experiences of Young Women in Bangladesh
Adaptation and Use of Mhealth Services
Method
Access to Health Information and Health Care Professionals
Use of Phone Calls, Sms and Websites
Use of Social Media
Use of Mobile Apps
Conclusion
References
11 Mobile and Digital Stories from Down Under: Tales from Rural Australia
The Context: Yarriambiack Shire
Background to the Digital Stories
How the Project Was Conceived
The Pre-production Process
Co-design and Co-creation of Yarriambiack Shire Tales
On Location: Capturing Rural Experiences
Smartphone Filmmaking, Drone and 360° Video Production
360° Video to Capture Scale and Ambiance
Making Use of Travel Time
The Yarriambiack Shire Tales
The Woods’ Farming and Heritage Museum
The Stick Shed
The Flying Doctors Story
Horsing Around Beulah
Hopetoun—A Lakeside Holiday Haven
Mobile Storytelling with Rural Australian Communities
Dissemination of the Stories
Conclusion
References
Part IV Creative Practice Research
12 Performative Storytelling: The Model of Setting-Based Mobile Music Creation
Methodology
The Model of Setting-Based Mobile Music
Pre-Production
Instrumental Performance
Mixing and Effect Processing
Algorithmic Composition
Sequencing
Post-Production
Narratives and Compositional Strategies
Challenges and Advantages
Reflections and Conclusion
References
13 Cine-Writing the Everyday: Walking with Agnes and Zelda
References
14 Embodied Interaction and Immersive Film Experiences on Smartphones
Introduction
Multisensory Encounters
Touch Interaction
Interactive Spectatorship
Immersive Narrative Experiences on Smartphones
Conclusion
References
Part V Mobile Education and Screen Storytelling
15 Chinese Middle School Students’ Beliefs About and Acceptance of Smart Mobile Devices
The Development of SMDs-Based Smart Education
The Technology Acceptance Module
Method
The Application of TAM on SMDs-Based Smart Education Research
Instruments
Procedure
Data Analysis
Results
Exploratory Factor Analysis
Confirmatory Factor Analysis
Conclusion
References
16 The Role of Learner Self-Regulation in Mobile-Assisted Language Learning
Introduction and Related Literature
Methodology
Findings
Aims and Impacts
Goal setting and Strategic Planning
Self-Observation and Self-Control
Self-Reflection
Further Discussion
Conclusion
References
17 The Multi-Specialist Moving-Image Practitioner: Reframing Film School for the Twenty-First Century
Film Work Today
What’s in a Name?
Portfolio Career Layers
More Tools, Makers, Uses, and Usages
Mobile in the Mix
Career Perspectives
Where to with Screen Education?
Art
Technology
Art + Technology
Design
Business
Advertising, Marketing, and Promotion
Agility
Challenges
References
Index

Citation preview

Mobile Storytelling in an Age of Smartphones Edited by Max Schleser Xiaoge Xu

Mobile Storytelling in an Age of Smartphones

Max Schleser · Xiaoge Xu Editors

Mobile Storytelling in an Age of Smartphones

Editors Max Schleser Department of Film and Animation Swinburne University of Technology Melbourne, Australia

Xiaoge Xu University of Nottingham Ningbo China Ningbo, China

ISBN 978-3-030-87246-5 ISBN 978-3-030-87247-2 (eBook) https://doi.org/10.1007/978-3-030-87247-2 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Palgrave Macmillan imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Contents

1

Mobile Storytelling in an Age of Smartphones Max Schleser and Xiaoge Xu

1

Part I Mobile Storytelling: Imperative Concepts and Innovative Cases 2

Mobile Storytelling: Changes, Challenges and Chances Max Schleser

3

Mobile Storytelling: Elements, Evolution, and Experience Xiaoge Xu

4

5

Mapping Through Mobile Filming: An iPhone Mapper’s Flirtation with Google’s Artificial Intelligence (AI) Krishna Sankar Kusuma Shoes and Taxis as Mobile Storytelling Tools: Stories from the Frontline About Immigration and Integration in Canada Gerda Cammaer

9

25

37

55

v

vi

CONTENTS

Part II Mobile News Storytelling 6

7

8

Multimodal Framing COVID-19 Stories: The Case of Douyin Qumo Ren

69

Twittering China’s Image: The Case of Xinhua News Agency Zhiying Zhang

85

Citizen News Going Mobile: Opportunities and Challenges Hao Liu

99

Part III Vulnerable Communities and Social Innovation 9

10

11

Mobile Storytelling with Seniors: Co-designing Smartphone Productions and Mobile Stories with Vulnerable Communities Diana Bossio and Max Schleser

119

Engagement with Mhealth: Experiences of Young Women in Bangladesh Naziat Choudhury

135

Mobile and Digital Stories from Down Under: Tales from Rural Australia Hilary Davis and Max Schleser

151

Part IV Creative Practice Research 12

13

14

Performative Storytelling: The Model of Setting-Based Mobile Music Creation Martin K. Koszolko

173

Cine-Writing the Everyday: Walking with Agnes and Zelda Liz Burke

191

Embodied Interaction and Immersive Film Experiences on Smartphones Kata Szita

205

CONTENTS

vii

Part V Mobile Education and Screen Storytelling 15

16

17

Chinese Middle School Students’ Beliefs About and Acceptance of Smart Mobile Devices Xiaoxiao Zhang

223

The Role of Learner Self-Regulation in Mobile-Assisted Language Learning Yanhui Zhang, Yuanyuan Liang, and Nancy Xiuzhi Liu

239

The Multi-Specialist Moving-Image Practitioner: Reframing Film School for the Twenty-First Century Daniel Wagner

253

Index

273

List of Contributors

Diana Bossio Department of Media and Communication, Swinburne University of Technology, Melbourne, Australia Liz Burke Swinburne University of Technology, Melbourne, Australia Gerda Cammaer School of Image Arts, Ryerson University, Toronto, ON, Canada Naziat Choudhury Department of Mass Communication & Journalism, University of Rajshahi, Rajshahi, Bangladesh Hilary Davis Social Innovation Research Institute, Swinburne University of Technology, Melbourne, Australia Martin K. Koszolko School of Music Collaboration, Melbourne, Australia Krishna Sankar Kusuma Jamia Millia Islamia, New Delhi, India Yuanyuan Liang School of Education, University College London, London, UK Nancy Xiuzhi Liu School of Education and English, The University of Nottingham Ningbo China, Ningbo, China Hao Liu Architecture and Art Department, Zhejiang Business Technology Institute, Ningbo, China

ix

x

LIST OF CONTRIBUTORS

Qumo Ren University of Nottingham Ningbo China, Ningbo, China Max Schleser Department of Film and Animation, Swinburne University of Technology, Melbourne, Australia Kata Szita Trinity Long Room Hub, Trinity College Dublin, Dublin, Ireland Daniel Wagner Unitec Institute of Technology, Auckland, New Zealand Xiaoge Xu University of Nottingham Ningbo China, Ningbo, China Xiaoxiao Zhang School of International Communication, Faculty of Humanities and Social Science, University of Nottingham Ningbo China, Ningbo, China Yanhui Zhang School of Education and English, The University of Nottingham Ningbo China, Ningbo, China Zhiying Zhang Faculty of Humanity Social Science, University of Nottingham Ningbo China, Ningbo, China

List of Figures

Fig. 1.1 Fig. 1.2

Fig. 9.1 Fig. 11.1

Fig. 12.1 Fig. 12.2

Participants at IMSC 2020 International Mobile Studies Congress Max Schleser’s keynote Mobile Storytelling in an Age of Smartphones at IMSC International Mobile Studies Congress Opera Smartphone Filmmaking production https:// opera.eclc.org.au Tales from Yarriambiack Shire http://bit.ly/Yarriambi ackTales. Screened at St Kilda Film Festival, June 2020 (Melbourne, Australia) and NYU Cellphone Cinema Showcase, June 2020 (New York, USA) Elements of setting-based mobile music creation model Setting-based model in action. Composing and performing during train commutes

2

5 121

154 176 179

xi

List of Tables

Table 10.1 Table 15.1 Table 16.1 Table 16.2

Helpfulness of online health information Mean, standard deviations analysis results Descriptive statistics for the goal setting and strategic planning Descriptive statistics for the self-observation and control and self-reflection

143 231 245 246

xiii

CHAPTER 1

Mobile Storytelling in an Age of Smartphones Max Schleser and Xiaoge Xu

In examining and exploring the currents and trends of mobile storytelling, we shall also take a look at the historical developments of mobile media and mobile stories. Mobile media as such is not new. The Greeks (mid-eighth century BCE) and Romans used wax tables (43–410 AD) for

This edited collection aims at sharing with its global audience the selected invaluable experience and insightful expertise in mobile storytelling presented at the inaugural International Mobile Storytelling Congress on 17–19 January 2020 at the University of Nottingham Ningbo China. M. Schleser Department of Film and Animation, Swinburne University of Technology, Melbourne, Australia X. Xu (B) University of Nottingham Ningbo China, Ningbo, China e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_1

1

2

M. SCHLESER AND X. XU

Fig. 1.1 Participants at IMSC 2020 International Mobile Studies Congress

documentation and communication. From then up to now, mobile media have been evolving in their respective forms, features and functions. By 2022, online videos will make up more than 82% of all consumer internet traffic—15 times higher than it was in 2017 (Cisco, 2020 online). By 2023, nearly two-thirds of the global population will have Internet access and over 70% of the global population will have mobile connectivity. Having experienced many changes, challenges and chances over centuries, mobile media have always been functioning differently to the same overall objective, that is, to create and cater to time-specific needs, tastes, preferences and styles in storytelling as mentioned in the second chapter of this collection (Fig. 1.1). In the context of constant innovation, one should remember that those who do not change and adjust to the changing condition can go out of business rapidly. Steven Sasson invented the world’s first digital camera while working at Eastman Kodak in 1975. It weighed 3.6 kg and shot a mere 0.01MP. The camera could store 30 images on a delicate cassette. The “Kodak moment” became a signifying moment in photography. In 1976, Kodak had 85% market share in cameras and 90% market share in film. In 2021, however, the company filed for bankruptcy. The first camera phone was introduced in Japan, the SHARP J-SH04 – 2000 and as indicated in the first chapter, most early mobile filmmakers used a

1

MOBILE STORYTELLING IN AN AGE OF SMARTPHONES

3

Nokia N95. In 2007, Nokia tried to develop everything, including software for mobile devices, operating systems and an online video platform, which was called OVI. Apple’s business model based on the App store changed the entire smartphone landscape. With the iPhone 4s, which introduced 1080p as broadcasting standard video, Apple has secured its market share by October 2011. Changes in media go hand in hand with changes in storytelling. To examine changes brought by mobile media, Mobile Media Making in an Age of Smartphones (Berry & Schleser, 2014) explored aesthetics, space and place, knowledge and stories and the self. The edited collection examined how representations and experiences are shaped by and through creative practices and digital ethnographies of “new visualities and socialities.” Mobile Story Making in an Age of Smartphones (Schleser & Berry, 2018) further developed this discourse and continues to define creative mobile media. “Story-making can contribute to formulating democratic processes and equity, which are imperative to create change and challenge traditional models of media production and consumption” (Schleser & Berry, 2018, p. 1). By means of embracing creating, collaboration and curating, this edited collection analyzed story-making, space-making and change-making. Mobile Storytelling in an Age of Smartphones, as the third edition, continues this dynamic exploration and discusses mobile news storytelling, vulnerable communities and social innovation, mobile creative art and mobile education and translation. Both the second chapter Mobile Storytelling: Changes, Challenges and Chances? and the third chapter Mobile Storytelling: Elements, Evolution, and Experience provide a platform for more specific discussions in the subject areas. As identified in 2014 by Xu, co-editor of this collection, mobile is constantly changing its forms, features and functions. “Never-ending new forms, features, and functions of mobile keep arriving, shaping our present and future” (Xu, 2014, p. xxi). Writing in Interdisciplinary Mobile Media and Communications: Social, Political and Economic Implications, Xu maps out the formation of “mobile studies” or “interdisciplinary studies of mobile media and communication” as a field. He traces the origin of the proposed field to Kristóf Nyíri book Mobile Studies: Paradigms and Perspectives, which was published in 2007. Nyíri described “mobile telephony” as an “interdisciplinary enterprise.” Viewing mobile as a medium and a communication, Xu identifies “fundamental components of mobile studies” and summarizes “its subjects, scope, and strategies” (Xu, 2014).

4

M. SCHLESER AND X. XU

In scholarship, one can note a body of literature emerging including Rheingold (2002), Castells et al. (2007), Jenkins (2008), Goggin and Hjorth (2009), Rieser (2011), Farman (2014), Atkinson (2014), Berry (2017), Prasad (2017), Baker (2018) and Schleser (2021). While these texts made a key contribution within their disciplines one should note the editors of this edition take more transdisciplinary and open approaches to Mobile Studies. In 2014 Xu published Interdisciplinary Mobile Media and Communications: Social, Political, and Economic Implications and Schleser et al. Mobile Media Making in an Age of Smartphones. The interdisciplinary approach continues to have an impact on other fields. With communication becoming increasingly audiovisual and storytelling more interactive, the process of convergence that Nyíri identified as a driving factor in 2007 is still paramount. At the time he was writing, the first Mobile Conferences and Festivals emerged: Mobilefest, which ran from 2006 to 2010 in Brazil and explored “how can mobile technology contribute to democracy, culture, art, environment, peace, education, health and the Third Sector.” After Schleser, the coeditor of this collection, established the Mobile Innovation Network & Association (MINA) in 2011 in Aotearoa/New Zealand (MINA, 2021), MINA has created interactions between communities, content and the creative industries (see Fig. 1.2). When Xu, the co-editor of this collection, launched Mobile Studies International (MSI) in 2012 in Singapore as a major platform to promote and conduct mobile studies globally, he observed that “[m]obile is reshaping not only who we are but also what we do in this mobile world” (Xu, 2020). The reshaping momentum has also been strongly indicated by the global major event of the Mobile World Congress (MWC), which was inaugurated in 2008 and is annually hosted even since then in Barcelona by GSMA—an industry trade body representing mobile network operators. Major network providers (such as Vodafone Group, Deutsche Telekom) and or key hardware and software companies such as Apple disseminate media and entertainment content via entertainment hubs such as Apple TV+ or Vodafone Entertainment streaming of Amazon Prime or Netflix via mobile devices and plans. As a mobile-first initiative and mobile-only streaming service, Quibi rushed to meet the homestricken users who could not commute or travel but could only consume mobile content due to the COVID-19 pandemic. Mobile short-form video established itself in the mediascape as its own format. As an ecology it has its own creators, summits (i.e., VidCon) and shapes its audiences.

1

MOBILE STORYTELLING IN AN AGE OF SMARTPHONES

5

Fig. 1.2 Max Schleser’s keynote Mobile Storytelling in an Age of Smartphones at IMSC International Mobile Studies Congress

In China, video platforms are a new media outlet. Within this context one can explore Douyin, Kuaishou, Tengxun Wei Shi, JiangBing, Pipi Xia, Pear video, Xiao Ka Xiu, Miao Pai or HaoKan Video. Next to the industry embracing mobile media, this edited collection particularly emphasizes the role of working with communities, i.e., young women in Bangladesh or mobile storytelling with seniors in Australia. Creative strategies for mobile music creation, cine-writing and embodies interactions are explored to emphasize inspirational approaches to craft mobile stories. The collaboration in an international context is supported by the section of mobile translation and shaping the next generation of mobile scholars, practitioners and researchers. A new generation of creatives is emerging, who grew up with mobile devices as other generation with TVs. This edited collection emphasizes that all ages, young or senior, all genders, and all areas, rural or urban, are part of this international and inclusive community.

6

M. SCHLESER AND X. XU

References Atkinson, S. (2014). Beyond the screen emerging cinema and engaging audiences. Bloomsbury. Baker, C. (2018). New directions in mobile media and performance. Routledge. Berry, M. (2017). Creating With mobile media. Palgrave Macmillan. Berry, M., & Schleser, M. (2014). Mobile media making in an age of smartphones. Palgrave Macmillan. Castells, M., Fernández-Ardèvol, M., Linchuan Qiu, M., & Sey, A. (2007). Mobile communication and society—A global perspective. MIT Press. Cisco. (2020). Annual Internet Report—White Paper (2018–2023). Retrieved from https://www.cisco.com/c/en/us/solutions/collateral/executive-perspe ctives/annual-internet-report/white-paper-c11-741490.html Farman, J. (2014). The mobile story: Narrative practices with locative technology. Routledge. Goggin, G., & Hjorth, L. (2009). Mobile technologies from telecommunications to media. Routledge. Jenkins, H. (2008). Convergence culture—Where old and new media collide. New York University Press. MINA. (2021). Mobile Innovation Network & Association. Retrieved from https://mina.pro/ Prasad, S. (Ed.). (2017). Creative mobile media: A complete course. World Scentific. Rheingold, H. (2002). Smart mobs: The next social revolution. Basic Books. Rieser, M. (2011). The mobile audience media art and mobile technologies. Rodopi. Schleser, M., & Berry, M. (2018). Mobile story making in an age of smartphones. Springer International Publishing. Imprint: Palgrave Pivot. Schleser, M. (2021). Smartphone filmmaking: Theory & practice. Bloomsbury. Xu, X. (2014). Interdisciplinary mobile media and communications: Social, political and economic implications. Hershey, Pennsylvania: IGI Global. Xu, X. (2020). Mobile Studies International (MSI). Retrieved from https:// www.msi.wiki/

PART I

Mobile Storytelling: Imperative Concepts and Innovative Cases

With the advent of the Internet, the Web, and mobile technologies, mobile storytelling has undergone constant changes in its forms, formats, features, and functions. Constant changes have brought about both challenges and chances to mobile storytelling. In its evolution, mobile storytelling has also experienced imperative changes in concepts, resulting in innovative cases of how the imperative concepts have been put into practice. The first two chapters in this part provided a concise overview of imperative changes in concepts ranging from short film production to community-engaged projects and from game changers to experienceoriented approaches. The last two chapters offered innovative cases of changing mobile storytelling. Significant changes in mobile storytelling were concisely captured and reviewed in Chapter 2. It offered an overview of the contributions of mobile media and mobile stories to short and feature film productions, mobile social media production, and community-engaged projects, to name just a few. Furthermore, it also elaborated different mobile storytelling options and its unique methods of representation and experience as a way to inspire mobile storytellers. To switch from online to mobile in storytelling is indeed a game change. To leverage such a game change to enhance mobile storytelling, Chapter 3 proposed a unique integrated approach, that is, to integrate traditional and innovative elements, to leverage storytelling evolution, and to maximize experience in mobile storytelling. Such an approach aims

8

PART I: MOBILE STORYTELLING: IMPERATIVE CONCEPTS …

to inspire more innovations in mobile storytelling and more efforts to enhance mobile storytelling in the context of experience economy. As an illustrative case, how to leverage mobile phones for better production and how to leverage locative media for effective optimization were well elaborated in Chapter 4. Going beyond locating a physical address, the author of this chapter argued that mobile phones could place a crucial part in providing creative location description with mobile filming of photos and videos. Mapping by Google maps using the iPhone and as geographies cannot be understood by artificial intelligence alone, sufficient and specific data are required to create more experiential and accurate geography. The author of this chapter further argued that collaborative user-generated content helps to strengthen the accuracy and detailing of maps. Another illustration of how changing concepts have been put into practice is provided by Chapter 5. This chapter descriptively analyzed three mobile cinema projects with immigrant communities in Toronto, Canada. Related in media, theme, and content and yet different in methodology, these three projects demonstrated that it was central to have some form of collaborative filmmaking and peer learning in artistic development. It also showed that mobile offers participants and viewers a real chance to experience a true ride-along. Through imperative concepts and innovative cases, the four chapters have truly demonstrated the power and popularity of mobile as a tool to enhance mobile storytelling in different areas for different purposes in different countries. Different in forms and yet similar in functions, mobile has been intimately inter-connecting all communities around the world through endless flows of mobile storytelling across time and space.

CHAPTER 2

Mobile Storytelling: Changes, Challenges and Chances Max Schleser

While there are existing theoretical frameworks for screen storytelling (Smith & Pearson, 2014; van den Oever & Christie, 2018) and digital storytelling (Alexander, 2011; Jenkins & Dunford, 2017; Lambert, 2013), mobile storytelling can expand screen storytelling concepts in form and function. Existing scholarship on mobile media making and mobile story making (Berry & Schleser, 2014; Schleser & Berry, 2018) explored mobile media through an interdisciplinary approach. The Mobile Story—Narrative Practices with Locative Technologies introduced “sitespecificity”, “urban makeup” and “creative misuse” (Farman, 2014, pp. 3 and 4). Since then, mobile stories continued to “reimagine our relationship to technology, place, and our own sense of self in the spaces through which we move” (Farman, 2014, p. 5). Mobile media is becoming more

M. Schleser (B) Department of Film and Animation, Swinburne University of Technology, Melbourne, Australia e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_2

9

10

M. SCHLESER

ubiquitous than ever and morphs audio-visual content into novel formats, merging online, short-form and site-specific authorship. Mobile stories can be crafted within existing screen industry structures and storytelling formulas. In a similar way, as a film is composed of shots, scenes, sequences and narrative structures Bernard (2016), filmmakers, screen producers and storytellers work with these formats to create engaging mobile stories. In addition, mobile media makers, storytellers, filmmakers, artists and designers further expand on existing story structures. As part of the Mobile Studies Congress in 2020, Schleser co-created the key-note presentation Imaginative Mobile Storytelling. As mobile media is based on a peer and network structure, scholarship should reflect the diversity and multiplicity in voices and approaches within mobile storytelling. This chapter will highlight the comments and positions by Camille Baker, Michael Osheku, Martin K. Koszolko, Krishna S. Kusuma, Anne L. Massoni, Adrian Jeffs, Patrick Kelly, Andrew Robb, Felipe Cardona, David Scott Leibowitz, Eugenio Tisselli, David Cowlard, Gerda Cammaer and Max Schleser, who are featured in this video (Schleser, 2020). Scholars responded to the prompt; what are the changes, challenges and chances for Mobile Storytelling. This chapter explores these mobile media researchers’ approaches within the Creative Arts to mobile screen production and how they approach storytelling with, on and for mobile devices. In a transdisciplinary research context, mobile media can bring together artists, designers, creatives, digital storytellers and researchers with various approaches to and perspectives on storytelling. This chapter further develops the prospect and potential for mobile media as a transdisciplinary field of study as mobile media can interconnect with various disciplines and approaches via storytelling.

Mobile Screen Stories: Encompassing Transmedia and Cinema Screens Andrew Robb is a mobile and smartphone filmmaker. His smartphone film Tofu Man is a great example of smartphone filmmaking and how mobile stories allows us to capture stories by various people and provide insights into their lives. His recent film Odd One Out utilised puppetry animation as the smartphone film was produced in lockdown. Next to his work as independent filmmaker, he also produces corporate videos. In the Mobile Studies Congress 2020 video he shows the first mobile

2

MOBILE STORYTELLING: CHANGES, CHALLENGES AND CHANCES

11

camera phone, the Nokia N95. Other mobile and smartphone filmmaking pioneers such as Felipe Cardona or Max Schleser started their journey as mobile storytellers with this device. Felipe Cardona participated in the very first Mobile Short Film Festival in Berlin in 2004 and, ever since then, has continued to drive a democratic approach to filmmaking. Nowadays, most people have a smartphone in their pocket that has the potential to create smartphone feature films (Schleser, 2020). At the International Mobile Storytelling Congress, Blue Moon was screened as an award-winning smartphone feature film. The story is driven by the financially troubled protagonist working at a gas station and when stolen drug money and affiliated troubles appear in his night shift. The 85 min feature film can develop an emotional engagement with the story through the main characters. The plot provides background information about the characters, and as the story develops, we realise that the protagonist and antagonist have met before. The beginning, middle and end follow in a linear path. One could argue that one could use existing storytelling structures for smartphone feature films such as Tangerine, Unsane or The Best Years of a Life, which are developed for cinematic release. When considering novel mobile story formats, i.e. for short-form video sharing sites, storytelling expands into new domains. Transmedia narratives provide a conceptual starting point for this investigation, but mobile stories do not necessarily transcend from the 4th screen (mobile devices) to the 1st (cinema), 2nd (TV) or 3rd (computer) screen (Goggin, 2012). Transmedia storytelling represents a process where integral elements of a fiction get dispersed systematically across multiple delivery channels for the purpose of creating a unified and coordinated entertainment experience. Ideally, each medium make its own unique contribution to the unfolding story. (Jenkins, 2006)

This chapter will, in particular, explore the unique mobile story characteristics. Writing in Computer as Theatre, Laurel refers to the computer as a medium. She argues that, like film, computers have developed their own unique methods of representation and experience and “have given them a new twist in terms of authorship, distribution, production methods, and interactivity” (Laurel, 2014, p. 150). Furthermore, she argues that the most important sensor in a smartphone is the camera. In order to understand mobile production methods as a media-specific

12

M. SCHLESER

form and format, it is key to study the creative processes and story structures. When mobile media is integrated into industry-standard production and industry formats, smartphones have reached a technical standard to operate very much within these industry conventions. Simultaneously mobile media can also further develop community-engaged, and network or peer-to-peer production approaches. The notion of an in-between space is characteristic of mobile media. A mobile or transmedia story can flow across a number of platforms and places. This mobile screen form characteristic is reflected in Ephemeral Media, which Grainge outlines as “transient and brief; in definitional terms, it describes anything shortlived” (Grainge, 2011, p. 2). He describes these semi-permanent artefacts through “brevity and evanescence” (Grainge, 2011, p. 3). Photographers like Anne Massoni with her work Y& M2 1 or Gerda Cammaer’s travel films such as Depart de Lille 2 explored these fleeting memories. Stories on social media platforms express an ephemeral quality. They are only saved for a short time, i.e. Facebook Stories, Instagram Reels and Snapchat stories will only be visible for 24 hours with the option of saving these to a story archive (Facebook) and re-shared via a post (Instagram). Unopened Snapchat stories are deleted after 30 days. Facebook stories can be of two minutes in duration, and on Instagram it is the same duration, but in intervals, and on Snapchat 60 sec only. TikTok, the new kid on the block, also has a 15–60 sec ratio. Grainge sees Ephemeral Media as exploring contemporary media specificities. He identifies remediation, “temporal immediacy and quotidian spectacle” as their unique characteristic. His edited collection focuses on the examination of “screen forms and relations that exist between, beyond and below” (Grainge, 2011, p. 10). In 2011 Grainge’s study focused on media “at the edges, within the junctions, or that sit askance to longer, bulkier entertainment content” and to “illuminate particular media phenomena that are simultaneously fleeting and increasingly ubiquitous in contemporary screen life” (Grainge, 2011, p. 12). If you like it or not, the media landscape has transformed in the last ten years, and screen stories are constantly morphing into new forms and formats. As Laurel pointed at the unique distribution format in computers, this transcends to mobile media. Most mobile social networks do not work in a broadcasting style of one to many but in a more rhizomatic and peer-structured network approach. The traditional script to screen production method is transformed into a shoot and share aesthetic (Schleser in Pearce & Daniels, 2013). As mobile stories are

2

MOBILE STORYTELLING: CHANGES, CHALLENGES AND CHANCES

13

arguably the most accessible form of storytelling, mobile stories can be more diverse. Mobile stories speak to niche audiences and can also be produced by them. Thus, stories can be crafted by these audiences or for these audiences. As much as social media is driven by personas, influencers and faces, compelling characters are key. Writing in Transmedia 2.0, Bernardo identifies three elements for transmedia projects: • Compelling characters • An exciting and convincing story world • A strong storyline with clearly defined plot points. These elements can be translated to mobile stories. The way to create engagement and to connect to a persona is via its “likeability”. There is enough disturbing hate speech online, but of course, a critical discourse can gain attraction in the same way. Whether the storyteller creates a character profile or casts from the world, personalities are key. They (characters) are the emotional link between your audience and your content. Not only do they introduce your story world, but they also authenticate the experience by acting and reacting to that world with all the quirks of human nature. … a strong character is the point through which a story transforms, from a passive ‘telling’ to a deeply personal shared experience. They demonstrate something of our own reality and reveal us to ourselves. (Bernardo, 2014, p. 45)

Whether mobile stories are scripted or capture the everyday life, most mobile storytellers choose to capture their own worlds. This reflexivity does not necessarily construct a “not reality, but a representation of it” (Lucas & Kelly, 2016). Characters speak to authentic engagement and allow capturing of unique personas. Writing in Connecting through Mobile Autobiographies: Self-Reflexive Mobile Filmmaking, Self-Representation, and Selfies, Schleser explores mobile stories for creating self-representation and applies a user-based interpretation to mobile autobiographies. Self-reflexive filmmaking, self-representation, and mobile autobiographies can create an engagement with their peers and the people involved in the production. (Schleser, 2014, p. 154)

14

M. SCHLESER

Characters can authenticate their own unique experience and show a transformation from their perspectives. They can also represent a community and thus provide an insider perspective, which means that members of a particular community can create mobile stories according to their agendas. Adrian Jeffs points out that he loves telling stories with heart (Schleser et al., 2020). And with that in mind, when interviewing people that are vulnerable and who would get nervous with a standard film crew pointing a camera at them, an iPhone can create an intimate relationship, and it allows them to really open their heart and tell their story. Patrick Kelly voices a similar idea pointing at the opportunity to continue to explore the kinds of stories and particularly that can be told using mobile media and smartphones as personal devices (Schleser et al., 2020). Mobile stories change what we see, as mobile stories can differ from what we usually see in mainstream media. Eugenio Tisselli calls this community memories, in which a group of people gets together to document and share specific aspects of their realities, of their concerns, of their views, their opinions and aspirations (Schleser et al., 2020). And in a mobile world, these communities are located online and in physical proximity. As much as Japan was years ahead 10–15 years ago with mobile camera phone, China now seems to provide an outlook into a possible future for video platforms. Here one can point at Douyin, Kuaishou, Tengxun Wei Shi, JiangBing, Pipi Xia, Pear video, Xiao Ka Xiu, Miao Pai or HaoKan Video as examples. “In this dual movement towards speed and storage, immediacy and archiving, the ephemeral has assumed a particular cultural and textual significance” (Grainge, 2011, p. 3). This duality also resonates in a new format of mobile stories: Screenlife. Screenlife is a new format of visual content that has grown from independent projects to full-length, world-renowned films, documentaries and TV shows. Its main idea is that everything that the viewer sees happens on the computer, tablet or smartphone screen. All the events unfold directly on the screen of your device. Instead of film set — there’s a desktop, instead of protagonist’s actions — a cursor. (Screenlife, 2020 online)

The Screenlife format incorporates the interface as part of the storytelling mechanism. It includes the smartphone (or desktop) interface as a window to the world. Inspired by the success of the blockbuster film Unfriend, Searching, and the snap chat series Dead of Night,

2

MOBILE STORYTELLING: CHANGES, CHALLENGES AND CHANCES

15

Russian Producer & Director Timur Bekmambetov set-up the Screen Life production method, including a HTML screen recorder which keeps the interactivity in Social Media feeds and websites while screen recording. The stylistic approach to create intimate and emotional stories resulted in a box office success for Unfriend $64 million and $75 million for Searching. Dead of Night had 15M views on snap chat. Bekmambetov’s signature is to develop innovative stories by exploring form and content and pushing the boundaries. One can point at Hardcore Henry firstperson view perspective or currently in production the first vertical video feature film V2. Escape From Hell.

Mobile Social Media Stories Mobile stories can take multiple forms and formats. Mobile media sometimes breaks conventions in terms of aspect ratio, such as the traditional landscape and cinematic 16:9 format is merged and re-formatted into Vertical video formats 9:16, or the Square Video, 1:1 aspect ratio as well known for Stories on Instagram. With an increased amount of content, stories need to stand out to receive recognition as more content choices are constantly being updated than ever before. Writing in the SEJ—Search Engine Journal, Trounce emphasises increased user engagement and reach of wider audiences as part of the rationale for a social media strategy based on storytelling. He argues that “temporary material” relates to the content and simultaneously to the viewers’ response or, in his words “our attention to it”. Trounce emphasises the immediate qualities of ephemeral media. The most prominent social media networks very much recognise this as a key part of their strategies. Beaulac quotes Mark Zuckerberg; Another important shift that we’re seeing across the industry is the growth of Stories. We expect Stories are on track to overtake posts in feeds as the most common way that people share across all social apps. That’s because Stories is a better format for sharing multiple quick video clips throughout your day. The growth of Stories will have an impact on how we build products and think about our business, including WhatsApp and Instagram, which are the #1 and #2 most-used Stories products in the world. (Beaulac, 2019 online)

16

M. SCHLESER

As part of the presentation, the interface is key, which is explicitly part of Screen Life. Writing in a recent study Ephemeral Storytelling With Social Media: Snapchat and Instagram Stories at the Brooklyn Museum, Villaespesa and Wowkowych argue that the ephemeral aspect “motivates minimal curation and editing”. They mention the social media’s capacity for stickers, video effects, tags, drawings, emojis, hashtags, geotags, polls or/and text comments in Snapchat and Facebook. According to them, these social media interactions lead to new narratives and meaningmaking in their study of the museum experience. With reference to the Museum experience, one can point at the work of David Leibowitz. The short film David Hockney at the Met using an app to create mobile digital art was inspired by David Hockney at the Met. While his work expresses immediate and ephemeral qualities, his work demonstrates that editing can play a key part in mobile production methods. With the availability of video editing applications, storytellers and filmmakers alike create mobile montages, ranging from free version like Quik (iOS and Android) or VN video editor (cross-platform, win, mac, iOS and Android) with limited capacity to more advanced options like Adobe Premiere Rush and Luma Fusion. Martin K. Koszolko’s creative arts practice and research into mobile music further expands this argument. Not only that mobile music can execute creative ideas much more rapidly, but he also points at the integration of algorithmic composition tools; mobile music allows producers to be musically creative in typically not very creative spaces. For example, during commutes on public transport or being able to record location sounds almost immediately anywhere. The aspect of mobility in mobile stories can be traced across space and time. In the context of the latter, one could also point at Anne L Massoni’s mobile photography project Mapping MiFi.3 Fifi, her great-grandmother, travelled from March to August of 1900. And Anne L Massoni followed this journey in 2018 again. Her great-grandmother used the popular camera of the day, a Kodak Bull’s-Eye No 2. Anne L Massoni used her iPhone 7, wanting to mimic the ease of technology respective to our times.

Mobile Montage The editing process in creating a mobile story is key. Montage is understood here in a Vertovian way4 (Vertov in Michelson, 1984). This means selecting images for sequences on location and then crafting these into a story concept can now be conducted on the same location. When

2

MOBILE STORYTELLING: CHANGES, CHALLENGES AND CHANCES

17

mobile stories are live-streamed or composed without editing in the individual takes, the interface and social media or video platform channel will provide some kind of sequencing. Thus, a linear approach might not be fruitful as online media is fragmented and especially mobile media affordances speak to micro-formats; smaller screens and handheld devices make the vertical video a conceptual choice for mobile viewing experiences and short-form video or ultra-short-form video as a choice rather than cinematic (90–120 min) or TV conventions (42 min–60 min) to shorter and more episodic content. Quibi shows made for mobile viewing (in the cinematic and vertical video) are seven to ten minutes.5 Network and database filmmaking has demonstrated how stories can operate as digital and computational formations in non-linear structures. Interactive documentaries have disrupted traditional creative processes and pointed out how storytelling can open up to more flexible approaches. The i-docs are categorised as conversational, hypertext, participative or experiential mode (Gaudenzi & Aston, 2014). In the conversational mode, the viewer is “in conversation” with the computer. The hypertext mode ‘as a closed video archive gives users an exploratory role’ (author italicised users in order to point at more active positions such as navigator or viewers). In the participative mode, the maker, creative, storyteller “is called upon to ‘stage a conversation’, with a community, with research subjects, with participants, co-producers and audiences as it counts on the participation to create an open and evolving database” (Gaudenzi & Aston, 2014, p. 127). Writing in Smartphone Filmmaking : Theory and Practice, Schleser developed these categorisations into the realm of mobile stories. He identifies the categories of the poetic and experimental mode, participatory and engagement mode, smartphone native mode, conversational mode and the “red carpet in your pocket” mode (Schleser, 2021). Storytellers and smartphone filmmakers can operate in more spontaneous and semi-permanent modalities. Productions might occur outside the production schedule, and stories can take shape over a more extended period of time. Mobile stories’ authenticity is anchored in its spontaneity when working with small teams, one-wo/man-band or integrating peers as co-creators. Mobile stories are also expanding the field of non-narrative approaches rather than focusing on the story as cause-and-effect experiences resonate in audio-visual forms. These traditions are anchored in surreal, constructivist or Dadaist art practices. Schleser (2021) discusses

18

M. SCHLESER

these smartphone films in detail, which provide a great example to emphasise the significance of editing to create an experience. Surreal dream sequences blur past and present, memory and actuality in Night Fishing, Sleep Has Her House or Absent VII a Homage to Maya Deren. Through paying attention to often-overlooked environments in nature, such as in N8 dreams or in the cities as in US stop, the mobile media art aesthetics (Baker et al., 2009) are celebrated. Mobile stories and smartphone filmmaking can facilitate experimentation and are most suitable to test new approaches in moving-image arts. Independent productions such as Sketch Three: Avant-Garde as much as production companies recognise the creative industries recognise this potential, here one can point at Experiments #ShotoniPhone (commissioned by Apple 2019, produced by Donghoon Jun and James Thornton of Incite). Michael Osheku confirms a similar development for the African International Smartphone Film Festival, which demonstrates the quantity and quality of smartphone filmmaking (Schleser, 2020). This is echoed by Krishna S. Kusuma, who sees mobile media as a part of everyday life, and it has become a culture in India. Gerda Cammaer highlights that filmmakers often act as historians of the present as they study how human beings situate themselves in the world, among all other things. They try to explain the political, social, cultural, ecological events by situating them in a broader historical context, often focusing on the times’ zeitgeist. The zeitgeist or the spirit of the times is the general intellectual, moral and cultural climate of an era (Schleser, 2020).

Mobile Stories For mobile stories that feature ‘talking heads’ or interviewees, new vertical or 1:1 video formats provide a suitable and nice ‘frame’. As a Creative Arts researcher, Schleser developed an InstaWorkshop concept for the Metropolitan Partnerships Inner South-east (Schleser, 2018) Victorian Department of Environment, Land, Water & Planning Victoria. The Instagram Channel6 provided a conversation starter that could also be re-edited into a two-minute YouTube video for the government department. By capturing various opinions and story prompts, an assemblage of voices captured their views and opinions regarding the liveability in the Inner-South East region. Similarly, the collaborative project 24Frames 24Hours used vertical videos by three different participants in three

2

MOBILE STORYTELLING: CHANGES, CHALLENGES AND CHANCES

19

different countries to create a montage to the cinematic 16:97 (Schleser, 2018). Writing in Digital Storytelling, Carolyn Miller discusses modular structures, which can be used to explore mobile stories. She discusses the Rashomon model, referring here to the Japanese movie with the same name. In this interactive narrative, viewers can move between storylines. She also draws on rhizomatic approaches. The rhizome is a metaphor that stems from an interconnecting root. Each story node can be connected to another story node. This decentralised model, which has no pre-determined beginning or endpoint, works well for mobile stories. Another model that mirrors mobile media’s viral approach is the “fractal structure”, in an interactive narrative context, this described for a story that does not advance but instead expands… it is a middle ground between the two ends of the spectrum: strict linearity on one hand, and unrestricted freedom on the other. (Miller, 2008, p. 126)

Miller also associated replayability with this model. Furthermore, Miller draws upon transmedia storyteller and creator Jeff Gomez, who identified the collective journey model. He refers here to Campbell’s Hero’s Journey but assigning participatory narratives, which allow for multiple perspectives. As mobile media is based on multiplicity, the collective characteristic is key. Miller’s digital storytelling approach is based on more traditional models such as Freyberg or Campbell, which are not necessarily suitable for micro-formats as these account for storytelling formats that need time to develop these narrative structures. Micro-movies “are conceptualised an alternative video format, as an experience rather than content” (Schleser, 2010, p. 10). Schleser positioned these micro-movies into nonscreen environments. “The mobile device is thus taking the images out of the cinematic experience onto the street, expanding spatial montage into the everyday environment” (Schleser, 2010, p. 109). The Game Design Concepts blog by Ian Schreiber provides a good overview for story structures discussing linear, branching, parallel path, threaded and dynamic object-oriented narrative. In his book Challenges for Game Designers, he specifies this as open-ended stories and dynamic object-oriented narratives, which are also termed Algorithmic Stories (Brathwaite & Schreiber, 2009 online). Brathwaite and Schreiber refer to the game Façade as an example.8 The linear and branching narrative follows a story arch and is thus linear. The branching narrative provides

20

M. SCHLESER

“multiple ways to get through a story with a possibility of multiple endings” (Brathwaite & Schreiber, 2009 online). Writing on Transmedia Digest—Narrative design across media, Peter von Stackelberg, points at the “Episodic Structure” and “Layered Episodic Structure”, which has “episodes published across multiple media”, and in the case of the layered approach, these are divided into primary and sub-stories. Florent Maurin, a former journalist and founder of The Pixel Hunt 9 formulates the concentric narrative, which has one starting and multiple endpoints. Maurin identifies this as a central hub, such as a map. Within this context, one could point at Viewfinders.gallery (Schleser, 2017), which takes a map as a central navigation hub to explore land and cityscapes. For David Cowlard, the critical reasons for making mobile film and photographs are the versatility of the camera and the possibilities for broader public architectural discussion and criticism brought about by mobile’s networked capabilities. Embedded GPS data, metadata and hashtags provide the basis for a rich layer of urban annotations that can further inform the development of new forms of database documentary. The parallel path or narrative has a pre-defined starting and endpoint. The multilinear or threaded narrative operates similarly but might have multiple starts or endings. Florent Maurin illustrates an option that progresses on a more linear path, which he refers to this a “Fishbone Narrative”. He exemplifies this through the i-docs Pine Point or Prison Valley. While these latter i-docs or AI-driven project like Façade will require a production team including creative technologist and coders, these computational developments might be a possible future for mobile stories. Smartphones are interconnected to the Internet of Things as much as entertainment hubs such as Netflix, HBO Max and/or Disney Plus. Camille Baker pointed at a possible future of mobile media and its integration into wearable media (Schleser, 2020). With more and more fragmentation in the mediascape, mobile stories will thrive and shape screen storytelling.

Towards Mobile Storytelling The Mobile Studies Congress 2020 mobile storytelling video includes the video prompts by creatives and scholars who participated in the annual Congress in China. The video emphasised examples and approaches to Mobile Studies and illustrated the scholars that are defining and shaping this story form and format.

2

MOBILE STORYTELLING: CHANGES, CHALLENGES AND CHANCES

21

This chapter introduced the multiple options for mobile stories and aimed to provide some inspiration to explore the various formats that define this story form in an in-between space. Like film or desktop computers, mobile stories have developed their own unique methods of creating representations and crafting experiences. Mobile stories can integrate with cinematic as much as interactive and digital story formulas. Social networks and network media further develop ephemeral media and semi-permanent narratives. Mobile Stories’ authenticity is driven by their intimate and immediate nature. Simultaneously Mobile Stories provide unique opportunities for community groups to create their own representations and tell stories beyond the canon of the mainstream media or blockbuster and high street cinema. This chapter is a snapshot in time that might require some updating in the future. Through demonstrating mobile stories capacity, this chapter emphasises Mobile Storytelling’s significance for screen and digital stories more generally.

Notes 1. http://www.anneleightonmassoni.com/ym2. 2. https://gerdacammaer.com/film-video/. 3. www.anneleightonmassoni.com. 4. “ I see”—“I kino-see”. Kino-eye is: Montage, when I select a theme (choosing one from among thousands of possible themes); Montage, when I make observations for a theme (choosing what is expedient from thousands of observations of the theme); Montage, when I establish the viewing order of the footage on the theme (selecting the most expedient from thousands of possible groupings of shots, proceeding from the qualities of the film footage as well as from the requirements of the chosen theme). 5. https://www.quibi.com/. 6. https://www.instagram.com/isemetro/. 7. https://youtu.be/dAePH00DeTk. 8. https://www.playablstudios.com/facade. 9. https://www.thepixelhunt.com/contact-2.

22

M. SCHLESER

References Alexander, B. (2011). The new digital storytelling—Creating narratives with new media. Praeger. Baker, C., Schleser, M., & Molga, K. (2009). Aesthetics of mobile media art. Journal of Media Practice, 10(2–3), 101–122. Baker, S. (Director). (2015). Tangerine. In. USA: Magnolia Pictures. Barley, S. (Director). (2017). Sleep Has Her House. In. UK: Ether Films. Beaulac, H. (2019). The future of ephemeral content on social media. Retrieved from https://www.acquisio.com/blog/agency/the-future-of-ephemeral-con tent-on-social-media/ Bekmambetov, T. (2020). Screen lifer. Retrieved from https://screenlifer.com/ Bernard, S. C. (2016). Documentary storytelling—Creative nonfiction on screen. Focal Press—Taylor & Francis Group. Bernardo, N. (2014). Transmedia 2.0—How to create an entertainment brand using a transmedial approach to storytelling. beActive Books. Cammaer, G., & Schleser, M. (2017). Viewfinders. Retrieved from http://www. viewfinders.gallery/ Campbell, J. (1990). The hero’s journey. Joseph Campbell Foundation & New World Library. Chaganty, A. (Director). (2018). Searching. In T. Bekmambetov (Producer). USA: ACME and Sony Pictures. Chan-kyong, P., & Chan-kyong, P. (Director). (2011). Night Fishing (Paranmanjang). In. South Korea: Moho Films. Dufresne, D., & Brault, P. (2009). Prison Valley. ARTE. Farman, J. (2014). The mobile story—Narrative practices with locative technologies. London. Fox, D. (Director). (2014). Sketch-Three: Avant-Garde (R.P.M. 2). In. United States: Independent Production. Gaudenzi, S., & Aston, S. (2014). Interactive documentary: Setting the field. Studies in Documentary Film, 6(2), 125–139. Goggin, G. (2012). The evolution of Australian mobile screens: New technology, new formats, new business models. Studies in Australasian Cinema, 6(3), 263–277. Grainge, P. (2011). Ephemeral media—Transitory screen culture from television to YouTube. British Film Institute & Palgrave Macmillan. Harris, S. (Director). (2018). Blue Moon. In. New Zealand: Dark Horse Films. Jenkins, H. (2006). Convergence culture—Where old and new media collide. New York University Press. Jenkins, M., & Dunford, M. (2017). Digital storytelling—Form and content. Palgrave Macmillan. Lambert, J. (2013). Digital storytelling—Capturing lives, creating community. Routledge—Taylor & Francis Group.

2

MOBILE STORYTELLING: CHANGES, CHALLENGES AND CHANCES

23

Laurel, B. (2014). Computers as theatre. Addison-Wesley & Pearson Education. Lelouch, C. (Director). (2019). The Best Years of a Life (= Les plus belles années d’une vie). In. France: Les Films 13, Davis-Films, France 2 Cinéma. Lopez, J. (Director). (2011). N8 Dream. In. Spain: Independent Production. Lucas, M., & Kelly, A. (2016). Documentary voice & vision—A creative approach to non-fiction media production. CRC Press, Focal Press and Taylor & Francis. Massoni, A. (Director). (2016). YM2. Maurin, F. (2014). Narrative structures in interactive documentaries. Retrieved from https://prezi.com/ilzwxzjz2t5p/narrative-structures-in-interactive-doc umentaries/ Meyer, S. R. (2021). Interactive storytelling for the screen. Taylor & Francis Limited. Michelson, A.‚ & O’Brien‚ K. (1984). Kino-Eye: The Writings of Dziga Vertov. University of California Press. Miller, C. H. (2008). Digital storytelling—A creator’s guide to interactive entertainment. Elsevier & Focal Press. Naishuller, I. (Director). (2016). Hardcore Henry. In. T. Bekmambetov (Producer). Sweden: Warner Bros. Pearson, R., & Smith A. (2015). Storytelling in the media convergence age— Exploring screen narratives. Springer & Palgrave Macmillan. Schleser, M. (2010). Mobile-mentary: Mobile documentaries in the mediascape. Ph.D. University of Westminster, London. Schleser, M. (2013). From ‘script to screen’ to ‘sh%t n share’. In C. M. Jill Daniels & G. Pearce (Ed.), Truth, dare or promise: Art and documentary revisited. Cambridge Scholars. Schleser, M. (2018). 24Frames 24Hours: An emerging form of workshop-generated documentary. Schleser, M. (Director). (2018). InstaWorkshop. Schleser, M. (Director). (2020). Mobile Studies Congress 2020 Co-Created Video. https://youtu.be/UFITHE_D_ps Schleser, M. (2021). Smartphone filmmaking—Theory and practice. Bloomsbury. Schleser, M. (2014). Mobile media making in an age of smartphones. Palgrave Macmillan. Schleser, M., & Berry, M. (2018). Mobile story making in an age of smartphones. Springer International Publishing: Imprint: Palgrave Pivot. Schreiber, I. (2009). Game design concepts. Retrieved from https://gamedesig nconcepts.wordpress.com/2009/07/ Schreiber, I., & Brathwaite, B. (2009). Designing a game to tell a story. In Challenges for game designers. Simons, M., & Shoebridge, P. (2011). Pine point. National Film Board of Canada.

24

M. SCHLESER

Smith, A., & Pearson, R. (2014). Storytelling in the media convergence age exploring screen narratives. Palgrave Macmillan. Soderbergh, S. (Director). (2018). Unsane. In. USA: Regency Enterprises and Extension 765. 20th Century Fox. Stackelberg, P. (2012). Transmedia digest—Narrative design across media. Story Structures for Transmedia Narratives. Retrieved from http://transmediadi gest.blogspot.com/2012/03/story-structures-for-transmedia.html Susco, S. (Director). (2018). Unfriended: Dark Web. In. T. Bekmambetov (Producer). USA: BH Tilt, Universal Pictures and Apollo Films. Tatham, J. (Director). (2016). Us Stop. In. France: Independent Production. Thornton, J., & Donghoon, J. (Director). (2019). Experiments. In. United States: Incite and Apple. van den Oever, A., & Christie, I. (2018). Stories screen narrative in the digital era. Amsterdam University Press. Weberg, A. (Director). (2013). Absent VIII a homage to Maya Deren. In. Sweden: Independent Production. Wowkowych, S., & Villaespesa, E. (2020). Ephemeral storytelling with social media: Snapchat and Instagram stories at the Brooklyn Museum. Social Media + Society, 6(1). Xu, X., & Schleser, M. (2020). Mobile studies congress. Retrieved from https:// www.nottingham.edu.cn/en/humanities-and-social-sciences/internationalcommunications/mobile-studies-congress-2020.aspx

CHAPTER 3

Mobile Storytelling: Elements, Evolution, and Experience Xiaoge Xu

Our evolving world is generating endless stories of all kinds, which are told in print, broadcast, multimedia, or any blended fashion. Ever since the advent of the Internet, stories have gone online. From offline to online, the change in storytelling is not a mere gear change, but a game change. Storytelling is no longer controlled by those who have the access to or the control over print, broadcast, multimedia, or any blended form of media. Instead, storytelling has become everyone’s business. The Internet has dramatically changed the storytelling landscape, in which stories can be told by both professional and amateur storytellers, generating more diversified stories. The Internet has also rapidly changed the power structure in storytelling, leading to more empowerment to more and more ordinary people in telling their stories. With the advance of mobile technologies, our online world with 4.66 billion active Internet users (Statista, 2021) has also gone mobile. In less

X. Xu (B) University of Nottingham Ningbo China, Ningbo, China e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_3

25

26

X. XU

than two decades, mobile has become the most popular medium with 5.25 billion unique mobile subscribers (GSMA, 2021) and 4.28 billion unique mobile Internet users (Statista, 2021) in the world. Going from offline to online and from online to mobile, storytelling has changed its strategies first from “online too” to “online first” and then from “mobile too” to “mobile first.” Among the increasingly diversified ways of storytelling, “mobile first” has become the dominant storytelling strategy. Mobile storytelling is different from the way stories have been told offline for centuries or online for decades. Mobile storytelling is faster anytime anywhere, easier to understand, and more multimodal than any other forms of storytelling to capture increasingly fragmented and competing attentions and to cater to increasingly diversified tastes, preferences, and styles in storytelling. Mobile storytelling is not, however, just a shift of a storytelling platform. It is not just another gear change. It is indeed another game change. To leverage the game change in mobile storytelling, this chapter elaborates elements, evolution and experience in mobile storytelling respectively and proposed a unique integrated approach to mobile storytelling enhancement. This approach is to integrate traditional and innovative storytelling elements, to leverage evolution in storytelling, and to maximize experience in mobile storytelling.

Integrating Storytelling Elements Traditional and Innovative Traditionally, among others, two representative categories can be used for illustration. The first category groups storytelling elements into the following five types: (a) action, (b) dialogue, (c) description, (d) inner monologue, and (e) exposition (Bunting, n.d.) The second category has the following seven types: (a) plot, (b) character, (c) theme, (d) dialogue, (e) melody, (f) décor, and (g) spectacle (Passi, 2019). Regardless of different categorizations, these storytelling elements remain essential in mobile storytelling although media and platforms have been changed. On top of the traditional storytelling elements, innovative storytelling elements brought about by mobile technologies include locative media, interactivity, collaborative features, augmented graphics, cloud computing, gamifications, and context-aware elements. Further examples

3

MOBILE STORYTELLING: ELEMENTS, EVOLUTION …

27

include animations and explainers, long-form scrollytelling, data visualization essays, expander format, user immersion and connection, mixedmedia, and personalized story formats (Amer, 2019). Still further examples include short documentaries, explainers, swipeable cards, blended media storytelling, listicles, smarticles, adapting stories, live blogging, automated storytelling, and 360-degree visuals, virtual and augmented reality (Ovaskainen, January 2, 2019). These innovative storytelling elements are popular and powerful in enhancing mobile storytelling. Both traditional and innovative mobile storytelling elements have to be integrated in order to enhance mobile storytelling to meet the changing needs, wants, tastes, and preferences of mobile users.

Reasons and Approaches Mobile storytelling has changed its strategy from “mobile too” to “mobile first” to cater to the changing needs of mobile users for ubiquitous stories. Furthermore, as mobile users’ attention can be fragmented and distracted by different people and things while on the move. Therefore, mobility in time, space, situation, and attention calls for short and different elements designed and delivered for short attention span among competing attentions in different situations. Mobile users tend to be multitasking with fragmented attention. Fragmented attention may also be prioritized for different mobile storytelling elements. For instance, while driving a car, a mobile user has to listen to audio story elements instead of video or visual ones. He or she may have sufficient time for visual elements while having snacks or drinking a coffee at a service area. A mobile user can switch to text or visuals when resting after listening to audio elements while jogging. While there might be many different ways of integrating mobile storytelling elements, there can be a sequential situationally determined approach to mobile storytelling elements integration: (a) to choose specific mobile storytelling elements, (b) to integrate the chosen elements, and (c) to prioritize them according to different situations.

Leveraging Storytelling Evolution To integrate mobile storytelling elements is just one of the effective ways to enhance mobile storytelling. Another way is to leverage the constant evolution in mobile storytelling. Storytelling in the mobile

28

X. XU

age has evolved, among others, from multimedia to omnimedia, from following stories to being followed by stories, and from being told to being experienced.

From Multimedia to Omnimedia Since the advent of the Internet and the Web, a story can be told by using a combination of different media elements, such as a text with an audio clip or a video with a text, accompanied by animated graphs or interactive maps. Storytelling in the age of the Internet and the Web features predominantly multimedia and interactive, enabling users to interact with other users, storytellers, and different story elements. In the mobile Internet age, smartphones and other types of mobile devices have further changed the way stories can be told. Stories can be told not only in a multimedia fashion but also in an omnimedia fashion. In other words, integrated multimedia stories can be told through a wholly integrated communication process, in which stories are disseminated by a combination of different producers, professional and amateur, such as radio stations, television stations, audiovisual publishers, film studios, publishing houses, newspapers, magazines, websites, social media, and we media. As they have all gone mobile, stories can be told anytime anywhere in any combination of media elements via any devices through any platforms.

From Following Stories to Being Followed by Stories Traditionally, stories used to be followed by readers, listeners or viewers as in the case of reading newspapers or books, listening to radio programs or watching TV shows. To follow stories via different media or platforms is the traditional practice in story consumption for centuries globally before the advent of the Internet. Since the advent of the Internet, especially with advances in mobile technologies, however, storytelling has been changed dramatically from following stories to being followed by stories thanks to mobile, artificial intelligence, and algorithm technologies. For instance, if we read a story in a mobile news app, more related stories will start to follow us wherever we go and whenever we read stories via the same app. The mobile,

3

MOBILE STORYTELLING: ELEMENTS, EVOLUTION …

29

artificial intelligence, and algorithm have enabled stories to follow anyone anytime anywhere.

From Being Told to Being Experienced Before the Internet, stories were told by professional storytellers in print or broadcast in a linear and non-interactive way, resulting in the traditional practice of having stories being merely told without being experienced. In the Internet age, however, stories have been gradually changed from being told to being experienced. The old fashion of “I report and you read” or “I broadcast and you watch or listen” is being supplemented or even replaced by a new mode of communication, which is typically represented by increasing importance being paid to experiencing stories instead of merely consuming them. Specifically, mobile storytelling is supposed to be a user-centered dynamic process and outcome of user interaction with it. Being no longer passive, readers, listeners, or viewers would like to interact with mobile stories. Behind their desire for interaction with mobile stories lies in the graduate shift to and emphasis on the importance of experience in mobile storytelling. In other words, they desire to be enticed, entertained, engaged, empowered, enlightened, and enhanced in mobile stories (Xu, 2018) in the age of experience economy.

Maximizing Storytelling Experience To experience stories instead of merely telling them is still another effective, probably, the most effective way. To experience stories in a mobile way is to interact with mobile stories, resulting in what is called mobile experience (Xu, 2018). As experience economy is taking the center stage and the world is increasingly dominated by mobile media and communication, mobile experience has become crucially important. In experience economy, work is a theater and every business is a stage (Pine & Gilmore, 1999) while experiences of all kinds can be created (Sundbo & Darmer, 2008) for consumers to be willing to pay for enjoying. The more enjoyable, memorable, and fun an experience is, the more willing consumers are to pay. This general trend has become even more apparent in the mobile world, where mobile experience is king although a content, a product, or a service remains fundamentally important. According to Pine and Gilmore, the passionate advocates

30

X. XU

of experience economy, our world is already at the stage of an experience economy, “where experiences supplant services as the predominant economic offering in terms of GDP, employment and especially actual value” (Pine & Gilmore, 2013, p. 26). The experience economy emphasizes selling experience (Pine & Gilmore, 1998) beyond its carrier, be it a product or a service. In modern societies, consumers expect more beyond a product or a service. Besides its four realms, i.e., entertainment, educational, esthetic, and escapist (Pine & Gilmore, 1998), experience also features heightened concentration and focus with all five senses being engaged, one’s sense of time being altered, being touched emotionally, being individually unique, contacting with raw and real things, being playful, being in control of the situation, a balance between the challenge and one’s own capacities, and being clearly goal-oriented (Boswijk et al., 2006). To satisfy these characteristics of experience is a must, according to Boswijk et al. (2006), for the creation and co-creation of meaningful experiences, which begins with focusing on the meaning of human experiences. Against this general backdrop, it is imperative to investigate experience in mobile storytelling in the context of experience economy. It is even more important and imperative to be equipped with knowledge and skills on how mobile experience can be leveraged and maximized to enhance mobile storytelling. In the following, three sections are devoted to (a) components of mobile experience, (b) factors shaping mobile experience, and (c) maximizing mobile experience.

Components of Mobile Experience Mobile experience can be defined as a process and an outcome. As a process, it refers to six different stages of mobile storytellers’ interaction with their audiences: (a) enticing, (b) entertaining, (c) engaging, (d) empowering, (e), enlightening, and (f) enhancing (Xu, 2018). In mobile storytelling, mobile forms, features, and functions can be fully leveraged to entice, entertain, engage, empower, enlighten, and enhance mobile users. The short and attractive forms and formats of mobile stories can entice their users while their immersive and interactive features can entertain, engage, and empower them. As an outcome, user experience refers to the results of these six stages of user interaction with a story: (a) enticed, (b) entertained, (c) engaged, (d) empowered, (e) enlightened, and (f) enhanced (Xu, 2018).

3

MOBILE STORYTELLING: ELEMENTS, EVOLUTION …

31

To be enticed is the very first essential step for us to enjoy a full-scale mobile experience. Without being enticed, it would be hard for us to start to enjoy other aspects of mobile experience. To be entertained is another important stage, where once we are entertained, we are closer to being engaged, which is another important step, without which we will move away. More important than being enticed, entertained, and engaged is to be empowered. This is especially true in the mobile age, where everyone can leverage mobile devices to be a storyteller or to contribute to storytelling about an event, issue, community, or a nation. Beyond being enticed, entertained, engaged, and empowered is to be enlightened and enhanced, the two final stages of the hierarchy of human desires for better experience. To be enlightened is to obtain a better understanding of what is being consumed so as to become a better consumer while to be enhanced is to enhance our consciousness, skills, and abilities as the final stage of mobile experience (Xu, 2018). As both a process and an outcome, mobile experience can be maximized through situationally prioritizing different dimensions or stages of mobile experience. As elaborated earlier, six stages or dimensions of user experience can be situationally prioritized in light of a specific situation a mobile user happens to be in. Situationally, a mobile user may prioritize a different stage or dimension of mobile experience. For example, while doing a fieldwork on the go, a mobile user has to prioritize search on mobile to be empowered or enlightened in terms of new information or knowledge.

Factors Shaping Mobile Experience Factors influencing mobile experience in general that have been identified by earlier studies include location, social context, mobility, battery, application interface design, application performance, phone features, routine, and lifestyle needs (Ickin et al., 2012); emotions and memories (Kujala & Miron-Shatz, 2013), ease of use, awareness, security, usefulness, availability, and accessibility (Sarmento & Patrício, 2012); mobile site optimization (Djamasbi et al., 2014); motor impairments and accessibility (Naftali & Findlater, 2014); trust and privacy (Krontiris et al., 2014); and device capabilities and settings, network performance (Patro et al., 2013), values, emotions, expectations, prior experiences, physical characteristics, personality, motivation, skills, usefulness, reputation, adaptivity (Arhippainen & Tähti, 2003), and attitude, social influence, media

32

X. XU

influence, perceived mobility, and perceived monetary value (Hong et al., 2008). Those factors can be generally described as internal factors, as they are closely related to all dimensions of mobile, that is, users, devices, apps, and networks. Besides the internal factors, there are other factors that may shape differences in mobile experience. They can be called external factors. They may refer to social structure (degree of Homogeneity, extent to which egalitarian, communication patterns with the outside world, gender, ethnicity, generational cultures, religion, education provision and the support for literacy, and language), temporal structures of daily life (subjective experience of time, societal time use structures and cultural expectations about time), values (openness to technological innovation, the degree to which societies are individualistic or group-oriented, other culture values), communication (communication patterns and expectations and low and high context cultures), and material cultures (special considerations, national differences in housing characteristics, and artefacts) (Thomas et al., 2005). Other external factors may refer to social factors (time pressure, pressure of success and fail, etc.), cultural factors (habits, norms, religion, etc.), and context of use (time, place, accompanying persons, temperature, etc.) (Arhippainen & Tähti, 2003). Being fully aware of identified factors that may shape mobile experience in one way or another is a precondition to be fully capable of maximizing mobile experience for enhancing mobile storytelling.

Maximizing Mobile Experience After locating different shaping factors, for effective and better maximization of mobile experience for enhanced mobile storytelling, special attention should be paid to three key areas: (a) how all indicators of mobile experience can be enhanced on the normative side, (b) how mobile experience can be enhanced on the empirical side, and (c) how mobile experience can be enhanced by narrowing the gap between the expected and actual mobile experience. Specifically, we should compare how all dimensions of mobile experience are properly embedded in a mobile storytelling app or a mobile site so that its users can choose, customize, and prioritize them according to their own desires, tastes, and preferences. The second key area is to compare how to leverage all changes in mobile users’ desires, tastes, and preferences for mobile experience by combining different methods such

3

MOBILE STORYTELLING: ELEMENTS, EVOLUTION …

33

as surveys, focus groups, interviews, and participant observations. The third area is to compare how the gap is narrowed between the normative and the empirical mobile experience. The results of the gap measurement should also be compared and triangulated to identify areas for enhancement. Further education, consulting, and services on mobile experience can be provided to mobile professionals and general publics in different countries. A general education on mobile experience should be offered to mobile users at different levels through different means. Mobile experience won’t be fully enhanced before mobile users and mobile content providers fully understand its changing forms, features, and functions as well as how it can be leveraged to enhance mobile communication. Once best integrated, mobile storytelling elements can play a crucial role in maximizing mobile experience in mobile stories. Specifically, among others, the following are three steps to be followed: (a) situationsensitive maximization, (b) stage-sensitive maximization, and (c) gapsensitive reduction. Situation-Sensitive Maximization: Highly situation-sensitive, mobile storytelling can be maximized according to specific situations. Different situations generate different needs and priorities for mobile experience in mobile storytelling. Therefore, the best way to maximize mobile experience is, first of all, to identify specific situations in which mobile storytelling is situated. Stage-Sensitive Maximization: As a process, mobile experience may differ among its six different stages. Mobile storytellers have to stay sensitive and responsive to those differences in mobile storytelling at different stages. Gap-Sensitive Reduction: Mobile experience can also differ at the normative and empirical levels. What experience should be in mobile storytelling can be different from what experience actually is. Wellequipped with this strong sensitivity of the gap, mobile storytellers should keep mapping and measuring the gap after stories are told. What is more important is to locate gap-shaping factors before securing more effective ways to narrow the gap as much as they can so that they can maximize mobile experience in storytelling.

Conclusion Mobile storytelling has experienced major evolution, resulting in being substantially different from its print, broadcast, and online counterparts.

34

X. XU

Being told anytime anywhere by anyone, mobile stories can also be told in an omnimedia fashion through an integration of different devices, media, and platforms. On top of being followed by their audiences, mobile stories can also follow their audiences anytime anywhere. Mobile stories are no longer just expected to be read, heard, or viewed. Instead, they are expected to be experienced by their audiences in the context of experience economy, where experience has become increasingly more important than content even though content remains essential. In the context of evolving mobile storytelling, on top of traditional mobile storytelling elements, more and more innovative elements have emerged and become popular in mobile storytelling. Mobile storytelling elements can become powerful when both traditional and innovative elements are situationally integrated. They can become even more powerful when they are situationally prioritized to cater to different needs, desires, preferences, and styles in mobile storytelling according to different situations. Keeping abreast with its evolution while integrating its traditional and innovative elements, mobile storytelling can be greatly enhanced through enhancing its experience. As both a process and an outcome of mobile users’ interaction with mobile stories, mobile experience can be reflected in its six components and stages, that is, enticement, entertainment, engagement, empowerment, enlightenment, and enhancement (Xu, 2018). Mobile experience can be maximized by prioritizing different components and stages according to different situations as well as by staying sensitive and responsive to the gap between what experience is expected and what experience is actually obtained. In a nutshell, to enhance mobile storytelling, we should be fully aware of, sensitive to, and responsive to evolving mobile storytelling. In enhancing mobile storytelling, different elements, traditional and innovative, should be integrated before we maximize different dimensions of mobile experience at different stages according to different situations. Ultimately mobile storytelling can be enhanced by bridging the gap between what experience should be and what experience is actually obtained in mobile storytelling in the context of experience economy.

3

MOBILE STORYTELLING: ELEMENTS, EVOLUTION …

35

References Amer, S. (2019, August 1). Why reinventing your mobile storytelling matter and how to do it. https://www.reutersagency.com/en/reuters-community/whyreinventing-your-mobile-storytelling-matters-and-how-to-do-it/ Arhippainen, L., & Tähti, M. (2003, December). Empirical evaluation of user experience in two adaptive mobile application prototypes. In MUM 2003. Proceedings of the 2nd International Conference on Mobile and Ubiquitous Multimedia (No. 011) (pp. 27–34). Linköping University Electronic Press. Boswijk, A., Thijssen, T., & Peelen, E. (2006). A new perspective on the experience economy. The European Centre for the Experience Economy. Bunting, J. (n.d.). 5 elements of storytelling. https://thewritepractice.com/showmore/ Djamasbi, S., McAuliffe, D., Gomez, W., Kardzhaliyski, G., Liu, W., & Oglesby, F. (2014, June). Designing for success: Creating business value with mobile user experience (UX). In International conference on HCI in Business (pp. 299–306). Springer. GSMA. (2021). Extensive datasets with a global reach. https://www.gsmaintellig ence.com/data/ Hong, S. J., Thong, J. Y., Moon, J. Y., & Tam, K. Y. (2008). Understanding the behavior of mobile data services consumers. Information Systems Frontiers, 10(4), 431. Ickin, S., Wac, K., Fiedler, M., Janowski, L., Hong, J. H., & Dey, A. K. (2012). Factors influencing quality of experience of commonly used mobile applications. Communications Magazine, IEEE, 50(4), 48–56. Krontiris, I., Langheinrich, M., & Shilton, K. (2014). Trust and privacy in mobile experience sharing: Future challenges and avenues for research. Communications Magazine, IEEE, 52(8), 50–55. Kujala, S., & Miron-Shatz, T. (2013, April). Emotions, experiences and usability in real-life mobile phone use. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1061–1070). ACM. Naftali, M., & Findlater, L. (2014, October). Accessibility in context: Understanding the truly mobile experience of smartphone users with motor impairments. In Proceedings of the 16th international ACM SIGACCESS conference on computers & accessibility (pp. 209–216). ACM. Ovaskainen, E. L. (2019, January 2). 9 types of visual storytelling on mobile. https://gijn.org/2019/01/02/9-types-of-visual-storytelling-on-mobile/ Passi, A. (2019, December 25). Elements of good storytelling. https://uxplanet. org/storytelling-elements-31b2a6a7e373 Patro, A., Rayanchu, S., Griepentrog, M., Ma, Y., & Banerjee, S. (2013, December). In Proceedings of the ninth ACM conference on emerging networking experiments and technologies (pp. 199–210). ACM.

36

X. XU

Pine, B. J., & Gilmore, J. H. (1998). Welcome to the experience economy. Harvard Business Review, 76, 97–105. Pine, B. J., & Gilmore, J. H. (1999). The experience economy: Work is theatre & every business a stage. Harvard Business Press. Pine, B. J., & Gilmore, J. H. (2013). The mobile experience: Past, present and future. In J. Sundbo & O. Darmer (Eds.), Handbook on the experience economy (pp. 21–44). Edward Elgar Publishing. Sarmento, T., & Patrício, L. (2012). Mobile service experience-a quantitative study. AMAServsig 2012. Statista. (2021). Global digital population as of October 2020. https://www.sta tista.com/statistics/617136/digital-population-worldwide/ Sundbo, J., & Darmer, P. (eds.). (2008). Creating experiences in the experience economy. Edward Elgar Publishing. Thomas, F., Haddon, L., Gilligan, R., Heinzmann, P., & de Gournay, C. (2005). Cultural factors shaping the experience of ICTs: An exploratory review. International collaborative research. Cross-cultural differences and cultures of research, COST, Brussels, pp. 13–50. Xu, X. (2018). Comparing mobile experience. In K. Norman & K. Kirakowski (Eds.), Wiley handbook of human-computer interaction set, volume 1 (pp. 225– 238). Wiley.

CHAPTER 4

Mapping Through Mobile Filming: An iPhone Mapper’s Flirtation with Google’s Artificial Intelligence (AI) Krishna Sankar Kusuma

Mapping “The Self” with Google AI My journey started three years ago in 2017 when I searched for the Dolls Museum in New Delhi on Google maps—I found great reviews and a preview of the place, complete with pictures. During my visit I kept taking good pictures, and while returning I uploaded all the pictures on maps as well as wrote a review about the place. I felt hopeful that someone will come across this information and be motivated to bring their children to the museum. I realized later that I am signed in with one of my Gmail accounts and my profile is created on Google maps. Later whenever I uploaded pictures and wrote reviews, I was able to get points that popped up on my screen and got added to the points that I had received as a reward for my contributions. With a gap of a few months, I started

K. S. Kusuma (B) Jamia Millia Islamia, New Delhi, India e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_4

37

38

K. S. KUSUMA

receiving monthly emails about my position, summaries of my contribution and I learnt that Google has awarded me the epithet of a “local guide.” On a lighter note, I keep telling my wife that I am working for Google. I used the iPhone SE for all the mapping activity. I have started delving deeper into mapping activity as it is also driving me to make use of more of its tools and techniques, which include introducing me to local guide networks, inviting me for a local guide buddy system. I slowly began to pick up the tips and tricks of mapping activity. I have started looking into other local guides’ points and their profiles, their way of writing reviews and so on. I also discovered that anyone can access my entire history of physical movement over all these years. Whenever I visit a place, it keeps telling me how many times I have visited the place earlier, with date, time and what was the typical duration that I had spent there. The “self” is left with no privacy even if one wanted in the game of mapping as the location or GPS should be kept open at all times for maps to continuously detect one’s position. The most common issue that I have encountered is the space that all this data occupies in the Google drive. The “Google One”,1 a cloud space is offered as a token reward only once, with free usage for one year to local guides to store the content on the drive. It is important to observe that Google never bothers to support users in any meaningful way, while they benefit from the user-generated data which is used for a variety of its experiments and innovation purposes. The phone memory is a rather pertinent issue, where one needs to be cautious about immediately uploading and erasing data once it is assured of being uploaded on the maps. The updating of 3G network to 4G has immensely transformed the participation in map making. Many times due to low bandwidth it is difficult to determine whether all the pictures meant for Google maps have been uploaded or not. As and when the bandwidth improves, it is often found that photos get uploaded multiple times. Interestingly, Google AI sometimes brands this as a malpractice. Additionally, it is also worthwhile to mention that the phone battery drains exponentially while using maps and GPS, causing hindrance to basic phone uses. Google mapping thus takes the form of a game and an addictive activity. The mapping users knowingly or unknowingly join the networked game play of live media content generation. “The performance of a game occurs in two, often simultaneous, domains: the player’s subjective experience, and the visible practice of playing” (Calleja, 2011) are the

4

MAPPING THROUGH MOBILE FILMING: AN IPHONE MAPPER’S …

39

distinctive features of digital gaming. Mapping is a social activity, where the user’s social and cultural background reflects in the aesthetics, spaces, perspectives of the media content. Google periodically declares the top 1% of the local guides in a city. While I have continuously been in top one per cent local guides, the struggles to sustain the activity of mapping continuously is a stiff challenge. The very fact that I have been continuously broadcasting myself in the public domain interacting through my phone with Google AI is a cause of sufficient concern. Invisibly, unknowingly I was interacting with cell towers, GPS satellites, bandwidth, antennas, Wi-Fi systems continuously. Real-time map data raising the concern to privacy of individuals, “my phone is also tracking my movements, often without me giving it permission to do so” (Farman, 2014). Information about sensitive locations is creating a space for terror attacks. “Snooping on the online activities of the activist or individual, tracking the accounts, phone using variety of technologies like Global Positioning Systems (GPS)” (Kusuma, 2018). The filming process imposes no notable restrictions on the aesthetics of the videos and photographs. Pictures have grains, shaky compositions, may sometimes be over exposed or under exposed, and it all depicts the state of the local guide where he/she might quickly manage to capture a picture on the go. Probably these kinds of glitch, error and jam gives birth to a new aesthetics of Google imagery. iPhone camera software is quick in capturing and processing to provide better images for Google. The communication between iPhone and Google maps is always in great sync to upload the media to it. iPhone’s computational photography adds not only metadata to its images but also enhances its creative elements. “The amateur image becomes immediately transformable, transmissible, transcoded, and taggable in real time” Chesher C. (2012). It has great capabilities of analyzing images metadata, algorithm and seamlessly communicating with other mobile applications. User interface, device integration and route quality of maps services differ from locality to locality and country to country. Some of the features don’t support some regions due to the limitations of the government and map companies’ collaboration about region-specific restrictions. Apple maps in iPhone and Google maps in android phones work efficiently with their maximum features. The ecosystem of native operating systems provides more ease of using and accuracy. Apple maps doesn’t have its presence in the android devices whereas Google maps is present

40

K. S. KUSUMA

in both the platforms. It is helping Google doubly from both the users base to improve its maps through crowdsourced data. In India the affordability of iPhone user base is smaller in front of massive android users. The accuracy like turn-by-turn navigation wise is something Google maps does well in India. When I visited Montreal and Toronto in Canada and Melbourne in Australia iPhone maps worked with absolute accuracy and the same with Google also. Exploring space beyond one’s location has been a defining trait of humanity. Brown M. C. (2006) rightly says, “Humans are spatial creatures.” The larger geo-political debates around boundaries have been over the sustained rift among nations as well as individuals. In the digital environment “locativity” is important, it is not only working as an evidence but also a powerful statement, a rich data. Botella C. L. (2012) argues that “images made with mobile devices (act) as a reflection of our mobile existence.” The map’s software embedded in mobile phones helps “locationalize” every movement. Maps have changed immensely for humans, animals, objects and the whole geography. Google Artificial Intelligence for instance has provided an immense boost to mapping technologies, helping both social interactions as well as corporations, for example, ridesharing, asset mapping, interactive gaming, media, health care, governance. Google Maps as a technology that “enables you to create applications that combine the mapping or earth information with your own set of data so that you can build customized views of information, all mapped to the geographical location of the items” (Brown, 2006). Google maps have grown in size and technology to incorporate information about the earth, sky and environment and hence become indispensable for people, from everyday users to space scientists. The ecosystem of Google constituted by corporate and public partnership makes it a huge event that is in a state of constant flux every second. This chapter is an attempt to present the users’ perspective of using mobile phone filming for mapping activity. “Film making was earlier confined to only a class of people. It was by and large a medium to communicate rich and elite individuals and groups to reach masses. With the advent of digital technology, particularly, smartphone, every user of this device is empowered to master the camera work or film making” (Bandi, 2020). I am placing myself in constructing these arguments as one of the active users as well as a contributor to Google maps and street view for examining the phenomenon from a set of holistic lenses.

4

MAPPING THROUGH MOBILE FILMING: AN IPHONE MAPPER’S …

41

Space is not static, it continuously changes both physically as well as digitally. It is the users who produce space. “Every society-and hence every mode of production…produces a space, its own space” Lefebvre H. and Nicholson-Smith D. (2009), through public photography in mapping, produces different perspectives of space. The digitally uploaded visual ecology of a space lessens the divides between private and public spaces, popular and less important, in fact every space is made public and their presence marked distinctly. The process brings into association a unique mode of ownership of imagery with the user’s profile identity.

Mapping as Everyday Public Activity Mapping activity is a new form of entertainment for users. The contributors derive fun from it and also find a social cause in making other users’ navigations and searches easier. Google maps is more than a company and software—it has emerged as the largest public networking engagement, where the public doubles as users as well as contributors and builds virtual cities and societies. As an end result it contributes to the making of big data, where much of the public does not know its real implications and ramifications. Knowing routes through navigation has been an increasing area of use for the public—knowing through maps about destinations, groceries, shopping and daily needs brings great comfort and essential skills to survive in a new place for the public. “The term ‘communication’ has had an extensive use in connection with roads and bridges, sea routes, rivers, and canals, even before it became transformed into ‘information movement’ in the electric age” (McLuhan, 1994). Digital maps enabled by the internet revolutionized physical or geographical communication. Mapping plays an important role in media content such as interactive documentary or i-doc, gaming, photography, locative journalism, locating crops, agriculture land and many more fields. Most of these interactions take place at a level that is far beyond one’s immediate awareness, but they are embedded in the practices of everyday locative technologies (Farman, 2014). The visual memories archived around a place and space by the state archives or individuals used to be a professional and specialized activity. The so-called ordinary public has no agency in its construction and their identities rarely find expression in these archives. Mapping activity helps to bring the digital socio-spatiality closer to upload one’s individual perspective of a place that can then be exhibited in the public domain through

42

K. S. KUSUMA

Google maps. India is a country typically characterized by caste discrimination. India is a caste ridden society, where disparities among castes are strong and always a hindrance to development and exercising democracy. “Caste is a notion, it is a state of the mind” (Ambedkar, 1936) which the entry to the Hindu temples is categorically restricted for dalits in the society. Dalits are marginalized communities belonging to constitutionally scheduled castes and tribals facing continuous discrimination whether it is in real space or digital space. Having said that, unlike reallife society, digital space brings more access and freedom to assert the dalit identity. The darsanic(vision) of God does not hold any caste restrictions for uploading or downloading any form of content. Maps provide digital access to the Gods and the temples’ architectural marvels. Initially there are restrictions to photograph certain portions like inner sanctum of temple, but now due to the penetration of mobile photography most of the imagery is available on Google maps. The access to the imagery of Gods and temples does not reduce the caste disparities. Modern and urban spaces like multiplexes, shopping malls create a new secular public space where entry and access is not restricted based on one’s caste identity, although class filters continue to exist even within these modern spaces. Google maps similarly creates a space where the freedom to mark a shop, whoever the owner may be, whichever caste he might belong. This doesn’t mean that urbanization and cities could eliminate caste, but the visibility of it will be significantly reduced but when context arises caste perspective will automatically overpower. Dalit participation as well as dalit localities in Google maps can be a worthy study. The mapping activity is largely participated in and contributed to by the anonymous public in building mega mapping data. It is worth studying further the class, caste dimensions of user profiles to understand who are these contributors. While mapping each one of us gets implicated within a “networked individualism” where we perform through our practices of continuous engagement of communication networking. It was described as an “operating system,” and that the new networked communication ecology is a result of the Triple Revolution which includes “social network, internet and mobile revolution” (Rainie & Wellman, 2014). Mobile has emerged as a powerful communication tool in everyday communication through the creation of self-styled media content. It was observed that in comparison to the traditional camera “camera phones capture the more fleeting and unexpected moments of surprise, beauty and adoration in the everyday” (Ito & Daisuke, 2003).

4

MAPPING THROUGH MOBILE FILMING: AN IPHONE MAPPER’S …

43

Methodology This study is confined to maps in mobile phones connected with Google map software. Maps are an integral part of many aspects of human life but the focus is on how individuals and the public at large are connected and how they internalize the process of mapping and mobilizing people with maps. There are integrated GPS services within the Google maps, street view and Google earth. This study concentrates singularly on Google maps. Street view is embedded with Augmented Reality and three-dimensional view is provided. But it is not so popular and accessible due to its high consumption of internet data and video playback. Collaboration and co-creation with user and mapping software creates new visual perspectives of physical society. “‘Co-creation’ as a broad term for the collaborations that emerge within that space” Aston et al. (2017). Google maps has a user-friendly design, which allows the iPhone creators to co-create and contribute media content on its platforms. This study seeks to explore how these rules of creativity have been emerging. One of the more significant questions discussed is in collaborative mobile filmmaking how Google is placing the user and creating content in the way it wants. Further, this chapter deals with the question of whether mapping can be looked at as a new entertainment digital platform for mobile filming. This study used a thick description of observations as a part of auto ethnography. Google maps have grown in size and technology about the earth, sky and environment to make it imperative for everyday users to space scientists. The ecosystem of Google sustained by corporate and public partnership makes it a body of data that expands every second. This study presents the users’ perspective of using mobile phone filming for mapping activity. I am placing myself in constructing arguments as one of the active local guides and reached level 9 of Google reviewers with the contributions consisting of 9586 photos viewed by 26,330,858, 1027 videos, 399 reviews and other additions to maps. This chapter discusses the mapping of public spaces for Google using the iPhone SE.

Mapping with Mobile Phone (iPhone SE) Mobile phones play a crucial role in mapping activities that an individual performs in collaboration with the internet and the global positioning system (GPS). This helps in the production and optimizing of individual

44

K. S. KUSUMA

viewer-specific “locative media.” The activity in question goes beyond locating a physical address in order to incorporate a creative description of the place enabled by mobile filming of photos and videos. One of the early predictions is the integration of maps and image processing and identification techniques in mobile phones is going to be of utmost necessity in the near future (Yang & Hsu, 2016). It was tested and proven that maps can be fruitfully integrated in third-party apps and various operating systems of mobile phones (Li & Zhijian, 2010), (Aldabbagh & Mohsen, 2014) and (Luo & Shen, 2009). Photography through mobile phones has dramatically changed conventional practices of taking pictures. The second camera and selfie mode together facilitate the creation of evidence-based visual imagery, for instance, the selfie confirms your presence in place at a particular time, this is prominently used for travel photography and trekking.2 The aura of a prominent place is included with the individual identity and unique perspective of an individual in the maps. Photos taken by the mobile phones reinforce the user’s individuality and the photographic product therefore being highly individualized, they have the ability to showcase the world uniquely (Gye, 2007). The vertical mode of photography facilitates an easy and quick mode of capturing a moment with one hand, which prominently provides a counter perspective for the horizontal/rectangular images uploaded in the Google maps. The physical understanding of the geographies cannot be made sense of in the same ways by AI technologies, in fact it needs a large amount of data to create more experiential and accurate geography. Collaborative user-generated content strengthens the accuracy and detailing of maps. The AI is fed every fraction of a second with the data. Introduction of augmented reality in maps is made possible with such data by Google. Google maps evolved keeping the requirements of the users and context produced by them over the years. One such feature is enabling the users to add four more types of traffic issues as well construction, lane closures, disabled vehicles, and objects on the road (Lekach, 2019). Mapping on the Google street view provides face blur by the software to conceal the identities of the people present in the visual uploaded about a place. It is an optional ethical given to the contributor who is identified by Google as “local guide.” This feature applies both to pictures as well as the 360° photos created by the inbuilt feature in street view. Google maps doesn’t provide the feature to mask or blur the identities of the people present while local guides capture the images. The images available

4

MAPPING THROUGH MOBILE FILMING: AN IPHONE MAPPER’S …

45

of a particular popular place are widely available with clear identities on display as well several instances of awkward situations depicted, and most importantly due to the speed or hurriedly clicked pictures create a lag in the images resulting in absurd images of people and public places. Collaboration is one of the key aspects in the mapping activity. The useful pictures can be liked and appreciated by the users who can further contribute by adding new images. The AI-enabled map activity provides approval of a place flagged by using different mappers. Once a local guide pins a new location which might be missing on the maps, is it subject to verification by other local guides to make sure the place exists and the depicted information is accurate. Mapping is going to be a new social media activity as well as an emerging content platform. Strategies of gamification and play are employed in mapping. The collaborative activity with users is implemented on a reward through points basis.3 Every contribution like a review, uploading a photo, video, road, correcting other mistakes in mapping is credited with points. There are different levels a user can reach, up to 10. Peer group training and interactions organized by Google local guide bloggers help increase the quality of content and mapping. To manage a large number of mobile filmmakers AI is used by Google to schedule the tasks, award the points, verify the repetitions in content and even check the user’s experience and credibility of contributors. AI is good at curating and archiving big data, it does the job of segregating/categorizing data swiftly. Google rewards each of the activities with points as follows 15 points for adding a place, 1 point for fact checking, 3 points for uploading a photo and 7 for a video. Apart from the above requirement in each level, one needs to meet a certain prescribed set of criteria, for example how useful your photos are or how many facts checking a local guide has done. Google system also shortlists the top “local guides”4 who are good at photography and videography for the reference and training of emerging local guides. Google decides on a date for meeting of the budding local guides to nurture their talents in recording better content with mobile phones. Local guides meet t at a pre-decided location and visit popular locations in the locality. For example a restaurant or a tourist spot to take the best visuals and post reviews under the supervision of expert reviewers and local guides. The collaborative activity with users is a reward through points basis. Every contribution like a review, uploading a photo, video, road,

46

K. S. KUSUMA

correcting other mistakes in mapping is credited with points. There are different levels up to 10. The peer group training and interactions will help increase the quality of content and mapping. Suggestions by the map can be viewed by the mapper to get some idea to which places need further data like an updated telephone number, email, timing of the shop, etc. Filming for the maps is generally networked and always in interaction with AI. However, very few filmmakers are aware of their everyday interaction being synced with Google AI. The analytics of the like feature, together with the best photos and videos help evolve the media sensibility of the mobile filmmaker. The general rules of photography and videography have been immensely re-defined with the coming of mobile filming for maps. The 360° spherical photo shooting with the help of dots simplifies the technology with inbuilt stitching options. The spherical cameras are new technology and involve new information and skills. Google made it simple and trained the users through gamification. The kinetics of the filmmaker seems suspicious and peculiar to others as they continuously follow augmented dots all over without changing their standing position. One’s body becomes a monopod and one’s hand works as a selfie stick while shooting 360° photos for Google street view‚ thus the body of the creator becomes part of the aesthetics of the content producer “…extensions of man—the technological simulation of consciousness, when the creative process of knowing will be collectively and corporately extended to the whole of human society, much as we have already extended our senses and our nerves by the various media…” (McLuhan, 1994). Internet technologies have come in bundled with the corporate aspirations, public participation, internalization of technology through “soft training” which means learning through peers and self. Every technology has its own political economy, they are not just technology alone, they are agendas, ideologies, tagged to larger geo-politics due to its global network and presence and data sharing. Google (AI) uses pattern recognition5 to make sense of images. Although the computer cannot understand pictures the way humans do, the AI uses a unique set of technologies to “make sense” through set patterns like face detection, color, common patterns.6 AI with 360/VR and AR help to give users better experiential visualizations. It is not limited to maps but also other forms recording and viewing devices. “Google Cardboard makes VR and 360° video possible in a communitybased context” (Schleser & Firestone, 2018). Integrating of maps data

4

MAPPING THROUGH MOBILE FILMING: AN IPHONE MAPPER’S …

47

through affordable VR devices provides a first-hand experience of the places in an immersive way. This study confirms that mobile phones are the best available devices to map a place and create a digital ecology of the landscape by adding audiovisual and textual descriptions of it. The interaction of maps is in sync with the iPhone SE camera software and provides a smoother interface to process the data faster. Once you have taken pictures and forgotten to upload them on the specific location, the moment you open a place on the Google map to upload data through access to the media gallery, it will remind you about the left-out images and videos to be uploaded.

Google Maps and AI Art Google with its vast data has ventured into experimenting with AI-based artwork through funding research scholars and individual artists alike by letting them use their resources. For instance, one of the projects aims to rebuild a 3D version of a city incorporating elements of its known past derived from crowdsourced platforms that use old maps both by Google as well as sourced from users, a temporal map server as well as a 3D experience platform Kiveris R. (2020). The transformation of the cityscape can be experienced virtually along with great nostalgic flavor. Nao Tokui in 2008 created a new way of experiencing the Google street view photos by adding suitable soundscapes with the help of deep learning models. The artwork is installed on a webportal where users can interact by experiencing the soundscape with Google visuals generated with the help of AI (Tokui, 2018a). In another project he has exhibited “Imaginary Landscape” (2018b) as part of the first AI art exhibition “Gradient Descent” at Nature Morte art gallery in New Delhi. The artwork is a simultaneous appearance of similarly identified streets, roads, windows of the world, derived from all the mega data available on Google Street View with the help of AI neural networks (Tokui, 2018b). The other experiment worth mentioning is that of creating artificial traffic jams by tricking Google maps. Simon Weckert, an artist from Berlin fed Google maps with misleading data by using a simple hack of keeping 99 phones in a cart and dragging in a less trafficked area Google maps mistook it and showed it as the road is jammed. This instance of how fake data affected Google and the users is an indication of massive data manipulation and chaos that such AI platforms are susceptible to (Weckert, 2020).

48

K. S. KUSUMA

Google has been experimenting with its crowdsourced data in many fields. Google Arts & Culture has been pioneering in bringing art to the virtual public within the digital realm. The initiative of digital museums with the combination of augmented reality and street view provides an intimate and interactive experience to the users. One such example is successfully bringing the works of the Museum of Modern Art (MoMA) from Manhattan, New York, United States.7 Fifty street artists from different parts of the world were brought to New Delhi by Google and painted the walls of colonial British architecture at Lodhi art district. The magnificent artworks create a sense of augmented presence of gigantic colorful paintings. Apart from the above, India has been witnessing art installations with Augmented Reality (AR), and AI-based video installations which are socially, politically and culturally motivated and reflexive of the hard reality. One such project is “Priya Shakti” comic series, a take on the rape culture, acid attacks oppression on women and other real-life genderbased atrocities. The project harnessed the power of multiple platforms, including both print, digital and location-based street art. The street art is a combination of thought-provoking imagery about gender stereotypes as well interactive through augmented reality app.

International Dimensions of Territory While on the one hand Google’s use of crowdsourced data can be seen as a certain form of utility, on the other hand, individuals provide their location, images, videos and description willingly allowing their privacy to be encroached by Google and other dependent mobile applications for free. Exploitation occurs in the form of using data meant for mapping elsewhere or misusing the data fed in other apps and artificial intelligence projects. There are provisions for multilayer verification to achieve accuracy of data, but sometimes the reportage of incorrect mapping locations might cost a person to lose his points and even data might get erased at one go. Mostly the hotel, restaurant, food items and tourism services that rely on user feedback are recommended by the app for customer comments. On my visit to Vietnam in 2018 most of the hotels, coffeeshops and other services were eagerly requesting customers to rate them on the Tripadvisor website. In India, Google through its reviews and ratings created

4

MAPPING THROUGH MOBILE FILMING: AN IPHONE MAPPER’S …

49

choices and a possible market both for the consumer and service operators. During my visit to Ningbo, China realized Google services are completely blocked and the Chinese apps and internet services are available with amazing facilities and high-end technology. “Baidu” maps are similar to Google but available only in Chinese language. Countries like China were able to realize the dominance and power of public mapping technology so it is a distant dream for Google maps to operate their data supremacy. It was reported in The New York Times that a generation of Chinese don’t know about Google, Twitter and Facebook (Yuan, 2018). The respondents for this article expressed that Chinese apps provide them with all that they need. On the other hand, iPhone maps work with perfect accuracy and precision available in both English and Chinese. Google’s Chinese version Google.cn started in 2006, could not survive more than four years and lost its market share. The political risk theory worked against Google (Stevens et al., 2015). Google often does not encourage mappers to provide low ratings or write a negative review. If any top tanked local guide is found to be consistently reviewing sentimentally and negatively, he/she might soon lose the points (these are unverified bits of information gathered from Google local guides blogs and www.localguideconnect.com where many complaints and speculations about loss of points were reported). Overall, criticism and lower rating for a service is mild and negligible by the local guides. Because once a local guide is locked in this system of points, ranks and likes it will be imperative for them to continue to contribute in a way that amplifies their credentials. Map data can be misused anonymously for criminal activities like kidnapping, riots and terrorist activities (Burney et al., 2018). Unlike previous years, elections have been increasingly fought using the data about each of the citizens. The geo-tagged houses along with other sorts of profiling based on caste, income and religion have the potential to create ample scope for the riots and internal conflicts of hate. Locating the physical assets fo the citizen Indian government has been gainfully using GPS services “geo-tagging is being used in a range of government schemes MGNREGA, Pradhan Mantri Awaas Yojana (Grameen), Pradhan Mantri Gram Sadak Yojana (PMGSY), for toilet construction under Swachh Bharat, as well as for highways and urban housing” (Tewari, 2019). Elections are another major event where maps play pivotal roles in household mapping of the voters, ideologies and mood of the voters. Political parties spending on Google products and services are steeply

50

K. S. KUSUMA

increasing in India (Khan, 2019) Google mapping increasingly intrudes not just within the intimate spaces of people and their cultures, but also becomes a tool in the hands of the state. Given the extent of its proliferation, Google maps/data shall continue to play a central role, indirectly or directly, in the local and global conflict management/operations, war and defence. The curious case of Jamia Nagar area in New Delhi where my university and large number of Muslims live often carries inaccurate as well as incomplete maps. Whereas the nearby upscale market place surrounding Surya Hotel, New Friends colony maps are accurate and provide 3D view. The incomplete maps carrying little or no information about streets and amenities deny the local Muslim community easy access to quick details about transport, eateries, medicine and other daily necessities.

Conclusion Crowdsourced data from mobile filming and user-generated content is not only useful for public good, but Google exploits its contributors in terms of authorship without paying any royalty, which amounts to an unethical practice of the company as the information is sold alongside premium services to the corporate clients. Google has become increasingly exploitative through Google maps both ideologically and economically. Google fails to infuse an ethical practice of mapping for the local guides as well in providing them certain safety measures while mapping. Aesthetically, mapping activity kills the aura of a place. It constructs the imagery from the multiple perspectives of varied users and results in a new way of looking at the place. It tampers with the uniqueness and “sanctity” of a place that is intimate and physical, almost like the concept of “Garba Gudi” or Garbhagriha (the innermost sanctum sanctorum of the god inside the temple) the site of which is strictly for physical visit of devotees not for any form of replication or photographing. Cultural AI is based on gathering and analyzing data about behaviors of millions of people. This evasive integration of AI in our cultural decisions has the power to change the very direction and future of culture. By and large mapping with Google has emerged as an extremely widespread, addictive and user-friendly engagement for users. Google maps has a potential to mobilize masses through its initiatives like mapping toilets, ATMs, COVID-19 layer which provides information

4

MAPPING THROUGH MOBILE FILMING: AN IPHONE MAPPER’S …

51

about the number of cases in a particular area (Banerjee, 2020). Information can be crowdsourced in rural India through citizen journalism. Such form of journalism tends to democratize the information which is warranted for the development of the marginalized sections of the society (Biswal, 2019). Google however fails to accommodate discriminatory spaces, social or cultural oppression and in providing equal clarity of maps in all places, for bandwidth poverty means being excluded from the making or using of maps. Google can improve its features by providing safer routes for women in India to reduce rapes and atrocities.

Notes 1. Google one is a commercial cloud space fo everyday users. Before the launch of new technologies by maps and other technologies it often tries with its mapping community. 2. Google maps introduced location stamp on the mobile photographs. See https://support.google.com/photos/answer/6153599?co= GENIE.Platform%3DAndroid&hl=en. 3. Points, levels, and badging—Local Guides Help. In: Google. https://support.Google.com/local-guides/answer/6225851? hl=en. Accessed 8 Jul 2021. 4. Google names the contributors to maps as “local guides” as they keep providing more visual data and mapping of the place. See https://maps.google.com/localguides/. 5. How Google uses pattern recognition—Privacy & Terms. In: Google. https://policies.google.com/technologies/pattern-recogn ition?hl=en. Accessed 16 Dec 2020. 6. Pattern Recognition. In: Computational Thinking. https://sites. google.com/isabc.ca/computationalthinking/pattern-recognition. Accessed 15 Dec 2020. 7. Museum of Modern Art, New York, United States—Google Arts & Culture. In: https://artsandculture.google.com/partner/ moma-the-museum-of-modern-art. Accessed 18 Dec 2020.

52

K. S. KUSUMA

References Aldabbagh, O., & Mohsen, K. J. (2014). Design and implementation an online location based services using Google Maps for Android Mobile. International Journal of Computer Networks and Communications Security, 2, 113–118. https://doi.org/10.47277/ijcncs/2(3) Ambedkar, B. R. (1936). Annihilation of caste; speech prepared for the annual conference of the Jat-pat-todak Mandal of Lahore, but not delivered. B.R. Kadrekar, Bombay, India, p. 31. Aston, J., Gaudenzi, S., & Rose, M. (2017). i-Docs: The evolving practices of interactive documentary (p. 7). Wallflower Press. Bandi, B. (2020). Foreword. In K. S. Kusuma (Ed.), Communication practices with mobile phones: A collection of essays in mobile media studies in India (pp. 3–5). Amazon. Banerjee, S. (2020). Navigate safely with new COVID data in Google Maps. In: Google. https://blog.google/products/maps/navigate-safely-new-coviddata-google-maps/. Accessed 20 Dec 2020. Biswal, S. K. (2019). Exploring the role of citizen journalism in rural India. Media Watch, 10, 43–54. Botella, C. L. (2012). The mobile aesthetics of cell phone made films: From the pixel to the everyday. KEPES, 9(8), 73–97. Brown, M. C. (2006). Hacking Google Maps and Google Earth (pp. 3–4). Wiley. Burney, A., Asif, M., Abbas, Z., & Burney, S. (2018). Google Maps security concerns. Journal of Computer and Communications, 06(01), 277–278. https://doi.org/10.4236/jcc.2018.61027 Calleja, G. (2011). In-game from immersion to incorporation (pp. 7–9). The MIT Press. Chesher, C. (2012). Between image and information: The iPhone camera in the history of photography. In L. Hjorth, J. Burgess, & I. Richardson (Eds.), Studying mobile media: Cultural technologies, mobile communication, and the iPhone (p. 99). Routledge. Farman, J. (2014). Locative media. In: The Routledge handbook of mobilities (pp. 253–262). Routledge. Gye, L. (2007). Picture this: The impact of mobile camera phones on personal photographic practices (p. 285). Continuum 21. https://doi.org/10.1080/ 10304310701269107 Ito, M., & Daisuke, O. (2003). Camera phones changing the definition of picture-worthy. Japan Media Review, 205–215. Khan, F. B. (2019). The game of votes visual media politics and elections in the digital era (pp. 181–182). Sage. Kiveris, R. (2020). Recreating historical streetscapes using deep learning and crowdsourcing. In: Google AI Blog. https://ai.googleblog.com/2020/10/ recreating-historical-streetscapes.html. Accessed 8 Jul 2021.

4

MAPPING THROUGH MOBILE FILMING: AN IPHONE MAPPER’S …

53

Kusuma, K. S. (2018). Media, technology and protest: An Indian experience. Language in India, 8, 120–123. Lefebvre, H., & Nicholson-Smith, D. (2009). The production of space (p. 31). Blackwell. Lekach, S. (2019). Let the traffic reporting begin for Google Maps users on iOS devices. In: Mashable India. https://in.mashable.com/tech/7736/letthe-traffic-reporting-begin-for-google-maps-users-on-ios-devices. Accessed 15 Apr 2020. Li, H., & Zhijian, L. (2010). The study and implementation of mobile GPS navigation system based on Google Maps. 2010 International Conference on Computer and Information Application, 87–90. https://doi.org/10.1109/ iccia.2010.6141544 Luo, R., & Shen, Y. (2009). The design and implementation of public bike information system based on Google Maps. 2009 International Conference on Environmental Science and Information Application Technology, 2, 156–159. https://doi.org/10.1109/esiat.2009.298 McLuhan, M. (1994). Understanding media: The extensions of man (p. 96). The MIT Press. Rainie, L., & Wellman, B. (2014). Networked: The new social operating system (p. 7). The MIT Press. Schleser, M., & Firestone, R. (2018a). Pasifka youth and health perspectives: Creative transformation through smartphone filmmaking and Digital Talanoa. In Mobile story making in an age of smartphones (p. 167). Palgrave Pivot. Schleser, M., & Firestone, R. (2018b). Pasifka youth and health perspectives: Creative transformation through smartphone filmmaking and Digital Talanoa. In Mobile story making in an age of smartphones. Palgrave. Stevens, C. E., Xie, E., & Peng, M. W. (2015). Toward a legitimacy-based view of political risk: The case of Google and Yahoo in China. Strategic Management Journal, 37, 945–963. https://doi.org/10.1002/smj.2369 Tewari, R. (2019). Geo-tagging—One of the secrets behind the success of Modi govt’s welfare schemes. In: ThePrint. https://theprint.in/india/geo-tag ging-one-of-the-secrets-behind-the-success-of-modi-govts-welfare-schemes/ 272165. Accessed 1 Jul 2021. Tokui, N. (2018a). Imaginary Soundscape. https://experiments.withgoogle. com/imaginary-soundscape.Exhibited on Oct, 2018. Qosmo, Inc. Accesssed 19 May 2019. Tokui, N. (2018b). Imaginary Landscape. https://naotokui.net/projects/ima ginary-landscape-2018/.Exhibited on 17 Aug to 15 Sep, 2018, Nature Morte, Dehli. Accessed 18 Dec 2020. Weckert, S. (2020). Google Maps Hacks. http://www.simonweckert.com/Goo glemapshacks.html.Accessed. Accessed 8 Jul 2021.

54

K. S. KUSUMA

Yang, S.-Y., & Hsu, C.-L. (2016). A location-based services and Google maps-based information master system for tour guiding. Computers & Electrical Engineering, 54, 57–105. https://doi.org/10.1016/j.compeleceng. 2015.11.020 Yuan, L. (2018). A generation grows up in China without Google, Facebook or Twitter. http://ir.westcliff.edu/. http://ir.westcliff.edu/wp-content/ uploads/2018/08/A-Generation-Grows-Up-in-China-Without-Google-Fac ebook-or-Twitter.pdf. Accessed 28 Dec 2020.

CHAPTER 5

Shoes and Taxis as Mobile Storytelling Tools: Stories from the Frontline About Immigration and Integration in Canada Gerda Cammaer

This chapter offers an applied description and analysis of three related mobile cinema projects with immigrant communities in Toronto, Canada. The projects are related in media, theme and content, but they differ in methodology. The first one, Taxi Stories (2017), is a project about immigrants who became taxi drivers (all men), filmed on mobile devices by student teams from the Masters in Fine Arts Program in Documentary Media. It was an initiative of the Harbourfront Art Centre and as such it was also a collaborative community project. The Shoe Project already existed since 2010. It was founded by Katherine Govier, with a purpose to give immigrant and refugee women in Canada a chance to develop their leadership skills through writing, storytelling‚ and public speaking. In 2019, I started with mobile filmmaking workshops with the aim to

G. Cammaer (B) School of Image Arts, Ryerson University, Toronto, ON, Canada e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_5

55

56

G. CAMMAER

give some of the women participants also basic skills as budding filmmakers. Different than in the Taxi Stories project, the women participants are the filmmakers themselves. Building on this experience, in 2020 we started the Shoe Project Shorts with the generous support of an RBC1 Immigrant, Diversion and Inclusion grant. In the true spirit of The Shoe Project, with weekly workshops over a period of three months, the women participants turned their own shoe stories into short films, developing various filmmaking skills in the process (at a more advanced level than what we are able to do within the mobile filmmaking workshops, which are only 2 days). Due to the Covid-19 crisis, the training happend online and in collaboration with Trinity Square Video, another local community organization.2 This project has thus become even more collaborative and community-based than originally intended. In the process, these three projects not only empower the participants through enhancing their communication skills, but also generate valuable knowledge about the experiences of new Canadians by making art about critical issues pertaining to the integration of immigrants and refugees into Canadian society.

Taxi Stories Taxi Stories was a project developed and executed with the main purpose to contribute in an original way to the Canada-150 celebrations in the summer of 2017. The project proposal was submitted for Government funding by the Harbourfront Centre, a non-profit cultural organization which offers a wide range of innovative programming in the arts, culture, education‚ and recreation, bringing together the indigenous arts of Canadian culture and the rich mosaic of other cultures in Canada as an immigrant country. The Centre presents, commissions‚ and incubates work of contemporary artists and delivers cultural, educational and recreational experiences. Its aim is to contribute to learning and innovation while strengthening Toronto’s identity as an international and multicultural city. Diversity has always been at the heart of Harbourfront Centre. Early in the process, the Habourfront Centre solicited my help to help develop and organize the Taxi Stories project. With my experience as a film educator, a mobile filmmaker and a mobile film scholar interested in migration and mobility, it was obvious to me that the best way forward was to do this as a series of short films, all shot on mobile. Since bringing different communities together is at the core of the Centre’s mandate,

5

SHOES AND TAXIS AS MOBILE STORYTELLING TOOLS …

57

I also quickly decided that this would be an interesting extra-curricular project for my students in the MFA Program in Documentary Media, who themselves are of diverse multicultural backgrounds. I also developed the form, look, presentation format and methodology for the short films, in addition to the films being shot on mobile devices by teams of two students during a taxi ride. In short, my role was to be the creative producer for the different films, as well as the curator and the organizer of the final gallery show at the occasion of Toronto’s multicultural day (June 27, 2017). In its final form, the project contained 15 short videos, shot by 7 student teams of 2, based on an interview with a taxi driver filmed and recorded during a random 20$ taxi ride in the city. All videos were filmed from the back seat, as a passenger, with the sound person sitting on the front seat (close to the subject) in a lower or a corner position as to stay out of the frame. The final videos vary between two and five minutes in length, and each has its own visual style and content. Some were filmed exclusively inside the taxi; others also have some images from the taxi on the road or even along the side of the taxi (with the smartphone camera mounted on a brace in the window). All of them show the taxi driver (and the car itself) “in action” and also in this sense this is a real “mobile” film project with lots of tracking shots and other moving camera images— including hand-held camera, which gives the films a more personal and intimate sensation. One specific image all teams were asked to shoot, was to film the eyes of the taxi driver in the rearview mirror. This had two purposes: this is how drivers always communicate with their passengers, but also, it allows to frame the taxi drivers in a quasi-anonymous way by not filming them from the front and full face. This was necessary to get the full participation of the drivers and to obtain the most honest accounts of their experiences. After all, the project was inspired by the reality that there is always something particular that happens when driving with a stranger, particularly when riding a taxi. Freer discussions seem to happen, as there is less at stake (one will probably never see this person again) so there is less danger when expressing possibly unpopular or controversial opinions, but also, there is the pressure of the time limit, as ultimately the only time one can talk to each other is during that one ride, creating a kind of “now-or-never” feeling. Another common experience is that taxi drivers tend to ask their passengers where they are from, particularly if they look or sound like a stranger: this is a very popular conversation opener. The Taxi Stories project was entirely based on this premise,

58

G. CAMMAER

turning it around and asking that same question to the drivers themselves. Most taxi drivers in Toronto are immigrants in Canada, and they see the city and the country through the eyes of newcomers. That often leads to interesting observations about what it means to live in Canada, including the trials and tribulations of starting a new life here. Interesting enough, almost none of these taxi drivers ever imagined that this would be their new profession. Most drive a taxi out of necessity, and with hopes that they can pave the road to a better life for their children. Another important aspect of the project has to do with its third partner: Beck Taxi company. This is the biggest taxi company in North America, and as such one of the biggest employers of immigrant workers. It was the company’s wish to do this project, particularly at the time of the Uber controversy in Toronto (and many other big cities around the world).3 The immediate and growing menace of cheap (and untrained) competition in the taxi market, inspired the company to join forces with Harbourfront Centre. They embarked on this mobile film project as to help create a positive image of taxi drivers, both to the community at large as to the taxi drivers themselves: giving them some recognition and encouragement when morale among their drivers was really low. Beck Taxi felt it was necessary to highlight the positive contributions of taxis and taxi drivers to life in the city. They agreed to contribute the drivers their time and pay for all the rides (all students were offered 20$ taxi vouchers) so that they would be able to show the person behind the wheel as a person, not just as a driver. It was agreed early on that the drivers would in no way be pressured by the company to either participate, or to say certain things. The aim, both for Beck Taxi company and for the Harbourfront Centre was to collect honest personal stories from these new immigrants. The 15 short films therefore contain free and frank conversations, not scripted in anyway by the language of diplomacy. The drivers were granted some anonymity by the method of shooting (they make eye contact in the rearview mirror, but we never see their full face) and by using only their first names or initials in the credits of the films. All the participants signed release forms (a requirement for any film shoot at the university), and for those as well, only first names were used. The drivers knew about all these “guidelines” beforehand, and I do believe it contributed to them being more open and honest with the student teams. A common theme was that most companies or organizations in Canada require that job applicants have Canadian experience. But since new immigrants are never offered a job in Canada this becomes a catch 22: they cannot acquire any Canadian experience hence they cannot get any Canadian job and vice versa. This is how all the experience, expertise,

5

SHOES AND TAXIS AS MOBILE STORYTELLING TOOLS …

59

and training they brought from their home countries, some of them even as doctors or engineers, gets lost. Their only choice was to take a job as a taxi driver to survive financially, and to be able to support their families. This stands in stark contrast to the image of Canada as an extremely welcoming country with an accepting and tolerant culture. In the end, the Taxi Stories project was also a fully mobile project in its final presentation. All the videos were uploaded to tablets which were then mounted on the back of the right front seat in a taxi (in front of where a passenger usually would sit in the back). This is an already familiar use of that seat, as many taxis in Toronto have a small screen mounted there to play commercials or offer video games to their passengers, which provides extra revenue. Three taxis were parked right in front of the Harbour Centre Art Gallery, of which the entire window wall was left open, as if the taxis were an extension of the gallery into the parking lot. Inside the gallery another show about multiculturalism was happening at the same time. Visitors could take place in any taxi, pick and choose a video of the playlist, and watch it from the comfort of the back seat. That the videos were also filmed in this manner, by a passenger on the back seat, contributed to this being an even more realistic and recognizable experience, and I am convinced that this contributed to a better understanding and deeper appreciation of the Taxi Stories. As such, a true multicultural exchange between Torontonians and new Canadians was successfully established with a project that was truly mobile, collaborative and community-oriented in every aspect of its creation and presentation.

The Shoe Project The Shoe Project was founded in 2010 and is directed by Canadian writer Katherine Govier.4 The Shoe Project gives immigrant and refugee women in Canada a chance to develop their leadership skills through writing, storytelling‚ and public speaking. Under the guidance of experienced writers, women recruited from the Greater Toronto Area (GTA) and across Canada write stories of their arrival in and adaptation to Canada through a pair of shoes. This is always followed by six weeks of performance coaching as the women turn their stories into a theatrical performance: they all read their stories on stage for a live audience. Women from around the world, different languages and cultures are inspired and strengthened by sharing their individual shoe stories with their peers.

60

G. CAMMAER

Shoes are a great icebreaker and a fantastic metaphor for the transformation that takes place in migration (to be on the move, moving from place to place, the journey, taking small or big steps toward integration, etc.). The Shoe Project thus takes each participant through a personal journey of critical reflection as she shares stories of her struggles in her country of origin, her manner of leaving and arriving, and her challenges in becoming a member of Canadian society. Each story is unique; some are sad, even heartbreaking, but all of them are entertaining, some are even funny. Creating these stories gives members a chance to laugh and cry in an empathetic environment. In this process, the project not only empowers the participants through enhancing their communication skills, but also generates valuable knowledge about the experiences of new Canadians, all by means of making art about critical issues pertaining to the integration of immigrants and refugees into Canadian society. The Canadian community thus discovers voices and issues related to immigration that are often left unheard; newcomer women’s stories of internal dilemmas, struggles, dreams, successes and resiliencies. These community arts projects thus contribute to the immigrant and refugee women’s lives as well as to the community through individual bonding and a common demonstration of resilience. In the fall of 2019, I volunteered to help these women to learn and develop filmmaking skills as the next step in their growth as community leaders and professional communicators. I did a mobile film workshop for a small group of Shoe Porject participants in Toronto. The workshop was in two parts: shooting and editing. For the shooting workshop, I started with a lecture about mobile filmmaking and micro-cinema, and by giving them some very basic technical tips. After a morning of in-class learning and lecturing, including showing many examples of short films shot on mobile by former students, established independent artists or amateur filmmakers, the workshop became purely practical. The women were sent out to record a short captivating moment. They each shot with their own smartphones and as so often with mobile cinema, this helped to demythologize the filmmaking process and lower the threshold. Inspired by the power of doc-moments and everyday filmmaking, the women were asked to go for a walk in the neighborhood and come back with a onetake or in-camera edited short video of something they observed and that for some reason touched them. Afterward, we all gathered to watch the mobile videos, and discuss the form and content. Their “doc-moments” each captured something the women had a connection to, something that

5

SHOES AND TAXIS AS MOBILE STORYTELLING TOOLS …

61

reminded them of “home” or that in some way they observed as “typical” images for Toronto (and by extension for Canada and/or North America). Sharing these images and discussing them together was the highlight of that first day. Each in their own way, these short mobile videos were small gems in everyday filmmaking and solid building blocks for the editing workshop that took place a couple of weeks later. The second workshop had a similar structure with first a lecture part illustrated with many examples and practical tips about image editing and sound, followed by a practical hands-on workshop. Each of the participants was allowed to choose the editing software they preferred, either on mobile, tablet or on a laptop. With the smartphone images they had shot during the first workshop, sometimes complemented by images they had shot on their own in the meantime, they each edited a short video that was supposed to be both captivating and poetic in nature. They were asked to pick an emotional state or mood for their work (funny, sad, contemplative, polemic, etc.), and then work toward creating that with the images and sounds they brought with them or found online. We were not able to finish the films within the timeframe of that oneday workshop, but the women were able to build some confidence as budding filmmakers. They experienced first hand how mobile filmmaking can be another instrument for them as storytellers, and especially how it is both an affordable tool and an achievable goal. For me, these very first mobile films edited by the women participants carried with them the promise to stand out in the mass flux of online videos simply by their critical reflection, emotional impact and by their valuable story content. Combining their skills as storytellers which they developed as part of the Shoe Project with the basic film skills explored in my mobile film workshops, these women took another step on their path to integration and social development as new immigrants and refugees in Canada.

The Shoe Project Shorts Storytelling and film are excellent tools to create a sense of connection to each other. Hence, Katherine Govier and I decided to take our collaboration to the next level and apply for a grant to turn the actual shoe stories into short films, all shot and edited by the women themselves. Our main goal was that the women in the process develop enough basic filmmaking skills such as shooting, editing, sound recording and sound mixing so that they can keep producing audiovisual content in the future on their own.

62

G. CAMMAER

The films also bring to the fore these women’s specific cultural and artistic traditions, since most cultures across the world have long established oral and visual traditions for story making and storytelling. Ultimately, we hope that this will help to make the shoe stories more accessible and share them more broadly (even internationally). Moreover, just as The Shoe Project has been instrumental in creating a new genre in Canadian Literature called “the short first-person immigration memoir” we similarly hope to expand the Canadian Film canon with the films that will be developed as part of The Shoe Project Shorts. Key here is the process of artistic development the women embark on, where they learn about the difference between story development for print, performance on stage, and screen writing, and this for a variety of short film genres (animation, fiction, documentary, experimental). Besides basic notions and techniques of script writing and development, a lot of attention is paid to the development of each filmmaker’s personal audiovisual style, with an emphasis on voice and storytelling as the cornerstones of powerful personal filmmaking. This is achieved by teaching the participants how to make and use story-boards, work with shot-lists, as well as by doing test shoots and test recordings and finally a lot of trial and error in the editing stage. The constant dialogue about personal voice and style is sustained throughout the entire production process. The two mentors, Serene Husni and myself (both also immigrants) played a pivotal role in encouraging the participants to tackle the sociocultural issues linked to their immigration and transition stories by using their own imagination, originality and creative engagement. The Shoe Project Shorts “film residency” was an intensive collaborative creative process, through mentoring, dialogue, exchange, empowerment, peer and group learning that ran over a period of three months. The main learning phase for the first group of six participants, were six intensive four-hour online workshops that offered some basic instruction about the different aspects of filmmaking from pre- to post-production. After these more intense training moments in the beginning, we had weekly conversations or “film clinics.” These were more informal gatherings (also all online) to which the women could bring their questions, problems, practical solutions they discovered, and share this with their fellow participants and the two mentors. The mentors were there mostly in their roles as film teachers and problem solvers, but also to offer mental and moral support, and extra encouragement when things didn’t go as planned. As the women progressed in their first film adventures, those weekly

5

SHOES AND TAXIS AS MOBILE STORYTELLING TOOLS …

63

conversations became extremely valuable collective learning moments, sometimes frustrating (especially due to the limitations of teaching online) but always enjoyable. The frequency and regularity of these “film clinics” contributed tremendously to creating a supportive and encouraging environment for the women, which compensated for the lonely process filmmaking can be, particularly in times of lock-downs and curfews. To select the first cohort of six women participants, we followed a two-step process. First, we read and checked all the existing shoe stories and selected those that had the possibility to be adapted for film. Second, we selected the twenty-five best stories, not only looking at the story itself, but also the availability of the women, and the probability that they would succeed (loosely based on the past experience of working with these women in The Shoe Project). These twenty-five “finalists” were contacted with an email that explained the demands and expectations for participation and soon we had six willing volunteers to embark on their first film adventure. Thanks to the RBC grant, we were able to offer the women a decent artist fee, besides all technical equipment they needed and extra technical help through the collaboration with Trinity Square Video, something we added to compensate for the loss of access to the university’s facilities and equipment due to the Covid-19 crisis and the lock-down(s). At the time of writing this chapter, five first short films were finished and screened online for Canada’s 2021 Muticultural Day celebrations, while the Shoe Project Shorts continues with new participants. To end, for both The Shoe Project and The Shoe Project Shorts monitoring has a high priority. This is mostly a self-reflexive process. Each participant and mentor was asked to fill out a self-assessment form and to reflect on what they learned in terms of filmmaking skills, how this is the same or different from what they learned as part of The Shoe Project, and how they plan to use their newly acquired skills and knowledge in the future. Similarly, the mentors are given a chance to reflect on their evolution as instructors/collaborators of The Shoe Project Shorts. Peer and group learning, as well as process-based filmmaking, are the cornerstones of this project, and this all the way through.

Conclusion In all three projects described here, artistic development through some form of collaborative filmmaking and peer learning was central. These

64

G. CAMMAER

projects were also driven by the idea that storytelling and film are excellent tools to create a sense of connection, even with complete strangers like taxi drivers. Every project aimed to generate a meaningful dialogue and foster social integration through forms of community-engaged art. No matter the differences between the Taxi Stories project and The Shoe Project or The Shoe Project Shorts in terms of methodology, the stories that emerged all somehow centered around similar issues of integration and assimilation to Canadian culture. A good example is the recurring complaint that new immigrants to Canada need “Canadian experience” to find work, but since they are never hired or have a really hard time getting internships, they are never eligible and end up having to take any job that can help pay the bills (for example drive a taxi) or re-school themselves (for example an eye surgeon who becomes a naturopath). These three projects thus expose cracks in the perfect picture of Canada as a multicultural and immigration friendly nation and “the Canadian community discovers voices and issues related to immigration that are often left unheard; newcomer’s stories of internal dilemmas, struggles, dreams, successes, and resiliencies” (The Shoe Project). These three projects also bring to the fore the advantages of mobile cinema, as its affordability and user-friendliness offers both the participants and the viewers a real chance to walk a mile in someone else’s shoes or experience a true ride-along. The three projects each in their own way confirmed the strengths of mobile filmmaking as a facilitator for storytelling, an enabler for story sharing, and as a cultural mediator, be it as a conversation starter or a conversation motivator. The mobile stories created in the process were all written and made with humility, honesty and respect. They are different from the mainstream, without being controversial or oppositional. They are different, because they are told with immediacy and intimacy. These are first-person stories, and firstgeneration immigrant stories, by first-generation filmmakers. Both the taxi drivers and the women of the Shoe Project have interesting stories to tell, giving us a personal and pertinent look at life in Canada as a new immigrant. Our main goal was the empowerment of these first-time filmmakers, which we achieved by letting them talk freely about their experiences and by giving them basic filmmaking tools. We thus proved that “we do not need to be shoemakers to know if our shoes fit, and just as little have we any need to be (film) professionals to acquire knowledge of matters of universal interest” (Hegel).

5

SHOES AND TAXIS AS MOBILE STORYTELLING TOOLS …

65

Notes 1. Royal Bank of Canada. 2. Trinity Square Video (TSV) is one of Canada’s first artist-run centers and the oldest media arts center. Their activities are guided by the goal to increase their members’ and audiences’ understanding of what media arts practices can be. Trinity Square strives to create supportive environments, encouraging artistic and curatorial experimentation that challenge medium specificity through education, production and presentation support: https://www.trinitysquarevi deo.com/about-tsv/. 3. The arrival of the smartphone application Uber has thrown the taxi industry in a state of disarray. First launched in June 2009 in San Francisco, the application has spread rapidly and can now be found in almost all major cities. It offers many advantages over traditional taxi rides, most importantly the lower rates, which had serious effects on the living conditions of taxi drivers. In many cities, including Toronto, taxi drivers blocked roads in protest. 4. Website: https://theshoeproject.online/.

PART II

Mobile News Storytelling

Mobile technologies have been reshaping the way news is gathered, designed, produced, and disseminated through different mobile devices and platforms, resulting in a new form of news communication, that is, mobile news storytelling. On top of its unique forms, features, and functions, mobile news storytelling has become popular and powerful in telling news stories in an increasingly innovative fashion. The three chapters in this part offer three important innovative forms of mobile news storytelling with mixed results and lessons from China. It is a very smart and innovative move for traditional mainstream news media and government agencies to switch to Douyin (TikTok’s version within China) to tell stories about COVID-19 during the pandemic. In Chapter 6, the author examined how Douyin has become a powerful platform to meet the challenge of reorganizing and redistributing, as the author described, “authoritative, scientific, credible and effective news” during the COVID-19 pandemic. On top of that, the author has located more innovative ways such as leveraging “fragmented, micro, prominent and ephemeral audiovisual storytelling units, especially in the dimensions of ‘concise viewpoints/facts’, ‘single scene focus’, ‘senses enhancement’, ‘emotional narration’ and ‘individual discourse amplification’” in successfully enhancing mobile news storytelling at a time of a health crisis. Another innovative form of mobile news storytelling after Douyin, which is largely for the domestic audience in China, is to tweet a national image by a national news agency by leveraging Twitter, which is blocked in China, as a unique way to join its national efforts to enhance China

68

PART II:

MOBILE NEWS STORYTELLING

storytelling and national image communication to the global audience. By analyzing official tweets from the official Twitter account of Xinhua News Agency, the leading national news agency in China, and its audience responses, Chapter 7 found that while Xinhua News Agency had done its best to construct a friendly and powerful identity of China, most of its audience remained neutral in their responses. It is innovative to use an internationally popular but domestically blocked mobile platform to tell national stories to the global audience. It is not effective, however, to rely on its innovative approach alone. Chapter 7 shows and shares the lesson from China that to enhance mobile storytelling, an innovative platform alone would not do the job. While media convergence has been proven to be effective as a strategy to boost rating and influence of citizen news programs, Chapter 8 located the incompatibility between the traditional news editorial process and the decentralization of Internet platform in China and its corresponding challenges to citizen news programs in reconstructing their journalistic practices, balancing between news editorial control and user-generated content, and developing more profitable business models in the context of Chinese citizen news going mobile. As demonstrated by these three chapters, mobile news storytelling still has a long way to go before it reaches its maturity. The experience and expertise in mobile storytelling from these three chapters have their value in sharing with their global audience in the globally shared efforts to enhance mobile news storytelling.

CHAPTER 6

Multimodal Framing COVID-19 Stories: The Case of Douyin Qumo Ren

In the age of attention economy and fragmented information, verbal, visual, and audio materials in any mobile storytelling format are competing and cooperating to attract our attention. As the world’s number one mobile short video app and the fastest growing mobile native social media platform, Douyin (Chinese short video app, known as TikTok in the global context) has been downloaded hundreds of millions of times. Against this backdrop, Chinese government agencies and news media have followed suit, trying to build a new force of Chinese discourse on Douyin. Research on Chinese news communication driven by mobile short videos, however, has been very limited. Through investigating targeted mobile short video news (MSVN) of COVID-19 from an interdisciplinary perspective of news framing and multimodality, this chapter examines how COVID-19 was multimodal-framed in Douyin’s fast-paced multimodal context.

Q. Ren (B) University of Nottingham Ningbo China, Ningbo, China e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_6

69

70

Q. REN

News Framing COVID-19 News media continue to play an irreplaceable role in the health crisis communication of the twenty-first century, which has been true during the Ebola, SARS, MERS, and now COVID-19 pandemics (Mutua & Oloo, 2020). Improper handling of news information may cause people to lose trust in news media and the government, and at the same time may also lead to economic crisis and high mortality rate (e.g., Chock & Kim, 2020; Deslatte, 2020; Mutua & Oloo, 2020). During the outbreak of the epidemic, the key role of news media is to provide instructive and trusted information (Ogbodo et al., 2020; Zhou et al., 2020). Gislason (2013) posits that the use of specific news frames in a public-health emergency has significant effects on societal understanding and responses to the epidemic outbreak. “News framing at this stage became an important aspect not only in shaping the public discourse about the outbreak but also in the communication of the disease outbreak management interventions” (Mutua & Oloo, 2020, p. 1). Regionally, for instance, Park et al. (2020) point out that at the beginning of the outbreak, the attribution or responsibility frame is the most used one in mainstream media in South Korea and that news containing medical frame has a stronger spillover effect. The whole pandemic development process is split into three periods in Spanish news media on Twitter—the pre-crisis period, the lockdown period, and the recovery period with key news frames such as state of alarm, livelihood, economy, and Madrid, etc. (Yu et al., 2020). Chinese media covered mainly the impact, eminence, and novelty frames mostly in positive tone and uncommonly affirmative, according to Gabore’s (2020) study. From a global perspective, human interest and fear/scaremongering frames dominated the global media coverage across mainstream media in the US, UK, France, Italy, China, and Nigeria, according to Ogbodo et al. (2020). Attribution of responsibility and health severity were two of the dominant frames among BBC, AL-Jazeera, People’s daily, and CNN at the pre-crisis phase and crisis stage (Mutua & Oloo, 2020). Besides generic-frames, more specific frames for this profound global event have also been detected and interpreted in the latest studies, such as containment, eminence, entertainment, ethnicization, fatality, health severity, hope, impact, medical, politicization, support/aids, and war frames (Benziman, 2020; Gabore, 2020; Mutua & Oloo, 2020; Nwakpu et al., 2020; Ogbodo et al., 2020; Park et al., 2020).

6

MULTIMODAL FRAMING COVID-19 STORIES: THE CASE OF DOUYIN

71

On the other hand, as for the framing effects, due to the inconsistencies and mis-framing churned out by the ruling party with its associated media channels in the US, constituencies had a greatly reduced awareness of the virus harm, and even resisted a series of preventive measures from the very beginning (Chock & Kim, 2020). Furthermore, in the US, public-health frames positively influence citizen preferences for avoiding unnecessary travel, however, economic frames increase the preference to make unnecessary trips to shop, and the appearance of federal messengers seems to strengthen the framing effect relative to expert messengers (Deslatte, 2020). On the contrary, containment and fatality frames were the most occupied story frames in Nigeria that helped Nigerians to take precautionary measures (Nwakpu et al., 2020). Currently, the reportage on COVID-19 has revitalized news framing. In dealing with a pandemic that the world has not experienced in more than a century, governments and news media have spared no effort to promote better resonance through various story frames (Benziman, 2020). Apparently, how do media highlight the reality of news stories through negotiating and selecting among ideologies, sources, rhetoric, visual stimulus, and other representative elements is becoming crucial to grab a mobile user’s attention. Although the emerging discovered news frames about the epidemic are eye-catching and short-format news has been a favored genre during the crisis (Tejedor et al., 2020), previous studies have not further explored news frames in a multimodal context driven by mobile medium (Knox, 2020), especially the globally trending format—MSVN. In addition, the whole world is surprised and puzzled by why China was able to control the epidemic in a short time (Burki, 2020). Scholars accordingly point out that the study of English media alone does not provide an indepth understanding of the initial success and effectiveness of China’s health communication strategy (Gabore, 2020; Li et al., 2020). The latest studies from China have shown that news information carried by mobile is the first option of a Chinese citizen for receiving COVID-19 news (e.g., Wei et al., 2020). Following this new research trend, it is very necessary to capture the audiovisual data of Chinese-based mobile social network for in-depth analysis, such as Douyin or TikTok (Li et al., 2020). Most of the previous studies used a timeline to analyze the news (from preselected news media) at each stage, without considering the relevant indicators of audience substantial engagement (e.g., like, share, comment). Therefore,

72

Q. REN

this chapter discusses those news frames and stories that audiences pay more attention to, and preliminarily probes the characteristics of these “filtered” news in the mobile multimodal context.

Multimodal News Framing Devices Gamson and Modigliani (1987) define a frame as a “central organizing idea or story line that provides meaning” to the presented event or issue (1987, p. 143). Researchers look for such frames through the use of five common devices: catchphrases, depictions, metaphors, exemplars, and visual images (Gamson & Modigliani, 1987). Similarly, according to Entman (1993), a media message “contains frames, which are manifested by the presence or absence of certain keywords, stock phrases, stereotyped images, sources of information, and sentences that provide thematically reinforcing clusters of facts or judgments” (p. 52). Yang and Van Gorp (2019) summed up some major framing devices so far: themes and subthemes (thematic and episodic), types of actors, actions, and settings, catchphrases, depictions, metaphors, exemplars, keywords, stock phrases, stereotyped images, contrasts, lexical choices, sources, episodic/thematic, etc. As shown above, we see that framing devices can be categorized or deployed at multidimensional levels. That is to say, they can be carried by different textual units (McLeod & Shah, 2015), message structures (Pan & Kosicki, 1993), and message components (Tankard, 2001). In the Web 2.0 age, such complex has become more manifested and intensified. Today, posts, comments, hashtags, picture/video, captions, internet memes, attached hyperlinks, and even social recommendation system itself have been discussed as framing devices (e.g., Liu & McLeod, 2019; Qin, 2015). Perhaps, Linstrom and Marais (2012) gave a more logical interpretation of this ambiguous concept. They sorted out and defined two genres of news framing devices. The first one is “Rhetorical and other written/grammatical devices,” such as metaphors, exemplars, key words, and so on. And the second one is “Technical devices (elements of newswriting, layout, visuals),” for example, headline, photographs, captions, sources, placement, layout, etc. What they mean is that we should consider framing in both language and semiotic measures. Not surprisingly, standing upon the watershed between traditional pure word-play and opened semiotic approaches, news framing studies have started to shift their direction into multimodality in recent years. For example,

6

MULTIMODAL FRAMING COVID-19 STORIES: THE CASE OF DOUYIN

73

specific metaphors with close-up visuals were studied as a combination to examine the audience’s behavioral responses (Powell et al., 2018). News web editor composed news bites through the choices ranging from such as nominalizations, whole-part metonymies to different uses of shots (Belmonte & Porto, 2020). In a television news framing study, Pentzold et al. (2016) investigated how criminal news is framed through anchor’s voice, camera angles/movements, image’s color and scale, etc. These examples indicate that different multimodal framing paradigms are sharing the common trend of further fining semiotics and meaning resources. Although multimodality has brought enlightenment to the development of news framing, the current discussions are largely based on the traditional media while mobile media are to be examined urgently(e.g., Borah, 2018; Harlow et al., 2017; Keib et al., 2018). In this vein, the study tentatively proposes a set of semiotic framing devices (technical ones) based on previous studies, trying to explore their performances in relation with user engagement (the number of likes).

Research Questions Recent research shows that the credibility of traditional media has been gradually weakened during the crisis (Chock & Kim, 2020). In the context of a decentralized, bottom-up, fairer competition between UGC and PGC driven by Douyin, the following research questions are raised to be addressed in this study: RQ1: What organizations or individuals tended to contribute to the most popular short videos on COVID-19? The latest studies have shown that the media use both generic and specific frames for reporting the outbreak, but little attention has been paid to the extent to which audiences interact with these frames. RQ2: What news frames are more attractive to users? Previous study on the crisis has preliminarily suggested that the visual information dominated by specific characters (e.g., experts, politicians, etc.) and their quotes (interviews, press conference, etc.) are more convincing and credible (Zhou et al., 2020).

74

Q. REN

RQ3: In the existing images, what types of social figures appear the most as a dominant narrative carrier? And how they are visually framed? The existing literature has not explored MSVN’s storytelling style of China in the pandemic. RQ4: What storytelling formats and audiovisual technique framing devices have been frequently used in packaging MSVN?

Method Douyin offers a comprehensive smart engine for users to search their preferences, through which news stories, documentaries, personal Vlog, and other trending types of information can be found. In Chinese, “Xingxingguanzhuang” and “Xinguan” are the meta-root words for novel coronavirus or COVID-19 in Chinese. Therefore, the platform presents balanced and mixed results of related concepts based on the two metaroot words if entered. The upside of this move is that search engines are giving more equal exposure to all kinds of related concepts, which is more objective than the method of exploiting in hashtags as some highly engaged videos did not contain or attach any hashtags. Together, a total of 100 videos were collected under the two meta-roots (50 for each) with the filtering conditions of “news” and “the most likes.” The study adopts content analysis. The pre-designed codebook has two major parts. The first one focuses on general information and textual elements, such as video’s attributes (account type, publisher level, original media type, etc.), topic interests (geographically), frames functions: diagnostic-centric frame (Walker et al., 2020) and solution-centric frame; diagnostic-centric frames: conflict, responsibility, economy, human interests, war, health severity, societal problems, legal issues, personal style and public opinion, and so on (e.g., Jackson, 2011; Mutua & Oloo, 2020; Pan & Meng, 2016), and solution-centric frames: prevention, education, support/aid, medical/science research, law, hope, mobilization, crisis management effectiveness, and so on were coded (e.g., Nwakpu et al., 2020; Ogbodo et al., 2020; Pan & Meng, 2016). Of these, 88 videos out of the 100 carried obvious news frames. The second part of the codebook contains semiotic variables. They include multimodal text structure (Lee, 2019), storytelling rhythm (video

6

MULTIMODAL FRAMING COVID-19 STORIES: THE CASE OF DOUYIN

75

duration, edit rate, single scene/single shot oriented story), storytelling format: quote-centric format (Pfurtscheller, 2020), scene descriptioncentric format, visual narrator (i.e., quote-related visuals), visual narrator’s prominence index: camera stand/position, medium close-up (MCU), eye contacts/frontal angle (e.g., Bednarek & Caple, 2017; CorrigallBrown & Wilkes, 2012), vertical screen style/at least 75% proportion (Corrigall-Brown & Wilkes, 2012), visual participants in image bites (Luth et al., 2013; Peer & Ksiazek, 2011; Tejedor et al., 2020), visual abstractness (Duan et al., 2017), entertainment, and prominence framing devices (such as background music bite, open style caption/subtitle) (Butler, 2019), transition (Peer & Ksiazek, 2011), vertical presentation style, and so on. If any of the above semiotic variables was absent, it would be coded as “0”. And it would be coded as “1” if any of the above semiotic variables was present.

Results Contributors of Popular Short Videos First of all, news short videos operated by news media and government agencies were the only two account types that users mostly responded to (news media = 89.8%, government agencies = 10.2%, n = 88). In details, news media got 118,516,000 likes (M = 1,500,202/SD = 1,625,604) and the total number of likes for government agencies were 7,787,000 (M = 865,222/SD = 271,942). In terms of publisher level, national accounts accounted for the highest share (72.3%), followed by provincial accounts (18.2%) and municipal accounts (6.8%). Among them, the proportion of news media types is TV = 59.1%, newspaper = 27.3%, original digital media = 12.5%, while the short video accounts with radio broadcast as the original media have almost never appeared. Further, short news videos led by CCTV (CCTV matrix = 44.32%, China Daily = 11.37%, People’s Daily matrix = 9.1%, Xinhua News Agency and others = 5.64%, n = 62) are far ahead of non-national news media and government agencies (n = 26). The results indicate that short video news generated by news media are more likely to be responded, especially those originated from national broadcast news media, e.g., China Central Television (CCTV). This is not surprising when considering their professions of shooting and editing

76

Q. REN

techniques and other audiovisual advantaged resources for “customizing” and “repurposing” news (Westlund, 2013). Attractive News Frames Geographically, users were most interested in topics about China (59%) and the US (26%). From the perspective of news frame, the occurrence times of diagnostic news frames (77, M/SD = 0.88/1.03) and solution news frames (76, M/SD = 0.86/0.71) were close to each other. Among all videos, 68.2% contained a solution frame, and nearly half (46.6%) of them independently presented by a single solution frame, most of which focused on China’s topics (45.5%). In stark contrast to this is the diagnostic-centric frame. The data suggests that diagnostic frames were mainly concentrated outside China, and most of them were in “solo-play” form (37.5%). China was the first country to frame the pandemic outbreak as a “war” in January 2020 (CGTN, 2020). As expected, Chinese mobile short video users paid more attention to those solution-oriented stories or constructive news in which positive psychologies were released on time (Hermans & Gyldensted, 2019) and that was extremely core during the beginning of the outbreak. Take a further look, solution-led frames such as education (19.3%), hope (14.8%), prevention (8%), and support/aid (8%) appeared more frequently in users’ preferences. These frames were interspersed into stories about improving public medical awareness, building hospitals, closing cities, assistance, good news on curing, national leaders’ encouragements, comforts, determinations, etc. Oppositely, diagnostic frames such as conflicts (14.8%), societal problems (11.7%), and health severity/threat (10.2%) tended to appear more regarding foreign outbreaks. They were centered mostly in China Daily’s posts, which also reflected the audience’s interests in overseas epidemic developments. In a word, in the public opinion dominated by short videos, audiences prefer news frames that contain substantial anti-epidemic measures, which can be considered as the remedies of the war frame. User Preferred Storytelling Patterns In the storytelling perspective, “text + moving picture + audio” was the most common multimodal text structure (76%). Spatially and temporally, the majority focused on the narrative rhythm of “16–30 seconds”

6

MULTIMODAL FRAMING COVID-19 STORIES: THE CASE OF DOUYIN

77

(44.32%) and the edit pace of “1–3 cuts” (78.41%). At the same time, keeping the visual content in the same scene (59.09%) or in the one-take format (47.73%) was one of the common features. Therefore, less than thirty seconds and three cuts may suggest that the efficient use of visual resources is correlated with user engagement. In terms of storytelling format, “quote-centric format” 57 (64.77) and “scene description-centric format” (13.92%) were the two most frequent storytelling formats, that is, the storyline is led by the presented quotes or the directly described scenes throughout. In mobile short video channel, users may be more interested in the intuitive, direct, and centralized visuals that are able to boost the story in a very short period. One or just a few centered or highlighted frames may be apparent enough to evoke users’ reaction when losing the story’s depth. This also echoes the findings in the second part and previous studies, which argued that a frame or its framing power should not be judged by its length (Liu & McLeod, 2019; McLeod & Shah, 2015). Public Figures as Narrators As mentioned, remarkably, quote-centric format was playing the absolute dominant role of storytelling, in which all kinds of video segments constituted the major component (89.7%). For instance, the top three were television news clips (talking head, 22.73%), news conference snippet (19.32%), and television interview bite (12.5%). These audiovisual bites mainly derived from domestic elites (57.95%, CCTV news anchors, Xi Jinping, Zhong Nanshan, officials, medical experts and crews, etc.) and foreign elites (14.77%, Trump, US politicians, US news anchors, etc.). Similarly, in non-quote-centric images (i.e., image bites), visual participants were mainly dominated by elites, but there were more diverse types of characters (i.e., domestic elites, foreign elites, domestic negative personalization, foreign personalization, etc.). Notably, 43% of the videos met all six criteria for the visual narrator’s prominence index, which may support the idea that the high narrator’s prominence has the potentials to trigger more interactions. When public figures serve as the storytellers, they tend to convey a national image of transparency, openness, fairness, unity, and responsibility as a great power. For instance, CCTV news anchors and government officials had more chances to communicate the real-time information and express the government’s standpoints toward the “war.” Medical experts

78

Q. REN

and crews in the front line provided more prevention and virus knowledge to the public. Xi Jinping and other national leaders emphasized China’s national interests and determination of winning the “battle.” However, when reporting or reviewing the pandemic in the West, among the images presented by public figure narrators are conflicts and irresponsible and personal styles, for instance, Trump administration announced that there’s no need for Americans to wear masks. The public (individuals) refused to wear masks for freedom. The government was holding back the real data of affected and death. People could not afford the medical expenditure. Frequent Framing Devices (Technique Ones) On the level of entertainment and prominence framing devices, the most striking result to emerge from the data is that every single video used open captions, of which 51.92% had diversified font colors (white, yellow and red appear most frequently). Open style subtitles can be placed anywhere in the image to increase the correlation between text and highlighted visuals as well as the visibility of key information, making them eyecatching and navigational (Butler, 2019). For example, “warm” colors (red, yellow, orange) are physiologically more arousing and alert than “cold” ones such as blue and green on the television screen (Jacobs & Suess, 1975), producing higher anxiety levels (Wilson, 1966). It is not difficult to understand the frequent use of such colored captions in the epidemic information. As for “soundscape” (Levy & Pinchevski, 2017), it’s hardly to see a video when it is accompanied by voice-over (5.68%), a traditional audio storytelling means in broadcast news. The majority kept the original redundant voices from the quotees (69.32%) and a few restored diegetic sounds (12.5%). Again, this indirectly evidences the solid performances of quote-centric format. Another significant trend is that 84.62% of the videos used background music, showing the use of background music is a conventional strategy in MSVN production and may function as emotion appeals, cognitive suggests, and vernacular creativity. This was studied as musical “contextualization” (Herget, 2019). In addition, significantly, non-abstract images almost monopolized all videos (98.08%), and most of them did not use any transition effects (88.46%). Meanwhile, visuals presented in full-vertical or nearly fullvertical screens reached 70.45%. This set of data reveals that, during the pandemic, the public is surely willing to receive true visuals that are

6

MULTIMODAL FRAMING COVID-19 STORIES: THE CASE OF DOUYIN

79

designed in a lifelike, immersive, and intimate form so that they can feel, experience, and participate in the stories derived from the front lines, instead of being passively consumed.

Conclusion Mobile short videos have developed into a phenomenon-level mobile communication carrier in the world today. And it is also a new traffic field that Chinese governments and media at all levels are scrambling for. Although a few previous researches have studied the epidemic information released on TikTok and the Chinese government’s management of crisis information on Weibo (Chen et al., 2020; Li et al., 2021), none of them involved the perspective of news framing or storytelling. The current study preliminarily explored the top-liked 100 epidemic information from Douyin, making up for the gap of China’s epidemic information in mobile short video dissemination. In this case study, Chinese mainstream news media remained the dominant sources favored by the public. The results show that although Douyin is an entertainment-oriented platform, the news media still kept communicating with the public in an authoritative and rigorous manner when dealing with the real-time tough events, and thus constructed an efficient government image. From this, the news media can also draw some enlightenment. With enhanced audiovisual sensory experience, the news gist can be quickly understood in a few seconds, thus realizing the concise dissemination and avoiding tedious and boring reading. The limitation of time puts forward higher requirements on the format and quality of the short video, which makes every second count for bringing in more direct impacts. This kind of high-density multimodal information, coupled with the creative and fast way of viewing, may encourage users to stay longer. Similarly, condensed and simplified facts and standpoints are also emerging to be one of the popular elements, especially when they are solely presented. Thus, many videos only presented a single frame. In general, contrary to the grand narrative of traditional mainstream media, MSVN attaches more importance to fragmented, micro, prominent, and ephemeral audiovisual storytelling units, especially in the dimensions of “concise viewpoints/facts,” “single scene focus,” “senses enhancement,” “emotional narration,” and “individual discourse amplification.” In the future, these variables need to be studied in a larger sample library and integrated into framing effects research. Mobile short

80

Q. REN

video has injected new blood into the news industry, making positive contributions to China’s efficient dissemination during the fight against COVID-19.

References Bednarek, M., & Caple, H. (2017). The discourse of news values: How news organizations create newsworthiness. Oxford University Press. Belmonte, I. A., & Porto, M. D. (2020). Multimodal framing devices in European online news. Language & Communication, 71, 55–71. Benziman, Y. (2020). “Winning” the “battle” and “beating” the COVID-19 “enemy”: Leaders’ use of war frames to define the pandemic. Peace and Conflict: Journal of Peace Psychology, 26(3), 247–256. Borah, P. (2018). Addressing theoretical and methodological challenges of doing news framing analysis in the contemporary media landscape. In Doing news framing analysis II: Empirical and theoretical perspectives (pp. 163–188). Routledge. Burki, T. (2020). China’s successful control of COVID-19. The Lancet: Infectious Diseases, 20(11), 1240–1241. Butler, J. (2019). The visual experience of accessing captioned television and digital videos. Television & New Media, 2019, 1–18. CGTN. (2020). Chinese President Xi Jinping: Control of new coronavirus is currently the primary task. https://news.cgtn.com/news/2020-01-25/XiControl-of-the-new-coronavirus-is-currently-China-s-primary-task-NxSgeH IBEI/index.html. Accessed 21 Sept 2020. Chen, Q., Min, C., & Zhang, W., Wang, G., Ma, X., Evans, R. (2020). Unpacking the black box: How to promote citizen engagement through government social media during the COVID-19 crisis. Computers in human behavior, 110. Chock, T. M., & Kim, S. J. (2020). US audiences’ perceptions of Covid-19 and conservative news frames. Partecipazione e Conflitto, 13(2), 1180–1189. Corrigall-Brown, C., & Wilkes, R. (2012). Picturing protest: The visual framing of collective action by First Nations in Canada. American Behavioral Scientist, 56(2): 223–243 Deslatte, A. (2020). To shop or shelter? Issue framing effects and socialdistancing preferences in the COVID-19 pandemic. Journal of Behavioral Public Administration, 3(1). Duan, R., Zwickle, A., & Takahashi, B. (2017). A construal-level perspective of climate change images in US newspapers. Climatic Change, 142(3), 345–360. Entman, R. M. (1993). Framing: Toward clarification of a fractured paradigm. Journal of Communication, 43(4), 51–58.

6

MULTIMODAL FRAMING COVID-19 STORIES: THE CASE OF DOUYIN

81

Gabore, S. M. (2020). Western and Chinese media representation of Africa in COVID-19 news coverage. Asian Journal of Communication, 30(5), 299– 316. https://doi.org/10.1080/01292986.2020.1801781 Gamson, W. A., & Modigliani, A. (1987). The changing culture of affirmative action. In R. G. Braungart & M. M. Braungart (Eds.), Research in political sociology (pp. 137–177). JAI Press. Gislason, M. K. (2013). West Nile virus: The production of a public health pandemic. Sociological Health Illness, 35(2), 188–199. Harlow, S., Salaverría, R., Kilgo, D. K., & García-Perdomo, V. (2017). Protest paradigm in multimedia: Social media sharing of coverage about the crime of Ayotzinapa, Mexico. Journal of Communication, 67 (3): 328–349. Herget, A. K. (2019). On music’s potential to convey meaning in film: A systematic review of empirical evidence. Psychology of Music, 1, 1–29. Hermans, L., & Gyldensted, C. (2019). Elements of constructive journalism: Characteristics, practical application and audience valuation. Journalism, 20(4), 535–551. Jackson, D. (2011). Strategic news frames and public policy debates: Press and television news coverage of the euro in the UK. Communications, 36(2), 169– 194. Jacobs, K. W., & Suess, J. F. (1975). Effects of four psychological primary colors on anxiety state. Perceptual and Motor Skills, 41(1), 207–210. Keib, K., Espina, C., Lee, Y. I., Wojdynski, B. W., Choi, D., & Bang, H. (2018). Picture this: The influence of emotionally valenced images, on attention, selection, and sharing of social media news. Media Psychology, 21(2), 202–221. Knox, J. S. (2020). Multimodal mobile news: Design and images in tabletplatform apps. In H. Stöckl, H. Caple, & J. Pflaeging (Eds.), Shifts towards image-centricity in contemporary multimodal practices (1st ed., pp. 233–252). Routledge. Lee, E. J. (2019). Traditional and new media: A comparative analysis of news outlets’ news feeds on Snapchat. International Journal of Interactive Communication Systems and Technologies (IJICST), 9(1), 32–47. Levy, H., & Pinchevski, A. (2017). Resounding news: The acoustic conventions of Israeli newscasts. International Journal of Communication, 11(19), 3355– 3373. Li, J., Xu, Q., Cuomo, R., Purushothaman, V., & Mackey, T. (2020). Data mining and content analysis of the Chinese social media platform Weibo during the early COVID-19 outbreak: Retrospective observational infoveillance study. JMIR Public Health and Surveillance, 6(2), 1–10. Li, Y., Guan, M., Hammond, P., & Berrery, L. E. (2021). Communicating COVID-19 information on TikTok: A content analysis of TikTok videos from

82

Q. REN

official accounts featured in the COVID-19 information hub. Health Education Research, 36(3), 261–271. https://doi.org/10.1093/her/cyab010 Linstrom, M., & Marais, W. (2012). Qualitative news frame analysis: A methodology. Communitas, 17 , 21–38. Liu, J., & McLeod, D. M. (2019). Counter-framing effects of user comments. International Journal of Communication, 13(2019), 2484–2503. Luth, W., Jardine, C., & Bubela, T. (2013). When pictures waste a thousand words: Analysis of the 2009 H1N1 pandemic on television news. PLoS ONE, 8(5), 1–10. McLeod, D. M., & Shah, D. V. (2015). News frames and national security: Covering big brother. Cambridge University Press. Mutua, S. N., & Oloo Ong’ong’a, D. (2020). Online news media framing of COVID-19 pandemic: Probing the initial phases of the disease outbreak in international media. European Journal of Interactive Multimedia and Education, 1(2): e02006. Nwakpu, E. S., Ezema, V. O., & Ogbodo, J. N. (2020). Nigeria media framing of Coronavirus pandemic and audience response. Health Promotion Perspectives, 10(3), 192–199. O’Brien, H. L. (2017). Antecedents and learning outcomes of online news engagement. Journal of the Association for Information Science and Technology, 68(12), 2809–2820. Ogbodo, J. N., Onwe, E. C., Chukwu, J., Nwasum, C. J., Nwakpu, E. S., Nwankwo, S. U., Nwamini, S., Elem, S., & Ogbaeja, N. I. (2020). Communicating health crisis: A content analysis of global media framing of COVID-19. Health Promotion Perspectives, 10(3), 257–269. Pan, P. L., & Meng, J. (2016). Media frames across stages of health crisis: A crisis management approach to news coverage of flu pandemic. Journal of Contingencies and Crisis Management, 24(2), 95–106. Pan, Z., & Kosicki, G. M. (1993). Framing analysis: An approach to news discourse. Political Communication, 10(1), 55–75. Park, H. W., Park, S., & Chong, M. (2020). Conversations and medical news frames on Twitter: Infodemiological study on covid-19 in South Korea. Journal of Medical Internet Research, 22(5): e18897. Peer, L., & Ksiazek, T. B. (2011). YouTube and the challenge to journalism: New standards for news videos online. Journalism Studies, 12(1), 45–63. Pentzold, C., Sommer, V., Meier, S., & Fraas, C. (2016). Reconstructing media frames in multimodal discourse: The John/Ivan Demjanjuk trial. Discourse, Context & Media, 12: 32–39. Pfurtscheller, D. (2020). More than recycled snippets of news: Quote cards as recontextualized discourse on social media. AILA Review, 33(1), 204–226.

6

MULTIMODAL FRAMING COVID-19 STORIES: THE CASE OF DOUYIN

83

Powell, T. E., Boomgaarden, H. G., De Swert, K. et al. (2018). Video killed the news article? Comparing multimodal framing effects in news videos and articles. Journal of Broadcasting & Electronic Media, 62(4), 578–596. Qin, J. (2015). Hero on Twitter, traitor on news: How social media and legacy news frame Snowden. The International Journal of Press/politics, 20(2), 166– 184. Tankard, J. W. (2001). The empirical approach to the study of media framing. In S. D. Reese, O. H. Gandy, & A. E. Grant (Eds.), Framing public life: Perspectives on media and our understanding of the social world (1st ed., pp. 95–106). Lawrence Erlbaum. Tejedor, S., Cervi, L., Tusa, F., Portales, M., & Zabotina, M. (2020). Information on the COVID-19 pandemic in daily newspapers’ front pages: Case study of Spain and Italy. International Journal of Environmental Research and Public Health, 17 (17), 6330. Yang, H., & Van Gorp, B. (2019). Framing the Asian infrastructure investment bank: A qualitative analysis of the political debate and media coverage on a China-led multilateral institution. The Pacific Review, 32(4), 603–634. Wei, N., Yang, C., & Wang, X. Z. (2020). The influence of the release of government information on public psychology during major public emergencies: Data analysis based on the COVID-19 pandemic. Jiangsu Social Sciences, 3, 10–23. Walker, H. M., Reed, M. G., & Fletcher, A. J. (2020). Wildfire in the news media: An intersectional critical frame analysis. Geoforum, 114, 128–137. Westlund, O. (2013). Mobile news: A review and model of journalism in an age of mobile media. Digital Journalism, 1(1), 6–26. Wilson, G. D. (1966). Arousal properties of red versus green. Perceptual and Motor Skills, 23, 947–949. Yu, J., Lu, Y., & Muñoz-Justicia, J. (2020). Analyzing Spanish News Frames on Twitter during COVID-19—A network study of El País and El Mundo. International Journal of Environmental Research and Public Health, 17 (15), 5414. Zhou, X., Mulay, A., Ferrara, E., & Zafarani, R. (2020, October). ReCOVery: A multimodal repository for COVID-19 news credibility research. In CIKM ‘20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 2020 (pp. 3205–3212). https://doi.org/10.1145/ 3340531.3412880

CHAPTER 7

Twittering China’s Image: The Case of Xinhua News Agency Zhiying Zhang

Twitter is regarded as the second important application among social media platforms for information diffusion and real-time communication across time and space in the political sphere (Bruns, 2012). Effective use of it can enhance image communication for organizations or individuals (Papacharissi, 2010). Image construction on Twitter can be expressed through digital marks such as pictures, texts (within 140 characters), and videos. At the same time, image communication on Twitter can invite responses from its audience through ‘favorites,’ ‘retweets’ and comments. This chapter investigates how China’s Xinhua News Agency Twitter account has been leveraged to enhance its efforts in communicating China image globally. Xinhua News Agency is a state-owned news agency to serve as the voice of the People’s Republic of China. With the development of media, Chinese media have managed to coordinate the

Z. Zhang (B) Faculty of Humanity Social Science, University of Nottingham Ningbo China, Ningbo, China e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_7

85

86

Z. ZHANG

relationship between the ruling party and capitalism (Zhou & Moy, 2007). In contrast to the Western media, which are considered as the ‘fourth organization outside the political institution as an extra monitor of the government branches’ (Stewart, 1975, p. 634), Chinese media have to serve the interests of both government and the public (Hua, 2000). Since its inception in February 2012, @XHNews has enhanced Xinhua News Agency’s role of serving as a representative of the Chinese government and a tool to monitor public opinion in China (Xin, 2006). To examine how China image was communicated on Twitter, this study used Mozdeh to collect tweets, rewets, comments and other relate data. This investigation also used sentiment analysis is to research how Twitter users responded to China image communication on Twitter. Three research questions were addressed in this study: (1) In what way did Xinhua News Twitter account frame China image? (2) How did Xinhua News Twitter account manage the global responses to its China image communication on Twitter? (3) How did its global audience respond to its China image communication on Twitter?

Framing China Images on Twitter Different approaches to constructing news can be regarded as specific features (De Vreese, 2005). As part of news-specific features, generic frames contained human interest, economic impact, conflict, responsibility, and morality (Semetko & Valkenburg, 2000). And frames can be found in framing devices, such as metaphors, exemplars, capture phrases, descriptions, and visual images (Gamson & Modigliani, 1989). Framing devices in this study refer to subheads, photos, source selection, quotes selection, statements, and paragraphs. And these framing devices, effectively used, would enhance national image communication on Twitter. It has become increasingly important and imperative for governments to leverage social media to boost their efforts in constructing and communicating national images nationally and globally (Szondi, 2010). Twitter has been widely leveraged for political use and image management as it allows users to disseminate information and to retweet posts from other users so as to reach a wider audience (Boiy & Moens, 2009). Twitter’s effect has been examined in various ways, including its sense of social reason and medical knowledge (Neiger et al., 2013), company social duty (Bae & Lee, 2012), international government organization

7

TWITTERING CHINA’S IMAGE: THE CASE OF XINHUA NEWS AGENCY

87

involvement (Sobaci & Karkin, 2013) and so on. In general, both politicians and governments use Twitter to achieve different objectives, as shown in previous research on this subject. Many studies state that political members and political parties have adopted Twitter to achieve self-promotion and advertise information (Golbeck et al., 2010; Grant et al., 2010; Vergeer et al., 2013; Waters & Williams, 2011). Thus, managing online image plays an important role for both individuals and organizations, including nations. Gulati (2004) found that politicians use different social media to shape various images that they present to the public. Few studies, however, have examined how organizations leverage Twitter to build a national image.

Research Methods To examine how China image has been framed on Twitter, this study collected related tweets via Mozdeh for sentiment analysis via SentiStrength, textual analysis, and thematic analysis. Tweets Collected Via Mozdeh Two kinds of tweets were gathered via Mozdeh: recent tweets related to queries and all tweets published by specific users. After the pilot tests, five queries about Xinhua News Twitter account were input into the data collection screen: Chinaxinhuanews, Xinhua News, xhsports, xhscitech, XHNews. ‘XHNews’ is the Twitter abbreviation of Xinhua News. The results of five queries collected 721 tweets from June 9th to June 18th in 2018. Mozdeh provides various keyword searches to select specific texts. It helps to research specific topics or events which are part of data mining and tweet filtering. Two typical keywords like ‘Dragon Boat’ and ‘Yulin’ were used to search for related tweets. Mozdeh can sort in ascending or descending order by date, retweet count or different sentiments, which can be seen as a quick overview of the tweets. For retweet database, tweets that had obtained over 10,000 favorites and over 50 retweets were chosen as samples. Besides, a function called trend detection in Mozdeh can review the change in topics through visual chart.

88

Z. ZHANG

Sentiment Research on Twitter By analyzing the comments, retweets, and tweets on Twitter, it is imperative to understand the sentiment related to a specific topic (Paltoglou & Thelwall, 2012). Wilkinson and Thelwall (2011) found that Twitter is suitable for researching attitudes about current events, for example, the scandal of Tiger Woods and the ceremony of the Oscars. Sentiment analysis is famous for extracting the attitude of the audience about specific events or themes by analyzing texts (Stieglitz & Dang-Xuan, 2013, p. 226). On the other hand, sentiment on Twitter can also be employed for forecast or evaluation in various fields, such as stock markets and political movements (Zeitzoff, 2011). For example, Twitter was used in detecting emotions regarding four candidates in Singapore’s 2011 presidential election (Wang et al., 2012). Sentiment analysis basically concentrates on positive and negative rather than blurred analysis (Bae & Lee, 2012). Scientists put words that are coded positively or negatively into a data set then computers recognize the number of words that belong to positive or negative fields automatically (Boiy & Moens, 2009). Zavattaro et al. (2015) claimed that this kind of sentiment analysis is more precise than human judgment. On social media, sentiment analysis is beneficial for the government to research the reactions and attitudes of the masses (Lee et al., 2013). SentiStrength was used in this study to understand daily communication between users and the Xinhua News Twitter account. Sentistrength mainly classifies the text of tweets from Mozdeh into positive and negative sentiments. The result of SentiStrength displays two scores, for example, the scores between 1 and −1 show neutral emotion. Scores from 2 to 5 mean positive sentiments which show stronger the higher the number, while scores between −2 and −5 mean negative emotion. The larger number means stronger emotions (Thelwall, 2017). Descriptive quantitative analysis was also used to analyze the frequency of different kinds of tweets with strong emotions. Textual Analysis and Thematic Analysis Text analysis was used for analyzing tweets in different categories. Tweets with high favorite scores were analyzed for the detailed content. Data from Twitter involves unstructured texts which contain several words, hashtags, links and so on. Sometimes, users retweet and add their

7

TWITTERING CHINA’S IMAGE: THE CASE OF XINHUA NEWS AGENCY

89

comments in new tweets which also show sentiment. Thematic analysis was used for analyzing hashtags, it contains frequency analysis which can focus on the reason for a prevalent theme.

Findings and Analysis The findings highlight that Xinhua News Twitter account uses generic frames to report news in order to frame a friendly and powerful identity of China. @XHNews Twitter account adopts Twitter functions such as hashtags and ego-retweeting to manage the impression of what it frames. In the process of identity construction, the attitude of Twitter users reflects the outcome. Most Twitter users express neutral emotion in relation to the way it is depicted on this account. Framing China Image on Twitter Xinhua News Twitter account show features of reporting styles. It mainly focuses on reporting international news and the theme of ‘Social Life.’ The form of ‘Text + Picture’ is the most frequent, while the form of ‘Text + Video’ attracted more attention from Twitter users. Xinhua News usually adds a URL at the end of tweets. There were 201 tweets covering international news, accounting for about 72%; while 78 tweets were about national news, accounting for 28%. According to different presentational forms, @XHNews mainly used text, video, pictures, and live broadcast. Most of the tweets used the form of ‘text + picture,’ which accounted for about 88%. There were five live broadcasts in the sample tweets, which were used to attract more attention than other forms. In terms of topics, 279 tweets were grouped into eleven types, including ‘Social Life, Politics, Technology, Economy, Military Affairs, Culture, Sports, Entertainment, Medical, Education and Others’. Tweets about ‘Social Life’ were the most common (39%). The percentage of tweets about ‘Politics’ was 20%, which ranks second; and the third main category was ‘Technology,’ which accounts for 10%. Xinhua News stresses concerns about other countries’ news and reflects the participation of China in international events by adoption of generic frames. It focuses on reporting international news, which is delivered approximately three times as often as national news. This is an example of adoption of ‘economic impact’ in generic frames.

90

Z. ZHANG

In thematic analysis, Xinhua News reported the news about cooperating with other countries which could frame a friendly and positive image. A tweet received 1.2 thousand favorites, which is about a political advisor encouraging more company help to development the economy in Uganda and to alleviate local employment problems. The high favorite count reflects agreement among Twitter users. According to the generic frame from earlier studies (Valkenburg et al., 1999), Twitter account adopted frames of ‘economic impact,’ ‘responsibility,’ and ‘Cooperation’ (Supadhiloke, 2012). These tweets used the similar phrases to report the relevant news about cooperation with Africa, such as ‘friendly and friendship,’ ‘cooperate and cooperation,’ and also use the repeat phrases such as ‘ties,’ ‘together,’ and ‘economic.’ Among the similar tweets, their common feature is that they show a friendly relationship between China and Africa. A metaphor was used as a framing device to describe the connections between China and Africa as ‘dragon boat teammates’ because that day happened to be the Chinese traditional festival ‘Dragon Boat Day.’ And it was also because rowing a dragon boat needs concerted efforts. The official account reports news about political, economic, medical, cultural, and social aspects continuously. Among these tweets, the last tweet gained a high favorite score at 1.1 thousand which means that the news received an appreciative reaction from Twitter users. Yang et al. (2011) indicated that a tweet with high favorite count may be popular with Twitter users who frame a successful social image. By reporting similar tweets, China frames a cooperative image to call for more countries to build joint relationship. Secondly, many tweets frame a positive role for China in international events and international conferences by using the form ‘Text + Video.’ For example, a major event about ‘The Shanghai Cooperation Organization’ (SCO) was presented during the time period. Xinhua News Twitter account presented the conference process in the host city of Shanghai through the form of ‘text + video.’ It can frame a positive host image to the world and promote the SCO theme. A popular tweet about this theme obtained 1.7 thousand favorites. The tweet contained a video of around 39 seconds and a short text, which gained 20.1 thousand views. The list of related tweets about SCO highlighted the Shanghai Spirit and called on SCO members to build up strength and mutual trust again. From the list of tweets, the form ‘Text + Video’ gained the higher favorite count than the form ‘Text + Picture.’ Regarding the international conference, video can illustrate the theme more intuitively than text which helps

7

TWITTERING CHINA’S IMAGE: THE CASE OF XINHUA NEWS AGENCY

91

the audience have more sense of participation. Combined with the live video trend, governmental accounts of social media would attract more users to attend the online international activities. People gain further understanding of Twitter content through video displays in the mobile era (Abisheva et al., 2014). Nation Branding in Hashtag Analysis Regarding some important tweets which can achieve nation branding, @XHNews used the hashtag as a ‘framing device’ to create discussion. When reporting the news about international events, Xinhua News Twitter account chooses to use hashtags to indicate the theme as ‘label’ (Gamson & Modigliani, 1989). Topics are pointed out by the mixture of a hashtag (#) and a key term. Using short-term approaches to mark tweets is similar to the use of ‘tags’ to classify tweet content (Boyd et al., 2010). Tagging attracted more attention to digital bookmarks (Golder & Huberman, 2006). Xinhua News Twitter account utilized various popular hashtags in tweets about famous events. 63 different hashtags were found in the tweets and appeared 112 times. The most popular hashtags include #Xinhuaspecial, #DailychinBrief, #SCO2018, #Trump, #XiJinping, #DragonBoatFestival, #KimJongUn, and so on. Two hashtags usually appeared in the tweets. In events related to SCO, four different hashtags are included in tweets. The hashtag ‘#SCO’ gained the highest frequency. The identification frame was mainly used to strengthen the important events. The hashtag ‘XinhuaSpecial’ and ‘#DailychinBrief’ are obviously designed to gather important news in official accounts which could help users search for brief information. These two hashtags are similar to the ‘title’ in news articles which play a reminder role in daily tweets. Xinhua News Twitter account also uses the names of politicians to mark the important characters in major events like #XiJinping, #KimJongUn, and #Trump. China President Xi Jinping is mentioned by each tweet which also gains high favorite on Twitter which means that the president is one part of nation branding. Other obvious hashtags include #DragonBoatFestival and #Duanwu.June 17 is a famous traditional festival called ‘Duanwu’ in China. In the Duanwu festival, one of the ways of celebrating is to row a dragon boat. Thus, the other name of the Duanwu festival is the Dragon Boat Festival. The content of the tweets shows how people celebrate the festival from all over the world. Among these tweets, Xinhua News

92

Z. ZHANG

account also used hashtags about the countries or places hosting events such as #Portugal and #Ljubljana. Using a number of hashtags helps to brand the nation. Culture is a strong soft power for strengthening Twitter users’ understanding of China. The American Marketing Association (AMA) said that a national brand stands for a wide range of elements and relevance including culture (Fan, 2006). Xinhua News used Twitter to promote offline activities in order to strengthen the perception of Chinese culture by hashtag mixture which again helped to brand the nation. Impression Management in Retweet Analysis Xinhua News Twitter account uses ego-retweeting to manage impressions. When it retweets an important tweet, Twitter users receive a deep impression about this retweet. It is beneficial for strengthening a standpoint to manage the national image. The users who do retweet do not stand for all kinds of Twitter audiences, as there are various motivations for using Twitter (Boyd et al., 2010). Among tweets about trade war events, Xinhua News is self-serving, stressing the standpoint of China, which received 161,142 favorites. Such retweet is regarded as a clicktivist action that raised awareness of related issues. Self-serving means a political aim to do self-promotion which is one of way to manage impression. (Halupka, 2014). The high favorite count from users indicates the way ego-retweeting is beneficial for managing the strong impression of China. During the ‘Trade War’ which originated by US President Donald Trump on March 22, 2018, the Chinese Ministry of Commerce restarted anti-dumping investigations on various products exported to the United States on June 16. The second retweet is a Xinhua News comment, while the third tweet is an ego retweet. The third tweet is the official announcement to America from the Xinhua News Twitter account. It expresses a tough attitude by China. When a Twitter account retweets messages that mention them, some scholars describe this as ‘narcissistic’ or ‘self-serving.’ Sentiment Analysis Results Among the effective data, the number of users that held neutral attitudes (−1,1) was 444. The number of people who expressed positive emotion (>1) was 114, while 198 tweets expressed negative attitude (< −1). It is interesting to note that the highest score for positive emotions was 4 while

7

TWITTERING CHINA’S IMAGE: THE CASE OF XINHUA NEWS AGENCY

93

the score for negative emotions was −5. In conclusion, the percentage of people who held negative emotions to be around threefold the proportion of user expressing positive emotions. Strong Emotions Tweets with strong emotions can reflect how Twitter users express their attitudes toward different events and news from the Xinhua News Agency Account. Xinhua News manages the national impression by selecting specific content to post. ‘YULIN FESTIVAL’ are retweeted by users which means that they desire to spread the topic when they agree with posts. The key comments are ‘cruel,’ and ‘killing’ which show negative emotion. Strictly speaking, Xinhua News Twitter account never posts tweets about this topic. But users shared and re-shared the protest tweets by retweeting, which also received many favorites. The conversation morphed due to different retweets. In Yulin, eating dog meat is a traditional custom celebration at a festival but in western countries, some vegetarians and/or dog owners think this is cruel behavior. The number of positive tweets (>2) was 45 while 76 tweets scored as negative tweets (< −2), both of which are worth key analysis. Among the positive tweets, the first tweet scored highest at 4 and the third sentence scored 3, both of which praised traditional Chinese customs. Related to the high favorite count at 5323 and 3179, it indicates that Chinese culture is an important influence on positive attitudes toward China by Twitter users. The second sentence is scored 3 and provides a proverb to show the opposing attitudes toward this ‘Trade War’ with China. The positive sentiment from Twitter users shows that they agreed with China’s reaction. The first sentence has been analyzed in the last section. The second sentence scored −4 which talks about the tough attitude of China toward the ‘Trade War.’ The word ‘hate’ (−3) and ‘war’ (−2) are the main negative terms. But the high favorite score showed that most people agreed with this tweet. Therefore, the score and the emotion of second sentence are contradictory. In conclusion, over half of Twitter users in the sample held a neutral attitude to China’s image. Most parts of tweets were scored highest for negative emotions, which actually show positive emotions scoring higher for favorites. Thelwall and Buckley (2013) claimed that some country

94

Z. ZHANG

names may be linked to strong positive or negative sentiment when discussing some themes.

Discussion and Conclusion The main way the image of China is framed was to focus on generic frames, which is similar to their use on conventional social media. As the type of generic frames was stated in the literature review, the ‘economic impact frame’ in generic frames was emphasized mostly, in order to report the economic help from China to other countries, especially Africa. @XHNews adopted a ‘human interest frame’ to report Chinese culture, such as the ‘Dragon Boat Festival,’ to catch the attention of Twitter users, which was the second most popular tweet. In a traditional way, journalists adopted a human interest frame to report emotional news and an economic impact frame to influence citizens, organizations, and countries (Wasike, 2013). The notion of media framing is often applied in news stories but is seldom used in Twitter research. The function of hashtag on Twitter is similar to a ‘headline’ in traditional media. Both use brief titles to summarize the theme, which makes it easier to achieve the framing image. On Twitter, a hashtag is similar to a ‘headline’ in framing devices, which is also good for creating a discussion about a specific topic. Xinhua News Twitter account used hashtags such as ‘#Duanwu,’ and ‘#DragonBoatFestival’ about Chinese culture, which is a kind of self-promotion. Culture is regarded as ‘soft power,’ which is beneficial to nation branding. By researching the feature of retweeting data, it is found that Xinhua News account mainly used egoretweeting to strengthen the presentation of the standpoint and attitude of China toward ‘Trade War’ to manage impressions. Some important standpoints were stressed by ego-retweeting, which also gained agreement from Twitter users. Boyd et al. (2010) suggested that ego-retweeting is self-serving. Jackson and Lilleker (2011) suggested that one of the methods of managing impressions is self-promotion, as used by MPs in the UK. The method of ego-retweeting and avoiding sensitive information helps Xinhua News to present a great image to Twitter users. It is also found that the @XHNews Twitter account also avoids reporting some sensitive news which is good for impression management. For example, tweets about the ‘Yulin Dog Festival’ took up the majority of negative emotions in the results from SentiStrength. However, @XHNews never reported the related news. The tweets on Twitter are from Twitter users

7

TWITTERING CHINA’S IMAGE: THE CASE OF XINHUA NEWS AGENCY

95

who were against the festival and expressed opinions online using ‘@’ to catch the attention of Xinhua News Twitter account. This is one of the approaches that can be used to achieve impression management. The main innovation in this research is the mixed use of Mozdeh and Sentistrength. However, in Sentistrength, the polarity analysis led to inaccurate results. It is better to add more specific words into dictionaries to increase the rate of accuracy. In SentiStrength, the overall emotion toward China was neutral. In case studies of Lee et al. (2013), they monitored the emotion of citizens toward government agencies. The results from detection show that the overall sentiments of tweets from the government agency were neutral. Most previous studies of sentiment analysis have talked about reviews of movies or products advertising (Gamon et al., 2005; Tang et al., 2009). Although a different tool was used to detect the emotion of Twitter users toward China on Twitter, the outcome remained the same. However, the strong negatives were not relevant to China. Twitter users showed more positive emotion to China, which is relevant to Chinese culture and to friendly relationships between China and other countries. The negative emotion of Twitter users was mostly due to different attitudes toward the ‘Yulin Dog Festival.’

References Abisheva, A., Garimella, V. R. K., Garcia, D., & Weber, I. (2014, February). Who watches (and shares) what on YouTube? and when? using twitter to understand YouTube viewership. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining (pp. 593–602). Bae, Y., & Lee, H. (2012). Sentiment analysis of twitter audiences: Measuring the positive or negative influence of popular twitterers. Journal of the American Society for Information Science and Technology, 63(12), 2521–2535. Boiy, E., & Moens, M. F. (2009). A machine learning approach to sentiment analysis in multilingual Web texts. Information Retrieval, 12(5), 526–558. Boyd, D., Golder, S., & Lotan, G. (2010). Tweet, tweet, retweet: Conversational aspects of retweeting on twitter. In 43rd Hawaii International Conference on System Sciences, IEEE (pp. 1–10). Bruns, A. (2012). How long is a tweet? Mapping dynamic conversation networks on Twitter using Gawk and Gephi. Information, Communication and Society, 15(9), 1323–1351. De Vreese, C. H. (2005). News framing: Theory and typology. Information Design Journal and Document Design, 13(1), 51–62.

96

Z. ZHANG

Fan, Y. (2006). Branding the nation: What is being branded? Journal of Vacation Marketing, 12(1), 5–14. Gamson, W. A., & Modigliani, A. (1989). Media discourse and public opinion on nuclear power: A constructionist approach. American Journal of Sociology, 95(1), 1–37. Gamon, M., Aue, A., Corston-Oliver, S., & Ringger, E. (2005, September). Pulse: Mining customer opinions from free text. In International Symposium on Intelligent Data Analysis (pp. 121–132). Springer. Goffman, E. (1974). Frame analysis: An essay on the organization of experience. Harvard University Press. Golder, S. A., & Huberman, B. A. (2006). Usage patterns of collaborative tagging systems. Journal of Information Science, 32(2), 198–208. Golbeck, J., Grimes, J. M., & Rogers, A. (2010). Twitter use by the US Congress. Journal of the American Society for Information Science and Technology, 61(8), 1612–1621. Grant, W. J., Moon, B., & Busby-Grant, J. (2010). Digital dialogue? Australian politicians’ use of the social network tool Twitter. Australian Journal of Political Science, 45(4), 579–604. Gulati, G. J. (2004). Members of Congress and presentation of self on the World Wide Web. Harvard International Journal of Press/politics, 9(1), 22–40. Halupka, M. (2014). Clicktivism: A systematic heuristic. Policy and Internet, 6(2), 115–132. Hua, X. (2000). Morality discourse in the marketplace: Narratives in the Chinese television news magazine Oriental Horizon. Journalism Studies, 1(4), 637– 647. Jackson, N., & Lilleker, D. (2011). Microblogging, constituency service and impression management: UK MPs and the use of Twitter. The Journal of Legislative Studies, 17 (1), 86–105. Lee, K. C., Kwon, O., Kim, C. R. et al. (2013). Factors affecting the intensity of emotional expressions in mobile communications. Online Information Review. Neiger, B. L., Thackeray, R., Burton, S. H., Thackeray, C. R., & Reese, J. H. (2013). Use of twitter among local health departments: An analysis of information sharing, engagement, and action. Journal of Medical Internet Research, 15(8), 177. Papacharissi, Z. (Ed.). (2010). A networked self: Identity, community, and culture on social network sites. Routledge. Paltoglou, G., & Thelwall, M. (2012). Twitter, MySpace, Digg: Unsupervised sentiment analysis in social media. ACM Transactions on Intelligent Systems and Technology (TIST), 3(4), 1–19. Semetko, H. A., & Valkenburg, P. M. (2000). Framing European politics: A text analysis of press and television news. Journal of Communication, 50(2), 93–109.

7

TWITTERING CHINA’S IMAGE: THE CASE OF XINHUA NEWS AGENCY

97

Stewart, P. (1975). Or of the press. The Hastings Law Journal, 2(6), 634. Sobaci, M. Z., & Karkin, N. (2013). The use of Twitter by mayors in Turkey: Tweets for better public services? Government Information Quarterly, 30(4), 417–425. Stieglitz, S., & Dang-Xuan, L. (2013). Emotions and information diffusion in social media—Sentiment of microblogs and sharing behavior. Journal of Management Information Systems, 29(4), 217–248. Supadhiloke, B. (2012). Framing the Sino–US–Thai relations in the post-global economic crisis. Public Relations Review, 38(5), 665–675. Szondi, G. (2010). From image management to relationship building: A public relations approach to nation branding. Place Branding and Public Diplomacy, 6(4), 333–343. Tang, H., Tan, S., & Chen, X. (2009). A survey on sentiment detection of reviews. Expert Systems with Applications, 36(7), 10760–10773. Thelwall, M., & Buckley, K. (2013). Topic-based sentiment analysis for the social web: The role of mood and issue-related words. Journal of the American Society for Information Science and Technology, 64(8), 1608–1617. Thelwall, M. (2017). The Heart and soul of the web? Sentiment strength detection in the social web with SentiStrength. In Cyberemotions (pp. 119–134). https://doi.org/10.1007/978-3-319-43639-5_7 Vergeer, M., Hermans, L., & Sams, S. (2013). Online social networks and microblogging in political campaigning: The exploration of a new campaign tool and a new campaign style. Party Politics, 19(3), 477–501. Valkenburg, P. M., Semetko, H. A., & De Vreese, C. H. (1999). The effects of news frames on readers’ thoughts and recall. Communication Research, 26(5), 550–569. Waters, R. D., & Williams, J. M. (2011). Squawking, tweeting, cooing, and hooting: Analyzing the communication patterns of government agencies on Twitter. Journal of Public Affairs, 11(4), 353–363. Wang, H., Can, D., Kazemzadeh, A., Bar, F., & Narayanan, S. (2012). A system for real-time Twitter sentiment analysis of 2012 us presidential election cycle. In Proceedings of the ACL 2012 system demonstrations, Association for Computational Linguistics (pp. 115–120). Wasike, B. S. (2013). Framing news in 140 characters: How social media editors frame the news and interact with audiences via Twitter. Global Media Journal: Canadian Edition, 6(1), 5–23. Wilkinson, D., & Thelwall, M. (2011). Researching personal information on the public web: Methods and ethics. Social Science Computer Review, 29(4), 387–401. Xin, X. (2006). A developing market in news: Xinhua news agency and Chinese newspapers. Media, Culture and Society, 28(1), 45–66.

98

Z. ZHANG

Yang, J., Morris, M. R., & Teevan, J., Adamic, L. A., & Ackerman, M. S. (2011). Culture matters: A survey study of social Q and A behavior. In Fifth International AAAI Conference on Weblogs and Social Media (pp. 409–416). Menlo Park. Zavattaro, S. M., French, P. E., & Mohanty, S. D. (2015). A sentiment analysis of US local government tweets: The connection between tone and citizen involvement. Government Information Quarterly, 32(3), 333–341. Zeitzoff, T. (2011). Using social media to measure conflict dynamics: An application to the 2008–2009 Gaza conflict. Journal of Conflict Resolution, 55(6), 938–969. Zhou, Y., & Moy, P. (2007). Parsing framing processes: The interplay between online public opinion and media coverage. Journal of Communication, 57 (1), 79–98.

CHAPTER 8

Citizen News Going Mobile: Opportunities and Challenges Hao Liu

When Internet technology prevails, its technological evolution also fundamentally influences the news industry. Among diverse journalistic practices, Chinese citizen news does not overmuch inherit ‘Citizen Journalism,’ which is designed to offer alternative journalistic practice and efficiently to shape the ‘fourth estate.’ On the contrary, it reconstructs journalism internally and externally in aspects of reporting narrative, audience interaction, and organizational forms. Analyzing differences between ‘Citizen Journalism’ and ‘Citizen News,’ this chapter focuses on the development of Chinese citizen news, especially the development of TV citizen news, and analyzes the government’s motives and strategies in media convergence under the influence of the mobile Internet. In the meantime, it also analyzes how various new media platforms such as Weibo, WeChat, and short videos put the impact on the production and dissemination of citizen news.

H. Liu (B) Architecture and Art Department, Zhejiang Business Technology Institute, Ningbo, China

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_8

99

100

H. LIU

Specifically, this chapter analyzes a provincial-level citizen TV news program and a municipal-level news APP as samples of observation. From the analysis of the advantages and disadvantages brought by their news reporting process and the technological transformation of new media, this chapter found that reconstructing citizen news into mobile is inevitable, as well as bringing diverse development of media platforms. But at the same time, citizen news still faces huge challenges including news production routine, opinion control, audience feedback, and profit growth.

Citizen News and Citizen Journalism It is important to first differentiate the concept of ‘citizen news’ in China from the concept of ‘citizen journalism’ that has been debated and explored for years outside of China. The concept of citizen journalism in the western context was initially related to individuals and organizations of journalistic practices, which strictly referred to the non-professional status of journalistic content creation, like what Örnebring defines citizen news as the ‘news produced by amateurs’ (2013, p. 36). The rise of citizen journalism is generally associated with the revolutionary application of internet and mobile communication technologies. The interactivity of the internet allows users to conduct broad journalistic practices such as writing current affairs blogging, diffusing news feeds on the tweet with pictures and videos captured by cameras or smartphones. For instance, individuals who experienced sudden, dramatic scenes like a natural disaster or terror attacks, their reactions of capturing field images and spreading them through online platforms are usually labeled as ‘witnessing’ (Allan, 2013), a form of the practice of citizen journalism. Even the actions such as re-posting, commenting upon online content, and tagging could be identified as citizen journalism, thus the boundaries of citizen journalism still need further exploration. Besides the discussions of journalistic practices, some scholars also integrate citizen journalism with broader concepts such as democracy and civil society (Campbell, 2014), believing that citizen journalism presents spontaneous opposition or challenges to dominating opinions of public affairs. The research of citizen journalism genealogy indicates that independent news institutes in some activist movements, such as Independent Media Center during the 1999 World Trade Organization protest and OhMyNews in South Korea, could be regarded as the predecessor of citizen journalism practice (Wall, 2005). Since the reports from those

8

CITIZEN NEWS GOING MOBILE …

101

agencies provided alternative views to stand for the welfare of certain groups, Friedland and Kim (2009, p. 297) also claim that citizen journalism is a ‘contribution to the discussion in the public sphere, whether in the form of simple information, synthesis reporting or opinion.’ The relationship of citizen journalism with mainstream media had been discussed by many scholars as well. Lewis noticed a tension between control of professional journalism and the need for rising citizen-generated content (2012). This tension could be demonstrated by arguments of professional journalists in terms of differentiating their journalistic routine from what citizen journalists approach in content creation, as they need to follow the high standards of news judgment, fairness, objectivity, and ethics (Allan & Thorsen, 2009). However, news agencies sometimes edit citizen-created content as components of news stories. For example, many news programs at professional journalistic agencies, such as iReport at CNN, launched separate spaces within daily episodes for citizens uploading their contents as part of the program as a whole (Domingo, 2011). Even though the presentation of citizen-generated content at mainstream media is noticeable, those contents are constantly controlled by professional news agencies. For instance, the format of presentation and narrative logic of news content are influenced by standards of mainstream news media, since the visibility of citizen-generated content on those platforms is inevitably under the procedure of editorial review (Ali & Fahmy, 2013; Semati & Brookey, 2014). The notion of ‘citizen news’ in China possesses completely different connotations from those of citizen journalism. Chinese media reform surged since 1992, straight after the pro-democracy movement in 1989. The reform itself could be observed as what Chan called the representation of ‘lagged diffusion of economic reforms into the ideological and cultural domain and the erosion of the Chinese Communist Party’s (CCP) control over ideology’ (Chan, 1993, p. 25.2). During the media reform, the element of capital has started to play an important role in remodeling the philosophy and routine of news media (Lee et al., 2007). One consequence of such re-modeling is the competition between nationowned media and regional media. For example, CCTV (China Central Television Network) and Chinese regional television stations encountered imbalanced development and mutual competition since the launch of the media reform, when regional television stations were initially regarded as the supplement mouthpiece of the nation, maintaining the influence of CCTV by expanding its local coverage. However, though still bearing

102

H. LIU

the party line, regional televisions have less motivation to obligate CCTV due to two major reasons. First is the absence of a top-down command system making the local governments reluctant to follow each instruction from CCTV. Secondly, regional media are now financially self-sufficient, which means there is no direct fund that regional news agencies could receive from the central publicity department or CCTV. Therefore, maintaining self-development and the function of a mouthpiece of regional government become the priorities of a regional TV station. The competition between CCTV and regional television stations prominently emerged at a territorial scale, since CCTV is legitimately authorized to provide the national coverage of news and entertainment while regional television networks broadcast within their respective administrative areas. It was under this political and economic environment that regional television stations started their exploration toward new formats and genres of programs, aiming to attract more preference and loyalty among local audiences. The launch of the TV news program ‘Nanjing Prime Watch’ was acknowledged as the event that marked the beginning of citizen news programs nationwide. ‘Nanjing Prime Watch’ is produced and broadcasted by City Channel, Jiangsu Provincial Broadcasting Corporation since 2002. Before the 1992 media reform, Chinese television news programs primarily focus on sustaining the authority of governments by delivering news pieces in a formal and less interactive fashion. Furthermore, the presentation of news content follows the hierarchy of governors who are involved in the news, or the agenda of local administrative departments. The attempts of increasing audience rating and potential advertising revenue inspire regional television producers to enhance programmatic innovation in either genre or presentation. Among the news practices of Nanjing Prime Watch, the function of media surveillance contributes to the unprecedented popularity of the program among the local audience, mostly as it establishes a special format of media surveillance in Chinese news practice. After receiving a news clue from the program hot-line, for instance, journalists will follow up by reporting the event, in which citizens have somehow suffered from inappropriate treatments by public sectors, or encountered disputes and conflicts within the community. By investigating the event and helping citizens out of the troubles, questioning the public sectors, or mediating disputes, Nanjing Prime Watch achieved its popularity among citizens. And such journalistic practices to some extents echo the impression of ideal citizen journalists in the western regime, as what Goode described

8

CITIZEN NEWS GOING MOBILE …

103

who ‘expose the failings of public and private institutions and their personnel, and sometimes become celebrated opinion-leaders, having circumvented the traditional journalistic career path.’ (2009, p. 1290). In this narrative, citizen news could even illustrate the implication of the ‘fifth estate’ (Cooper, 2006). Apart from media surveillance, Nanjing Prime Watch also establishes a special section in the program named ‘field survey.’ The presenter announces the theme of a daily survey at the beginning of the show. And qualitative and quantitative results of the daily survey are posted before the end of a daily live show. After the overview of the survey, the program presenter generally delivers personal comments on that particular theme. The presence of the survey in news program provides a virtual space for gathering public opinions and even influencing the policymaking of a local government in turn. Such programmatic innovation implies the notion of ‘Public Sphere,’ in which an individual citizen could act on behalf of the public interest rather than his or her private interest, therefore fostering an unprecedented atmosphere for journalism to become a kind of ‘conversation’—the notion that resonates with ideals from Jürgen Habermas (1989) and James Carey (1998). On the other hand, through agenda setting, news media accomplish the construction of opinion platform on which the expression and confrontation of diverse opinions become visible and negotiable, and such negotiation naturally leads to multilateral communication among media, citizen, private and public sectors, and eventually the reflection of civilization and modernity. Such practice of ‘media surveillance’ and interactive survey in citizen TV news program partly find the historical echo with public journalism which emerged in the early 1990s. Facing to the decline of citizen participation in the democratic process, many news organizations began to experiment with various ways such as focus groups, community meetings, and other programs directed by corporate news outlets to enhance civic commitment and participation in the democratic process (Ryfe & Mensing, 2010). The practice of public journalism became a sort of US-based movement in which mainstream news institutions played the leading role. Though the presence of news content still comes from mainstream journalists rather than ordinary citizens, the notion of public journalism started to re-identify audience from information ‘consumers’ to event participators, who actively construct conversation among each other, while engaging public affairs through such attention, or ‘conversation’ (Haas & Steiner, 2006).

104

H. LIU

The success of Nanjing Prime Watch was phenomenal since it obtained the highest rating among all TV programs within the Nanjing city area only 8 months after its debut. And it furtherly achieved its record of rating at 17.7% in the following year, which was unprecedented for a regional TV news program. The upcoming advertising revenue was also prominent. According to official statistics of City Channel, Jiangsu Broadcasting Corporation, the average advertised price of peak time of channel escalated from 7000 RMB to 12,000 RMB per 30 s after the debut of Nanjing Prime Watch, achieving more than 70 percent of the growth in one year, due to the success of this particular citizen news program, and average rating of channel experienced an increase as well. Since then, Nanjing Prime Watch has been publicly recognized as the pioneer, starting the era of citizen news within Chinese media evolution. And its success stimulated the regional stations all over the country to urgently establish their respective local versions of the citizen news program. Besides exploring new boundaries of conventional news report systems, the preliminary consideration behind this trend is driven by market competition.

Citizen News and Media Convergence The concept of media convergence is prominent in media communication research since the application of the Internet and digital technologies to traditional media organizations. Kolodzy (2009) categorizes media convergence into technological, economic, and journalistic aspects. While the technological aspect focuses on technological changes in media practices and digitalization of media working flow, the economic aspect refers to the impacts of convergence upon media organization, especially on market. Moreover, the journalistic aspect concerns the fusion of journalists, including the forms of journalistic practices, reconstructing traditional journalistic routine and media sphere (Kolodzy, 2009). The empirical researches on media convergence illustrate different perspectives from researchers. Verweij (2009) declares that the trend of media convergence is changing the character of journalists from ‘lone wolf’ to multi-team participants. And the transformation of the working flow of journalistic practice under media convergence also leads to new professional requirements. As Robinson (2009) discovers, it also imports technological convergence to enhance digital skills of journalists and to cultivate a new mindset of reporting stories that differs from traditional journalistic reporting.

8

CITIZEN NEWS GOING MOBILE …

105

The media convergence in China, unlike the spontaneous evolution of western media organizations driven by technology and journalistic routine, is mainly a response to the challenges that social media impose on traditional media agencies, especially the competition from Microblog (Chinese version of Twitter). For microblog, its very nature of individual opinion diffusion has gradually transformed this social media into an ideological battlefield of many public events. For example, a series of public demonstrations against the presence of Para-xylene (PX) chemical factories in several Chinese cities have broken out among 2007–2014, such protests generated unprecedented online discussion on Microblog, and even partly change the governmental agenda on PX factory construction. From the perspective of the central government, the media convergence not only strengthens the online voice of the party-state but also increases the possibility of keeping the focus of online opinion in line with the CCP. The government believes the media convergence can achieve such goals due to the effects of digital media on information diffusion and influencing public perception on particular agenda, therefore it is regarded as the newly emerging battleground requiring ideological control. Moreover, the market potential of digital media is also promising benefit to Chinese media competitiveness in the global area. At the national level, top leaders have strengthened the importance of media convergence on different occasions. Liu Yunshan, formerly one of the Politburo Standing Committee members and the head of the Party Central Committee’s Propaganda Department, spoke in 2011 at the establishment ceremony of the People.cn Co. Ltd and expressed the wish that People’s Daily Online would ‘hold tightly to the correct public opinion guidance, continuously strengthen and broaden the online mainstream battlefield of public opinion’ (Xinhua, 2010). More importantly, Chinese President Xi Jinping has specifically mentioned the importance and strategies of media convergence in the speech at the fourth conference of the Central Leading Group for Comprehensively Continuing Reform, by stressing the necessity of ‘promoting deep convergence between traditional media and new media at the level of content, channel, platform, management, and administration,’ to build ‘some newly mainstream media that diversified, advanced and competitive; several new media groups with strong capacity of communication, credibility and influence’ (Zhao & Yan, 2014). Xi’s speech authorizes the legitimacy of media convergence and therefore accelerates the convergence among Chinese media agencies.

106

H. LIU

The models of convergence that the Chinese central government encourages are diversified. Traditional media such as newspaper and television network are scrambling to register their official accounts on dominant new media platforms, like Weibo, WeChat, and Douyin (Chinese Tiktok), distributing the contents that are recreated from what have been posted on the traditional channel, or even direct link leading to their websites. On the other hand, many government-controlled media, mostly at the national level, even invest heavily in developing news apps of their own, aiming to transfer readers and audiences from conventional channels to the online world. For example, the newspaper People’s Daily, the mouthpiece of the central government, launched the website of People’s Daily Online in 1997, and its mobile version in 2007, co-establishing People’s Video in 2010 for targeting mobile internet users. Similarly, the Chinese Central Television Network (CCTV) established an official website in 1996, a mobile version in 2006, and developed an all-platform news app CCTV Video in 2015. The prevalence of media convergence reached its peak right after Xi’s speech in 2014. The scale of convergence expands nationwide, with all forms of media proclaiming their existence in a digital context. Therefore, traditional media agencies gradually establish their accounts on different online platforms and APPs, such as the public account of WeChat, the official Weibo account, and the short video of Douyin (the original version of TikTok). Not only the presentations of news contents are modified for catering to the preference of new communication platform users, but also the sources of news are also extensively diversified by online communication platforms. Starting from People’s Daily, the model of ‘Central Kitchen’ in news production becomes widely accepted by other news media as a sign of convergence. The concept of ‘Central Kitchen’ refers to the news production model through which journalists collect news materials including footage, pictures, interview clips and upload them to the database, despite the formats of news reports differentiating each other. The affiliated media and diverse channels under one particular media group can adopt the same news materials from a database and produce news reports differently, according to the characteristic of media. This method becomes popular through media agencies nationwide, and is described as the mode of ‘Omnimedia’. However, the economic pursuit of media convergence in China does not meet the expectation. Generally, media convergence in China is government-driven, especially in the area of television and radio networks.

8

CITIZEN NEWS GOING MOBILE …

107

The cost of media convergence is usually huge and the cases of profitable media under convergence strategies are rare. Given the hierarchical structure of Chinese media, only those who are directly owned or controlled by the central government with national influence may obtain specific funding from the Ministry of Finance for compensating their loss in promoting media convergence. However, local media agencies at the municipal level could hardly receive funding continuously from either central or local governments, and those local agencies are coincidentally, as what we analyzed previously, the major players who have been leveraging citizen news as a tool for competing with CCTV and national news agencies. Amidst the application of convergence trends and mobile communication technologies, local citizen news programs encounter unprecedented changes.

Citizen News Going Mobile Technically, the most salient characteristic of a citizen news program amid media convergence and mobile Internet communication is to go mobile at a new media platform. The first wave of launching new media accounts among Chinese traditional media is Sina Weibo, as its functions of information diffusion and opinion interaction basing on users’ comments possess similar features of public media. Later on, the appearance of WeChat and its Official Accounts service allows governments at different levels, enterprises, social organizations, and individuals to ‘promote their brands to billions of WeChat users [ …] thus reducing propagation costs, raising brand popularity, and building up more influential brand images’ (WeChat, n.d., n.p.). This service provides the media agencies alternative channels to reach their possible users on the internet. Due to its technological innovation on long content creation, if comparing with the word and picture limitation of Weibo (144 words and 9 pictures only for each post), WeChat Official Accounts offer users the autonomy to establish diverse contents and build remarkable images, which are significant effects that media agencies highly value. In this case, television citizen news programs always generate individual online contents that originating from their regular programs and post them on WeChat official accounts. The emergence of short-form video sharing applications like Douyin (TikTok) further expands the boundary of citizen news. Short-form video applications like Douyin were initially dominated by User-Generated Content (UGC), which particularly records people’s offline life such

108

H. LIU

as fancy food, cultural activities, places of interest, and those contents are accepted by users as a sign of fashionable lifestyle. Technically, an interest-based algorithm of short-video applications assures the frequency of similar video contents being pushed to subscribers, and the presentation of short-video including time-length and the visual format also guarantees easier user access and immersion. These technical innovations are utilized by media agencies to strengthen the influences of their media platform and programs, in which citizen news TV programs are popularly recreated their news reports into short videos, due to their high relevance with daily life. As mentioned above about the incompatibility between state and local television networks during the Chinese media reform, the citizen news program finds its autonomy and popularity in the local television market due to economic and contextual preference. Since the close bond with the ordinary life of local citizens, Chinese citizen news programs are generally established by the municipal and provincial television network. One television citizen news program and one mobile news APP are selected for observation, to introduce their regular news practice under the influence of mobile technology and media convergence, while partly illustrating the opportunities and challenges that television citizen news encounter at the current stage, as well as the possible evolution under technological innovations in future. 1818 Golden Eye Established in 2004, 1818 Golden Eye followed the path of Nanjing Prime Watch and became the first citizen television news program in Zhejiang Province, China. It is now ranked as the most popular program among all news agencies at the provincial level. Facing the emergence of mobile internet, the media agency swiftly explored the potential of program under the guide of media convergence from the central government, and led to establishing official accounts on Sina Weibo and WeChat Official Account, for expanding programmatic recognition. However, diversified channels at social platforms don’t contribute to the rise of program popularity. Like many traditional media agencies who create new media accounts, 1818 Golden Eye has not changed the way of producing news except uploading individual pieces of re-edited news on a social media platform. But its ambition of expanding influence unexpectedly came true as short-form video platforms like Douyin (TikTok)

8

CITIZEN NEWS GOING MOBILE …

109

and Bilibili entered into the market competition. According to the latest official report of Chinese Internet development, the number of users of short-form videos has reached 818 million, which equals 87% of the overall Chinese Internet users. The report also indicates that short-form video has already become a new channel for producing and distributing news stories. For example, the users of Weibo have uploaded more than 2.25 million video clips with 84.2 billion views during the COVID-19 epidemic period, in which many of them have been integrated as either contextual elements or news stories of the COVID-19 theme report (CNNIC, 2020). Since major content of 1818 Golden Eye is about media surveillance that focusing disputes between ordinary citizens and commercial sectors (sometimes public sectors), to solve problems in favor of ordinary citizens, the daily reports of the program are usually filled with confrontational scenes that journalists record between ordinary citizens and public/commercial sectors, sometimes among citizens. On the Bilibili.cn, one of the biggest UGC online community in China, active users choose news footages from 1818 Golden Eye and edit the video clips into an amusing way by adding music, subtitles, and even voiceover to re-create sort of ‘deviant’ and ‘entertaining’ viewing experiences. This kind of user re-creation on news content attracts huge positive feedback from other Bilibili users, and the popularity of 1818 Golden Eye is therefore dramatically strengthened nationwide. After its success in leveraging a short-form video platform, 1818 Golden Eye launched its official accounts on Bilibili.cn (since mid-2018) and Douyin (since mid-2019), uploading clips of news daily. It even cooperated with Bilibili.cn to create a new online program integrating news stories and online comments. Under the policy of media convergence and the pressure of rating competition, 1818 Golden Eye cooperates with short-form video platforms to strengthen its popularity as a ‘citizen-friendly’ news program. However, such practice of ‘going mobile’ and ‘embracing new media’ does not fundamentally solve the difficulties that the program itself encountering currently and brings obvious uplift in audience rating. Since the program is broadcasted within provincial area, even high exposure on the internet could not lead to higher TV program rating. Moreover, high exposure of certain amusing or ‘deviant’ video clips on short-form video platforms to some extent undermines public perception of 1818 Golden Eye as a serious citizen news program, but an entertainment-oriented program instead.

110

H. LIU

Ningbo Connection News APP Ningbo Connection is the official news app of Ningbo TV & Radio Group in the city of Ningbo, Zhejiang province. This news app was launched in February 2019, integrating affiliated TV networks and Radio channels of Ningbo TV & Radio Group into a comprehensive mobile platform. It was intentionally designed to expand media market share in the online communication field. The establishment of an online news app does not only illustrate the strategic change of local media agencies but also the compulsory duty to implement a policy of the central government. After Xi Jinping called for further efforts to boost media convergence at the fourth conference of the Central Leading Group for Comprehensively Continuing Reform, National Radio and Publication Administration (NRTA) further set a national policy to accelerate the integration and development of traditional media and new media (NRTA, 2016). The framework of Ningbo Connection consists of three parts. The major content on this news app is the collection of daily news from affiliated channels, selected by editors and presented on its home page. The citizen news and short-form shared videos are on the other two salient pages, respectively. The presentation of citizen news on this news app is different from what television news programs present. It allows authorized correspondents to post articles, pictures, and video clips relating to urban life and citizen activities on the platform directly. Editors of the platform could select certain valuable information for further journalistic probe, such as investigative reports. In this sense, what is presented on a news app has the potential for regenerating news value. Moreover, to preserve ‘media surveillance’ of citizen news, the app even lists personal social accounts of their journalists on a single page, helping users to set private communication channels with journalists and offer news clues, which might lead to producing valuable exclusive citizen news. What needs to be highlighted is that, unlike regular broadcasting of TV network news programs, citizen news journalistic practice on news app can overcome the limitation of broadcasting schedule as distributing complete reports on the app anytime, which also gives users the autonomy to repost news contents through social network conveniently. Although news reports on the app are produced and distributed in a mobile way, the internal procedure of news reporting still follows the routines of traditional TV citizen news programs, which challenge

8

CITIZEN NEWS GOING MOBILE …

111

the immediacy of Internet communication. There are seven steps before reports are officially presented on the app. Firstly, a journalist submits a proposal of the news report to his or her editor, but won’t be able to implement it until the proposal is approved. The next step is having news footage through regular interviews. Editing could not begin unless the script is completed and approved by a chief editor. Final editing of the news report still needs to be reviewed by program producer, with possible amendments before the official online presentation. Apart from a torpid hierarchy of journalistic practice, the technical standards of news agencies also require journalists to produce news footage by professional camera and editing workstation, which is time-consuming as comparing to producing UGC short videos on a smartphone. Furthermore, citizen news on apps indeed benefit from flexible distribution and interaction of the Internet, but such technological innovation also generates an unexpected dilemma. The journalists and editors of Ningbo Connections App complain that the high volume of views and reposts of a particular citizen news story does not necessarily lead to more favorable public perception of news App. Because online viewers simply enjoy those reposted contents at their regularly accessed platforms instead of the Ningbo Connection App. The App itself is thus regarded as communication platform but not content creator.

Conclusion The notion of ‘citizen news’ in western journalism is normally related to diversity and alternativeness of an existing journalistic practice, which focuses on the public participation in news communication, offering platforms to those who are ‘unseen’ and ‘unnoticed’ to make their voices heard. However, such public participation and journalistic democracy of citizen news are questioned by scholars as they believe mainstream media still influence the procedure and presentation of ‘citizen news.’ On the other hand, citizen news in the Chinese journalistic context is completely different from its western counterpart, since it is a form of news practice led by state-owned media. The contents of local ordinary urban living nourish local media to build their journalistic moat and preserve rating in the competition with national media who are financially supported by the central government. The function of media surveillance has been applied to Chinese citizen news programs. As what Nanjing Prime Watch practices in the daily report, media surveillance, and the consequent rating success

112

H. LIU

led to the prevalence of similar news programs all over the territorial television networks. The rise of mobile internet technology and its applications on entertainment, e-commerce, and social media escalate the Chinese media competition, and partly result in the decline of traditional media agencies. The top-down policy of media convergence advocated by the central government has contributed to the expansion of online presentation of state-owned media, through establishing official accounts on different social media and short-form shared video platforms. Moreover, the organizational structure of news departments also experiences centralization under this policy. TV citizen news, therefore, began the strategy of ‘going mobile’, driven by such external pressure as well as internal motivation. However, our findings illustrate that mobile Internet indeed increases diffusion channels of news programs while making limited contributions to news practices. Instead, it might even undermine the credibility and unity of news programs. On the other hand, the emergence of news apps to some extent creates a flexible channel of gathering news sources and integrating individual comments and professional journalistic practices, as well as distributing news on unified mobile platforms. But decentralized networks could hardly guarantee the everlasting recognition of particular citizen news programs among Internet users, unlike what conventional TV networks operate at the territorial level. Besides, the efforts of ‘going mobile’ for citizen news programs, whether on television or news apps, could not change the routine of journalistic practice within news agencies, even amplify the hierarchy of editorial control instead. Going mobile is an inevitable trend for Chinese news agencies, especially citizen news, as it is widely accepted that mobile Internet and social media could strengthen information diffusion, as well as audience trust. But current developments of territorial citizen news on the Internet still have unsolved problems in terms of reconstructing journalistic routine into ‘mobile’ nature, balancing the organizational control with user generation on news content, unifying user perception of news program online and offline, as well as exploring more profitable operation modes.

8

CITIZEN NEWS GOING MOBILE …

113

References Ali, S. R., & Fahmy, S. (2013). Gatekeeping and citizen journalism: The use of social media during the recent uprisings in Iran, Egypt, and Libya. Media, War & Conflict, 6(1), 55–69. Allan, S. (2013). Citizen witnessing: Revisioning journalism in times of crisis. John Wiley & Sons. Allan, S., & Thorsen, E. (Eds.). (2009). Citizen journalism: Global perspectives, (Vol. I). Peter Lang. Campbell, V. (2014). Theorizing citizenship in citizen journalism. Digital Journalism, 3(5), 704–719. Carey, J. W. (1998). Communication as culture: Essays on media and society. Unwin Hyman. Chan, J. M. (1993). Commercialization without independence: Trends and tensions of media development in China. China Review, 1993, 25.1–25.21. https://www.jstor.org/stable/23451606 CNNIC. (2020). The 46th China Statistical Report on Internet Development. http://www.cnnic.cn/gywm/xwzx/rdxw/202009/W02020092934312574 5019.pdf Cooper, S. D. (2006). Watching the watchdog: Bloggers as the fifth estate. Marquette Books. Domingo, D. (2011). Managing audience participation: Practices, workflows, strategies. In J. B. Singer, D. Domingo, A. Heinonen, A. Hermida, S. Paulussen, T. Quandt, et al. (Eds.), Participatory journalism: Guarding open gates at online newspapers (pp. 76–95). Wiley Blackwell. Friedland, L. A., & Kim, N. (2009). Citizen journalism. In: D. C. Whitney, and C. H.Sterling (Eds.), Encyclopedia of Journalism ( pp. 297–302). SAGE. Goode, L. (2009). Social news, citizen journalism and democracy. New Media & Society, 11, 1287–1305. Haas, T., & Steiner, L. (2006). Public journalism: A reply to critics. Journalism, 7 (2), 238–254. https://doi.org/10.1177/1464884906062607 Habermas, J. (1989). The structural transformation of the public sphere: An inquiry into a category of Bourgeois society. MIT Press. Kolodzy, J. (2009). Convergence explained. In A. Grant & J. Wilkinson (Eds.), Understanding media convergence (pp. 31–51). Oxford University Press. Lee, C. C., He, Z., & Huang, Y. (2007). Party-market corporatism, clientelism, and media in Shanghai. International Journal of Press/politics, 12(3), 21–42. Lewis, S. C. (2012). The tension between professional control and open participation: Journalism and its boundaries. Information, Communication & Society, 15(6), 836–866. https://doi.org/10.1080/1369118X.2012.674150 National Radio and Television Administration. (2016). Opinions on further accelerating the integration and development of radio and television media and

114

H. LIU

emerging media (in Chinese). http://www.nrta.gov.cn/art/2016/7/2/art_ 3592_42309.html Örnebring, H. (2013). Anything you can do, I can do better? Professional journalists on citizen journalism in six European countries. International Communication Gazette, 75(1), 35–53. Robinson, S. (2009). If you had been with us’: Mainstream press and citizen journalists jockey for authority over the collective memory of Hurricane Katrina. New Media & Society, 11(5), 795–814. Ryfe, D. M., & Mensing, D. (2010). Citizen journalism in a historical frame. In J. Rosenberry, and B. St. John III. (Eds.), Public journalism 2.0: The promise and reality of a citizen-engaged press (pp. 32–44). Taylor & Francis. Semati, S., & Brookey, R. (2014). Not for Neda: digital media, (citizen) journalism, and the invention of a postfeminist Martyr. Communication, Culture & Critique, 7 (2), 137–153. Verweij, P. (2009). Making convergence work in the newsroom. Convergence: The International Journal of Research into New Media Technologies, 15(1): 75–87. Wall, M. (2005). Blogs of war: Weblogs as news. Journalism, 6(2), 153–172. WeChat. (n.d.). Official account admin platform registration instructions (in Chinese). https://mp.weixin.qq.com/mp/opshowpage?action=newoplaw Xinhua. (2010). The establishment ceremony of the People’s Net held in Beijing (in Chinese). http://news.xinhuanet.com/politics/2010-06/20/c_1 2240008.htm. Accessed March 2013. Zhao, G. X., & Yan, S. (2014). Xi Jinping: Promoting the integration and development of traditional media and emerging media (in Chinese). The People.cn. http://media.people.com.cn/n/2014/0818/c120837-25489622.html

PART III

Vulnerable Communities and Social Innovation

Many communities have complex layers of dis/advantage depending on a mix of geographic, socio-economic, population-based, digital, or other factors. The following three chapters provide examples of specific communities that include vulnerable members: older Australian people, young menstruating women living in Bangladesh, and rural Australian communities. These diverse community groups experience specific challenges. Digital technology, particularly digital storytelling offers social innovative opportunities for those living on the digital fringe (Davis et al., 2019; Hespanhol et al., 2018). The chapters focus on how researchers have sought to capture and meet some of these challenges through socially innovative practices that aim to meet the social needs of the communities and pose alternatives to existing solutions. It is hoped that these examples of social innovation have helped extend and strengthen these communities. Older people aged 65 years and above are among the most digitally excluded groups in Australia. They are often impeded by lack of access to digital technology, limited internet connectivity, and lower levels of digital literacy. Older people have less representation in mainstream and social media, despite research that shows that older people benefit from engaging in social, creative, and fun activities, and that these can be crucial for enhancing and maintaining well-being in old age (Greaves & Farbus, 2006). Chapter 9, Mobile Storytelling with Seniors: Co-designing Smartphone Productions and Mobile, seeks to redress this balance. It focuses on case studies from a variety of mobile storytelling projects with seniors

116

PART III:

VULNERABLE COMMUNITIES AND SOCIAL …

that have created opportunities for older people to engage with digital literacies and develop online representation. The projects are founded on lengthy, trusted relationships between researchers, video producers, and participants. The digital stories represent elders’ community perspectives. The case studies shared in this chapter include a combination of interest-based learning approaches, co-designed storytelling, and online sharing via social media platforms, Facebook and Instagram, over the course of the projects and beyond. Bossio and Schleser argue that there is much evidence that seniors benefited from participating in co-creating the digital stories and that they positively contribute to the representation of seniors in digital spaces. Chapter 10, Engagement with Mhealth: Experiences of Young Women in Bangladesh explores gender-based disadvantage during menstruation. There are specific social and religious taboos related to women’s health in many countries, including Bangladesh. During the menstruation period, women are confined to their homes for seven days, must follow a specific diet, sleep alone, may not see a doctor nor attend social meetings, and adopt specific behaviors relating to hygiene. These factors lead to social disadvantage as young women cannot seek medical assistance, attend school, and engage in normal life during this time. Further, Bangladesh despite recent initiatives seeking to promote healthcare change, has a critical shortage of health resources, including medical staff who can support young women. Choudhury argues that mobile phone technology which is widely available and has good coverage in Bangladesh has promise as a healthcare support due to its accessibility and affordability. Through social networking and using a snowball sampling approach, Choudhury interviewed 54 young unmarried women who used smartphones for mhealth (mobile health) purposes. She mapped out young women’s usage patterns of mhealth, finding that mobile technology including social media and mobile apps, offered positive approaches for information seeking, more detailed explanations of symptoms and management, specialized support services, alternative treatment pathways, appointment booking, and others. Young women felt that managing privacy issues was one space in which mhealth provided value over traditional visits to the doctor, where patients must often speak in front of many other patients. However, Choudhury argues, this must be balanced alongside concerns about the reliability and authenticity of mhealth information in addition to privacy concerns about data storage and usage.

PART III: VULNERABLE COMMUNITIES AND SOCIAL …

117

Australian communities are diverse, with layered levels of dis/advantage, depending on a mix of geographic, socio-economic, population and digital challenges (Lawrence-Bourne et al., 2020). However, rural communities benefit from significantly higher levels of social networks, civic participation, and social cohesion when compared with urban areas. Various studies have found higher levels of involvement in community organizations, neighborhood connections, and volunteering in rural areas compared with cities (Ziersch et al., 2009). Despite this, Australian rural communities are spaces which are often overlooked in mainstream and social media. Chapter 10 illustrates how understanding and working with these community groups can lead to social impact opportunities. Chapter 11, Mobile & Digital Stories from Down Under: Tales from Rural Australia showcases a collection of mobile digital stories from five rural Victorian communities. Co-created with local community members, each digital story was produced with the intention of raising awareness and showcasing a local attraction, which was largely unrecognized outside of the surrounding communities. In this chapter, the authors recount the project processes including engaging key community members and organizations; outlining the role of smartphone, 360 cameras, and other technology in capturing key storylines; and co-creating the mobile stories. Authors Davis and Schleser, note that working closely with community organizations, volunteering and community groups generated a range of positive outcomes. These included an enriched researcher experience, community capacity building in the use of digital technology for project networking, and ultimately a set of mobile stories capturing authentic Australian rural community attractions. Now uploaded to social media, the mobile stories have been viewed widely. Readers can access them via this YouTube Channel: http://bit.ly/YarriambiackTales.

References Davis, H., Waycott, J., & Schleser, M. (2019). Digital storytelling: Designing, developing and delivering with diverse communities. In Satu Miettinen & Melanie Sarantou (Eds.), Managing complexity and creating innovation through design, 1st ed., Chapter 1 (pp. 131–140). Greaves, C. J., & Farbus, L. (2006). Effects of creative and social activity on the health and well-being of socially isolated older people: Outcomes

118

PART III:

VULNERABLE COMMUNITIES AND SOCIAL …

from a multi-method observational study. Journal of the Royal Society for the Promotion of Health, 126(3), 134–142. Hespanhol, L., Davis, H., Fredericks, J., Caldwell, G. A., & Hoggenmuller, M. (2018). The digital fringe and social participation through interaction design. The Journal of Community Informatics, 14, 4–16. Lawrence-Bourne, J., Dalton, H., Perkins, D., Farmer, J., Luscombe, G., Oelke, N., & Bagheri, N. (2020). What is rural adversity, how does it affect wellbeing and what are the implications for action? International Journal of Environmental Research and Public Health, 17 (19), 7205. Ziersch, A., Baum, F., Darmawan, I., Kavanagh, M., & Bentley, R. (2009). Social capital and health in rural and urban communities in South Australia, Australian and New Zealand. Journal of Public Health, 33(1), 7–16.

CHAPTER 9

Mobile Storytelling with Seniors: Co-designing Smartphone Productions and Mobile Stories with Vulnerable Communities Diana Bossio and Max Schleser

This chapter focusses on co-created mobile storytelling examples and case studies that have engaged participants in collaborative processes through most of the production process. While some are more focused on the collaborative nature of production, others emphasise the story more significantly, with each method producing very particular impacts on the production, the producer and the participant. Using case studies

D. Bossio (B) Department of Media and Communication, Swinburne University of Technology, Melbourne, Australia e-mail: [email protected] M. Schleser Department of Film and Animation, Swinburne University of Technology, Melbourne, Australia

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_9

119

120

D. BOSSIO AND M. SCHLESER

from a number of digital storytelling projects including 60+ Online (McCosker et al., 2018), May Days (Prasad, 2012), Oma’s Applesauce (Van Genderen, 2020), Grey Matter AR (Vanderborght, 2019) and The OPERA Project (Bossio et al., 2019a, 2019b), we reflect on the role of co-design in mobile screen production. Further to this, we also discuss the impact of co-creation processes during the OPERA Project (Schleser & Bossio, 2019) on digital and online representation of vulnerable communities. The project used in-depth co-creation relationships between researchers, video producers and participants over a long period of time and we argue that engaging this methodology shifted the conversation from a horizontal to a vertical plane of co-producers and creators. This allows a conceptual shift to take place (Schleser & Berry, 2018, p. 2), which results in stories that are told from a community perspective, rather than an author or auteur focusing on story elements and structures. The personal stories empower older people to engage with new technologies by a mix of interest-based learning, co-design storytelling and online sharing to make a positive contribution to representation of older people in online and digital spaces. Furthermore, surveys conducted after the project was completed showed that participants valued the intimate smartphone production framework and the extended co-design consultation phase and community engagement activity, which resulted in participants’ reporting of increased confidence to use and engage with digital technologies and modes of online representation.

Co-designing Mobile Stories Digital storytelling is defined by narrative formats that are created and shared using digital tools and formats (Davis et al., 2018). Digital storytelling generally produces a short video (up to five minutes) and can be used to combine elements such as narration, music, photography and even animation (Lambert, 2009). Digital storytelling has been used in a number of different contexts including community development, education, mental wellbeing and therapeutic purposes, and more recently, as a research method (De Jager et al., 2017). Smartphone digital storytelling differs slightly in that it uses low-cost smartphone equipment and editing software to create short, personal stories made predominantly for online publication and distribution (Meadows, 2003). Mobile devices and editing apps have become an increasingly important part of the digital storytelling process, in particular the “keitai” aesthetic of

9

MOBILE STORYTELLING WITH SENIORS …

121

digital stories and mobile-mentaries (mobile documentaries) (Schleser, 2011), which emphasises the way camera phones shape filmic experiences of the location, time and subject (Schleser, 2013). Given the increased potential for intimate connection with audiences (Waycott et al., 2015), smart phone-enabled digital storytelling has also grown into an important community engagement tool. In particular, co-designed mobile storytelling has been presented as an important method for working with otherwise marginalised communities, for developing digital literacies, and for supporting social impact projects (Davis et al., 2019; Schleser, 2012; Schleser et al., 2018) (Fig. 9.1). A co-design methodological approach for digital storytelling is defined by its inclusion of the participants of study in the process of designing and implementing the outcomes. Thus, a co-design process is distinguished by its non-hierarchical method of designing a project, where the traditional power imbalance between the subject and the creator is mitigated by the more active role participants take in directing and

Fig. 9.1 Opera Smartphone Filmmaking production https://opera.eclc.org.au

122

D. BOSSIO AND M. SCHLESER

sometimes in producing the project (Davis et al., 2019). The co-design process is valuable in ensuring participation among diverse—and sometimes marginalised—communities, but it also requires a great deal of researcher flexibility. Complexity can arise in the production process when working within the limits of institutional or organisational expectations, as well as catering for the unique requirements of individual groups. The co-design best practice report by CO.CREATE, an Erasmus + Strategic Partnership Project in Europe, emphasises that “a key tenet of co-design is that users, as experts of their own experience, become central to the design process” (CO.CREATE, 2017, p. 5). The report identifies eight elements as significant for an “open mind set” of production: skills of the facilitator, clear needs and shared pains, building a healthy environment for co-creation, diversity of the team, common vision and shared values, individual roles for individual goals, handling conflicts and interests, and reflection and evaluation (CO.CREATE, 2017). The co-design process thus needs to be flexibly designed and continually redeveloped in response to diverse communities and their experiences (Davis et al., 2018, 2019), and this requires a great deal of researcher/producer engagement. The co-design process is particularly useful in working with marginalised or vulnerable communities because it produces stories “with” rather than “on” participants (Conrad & Sinner, 2015) which can make it a more socially and culturally appropriate approach. The engagement of participants in story generation as well as in feedback opportunities, including community consultation, workshops and screenings, demonstrates how the engagement of storytellers commences before the first interviews and continues after the production is wrapped. The involvement of vulnerable participants such as older people works against issues such as paternalism. Burns and Masoodian (2018, p. 349) identify a gap in the current research for older people who also need “meaningful engagement in terms of entertainment, recreation and social connectedness”. They discuss storytelling as a “medium”, which allows older people to overcome technological barriers and more meaningfully connect to services. Following Randell (in Burns & Masoodian, 2018, p. 353), who defined this approach as a practice of “narrative care”, they suggest the needs of traditional subjects of the film are centred as part of the production process as an ethical and impactful mode of producing digital stories (Burns & Masoodian, 2018, p. 353).

9

MOBILE STORYTELLING WITH SENIORS …

123

Mobile Storytelling with Older People: Reflective Case Studies Much mobile storytelling with older people has been focused on challenging traditionally negative representations of older people as frail, cognitively impaired or conservative by highlighting the unique diversity of their lived experience. To do this, digital storytelling methodological strategies focus firstly on producers working with participants over an extended period of time, allowing continual co-design discussion, reflection and reiteration to refine the storytelling themes and event to engage with digital production based on personal interest. Secondly, documentary designers work as reflective practitioners (Schön in Keen, 2018) to reflect on what is made and use this reflection to inform what is made next. This means that the mobile storytelling process is necessarily intimate and iterative, working on building relationships with participants and working through reflection and refinement of stories over time. For example, the 60+ Online project demonstrates how older people can engage with digital literacies to create meaningful stories to share with their peers. The 60+ Online project fostered digital inclusion among 22 Australian older people (64–86 years), with varied digital skills, and socio-economic/cultural backgrounds. Within workshops, seniors were encouraged to draw upon personal and community interests to inform storyboarding and digital story development. Digital stories were generated using iPads and smartphones and edited using Adobe Premiere Clip. Social media sites Facebook and Instagram facilitated shared digital skills development, supported by workshop participants and researchers. Regardless of their skills at the outset of the project, every participant was able to produce their own digital story. These were showcased at festivals, City Council events and hosted on YouTube (McCosker et al., 2018). While the 60+ Online project illustrated the use of digital storytelling for learning digital skills, other forms of mobile story production indicate the therapeutic and artistic value of intimate and iterative storymaking. For example, May Days is a short mobile-mentary created by Sylvie Prasad, a British visual artist and photographer, about her mother May who lives in a care home in New Forest (Prasad, 2012). May has Alzheimer’s Disease, so Prasad used the mobile phone production process to ‘stand-in for the short-term memory’ (Prasad, 2017, p. 19). Prasad argues that the mobile phone was not only a recording device but a therapeutic tool. She used the mobile phone to start conversations and as a

124

D. BOSSIO AND M. SCHLESER

way to bring her and her mother together. These mobile interactions then began to form a reflexive autobiography. Prasad would edit the mobile video after visiting her mother and would then show her the edited material on her mobile device at the next visit, prompting further discussion. This kind of prompted storytelling has been shown to be therapeutically beneficial for those suffering cognitive decline (George & Houser, 2014). As a visual diary, the work also created a meaningful record for her family. She envisioned May Days as a digitally shared family album, where “social media is largely about visual culture and operates like a form of extended family album, albeit with a global reach” (Prasad, 2017, p. 58). Similarly to Prasad, the Australian smartphone filmmaker Jason Van Genderen created the Facebook series Oma’s Applesauce (Van Genderen, 2020) about his mother Hendrika Van Genderen, who suffers from Dementia. The filmmaker wanted to reach out to other people caring for a loved one with Dementia through powerful but intimate storytelling. Referring back to Prasad’s notion of the global family album, Van Generen’s family created a supermarket for Oma (“grandmother” in Dutch) in their home during the COVID-19 lockdowns in NSW and captured the Oma’s “visits” in short smartphone films for their social media channel. The videos became a viral hit and were featured in multiple news broadcasts around the world. Van Genderen and his team at the Treehouse production company continued to produce videos documenting the everyday life of their family with a focus on Oma. Van Genderen suggested that producing the stories not only brought his family together, but also created an informal peer support network for carers of older people with dementia through the social media channel. Both Oma’s Applesauce and May Days indicate how the prioritisation of lived experience and intimate quality of mobile filmmaking can impact on a more positive representation of older people online, bringing audiences together to share in the intimate, but everyday experiences of families. Finally, mobile storytelling has been used in Arts-based practice and research to provide opportunities to share and innovate in online environments. For example, Grey Matter AR by Karen Vanderborght uses Augmented Reality and Social Media as an artistic platform. Featured at MINA’s 2019 International Mobile Innovation Screening (Schleser, 2019, online) program,1 Grey Matter AR is a portable version of a Magic Mirror, flipping selfies into self-reflection, with the help of older people. To do this, Vanderborght filmed and interviewed ten Canadian seniors from diverse backgrounds to provide a unique source of life advice

9

MOBILE STORYTELLING WITH SENIORS …

125

for younger generations. Using an AR lens and Snapchat2 these experiences are shared in exhibition spaces, aiming to build intimate connection between ages and communities. These examples demonstrate a key element in mobile storytelling which brings people together and shares human experiences. Beyond the overly quoted “campfire experience” of sitting together to share stories, digital sharing of mobile filmmaking prioritises social connection in both the distribution and engagement with the stories. Whether this in community workshops as in 60+ online or Gray Matter AR, or family settings as in May Days or Oma’s Applesauce, smartphone filmmaking capture these experiences but also draw about social sharing techniques to engage via social media with relevant communities. As storytellers, older people can draw upon their lived experience and personal experiences as a gateways to story worlds. When the personal connection to a story is authentic, it creates empathic engagement with audiences regardless of age. A co-designed and participant-driven storytelling approach relates to Randall’s formulation of “biographical encounters” where older people can support their peers through new skills and share their stories to enhance “recreation and social connectedness” (Randell in Burns & Masoodian, 2018, p. 353). Working with traditionally marginalised communities to empower their engagement has many positive impacts, but also requires deliberate planning for flexibility within the methodological approach, both for the collection of data from participants and the production of the digital stories. This can be encapsulated by Keen’s design thinking approach to digital storytelling production, which encourages “reframing a problem to understand and define it” (Keen, 2018, p. 53). Within Keen’s 21 overall propositions for designing documentary production, three key ideas were adapted during the OPERA Project for approaches to mobile storytelling with older people or other marginalised communities (Keen, 2018, p. 61). These include: 1. “The documentary designer works iteratively as a reflective practitioner (Schön, 1983) to reflect on what is made and use this reflection to inform what is made next; 2. The documentary designer iteratively design tools and production frameworks for these tools, and,

126

D. BOSSIO AND M. SCHLESER

3. The documentary designer understands that work may not be viewed in its entirety and may have multiple viewings (Keen, 2018, p. 61).” During the OPERA Project, a digital storytelling project for social impact on community perceptions of ageism, the researcher/producers of the digital stories worked within Keen’s framework to understand firstly, how ageism is perceived and experienced by older people in the region, and secondly, how that evidence could be used to frame a co-designed digital storytelling intervention into community experiences of ageism. While many mobile storytelling projects have been focused on challenging traditional representation of older, this project also used methodological strategies that engaged and empowered participants to co-design the production itself.

The Opera Project: Digital Storytelling for Social Impact The OPERA Project (Older People: Equity, Respect and Ageing) was a co-designed digital storytelling project produced in partnership with a community legal centre located in the eastern metropolitan region of Victoria, Australia. The project team of four producer-researchers used co-design methodologies to foreground the voices of older people as part of a community-led primary prevention strategy to bring awareness to the stories of active older people in the Eastern metropolitan region of Melbourne. The OPERA project was conducted in two phases; the first engaging a comprehensive consultation process with a diverse range of older people across the Eastern Metropolitan Region (EMR) of Victoria to understand how they experience ageism in their communities. The second phase of the project used the evidence gathered to produce a digital intervention into community experiences of ageism by creating a counter narrative that was co-designed by older people and foregrounded positive experiences of ageing. The aim of the OPERA Project was to facilitate older people’s leadership of a digital storytelling strategy that focused on framing ageing as a series of transitions, rather than an experience of continual decline. The digital stories aimed to challenge ageism with messages about a positive journey of ageing. Participants told stories based on their diverse

9

MOBILE STORYTELLING WITH SENIORS …

127

backgrounds and experiences but overwhelmingly represented ageing as a positive experience and older people as active, positive and socially connected, despite having to combat negative perceptions and dismissive behaviours towards them. Therefore, the process of co-designing, creating and viewing the OPERA digital stories was an important methodology both to reveal the complexity of older people’s experiences of ageism, but also to ensure that older people’s lived experience was not positioned as representation of “victims”. The co-design process was facilitated using six community workshops held at a local town hall in 2019. Researchers recruited interested participants through existing community networks including the local Elder Abuse Network (EAN) and local council Active Ageing networks. The project recruited 27 participants with an equal split between the 61–70 and the 71–80 age groups. Participants were predominantly female (21 females to 6 males) and retired, receiving a pension and living in their own home. The project worked with a diverse group of older people living in the eastern region of Melbourne, including local Aboriginal groups, CALD communities and participants with a disability. Participants were incentivised with a voucher to attend both workshops. During the second phase of the project, co-design workshops were conducted with participants to design and evaluate the key messages and format of the videos. Key themes identified from Phase one community consultations around the expressions and experiences of ageism centred on: images and stereotypes about ageing, social and infrastructure barriers to equal participation in the community; and community attitudes and behaviours that directly or indirectly discriminate against older people. These themes were used to inform the structure of workshops in the second phase of co-design, where researchers worked with participants over an extended period to discuss and refine the storytelling themes and messaging. To do this, the co-design process was designed so that each participant engaged with a series of structured activities, followed by in-depth discussion. For example, the co-design workshop activities included: “Storyboard your life as an older person”, where participants were encouraged to use images, words, colours, descriptions or any other creative approach to represent their current lifestyle as a senior. This could include personal or family life, social time, hobbies, responsibilities, jobs, etc.—anything they felt was significant in their life. Other activities included “What does being older look like?”, where participants were encouraged to create a collage of words, images and experiences

128

D. BOSSIO AND M. SCHLESER

for and “Preparing for getting older” where participants were asked to devise an ageing “cheat sheet” that explained what they might say to prepare family and friends for getting older. Participants shared what they had produced once activities were completed. This also allowed more direct questioning aimed at eliciting more information or were otherwise aimed at co-designing the video production approach more directly. After the first workshops, key messages, stories and themes were developed into potential video story worlds. The workshop participants returned to discuss and give feedback on the story themes that the Swinburne research team formulated from the findings of the first workshop. This is an important part of a co-design process, where feedback and agreement are given throughout the development process. Rather than just informing the researcher’s findings, the co-design process allows community attitudes and experiences to be represented throughout the design, production and editing process. Almost all participants described the importance of regular connection with family and friends, including taking on some caring responsibilities for grandchildren, or pastoral care duties for family or neighbours. Others suggested that they needed to carefully plan a new life after losing an intimate partner and gave examples of important connections to community. In the workshops, participants wanted not only to set the agenda, but also be included in the video production. The research team ensured that a diversity of stories and even socio-cultural representation was established. This approach reflects Schön’s “reflective designing” (Schön in Keen, 2018) thus coincides with self-reflective approach in mobile storytelling. “Collaborative mobile filmmaking shifts away from looking at film as an end product and towards examining the processes that can be revealed, …audiences have agency and can take part in deciding about the final form of the projects”. (Schleser, 2012, pp. 408/9). The most important consideration of the OPERA’s co-design methodology was reflexivity in the practitioner’s work with older people. A flexible feedback model allowed researchers to change and develop video messaging and production style to accommodate the participant perspectives. In designing this methodology as a primary prevention strategy for elder abuse, it was important that the co-design process was structured to allow researchers to be flexible and responsive to the participants’ suggestions and experiences for the digital intervention. The digital storytelling process was shaped through interaction between researchers and participants and was, therefore, an emergent process, guided through a cyclical

9

MOBILE STORYTELLING WITH SENIORS …

129

process of participants’ initial input, researcher translation and then participant feedback. Allowing this flexibility, prioritised the voices and agency of the participants and ensured a shared approach to the creation of the intervention. This provided the researchers with an opportunity to be responsive to community, to celebrate and acknowledge participant involvement in the production process and ensure community perception of “ownership” of the project—thereby increasing the likelihood of continued sharing and discussion of the key messages in the videos after the project’s completion. During the community workshops particular themes emerged for discussion in the digital stories, while participants were chosen for the videos based on their own personal stories and willingness to be filmed. This led to the second consideration for this project—the iterative design of the production. Schleser (2012) argues that a reflexive process results in the engagement and creates a sense of presence and sociability. Mobile devices are lens-based communication tools, thus the creative process embraces a more communicative and connected approach. During the production process, participants were able to link stories and experiences discussed in the workshop with objects and spaces in a more personally significant environment, such as their homes, their gardens or important community spaces. They were involved in the choice of filming location, brought treasured objects into the space and decided how they would appear during the story. Some chose a traditional interview format, while others performed daily rituals, such as working in the garden, doing exercise or even performing spiritual activities. These decisions were iterative and required a great deal of producer flexibility in considering how the needs of the participant could be incorporated into a video with professional filmic qualities, including lighting and sound. In considering production elements in a co-design context, mobile filmmaking is uniquely positioned to provide an intimate, non-intrusive, yet high quality production. Seth Keen (2018, p. 63) suggests that it is “important to understand the technical limits of different types of video cameras” as it impacts on the way producers interact with participants in a digital and mobile storytelling production. The OPERA stories were filmed with a two-camera set-up using iPhone 10. The two-camera setup gave the appropriate options in the editing. The producers chose to work with radio mics and natural light in order to create a natural and authentic feel. The mobile “tools” meant that production could adapt to the senior’s environment and position them in a way they wanted to see

130

D. BOSSIO AND M. SCHLESER

them, i.e. in their gardens, community settings, parks, kitchens or living rooms. This not only meant that the interviewees were comfortable but also showed them as active participants in their communities caring for their peers and others. This made working with an iPhone important when working with vulnerable communities because the small camera and ability to create an intimate set up took pressure off interviewees. The crew was accompanied by an Assistant Director who explained the filming process to the participant while setting up and introduced the crew. While the production turn-around was similar to the operation of a television crew, producing each story in less than half a day, the team still took time to engage in conversations with participants. The participants had been previous participants in the workshops, which created trust and gave the seniors confidence in the vision of the project. Based on the co-design methodology, the team filmed, edited and produced nine 5-min videos showcasing positive stories about ageing, as told by thirteen older participants from the Eastern Metropolitan Region of Victoria. The digital stories focused on the voices of older people, talking about their everyday lives to an audience envisioned as other older people and their families. The stories were based on a direct question and answer interview style of production, based on the time researchers spent in the workshopping process learning about participants’ lives. While most of the stories talked about somewhat banal topics such as finding new hobbies, gardening and keeping active at an older age, the stories also openly acknowledge some of the difficult transitions associated with ageing including disability, bereavement and transition into retirement but also positioned these as surmountable difficulties in what could be an otherwise active and independent lifestyle. These types of stories helped to disrupt ageist stereotypes by normalising the ageing process as a series of transitions that active, happy and socially connected people can plan for, and enjoy. Finally, the digital stories were produced in the context of online sharing and local community consumption and engagement. During the production process, a story formula was refined and by means of working with a two-smartphone interview set-up, the transition between shots could create a recognisable rhythm in each short story. The videos were created with the same visual ID as the website and thus provide a seamless experience when watching on the dedicated project website, or when

9

MOBILE STORYTELLING WITH SENIORS …

131

shared via social media create a visual link to the projects’ visual communication design. The videos were edited with Adobe Premiere Pro by the 3- person production and editing team. The intro and outro jingle was composed by Simon Longo to give the project and engaging and uplifting experience. This meant that the producers assumed that not all stories would be viewed in one sitting but were instead made with online engagement and sharing in mind. The videos are presented on the Eastern Community Legal Centre’s Vimeo channel3 and linked to the OPERA Project’s dedicated website,4 which also provides further information about age discrimination and elder abuse. A coherent web format was created in order to provide thematic continuity across the diverse interview subject matter that was covered. The videos were organised under thematic messages of positive ageing including: sharing kinship and companionship; healthy minds, bodies and spirit; living independently; finding new friends and communities, among others. This conceptual frame was visualised through an initial montage matrix seen on the homepage of the website, which brought older people’s stories together in one “hero” video (Schleser & Bossio, 2019).5 Using a split screen, multiple voices were combined into a hero video that represents the OPERA project and the values of senior citizens. The following initial prompt: “We are still here. We have survived. We still ask questions. We are still learning. We still aspire to more. Listen to us. Life hasn’t stopped” characterises the disruptive quality of each individual story within a traditional narrative of ageing as decline. These representational decisions reflected what was discussed in the co-design workshops as people recounted the changes they had made in their own lives as they aged. The participants, their families and their peers also came to the screening of the films and became advocates for the positive journey of ageing. This was shown in pre- and post-screening surveys that indicated an increase in awareness of ageism among audience members after engaging with some of the digital stories.

Conclusion: A Storytelling Framework for Community Engagement The mobile storytelling projects discussed in this chapter demonstrated how co-creation methodologies can help empower older people to create their own representations “as experts of their own experience”

132

D. BOSSIO AND M. SCHLESER

(CO.CREATE, 2017). While there are many examples of how older people’s stories can provide positive representations of ageing, the OPERA project developed a co-design approach that also provided a framework for empowering older people. The community collaboration in the form of workshops and ideation for the mobile stories provides insights into the communities’ perspectives and prioritises older people as experts of their own experience. By means of further developing digital storytelling into the realm of mobile and smartphone filmmaking, new opportunities to work with communities through long-term co-creation relationships were also established. The approach is sustainable as the cocreation workshop model allows community members to become part of the creative process, and equipment in the form of smartphones is accessible for ongoing video production in the community. These community “visions and shared values” (CO.CREATE, 2017) are then translated into a story structure where community members set goals and craft representations from their viewpoint. Thus, mobile storytelling is an innovative method for community engagement and as a reflective tool ensures that representations are created in a way that communities would like to see these on screen and social media. A shared screening at the end of the projects is also key to validate the mobile stories and celebrate the collaborative work, assigning a leadership role to participants, rather than producers. Mobile storytelling thus develops a reflexive process which allows participants to understand their position and the influence their stories have on others. Acknowledgements We would like to acknowledge and thank participants in the OPERA consultation workshops, and digital storytelling participants, who have been deidentified for this article. The OPERA Project website and digital stories can be found at www.opera. eclc.org.au. We acknowledge the following contributors to the OPERA project referred to in this article: Filming and editing: Max Schleser. Filming and post-production assistants: Shuai Li and Felix Gyebi Amofa. Sound production: Simon Longo. Research Assistance: Ivana Randjelovic. Research Team: Diana Bossio, Max Schleser, Hilary Davis, Anthony McCosker.

9

MOBILE STORYTELLING WITH SENIORS …

133

Notes 1. MINA 2019 program: https://mina.pro/screenings/2019-2/. 2. Snapchat https://www.snapchat.com/l/en-gb/. 3. ELC - Eastern Community Legal Centre, https://vimeo.com/use r5005671 and https://vimeo.com/360950984. 4. https://opera.eclc.org.au/. 5. OPERA Stories: https://opera.eclc.org.au/stories.

References Bossio, D., Exon, J., Schleser, M., McCosker, A., & Davis, H. (2019a). The OPERA Project: Community co-design of digital interventions for primary prevention of ageism and elder abuse. Swinburne University of Technology. Bossio, D., Schleser, M., McCosker, A., & Davis, H. (2019b) online. Retrieved on April 9, 2020. https://opera.eclc.org.au Burns, L., & Masoodian, M. (2018). Storytelling: A medium for co-design of health and well-being services for seniors. In Entertainment Computing – ICEC 2018 (pp. 349–354). Conrad, D., & Sinner, A. (Eds.). (2015). Creating together: Participatory, community-based, and collaborative arts practices and scholarship across Canada. Wilfrid Laurier University Press. Davis, H., McCosker, A., Bossio, D., & Schleser, M. (2018). 60+ online: Enhancing digital inclusion of seniors via mobile digital stories and social media participation. The Journal of Community Informatics (Special Issue: Designing Participation for the Digital Fringe), 14(1), 38–59. Davis, H., Waycott, J., & Schleser, M. (2019). Digital storytelling: designing, developing and delivering with diverse communities. In Managing complexity and creating innovation through design (pp. 15–24). Routledge. De Jager, A., Fogarty, A., Tewson, A., Lenette, C., & Boydell, K. M. (2017). Digital storytelling in research: A systematic review. The Qualitative Report, 22(10), 2548–2582. George, D. R., & Houser, W. S. (2014). “I’ma Storyteller!” exploring the benefits of timeslips creative expression program at a nursing home. American Journal of Alzheimer’s Disease & Other Dementias, 29(8), 678–684. Keen, S. (2018). The documentary designer: A list of propositions for interactive documentary practice online. Palgrave MacMillan. Lambert, J. (2009). Where it all started: The centre for digital storytelling in California. In J. Hartley & K. McWilliam (Eds.), Story circle digital storytelling around the world (pp. 79–90). Wiley-Blackwell.

134

D. BOSSIO AND M. SCHLESER

McCosker, A., Bossio, D., Holcombe-James, I., Davis, H., Schleser, M., & Gleeson, J. (2018). 60+ online: Engaging seniors through social media and digital stories. Swinburne Social Innovation Research Institute, Swinburne University. Meadows, D. (2003). Digital storytelling: Research-based practice in new media. Visual Communication, 2(2), 189–193. Prasad, S. (2012). May days independent production. Prasad, S. (Ed.). (2017). Creative mobile media: A complete course. World Scientific. Schleser, M. (2011). Mobile-mentary: Mobile documentaries in the mediascape. LAP LAMBERT Academic Publishing. Schleser, M. (2012). Collaborative mobile phone filmmaking. In E.-J. Milne, C. Mitchell, and N, de Lange (Eds.), Handbook of participatory video. Altamira Press Schleser, M. (2013). From ‘Script to Screen’ to ‘sh%t n share’. In J. Daniels, C. McLaughlin, and G. Pearce (Eds.), Truth, dare or promise: Art and documentary revisited. Newcastle Upon Tyne. Schleser, M. (2019). MINA International Mobile Innovation Screening 2019 online. Retrieved on April 9, 2020. https://mina.pro/screenings/2019-2/ Schleser, M., & Berry, M. (Eds.) (2018). Creative mobile media II – Making a difference. In Mobile story making in an age of smartphones. Palgrave Pivot. Schleser, M., & Bossio, D. (2019). OPERA stories online. Retrieved on April 9, 2020. https://opera.eclc.org.au/stories Schleser, M., McCosker, A., Bossio, D., & Davis, H. (2018). 60+ Online. Retrieved on April 9, 2020. https://www.youtube.com/channel/UCr8mS qSCsXMQPjbTTOQop5w Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books. Sneeuw, A., Retegi, A., Predan, B., Spisakova, B., Enrique, T., Tremetzberger, G., et al. (Eds.) (2017). Co-design best practice report. Co Create. Online. Retrieved on April 9, 2020. http://www.cocreate.training/wp-content/upl oads/2017/07/co-design_best-practice-report.pdf Van Genderen, J. (2020). Oma’s Applesauce. Independent Production Tree House Productions (Online). Retrieved on April 9, 2020. https://www.fac ebook.com/omasapplesauce/ Vanderborght, K. (2019). Gray Matter AR. Independent Production. Retrieved on April 9, 2020. https://www.behance.net/gallery/80724755/Grey-Mat ter-AR Waycott, J., Davis, H., Thieme, A., Branham, S., Vines, J., & Munteanu, C. (2015, April). Ethical encounters in HCI: Research in sensitive settings. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (pp. 2369–2372).

CHAPTER 10

Engagement with Mhealth: Experiences of Young Women in Bangladesh Naziat Choudhury

Social and religious taboos, discrimination and misinformation related to women’s health are prevalent in Bangladesh. One example of this is the myths and taboos around menstruation. During the menstruation period, many women are confined in their homes for seven days and must follow a specific diet due to “the widespread beliefs that menstrual blood is polluting and dangerous, and that the menstruating body is weak and shameful, lead[ing] to behaviors that expose women to health risks” (Seymour, 2008, p. 1). Menstruating women are required to sleep separately, must avoid using sunlight to dry the cloths used during their menstruation, are not permitted in their kitchen and they cannot see a doctor or attend any social meetings (Mondal et al., 2017). Various forms of health complications develop from such unhealthy conditions, such as urinal infections, irregular pains and increase risk factors during pregnancy (Wasata, 2018). The lack of sex education in schools results in building

N. Choudhury (B) Department of Mass Communication & Journalism, University of Rajshahi, Rajshahi 6205, Bangladesh

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_10

135

136

N. CHOUDHURY

taboos around puberty which significantly impacts young adults as they go through the changes of puberty. These social and cultural practices negatively affect girls and women’s health while also creating a barrier for their social and economic development in the long run, as women are more reluctant than men to seek help for their health issues. Mobile phones can be a supportive tool in health care; as studies have shown that due to easy access, affordability (Shankar et al., 2015) and cost effectiveness (Whittaker, 2012), mobile phones can play a significant role in health care services in developing countries. Information made accessible through these technologies can detect, prevent and treat common health problems among women in Bangladesh. For example, non-government organizations such as BRAC, Dnet and the SAJIDA Foundation provide health care support to pregnant women, new mothers and their newborn children by connecting local health workers with women through mobile phones (Alam, 2018). This paper examines the unexplored area of mHealth (mobile health) use by young women who are yet to experience motherhood. Through semi-structured interviews, this research maps out young women’s usage patterns of mHealth facilities and seeks to understand its significance in their lives. The paper also explores the ways young women collect health information. This will offer an insight into their experiences with mHealth and addresses a clear lack in research related to the use of mHealth by women in the Bangladesh context. The health care system in Bangladesh is weak. Impediments to strengthening the health care system include corruption, inaccessibility, mismanagement, insufficient number of doctors and nurses and a lack of resources (Islam & Biswas, 2014; Mahmood, 2012). For instance, data from 2018 shows that there were 0.6 physicians (World Bank, 2021a) and 0.4 nurses and midwives per 1000 people in Bangladesh, whereas WHO set the minimum standard to 2.5 health workers (which includes physicians, nurses and midwives) per 1000 people (World Bank, 2021b). The unequal distribution of medical services is common (Islam & Biswas, 2014). However, there have been some positive changes implemented from late 1980s and early 1990s (Jeji, 2019). Mass awareness campaigns on national television and radio channels were launched to educate women on nutrition, vaccination and pregnancy. Female health workers were also employed to go door to door to educate women on reproductive health. These initiatives helped to reduce the child mortality rate by more than 200 percent and fertility rate from seven births per

10

ENGAGEMENT WITH MHEALTH …

137

woman in 1971 to 2.3 in 2010 (Jeji, 2019). Part of this development has been the introduction of mHealth. The government and non-government organizations have realized that the wide accessibility of mobile devices can contribute to and transform health care services, which face ongoing infrastructural and financial challenges in the country. Bangladesh is a country which has witnessed unprecedented growth in mobile technologies. Around 98 percent of the total population are covered by the cellular network (WHO, 2011) and in 2019 there were 160 million mobile subscribers (BTRC, 2019). In 2018, the total number of Internet subscribers in Bangladesh was 80.829 million and most of them (75.396 million) were accessing it with their mobile phones (BTRC, 2018). According to the Bangladesh Bureau of Statistics (BBS), in 2019, 71 percent of women between the age of 15 and 49 admitted to possessing a mobile phone; however, the BBS believe that the actual number was much greater (Chowdhury, 2019). Seeing the potential environment for mHealth services to succeed, several initiatives were developed. Since 1990, around 225 types of eHealth and mHealth projects have been introduced and in 2012 there were 26 functioning mHealth projects (Alam, 2018). Non-government organizations such as Synesis IT, BRAC, Dnet, CRP Telemedicine and Pulse Health Services Limited are also investing in mHealth services. However, most mHealth initiatives have been implemented by private organizations (Ahmed et al., 2014). Despite the infrastructural and financial constraints, Bangladesh is gradually incorporating a number of mHealth features along with their existing health care services. This article investigates whether young women are taking advantage of these mobile-based health facilities.

Adaptation and Use of Mhealth Services Mobile health (mHealth), a new area of eHealth, is the third wave of digital technology in health care which. moves towards the digitisation of as many elements of healthcare as possible and the interaction and exchange of data between different institutions and systems: an overarching framework that incorporates data not only from healthcare institutions and systems but actors and agencies outside this sector. (Lupton, 2015, p. ix)

138

N. CHOUDHURY

The term “mHealth” is used to explain the process of using the Internet and other technologies through mobile devices to provide health care support and services to the public. Technology is advancing at a rapid pace and the existing technological infrastructures are being strengthened by powerful networks which enhances the connectivity. These powerful networks of mobile technologies are creating possibilities for offering individualized medical services (WHO, 2011). mHealth is also “expediting the decision-making process of medical professionals without affecting their role in diagnosis” (Sun et al., 2016, p. 3). WHO (2011) defines mHealth as “medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, personal digital assistants (PDAs), and other wireless devices” (p. 6). In the context of mHealth in Bangladesh, mobile phone services are used in the form of phone calls, SMS, the Internet and Bluetooth, which are used to receive information and treatment and also to provide support to health care professionals. Although mHealth is a new area of research, its various services and users of these services are being studied in developed and developing countries. A common feature of mHealth is using mobile technologies for health management and monitoring programs (Anderson et al., 2013; Katz et al., 2012; Sun et al., 2016). mHealth initiatives are being used to support senior citizens in China for age-related medical concerns (Sun et al., 2016). Information is immediately transmitted through wearable devices to health workers, which expedites the treatment process and also assists in the rehabilitation of the patients. Sun et al. (2016) argue that “mHealth could also enhance the international cooperation of medical providers in different time zones and the sharing of high-quality medical service resources between developed and developing countries” (p. 1). In addition to wearable devices, text messages/the short message service (SMS) has also been an effective tool in behavior change interventions (Dale et al., 2016; Hall et al., 2015; Krishna & Boren, 2008). For example, for patients with cardiovascular diseases, awareness through text messaging impacted their regular intake of medication and increased their physical mobility (Dale et al., 2016). Social media is also being used by patients to share experiences and gain support. In addition to medicine-based treatment, patients often need more psychological and informational support than that which is provided by friends and families. Social media contributes to building

10

ENGAGEMENT WITH MHEALTH …

139

this support through joining online support groups and finding information online. Thus, social media has been successfully used for disease management and in the improvement of quality of life (Alanzi, 2018; Fernandes et al., 2018; Ventola, 2014). Smailhodzic et al. (2016) suggest that social media plays a supportive role for medical professionals, noting that patients were sharing their inner feelings through social media which strengthened their emotional state (Smailhodzic et al., 2016). Within health management, tracking health data through apps is another example of mHealth. Mobile technologies provide 24 hours of monitoring and tracking health measures, which can be used for the improvement of physical health and mental wellness (Lupton, 2016; Neff & Nafus, 2016; Rooksby et al., 2014). Self-tracking methods, include collecting, monitoring and analyzing health data, have been well received by the users of these apps; however, fitness and nutrition-based health-related mobile apps were used predominantly (Bert et al., 2014; Krebs & Duncan, 2015). These apps are used more frequently by younger generations than older generations, as younger people prefer more personalized apps (Krebs & Duncan, 2015). All around the world, various aspects of mHealth services, such as phone calls, SMS, social media and apps, are being used for health maintenance, lifestyle management, building awareness and also for improving medication adherence. However, although there are advantages associated with mHealth, its ability to provide adequate treatment and ensure the security of patients’ personal data has raised questions (Becker, 2016; Malhotra et al., 2014). These factors affect the use and acceptability of mHealth services. Studies regarding mHealth services in Bangladesh primarily focus on maternal care (Dahdah et al., 2015; Feroz et al., 2017) and the current status and use of mHealth technologies (Alam, 2018; Khatun et al., 2014). A survey conducted among nearly 5000 respondents in a rural area of Bangladesh found that 81 percent of households had mobile phones. However, despite this high usage of mobile phones, there was a lack of knowledge about the availability of mHealth care, which played a significant role in the low usage of these services in Bangladesh (Khatun et al., 2014). Only 31 percent of the respondents knew about mHealth care and, among these, 39 were male and only 26 percent were female. Interestingly, these services were used primarily by women (2.4 percent of respondents) rather than men (1.4 percent). Khatun et al. (2014) explain that.

140

N. CHOUDHURY

[this] reflects the influence of the cultural factor that limits female mobility in this rural area; females are only taken out for healthcare when all other options are exhausted. (p. 7)

Initially, the type of mHealth in Bangladesh was mainly informationbased services such as call centers through which tele-consultations were provided (Ahmed et al., 2014). But, as communication technologies were made more easily accessible, more mHealth features were introduced. Karim et al. (2016) found that around 180 apps have been developed for Bangladeshi users specifically, with the majority (65) of them focused on providing “health care information” such as seeking information on blood donations and medication. The second highest number of apps developed (18) were for fitness and for supporting pregnant women and babies (Karim et al., 2016). With the development of a diverse range of programs and services, the number of users increased as well. But, are younger women between the ages of 18 and 25 taking advantage of these facilities? These younger women are the ardent users of mobile technologies and one way to ensure better and easier access to medical assistance is usage of these technologies. There is little research that focuses on young women’s use of mHealth services. The existing literature in the context of Bangladesh also shows a lack of research on these groups of women and on the features of mHealth they use the most. This paper aims to fill this gap.

Method mHealth has created opportunities for new ways to access health care support. This paper examines how young women residing in a developing country take advantage of these technologies. Participants were initially invited through the author’s social network, with other participants selected through snowball sampling. This method was selected because finding women who use mHealth services through any other sampling technique is difficult as there are no data available on young generations’ mHealth use. Fifty-four young women who were users of mHealth services were selected for semi-structured interviews, as this interview format empowers young women to share their stories. The young women participants were between the age of 18 and 25, were not married and had not experienced motherhood. During the interviews, both open and close-ended questions were asked to explore how and for

10

ENGAGEMENT WITH MHEALTH …

141

what purposes the young women were using the mHealth services. The interview consisted of four sections, each of which dealt with one topic: (a) access to health information and health care professionals; (b) the use of phone calls, SMS and websites for health purposes; (c) the use of social media for health purposes; and (d) the use of apps for health purposes. This process provided information and reflection about the choices these women made regarding various aspects of mHealth services and also their experiences of access to these services. Each interview lasted for approximately 25–30 min. The respondents were reassured that the information provided would only be used for research purposes. Descriptive statistics, in this case percentages, was used to analyze the close-ended questions and thematic analysis was conducted for the open-ended questions.

Access to Health Information and Health Care Professionals The majority of the respondents (22 percent) were 20 years old. Around 15 percent were in the 22- to 23-year-old age group. These young adults were mainly students (74 percent), while others were engaged in professions such as teaching, business, administration and accounting. At the initial stage of the interview, an attempt was made to understand the reasons for their use of mobile technologies in relation to information on health and health care professionals. The majority of the young women (70 percent) declared that they sought information from the Internet and social media first, in preference to asking those within their own social network. Consultations with family, friends and doctors varied depending on their health conditions. It was only when symptoms worsened that respondents made an appointment to talk with a physician. The main reason for their reluctance to visit a doctor was a lack of privacy. Respondents reported that some doctors saw all of their patients in one room, speaking to them one by one, while other patients were within hearing range. Disclosing information about their illnesses in such a scenario was discomforting for these women, especially when dealing with sensitive issues such as menstruation. They noted that mobile technologies offered an alternative solution that functioned as a path to connect with the doctors. Furthermore, respondents stated that information obtained from mobile phones had replaced that of friends and elder sisters, who had worked as a peer support system in the past. These Internet tools were working as their peer support system. This finding contradicts research

142

N. CHOUDHURY

that suggests that social networks were relied on for health information (Owusu-Addo et al., 2016; Riahi et al., 2015). The women argued that the Internet and social media were their preferred media as they could hide their identities (55 percent). One of the respondents (age 24) described the scenario as follows: Some issues are still a taboo or a topic for gossip in our society. To seek medical help for such conditions, for us we look for tools through which we could hide ourselves. The Internet and social media provide us that opportunity.

In relation to hiding their identities, being able to express themselves freely through these media without leaving personal information was cited as important for 35 percent of respondents. A 19-year-old woman explained, “At our age we continue to go through physical changes which we feel uncomfortable speaking to a doctor about it in a face-to-face context. We can answer our queries with ease through social media.” Convenience was also another reason for choosing the Internet and social media (10 percent). These advantages were missing in physical in-person contexts, as personal information shared with family and especially friends could be shared without their permission, resulting in losing respect in society. The specific features of mHealth used to seek such advantages are discussed in next sections.

Use of Phone Calls, Sms and Websites SMS was a popular mHealth service and was used for receiving messages related to health issues. About 72 percent of the respondents received messages from leading mobile service providers and their health-related helplines in Bangladesh such as Grameen Phone, Banglalink, Robi and Teletalk. Only a small number (13 percent) called these helplines for advice. But, the young women actively used websites to access health information, with around 60 percent of respondents visiting mHealth websites between one to five times per week. The international websites they visited included the Mayo Clinic, Healthline, WebMD and Flo, a period tracking app. A small number of them mentioned that they occasionally accessed national websites such as Maya and Square Hospital. They searched online for apps (39 percent), reviews on doctors (20 percent), symptoms of illness (20 percent), temporary treatments, diets

10

ENGAGEMENT WITH MHEALTH …

143

and tips on healthy living (17 percent) and home remedies for certain health issues (4 percent). After getting primary information online, more than half (53 percent) consulted a doctor. They visited doctors when they suffered from high fever, chest congestion and severe sore throats (50 percent), weakness or tiredness (25 percent), muscle pain, broken arms or legs (10 percent), irregular menstruation cycles (10 percent) and for other reasons (5 percent declined to comment). Among those who consulted doctors, 45 percent of them reported that they relied on the websites and apps to understand their health issues, as some doctors were reluctant to provide details about their concerns. A 22-year-old respondent explained, “The doctors only mentioned the health problem and the treatment for it, but I needed more information as to why and how of the disease.” This experience was reflected in findings, where 70 percent of the respondents identified the information found online to be helpful. The reasons stated are discussed in Table 10.1. Unfortunately, the respondents in this study did not question the reliability of online health information. One respondent explained, “Why worry? Having access to so much information is a blessing.” The authenticity and accuracy of health information found online has been a concern for health professionals around the globe (Vicario et al., 2016). The absence of digital and information literacy could be a contributing factor for Bangladeshi women’s lack of knowledge about the issue of the credibility and reliability of online information. Limitation of digital literacy should be a concern for these young women, as they not only rely on websites but also social media for information, which is also inundated with fake and misinformation. Table 10.1 Helpfulness of online health information Rationales for finding online information

Percentage

Detailed explanation of disease; helps to understand the symptoms Find specialized hospitals and doctors at home and abroad Herbal treatment Book appointments Saves time and money Free diet plans and exercises Healthy lifestyle tips

51 17 11 9 6 3 3

144

N. CHOUDHURY

Use of Social Media Social media is the most widely used feature of the Internet. In 2019, 28 million Internet users in Bangladesh visited Facebook (Internet World Stats, 2019). Seventy-six percent of the young women in this research reported accessing Facebook to look for health and other medical information. More than half of the respondents visited these pages less than five times per week. Searching for health information was their most frequent activity (51 percent). For 25 percent of young women, Facebook was a tool for gaining emotional support during a health crisis. The other 14 percent relied on Facebook to consult doctors and to find recommendations for them and 10 percent to watch videos on healthy foods and lifestyles. Around 43 percent of the young women visited Facebook groups that were created to locate doctors or hospitals. Among these, 64 percent claimed that they received helpful and instructive information from those pages. According to them, along with important health tips, they also received information on herbal and alternative medications. The rest of the young women (57 percent) who did not collect information from these groups complained that these pages contained limited information and the hospitals or the medical staff who ran the Facebook pages were reluctant to respond to their queries. The participants also claimed that these Facebook pages carried mainly commercials for the hospitals or doctors. International Facebook health-based groups or pages were often visited, namely Dr Mehmet Oz, Health Digest, Health + , Time to Change and Apollo Hospitals. Compared to local websites, local Facebook health pages or groups were visited more often when information on local doctors and health care services was available. Nearly half of the young respondents (45 percent) chose to visit national pages such as Maya and Pulse Healthcare Services. In 2019, Pulse had 2291 Facebook followers, whereas Maya had 319,040. The Maya Facebook page focuses on physical health, mental health, lifestyle and beauty-related content and they have gained popularity among a wide audience. Pulse, in contrast, is dedicated to mainly health information and consultation with health professionals. Similar to the reliability of information found on various websites, the respondents were also not concerned about the accuracy of health content on Facebook. As stated by one 24-year-old respondent, there was a common belief that “if found online or on Facebook then it must be true.” However, as anyone can create and share content online, health

10

ENGAGEMENT WITH MHEALTH …

145

information found on social media can often not be trusted (Moorhead et al., 2013; Ventola, 2014). Researchers, health practitioners and other health service providers often stressed that health-related information found on social media “needs to be monitored for quality and reliability, and the users’ confidentiality and privacy need to be maintained” (Moorhead et al., 2013, p. 2). The participants of this research indicated their reluctance to verify fake and misinformation found on Facebook pages and groups. Nevertheless, the respondents did not limit their use to Facebook only, mobile apps were also a consulted entity of mHealth.

Use of Mobile Apps Familiarity with health-based mobile apps was noticed as 57 percent of participants had five or fewer apps which were accessed less than five times per week. The cost of the apps played a deciding factor on whether to download an app and thus the most favored apps were free. The most frequently cited apps were Flo, Pedometer, Daily Yoga, My Calendar and Calm. The only local apps that had managed to attract this young group of women’s attention were Maya and Pulse. Pulse is available through both Google Play and Apple’s App Store, whereas Maya is available only through Google Play. The discussion of the advantages of these apps stirred a mixed reaction among participants, with 51 percent finding such apps beneficial while 49 percent disagreed. The national and international apps used by these respondents helped them to track their menstrual cycle information (70 percent), physical activity (20 percent) and overall healthy lifestyle (10 percent). In addition to these conveniences, some found these apps effective for gaining support, with around 43 percent of respondents mentioning that these apps worked as a peer support system. As friends and family members were not always with them, these apps provided the support which would otherwise have come from friends and family. For instance, the constant reminder through app notifications to move and eat well encouraged participants to take extra steps or participate in a sports activity which they would otherwise have avoided. About 16 percent of the respondents added that they decided to join a gym or sports club in their institution and, according to them, these apps worked as a motivator. A 23-year-old woman noted, “As the multitasker generation, we often forget to look after ourselves, hence we depend on apps to remind us to work out and relax.” Meditation-based health apps were also used by a small number of respondents (10 percent). Although

146

N. CHOUDHURY

not statistically significant, their opinions carried value. While traveling to or returning from work or university, these women used apps to help them meditate and calm their minds. Some even used it during lunch periods which, as a 24-year-old respondent claimed, brought some peace from the everyday chaos. A 25-year-old woman explained, “As a working woman, work and family responsibilities cause mental and physical pressure. Although the meditation apps are not always helpful, as they charge money for better programs or need to find peaceful space and time, but still these meditation apps provide a little break.” Another 25-year-old woman pointed out that “some of these apps can be used offline which makes them more convenient for women like me who is always on the move.” Household chores and infrastructural obstacles limit the use of mobile apps; however, the ability to use these offline helped to maintain psychological well-being. Fitness, nutrition and period trackers were the most common categories of health apps used. A study in England (Kayyali et al., 2017) and in the United States of America (USA) (Krebs & Duncan, 2015) also found that health and lifestyle-based apps were more popular among mHealth apps users. Along with tracking menstrual cycles and physical activity, mHealth apps were used for compiling health data, identifying the symptoms and prevention methods of disease, communicating with health professionals and keeping updated on medication (Karim et al., 2016). The Bangladeshi young women in this study had not yet explored the vast options provided by these health apps. For instance, none of the respondents tracked their sleeping patterns, level of sugar intake or blood pressure. When asked if they worried about their data safety, they replied in the negative. Trusting these app companies with their personal data was not a concern for them. None of them expressed a fear of losing personal information to a third party.

Conclusion mHealth can play a supporting role alongside health care professionals to encourage young women to seek help related to their physical and mental health and contribute to improved well-being. The main contribution of this paper is highlighting the pattern of young women’s usage of mHealth services and its significance in their lives, an area overlooked by researchers in Bangladesh. Facebook was found to be the most popular aspect of

10

ENGAGEMENT WITH MHEALTH …

147

mHealth for young women. The second feature that attracted the attention of mHealth users were mobiles apps, specifically Flo, Pedometer, Daily Yoga, My Calendar and Calm. But, previous research has shown that phone calls and SMS were the widely used means of mHealth. Another significant finding of this research is that the young women in Bangladesh are realizing and utilizing the potentials of mobile technologies for health and health care information. Facebook and mobile apps, excluding interaction with the immediate social network, were working as a peer support system for these women, as they heavily relied on them for information. However, access to authentic and reliable information is essential in bringing about changes in people’s belief systems and behavior. Unwillingness to recognize or trace fake and misleading information could lead to serious negative health outcomes for these women. This suggests that these issues were overshadowed by the advantages of having a readily available and accessible health information source. Digital and health literacy in young women needs to be improved so they can critically evaluate health information for mHealth facilities to become widely and successfully adopted and to avoid health complications in future.

References Ahmed, T., Henry, L., Khan, A. S., Islam, R., Bhuiya, A., & theIqbal, M. (2014). eHealth and mHealth initiatives in Bangladesh: A scoping study. BMC Health Services Research, 14, 1–9. Alam, M. Z. (2018). mHealth in Bangladesh: Current status and future development. The International Technology Management Review, 7 (1), 112–124. Alanzi, T. (2018). Role of social media in diabetes management in the middle east region: Systematic review. Journal of Medical Internet Research, 20(2), 1–13. Anderson, C., Henner, T., & Burkey, J. (2013). Tablet computers in support of rural and frontier clinical practice. International Journal of Medical Informatics, 82(11), 1046–1058. Becker, D. (2016). Acceptance of mobile mental health treatment applications. Procedia Computer Science, 98, 220–227. Bert, F., Giacometti, M., Gualano, M. R., & Siliquini, R. (2014). Smartphones and health promotion: A review of the evidence. Journal of Medical Systems, 38(1), 1–11. BTRC. (2018). Internet subscribers. Retrieved October 25, 2018, from http:// www.btrc.gov.bd/content/internet-subscribers-bangladesh-january-2018

148

N. CHOUDHURY

BTRC. (2019). Mobile phone subscribers. Retrieved September 02, 2019, from http://www.btrc.gov.bd/content/mobile-phone-subscribers-bangladeshmarch-2019 Chowdhury, A. (2019). 71 percent women with mobile phones in hand. The Daily Ittefaq. Retrieved October 23, 2020‚ from https://www.ittefaq.com. bd/national/108273 Dahdah, M. A., Lou, A. D. D., & Méadelc, C. (2015). Mobile health and maternal care: A winning combination for healthcare in the developing world? Health Policy and Technology, 4, 225–231. Dale, L. P., Dobson, R., Whittaker, R., & Maddison, R. (2016). The effectiveness of mobile-health behaviour change interventions for cardiovascular disease self-management: A systematic review. European Journal of Preventive Cardiology, 23(8), 801–817. Fernandes, L. D. S., Calado, C., & Araujo, C. A. S. (2018). Social networks and health practices: Influence of a diabetes online community on adherence to treatment. Ciênc. Saúde Coletiva, 23(10), 3357–3368. Feroz, A., Perveen, S., & Aftab, W. (2017). Role of mHealth applications for improving antenatal and postnatal care in low and middle income countries: A systematic review. BMC Health Services Research, 17 (1), 1–11. Hall, A. K., Cole-Lewis, H., & Bernhardt, J. M. (2015). Mobile text messaging for health: A systematic review of reviews. Annual Review of Public Health, 36, 393–415. Internet World Stats. (2019). Internet usage in Asia: Internet users, Facebook subscribers & population statistics for 35 countries and regions in Asia. Retrieved November 15, 2019, from https://www.internetworldstats.com/ stats3.htm Islam, A., & Biswas, T. (2014). Health system in Bangladesh: Challenges and opportunities. American Journal of Health Research, 2(6), 366–374. Jeji, D. (2019). Healthcare in Bangladesh: Women to lead the way. The Daily Star. Retrieved November 03, 2019, from https://www.thedailystar.net/opi nion/news/healthcare-bangladesh-women-lead-the-way-1711867 Karim, M. M., Islam, M. N., Priyoti, A. T., Ruheen, W., Jahan, N., Pritu, P. L., et al. (2016). Mobile health applications in Bangladesh: A state-of-the-art. Paper presented at the 2016 3rd International Conference on Electrical Engineering and Information Communication Technology (ICEEICT), 22–24 September, Dhaka Katz, R., Mesfin, T., & Barr, K. (2012). Lessons from a community-based mHealth diabetes self-management program: ‘It’s not just about the cell phone’. Journal of Health communication, 17 (Suppl 1), 67–72 Kayyali, R., Peletidi, A., Ismail, M., Hashim, Z., Bandeira, P., & Bonnah, J. (2017). Awareness and use of mHealth apps: A study from England. Pharmacy, 5(2), 1–14.

10

ENGAGEMENT WITH MHEALTH …

149

Khatun, F., Hanifi, S., Iqbal, M., Rasheed, S., Rahman, M. S., Ahmed, T., Hoque, S., Sharmin, T., Khan, N. U. Z., Mahmood, S. S., Peters, D. H., & Bhuiya, A. (2014). Prospects of mHealth services in Bangladesh: Recent evidence from Chakaria. PLoS ONE, 9(11), 1–9. Krebs, P., & Duncan, T. D. (2015). Health app use among U.S. mobile phone owners. Journal of mHealth and uHealth, 3(4), e101 Krishna, S., & Boren, S. A. (2008). Diabetes self-management care via cell phone: A systematic review. Journal of Diabetes Science and Technology, 2(3), 509–517. Lupton, D. (2015). Preface. In D. Lupton (Ed.), Beyond techno-utopia: critical approaches to digital health technologies (pp. IX-XIII). MDPI. Lupton, D. (2016). The quantified self . Polity Press. Mahmood, S. A. I. (2012). Health systems in Bangladesh. Health Systems and Policy Research, 1(1), 1–4. Malhotra, N., Kim, S. S., Kim, S. S., & Agarwal, J. (2014). Internet users’ information privacy concerns (IUIPC): The construct, the scale, and a causal model. Information Systems Research, 15(4), 336–355. Mondal, B. K., Ali, M. K., Dewan, T., Tasnim, T. (2017). Practices and effects of menstrual hygiene management in rural Bangladesh. Paper presented at the 40th WEDC International Conference, Loughborough, UK Moorhead, S. A., Hazlett, D. E., & Hoving, C. (2013). A new dimension of health care: Systematic review of the uses, benefits, and limitations of social media for health communication. Journal of Medical Internet Research, 15(4), 1–34. Neff, G., & Nafus, D. (2016). Self-tracking. MIT Press. Owusu-Addo, S. B., Owusu-Addo, E., & Morhe, E. (2016). Health informationseeking behaviours among pregnant teenagers in Ejisu-Juaben Municipality, Ghana. Midwifery, 41, 110–117. Riahi, A., Hariri, N., & Nooshinfard, F. (2015). Study of health information needs and barriers to access among Afghan and Iraqi immigrants in Iran. Journal of North Khorasan University of Medical Sciences, 7 (3), 597–610. Rooksby, J., Rost, M., Morrison, A., & Chalmers, M. (2014). Personal tracking as lived informatics. Paper presented at the CHI 2014, Toronto, ON, Canada Seymour, K. (2008). Bangladesh: Tackling menstrual hygiene taboos. Retrieved November 01, 2019, from https://www.unicef.org/spanish/wash/files/10_ case_study_BANGLADESH_4web.pdf Shankar, M., Ananthakrishnan, A., Muraleedharan, V. R., & Venkatesh, G. (2015). An exploratory study of mobile health [mhealth] technology in the Southern States of India. Retrieved September 10, 2019, from https://www.researchgate.net/publication/298346352_Use_of_ Mobile_Technology_in_Healthcare_case_of_Southern_India

150

N. CHOUDHURY

Smailhodzic, E., Hooijsma, W., Boonstra, A., & Langley, D. J. (2016). Social media use in healthcare: A systematic review of effects on patients and on their relationship with healthcare professionals. BMC Health Services Research, 16(1), 1–14. Sun, J., Guo, Y., Wang, X., & Zeng, Q. (2016). mHealth for aging China: Opportunities and challenges. Aging and Disease, 7 (1), 53–67. Ventola, C. L. (2014). Social media and health care professionals: Benefits, risks, and best practices. Pharmacy and Therapeutics, 39(7), 491–499. Vicario, M. D., Bessi, A., Zollo, F., Petroni, F., Scala, A., Caldarelli, G., Stanley, H. E., & Quattrociocchi, W. (2016). The spreading of misinformation online. Proceedings of the National Academy of Sciences, 113(3), 554–559. Wasata, R. (2018). Through the lenses of rural health disparities: Menstrual hygiene management (MHM) in rural Bangladesh. Unpublished paper. Whittaker, R. (2012). Key issues in mobile health and implications for New Zealand. Health Care and Informatics Review Online, 16(2), 2–7. WHO. (2011). mHealth new horizons for health through mobile technologies. Retrieved September 11, 2019, from https://www.who.int/goe/pub lications/goe_mhealth_web.pdf World Bank. (2021a). Physicians (per 1,000 people). Retrieved November 03, 2019, from https://data.worldbank.org/indicator/sh.med.phys.zs World Bank. (2021b). Nurses and midwives (per 1,000 people). Retrieved November 03, 2019, from https://data.worldbank.org/indicator/SH.MED. NUMW.P3

CHAPTER 11

Mobile and Digital Stories from Down Under: Tales from Rural Australia Hilary Davis and Max Schleser

The Context: Yarriambiack Shire Yarriambiack Shire is a local government area in Victoria, Australia, located in the north-western part of the state, more than four hours’ drive northwest from the capital city of Melbourne. It covers an area of 7158 square kilometres and, at the 2016 Census, had a population of 6674 people. Yarriambiack Shire includes a number of towns—five of which are included in these mobile-mentaries, that is, the towns of Hopetoun, Murtoa, Rupanyup, Beulah and Minyip. The populations vary for the townships, the largest is Murtoa (865 people), and Beulah has the smallest population (429 people) (Australian Bureau of Statistics,

H. Davis (B) Social Innovation Research Institute, Swinburne University of Technology, Melbourne, Australia e-mail: [email protected] M. Schleser Department of Film and Animation, Swinburne University of Technology, Melbourne, Australia

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_11

151

152

H. DAVIS AND M. SCHLESER

2017). The area is known for being the heartland of grain production in the Wimmera and Mallee region with agriculture being the region’s main industry. Many people living in the area have a shared history, as some families have lived in the township for generations. This connection between the land and the people that live there, and their sense of pride in their shared history, are embodied in the digital stories. We encounter examples of this shared history in images and conversation about, for example, personal experiences of working in a grain “stick shed” over fifty years ago; local community members telling stories of the history of the founding of the township and key community members depicted in a town mural; community members developing a lake for recreational activities; donations to a collection of farming equipment for display in a farming museum and an archive of horse-related memorabilia. In all the digital stories, the storytellers fondly recount past memories, the significance of artefacts and key community members who contributed to the shared sense of history. These digital stories were funded by the Yarriambiack Shire council for the purposes of promoting local, rural and unique places and spaces of interest with the aim of increasing recognition of local attractions and encouraging visitors to the area.

Background to the Digital Stories We are drawn to mobile storytelling, because this visual medium of sharing stories supports diverse content and foregrounds experiences that may otherwise remain hidden. Mobile Storytelling leverages social media to create connections within the community and beyond. The Yarriambiack Shire rural communities were consulted about where and how they wanted to share the digital stories. Mobile storytelling work utilises similar approaches and frameworks to Digital Storytelling, but is more accessible, as everyday technology such as smartphones are utilised for filming. Writing in Documentary Across Platforms, Zimmermann (2019) develops an inspiring account that demonstrates a case for mobile storytelling interventions. In the chapter Toward a Theory of Participatory New Media Documentary Patricia Zimmermann (2019) argues for an engagement with community texts. Her study provides recognition for a diverse documentary ecology, which thrives beyond the character-driven, narrative arc, the genre-infused feature film, the heavily resourced and

11

MOBILE AND DIGITAL STORIES …

153

corporatised interactive documentary and the documentary infusion in galleries (Zimmermann, 2019, p. 231). She says; “We are living in one of the most exciting periods of documentary: the documentary turn in the biennales, the participatory turn, the digital turn, the postcolonial turn, and the anthropocene turn churn up our theories and histories.” (ibid). Zimmerman defines her argument for small-scale, local, sustainable, modestly resourced, long-time-frame collaborative projects often ignored by festivals, museums, new media convenings and scholars as “utopian imaginary” (Zimmermann, 2019, p. 232). Only if we can imagine these possibilities can we work in this direction. As a filmmaker working with smartphones and social scientist with a focus on rural communities we hope that this work can contribute to “the unresolved realm of participatory new media collaborative documentary built on collaboration, collectivity, engagement, politics, and process.” (Zimmermann, 2019, p. 232). These characteristics are reflected in Tales from Yarriambiack Shire. As a mobile storytelling production and research project we prioritised collaboration with small rural Victorian communities. As a participatory new media project, Yarriambiack Shire Tales is defined by horizontal, place-based production. A number of meetings were conducted online and in person before filming. During these meetings, in most cases, we trained participants how to work with video-conferencing tools, Skype and Zoom. This meant that a telephone conversation was needed to explain how to connect to Wi-Fi, download the software, operate the camera and other elements necessary for digital engagement. Another aspect was to recognise that every community had a particular focus. We thus included multiple dialogues, which Zimmerman defines as another key characteristic of participatory new media documentaries (Zimmermann, 2019, p. 233). In order to capture various ideas and ideals about Yarriambiack Shire, we worked with a story canvas model to map out the collaborative smartphone documentaries in form and content. The story canvas tool is developed by Digital Storytellers1 and is available via Creative Commons. Mobile Storytelling focuses on smartphone filmmaking for social innovation. Within the context of social change one can note work by organisations such as Digital Storytellers,2 Insight Share,3 Video 4 Change,4 Film for Change Aotearoa5 or Engage Media.6 The #MINA2018 screening featured KUUJJUAQ, a video essay about the perspective of an indigenous teenager on his hometown, filmed on a Go Pro Camera. KUUJJUAQ, as much as the mobile-mentaries described in this chapter,

154

H. DAVIS AND M. SCHLESER

make a difference for the way communities represent themselves. The qualities of smartphone filmmaking, which cannot be replicated by DSLR, Blackmagic or Red cameras (and any other standard size digital video camera), defined as intimate, immediate and referencing the significance of location are key in work with rural and often marginalised communities (Fig. 11.1). In collaboration with Yarriambiack Shire Council the idea of cocreating digital stories showcasing local rural innovation was initiated. From initial meetings, a relationship was forged with the Council, who were keen to collaborate on projects that showcased local communities and utilised digital participation techniques. The stories in the mobile-mentary were co-created by key members of these communities, including long-term residents, historical society members, committee members from Probus Club, local business people and others. These

Fig. 11.1 Tales from Yarriambiack Shire http://bit.ly/YarriambiackTales. Screened at St Kilda Film Festival, June 2020 (Melbourne, Australia) and NYU Cellphone Cinema Showcase, June 2020 (New York, USA)

11

MOBILE AND DIGITAL STORIES …

155

people consider themselves guardians of local history, and advocates for these townships and their residents, which despite declining populations, still have much to offer to visitors. Due to the long drive to rural areas, the authors supported at least one local person in each of the six communities to use video-conferencing software such as Skype or Zoom. Typically this involved providing instructions by email in advance of the meeting, and talking to community members over the phone immediately prior to the meeting, to support them through the teleconferencing setup and connectivity process. Typical issues included downloading software, creating a user account and password, setting up an external camera (most used older laptops without internal cameras) and connecting to local WiFi. This was necessary to meet-and-greet local people, support meetings over the distance, as well as arrange future onsite workshops. The latter workshops took place in locations in the Wimmera, including hotel foyers, cafes and community centres. In addition, we visited the location of the stories, and stayed overnight, for example, at the site of the horse ranch for Horsing around Beulah, and at an old railway hotel outside Murtoa. We sought advice from the community about what and where things should be filmed for particular purposes and local insights. The mobilementaries featured local musicians, interviewees and voice-over narrators. While these digital stories were completed on a very tight budget, which included generous donations of unpaid time from the researchers, and time and effort from the township members, we, as researchers, feel enriched by the friendships we have made through the co-creation of the Yarriambiack Shire Tales. The collaborative approach was more inclusive and prioritised community requests over story structure or aesthetic considerations. In the edits for the film festival the focus on story-structure and aesthetic decisionmaking was prioritised. In both approaches people and places were the key characteristics. This place-based quality, which Zimmermann identified as another key element in participatory new media documentary (Zimmermann, 2019), involves participants from different social sectors across communities in regional Victoria and featured lived experience in small communities. Aston and Odorico in The Poetics and Politics of Polyphony: Towards a Research Method for Interactive Documentary advocate for an approach which “consists of independent voices which are fully equal, and which become subjects of their own right, without needing to serve the ideological position of the author” (Aston & Odorico, 2018, p. 64). They

156

H. DAVIS AND M. SCHLESER

define as a “road” where different components and subjects of projects can meet and interact in a process of “branching out.” Their key provocation for this research is the Interrogating the multiple; i.e. multiplicity of aesthetics, narratives, authors, realities and screens (Aston & Odorico, 2018, p. 73). Interrogating the multiple is something that one could relate to smartphone filmmaking and mobile storytelling within the contemporary mediascape. Yarriambiack Shire Tales merged multiple aesthetics, collaborative mobile filmmaking, Cinematic VR and short form documentary into one web series. The narratives are diverse, as are the people and places in rural Victoria. A range of people including community members, council members, local historians as well as researchers contributed to creating the Tales from Yarriambiack, which seek to represent the community according to their views and visions. Tales from Yarriambiack provides glimpses into the everyday life and realities of some rural Victorian towns, using a combination of resources, including smartphones as filmic devices. In collaborating with, and codesigning these digital stories with local community members, we hoped to inspire the community to continue and grow this approach. As Aston and Odorico emphasise: “all (of) this is also about building relationships of understanding within a specific complex system and environment” (Aston & Odorico, 2018, p. 78). As part of the production we visited these rural communities numerous times and formed opportunities for feedback. While this meant an extensive amount of time in the field, we sought to provide authenticity of rural Australian experiences, capturing stories from rural Australia as a lived experience (Davis et al., 2020). The aim of mobile storytelling is to create a community approach and its objective to drive engagement and create an open dialogue on issues emerging out of the storytelling process. Digital storytelling is multi-purposeful—it has been used for educational purposes, such as developing skills in digital literacy and creativity (Davis et al., 2018; McCosker et al., 2018), and it has been used for supporting social work practices (Lenette et al., 2015) and for exchanges of intergenerational knowledge between Indigenous communities (Edmonds, 2014) among others. With support, people from a variety of backgrounds, including people from diverse or marginalised communities (Davis et al., 2019) may be empowered to create and share stories of their experiences, and the spaces and places in which they are shaped.

11

MOBILE AND DIGITAL STORIES …

157

Writing in Local Content Producers: Co-Creating Communal Stories and Community in the Big Stories, Small Town Participatory Documentary Project, Martin Potter discusses the Big Stories Small Town project, which included filmmaker residencies within communities. He identifies the filmmaker-in-residence with bringing a professional toolkit of aesthetic, narrative and technical languages (Potter, 2019, p. 199). The article points at the collaborative relationship with professional outsiders as “technical facilitators” and local content producers and argues that “for the professional creator in general, the role of active listener must involve critical thinking and critical engagement.” (Potter, 2019, p. 204). The filmmaker in Yarriambiack Shire Tales, Schleser, not only had experience in having worked on major participatory film projects such as Pangeda Day 7 as a commission through Nokia, initiated communitybased international collaborations as in the case of 24Frames 24 h 8 and Viewfinders,9 working with cultural organisation like the Goethe Institute on #Nucleus,10 but also worked with co-author Davis, who has extensive expertise and connections with rural communities. Schleser based his production approach on “localised aesthetic as an art of engagement” (Schleser, 2012, p. 405), which is inspired by the community artist Peter Dunn.

How the Project Was Conceived One of the authors of this chapter, Davis, had spent much time in regional and rural Victoria conducting research and program evaluations. During these times, she came to appreciate the warmth of the local people, the beauty of the landscape and the unique attractions, such as the Silo Art Trail—large-scale mural portraits painted onto grain silos. The Silo Art Trail is Australia’s largest outdoor gallery stretching 200 kms across the Wimmera Mallee region.11 With members of the local community, casual conversations arose about local attractions in the area, and how these could be best promoted to bring visitors to the region. The researcher felt that mobile digital stories—an approach the authors had used in other projects in the region—had potential for promotion of local attractions. On this basis, the author wrote a two-page project outline, submitting it to Yarriambiack Shire, where it caught the attention of a council member. Together, over lunch in the central township of Horsham, this project was conceived. The council member wrote to key local people connected with

158

H. DAVIS AND M. SCHLESER

local attractions in the Shire, asking for expressions of interest in partnering with the project. Subsequent meetings with the council member further identified which stories might be told, by whom, when and in what order. These details were significant to ensure authenticity of storytelling to reflect both the history of the attractions and the key people involved. The timing of stories had to be planned around key local events such as planting, harvesting and festivals, when local residents worked long hours to meet the needs of the community. Once a trusted relationship had developed through Yarriambiack Shire Council connections, the authors contacted the residents via telephone, email, skype or met face to face. The method of connectivity used depended upon a range of factors including whether the meeting could coincide with a visit to the region, which technology the community member was most comfortable with, their access to technology and digital connectivity. In some of the areas, broadband connectivity was inconsistent, which was frustrating for both researchers and community members, as it impeded conversations.

The Pre-production Process The pre-production process was based on discussions between the filmmaker, social researcher and interested community members. We used the Storycanvas method, licenced under Creative Commons, to map out the narrative for each story. The Storycanvas method utilised a Storycanvas template which was workshopped by participants and facilitators, to define key elements of the story including story structure, location of filming and selection of interviewees as well as questions asked during the mobile stories. These included such things as: Who is the intended audience for the digital story? What key messages do you want to share? What kind of story shall we tell? For example, a high impact vision piece, a colourful explainer or perhaps a more personal journey? How should the digital story start, how should it end and what memorable moments happen in between to keep us engaged? Who will feature in the story and what locations will be used? Use of the Storycanvas methodology is a tested methodology that “provides a lens for telling stories that resonate with audiences and (can) build into successful campaigns.”.12 Our intention with these digital stories, therefore, was, with local community members, to use the digital story medium to authentically capture and communicate rural life in the Wimmera that resonated with both local and non-local audiences, thereby creating recognition and change (Davis et al., 2020).

11

MOBILE AND DIGITAL STORIES …

159

Co-design and Co-creation of Yarriambiack Shire Tales Following consultation with members of the Yarriambiack Shire Council and Yarriambiack Shire community, we chose to apply a TV style interview approach because this would be a recognisable format for viewers and potential audience, in combination with the participatory documentary approach (Nichols, 2010). Nichols characterises this approach through “engagement, strong investment in the encounter with others or in presenting a historical perspective” (Nichols, 2017, p. 109). Both accounts define the approach used in the Yarriambiack Shire Tales. Within the historical perspective it is key to note that communityengaged projects are driven by the community members. The Wood’s Farm and Heritage Museum is an initiative driven by two brothers, the Stick Shed is managed locally by a Committee of Management and supported by the Murtoa Lion’s Club volunteers and Minyip’s residents retain pride in their town appearing in a 1980’s TV series by upkeeping and displaying props from the show. This connection to a particular time in history became a key story element and a signifier for the rural communities, and was further developed through time-lapses and personal archive material that was collected by the interview participants.

On Location: Capturing Rural Experiences The digital storytelling process encourages self-expression—storytellers are encouraged to select the artefacts which best represent their story. Typically upon arriving on location, the storytellers had already selected specific items which they hoped would feature in their story. In Beulah, for example, a large room in the neighbourhood house had displays of mounted photographs, patch-work quilts, horse racing cups and other memorabilia on display, and chairs strategically placed around the room for filming. We needed to ensure that we authentically captured this information, as it was important to local rural people. Thus, a range of artefacts feature in The Tales from Yarriambiack Shire including photographs and newspaper clippings from public and private collections (Beulah, Hopetoun), household items and farming equipment no longer in use (Rupanyup, Hopetoun) and items from pivotal moments in the town’s history such as television production sets (Minyip) and murals (Hopetoun, Beulah). Further, a deep sense of connection with

160

H. DAVIS AND M. SCHLESER

local surrounds including fauna, water and animals is embedded in the digital stories. The showcasing of these artefacts represented an opportunity for the communities to share this sense of connection and history with the audience. Thus, while the digital storytelling took place in spaces initially carefully chosen by Yarriambiack Shire council members, they were mediated by local township residents to focus on specific events, artefacts and people.

Smartphone Filmmaking, Drone and 360° Video Production We utilised smartphones for filmmaking, allowing the films to be conducted in a nonintrusive and personal style. Thus communities may have felt they had greater participation than if we had used large television style cameras, as community members felt comfortable asking the filmmaker to stop recording, or reshoot something of significance. The smaller scale of the smartphones helped ease community members into conversation, as many were familiar with the technology, and the smartphones were less intrusive for face-to-face interviewing. We used small tripods for the smartphones using a beastgrip rig. We filmed on an iPhone 7 and sourced an iPhone during production. Natural light was used, and working with Filmic Pro we could adjust to accommodate difficult but beautiful lighting conditions (such as in the Stick Shed). Using Rode’s i-XLR we were able to use XLR microphones from Swinburne University of Technology for sound purposes. These were placed beneath the top layer of the clothes of the storytellers. The production process also included use of a drone for the Flying Doctors episode to simulate the idea of a plane flying into the township. The drone footage was also reminiscent of the television series, when in the opening sequences, a plane flies over the township. In the 1980s, a helicopter or plane would have been needed for Arial photography (as in the original TV production by Crawford Productions from 1986 to 1993). Use of a DJI Maveic drone mimics the flights of the Flying Doctors plane into the township, but also showcases the distinctive but flat rural Australian landscape. The intro and outro sequences are thus quite significant in establishing a sense of place.

11

MOBILE AND DIGITAL STORIES …

161

360° Video to Capture Scale and Ambiance The Stick Shed 360 digital story was not originally included in the Yarriambiack Shire Tales program. However, the Stick Shed storytellers and filmmaker Schleser felt that a 360 camera could most effectively capitalise on the size, scale and ambiance of the Stick Shed. Writing in Mobile Cinematic VR, Schleser outlines spatialised storytelling, embodied stories and imaginative spaces as key considerations for mobile cinematic Virtual Reality [VR] (Schleser, 2019). The Stick Shed 360 is an exemplary example of this. The ambient lighting, the scale of the poles and the sheer vastness of the interior of the building would be difficult to capture utilising standard videoing techniques. The 360 camera was placed at eye level in the shed and thus could capture the architectural experience. By means of positioning the camera outside the stick shed as well as in various places inside, a range of lighting could be captured. The VR experience extends and amplifies the experience captured in the Stick Shed video, potentially sparking a more authentic representation of the space, and an interest in experiencing the Stick Shed in person.

Making Use of Travel Time During the production of our mobile stories we used the four-hour drive as pre- or post-production meetings to review, critique and reimagine the post-production outcomes. Adobe Rush software was used to support editing of audio–visual clips when driving back to Melbourne. During the four-hour drive we would review the rushes and select the best takes. Further, once back on campus and in the studio, these were synced through the Creative Cloud and imported into Premiere Pro for further editing towards the fine cut.

The Yarriambiack Shire Tales The following sections provide an introduction to the digital stories and provide a synopsis of the townships in which they are located. Yarriambiack Shire is known for being the heartland of grain production in the Wimmera and Mallee region of the state, with agriculture being the region’s main industry. This theme is central to the stories. The digital stories can be viewed through the links below. YouTube Channel: http://bit.ly/YarriambiackTales.

162

H. DAVIS AND M. SCHLESER

Playlist: https://www.youtube.com/playlist?list=PL-MF8DMBFbp ezwH7b6WjSW5NGnvPiCZgg.

The Woods’ Farming and Heritage Museum The Woods’ Farming and Heritage Museum, located in Rupanyup, Victoria is a memorabilia collectors dream. It was founded by local brothers, Michael and John Woods, who started collecting farming memorabilia in the early 1980’s. To house their burgeoning collection, the brothers purchased a large shed which they later opened as a museum and entrusted to the public in 2007. A second shed was purchased and opened in 2010 to showcase their extensive memorabilia collection. Much of the collection is displayed in discrete settings or rooms. For example, there is a dedicated hospital room, displaying medical equipment used in Murtoa Hospital. The hardware shop contains entire collections of spanners and various tools donated to the museum or purchased from swap meets (gatherings where people exchange and trade items). The museum’s “collectible room” is a room bursting with colour, containing many iconic pieces of Australiana memorabilia such as the classic Arnott’s and Anzac biscuit tins and for the younger generation an exciting display of M&M chocolate merchandise. Naturally, the Woods Museum showcases a number of tractors, stationary engines and farming machinery. The Ford and Fordson tractors date from 1918 well into the 1980s. The Woods Museum is managed by a community group “Friends of the Woods’ Museum.”

The Stick Shed Stick Shed 360 Story https://youtu.be/xwOf-18NMro. The Murtoa Stick Shed was a massive grain repository, at its height of production from the 1940s. The Stick Shed digital story is introduced by David Grigg, the Secretary of Management of the Stick Shed, and Bob Petschel, who worked in the Stick Shed in the 1960’s. The Stick Shed contains 516 Mountain Ash wooden poles derived from forests in the Dandenong Ranges. The poles run vertically from ceiling to floor, the highest one in the centre of the shed is 19 m (over 62 foot) tall. Grain was emptied from nearby grain silos by rail trucks, and once at the Stick Shed site were loaded onto a “hopper” (a cone-shaped container) via a large bucket. The grain was poured into the shed and reached the

11

MOBILE AND DIGITAL STORIES …

163

top of the walls. The Stick Shed is now out of commission, but at one time, the Stick Shed could hold 92,000 tons of grain. Up to 70 men were employed in the Stick Shed at the height of its production. While the Stick Shed is empty, the concrete floors, beautiful Ash poles, diagonal ropes from ceiling to floor and ambient lighting highlighting particles of dust are all evocative of past times. Working with David Grigg, Bob Petschel and members of the management committee, two digital stories were created. The first contains a conversation between the two locals about experiences and working conditions in the Stick Shed when it was at its peak of production in the 1960s. The second, the 360 video, is used to better capture the scale of the building and has a local radio announcer voice-over talking about the construction of the Stick Shed.

The Flying Doctors Story Minyip is the home of The Flying Doctors,13 a popular television series first showcased in 1985, that was broadcasted for nine seasons in Australia. The television series was located in the township of Minyip or “Coopers Crossing” as it was affectionately known by fans of the program. Consequently, Minyip has become a tourist destination for fans of The Flying Doctors from all over the world. Further, due to its location, flat terrain and residents welcoming of filmmaking, Minyip has been used as a location for other iconic Australian films, such as The Dressmaker 14 and “The Dry” 15 filmed in 2019. The Minyip digital story features memories of people—including family members of actors still resident in the town— about filming The Flying Doctors television series, and how this has helped to shape a sense of local history.

Horsing Around Beulah The township of Beulah has a deep connection with horses. According to the locals, early settlers first came into Beulah on horses. With the help of horse-power these settlers began to cultivate the land and build the township. Pony Express mail deliveries, a mail service delivering messages, newspapers and mail using relays of horse-mounted riders operated between Horsham to Beulah, a distance of 100 kms (62 miles). Children were sent to school on ponies, and adults would head into town on horses to buy their groceries. Although times have changed since then,

164

H. DAVIS AND M. SCHLESER

Beulah still maintains a strong connection between horses and community. Local resident Len Coffey, or “Squatter” as he is known to locals, has established a horse ranch where people can learn how to ride a horse and enjoy the nearby riding trails around the township. Squatter is well known locally, and he has been painted with a horse on a silo outside the nearby town of Rosebery, as part of a “silo art trail,” Australia’s largest outdoor gallery that stretches over 200 kms in the Wimmera. The Silo Art trail links small townships such as Brim, Rosebery, Sheep Hills and Rupanyup (focus of the Woods’ Farming Museum digital story). Riding horses is an enjoyable experience for visitors to Squatter’s ranch, in school holidays and at other times. Children and adults alike, can bring a “swag” (a sleeping bag with mattress) and sleep under the stars around a camp fire, to enjoy an authentic rural Australian experience. The digital story depicts this connection through personal accounts of growing up with horses, horse memorabilia and photographs (including a local horse mural) and families horse-riding at the ranch.

Hopetoun---A Lakeside Holiday Haven Hopetoun is a town on the Silo Art Trail on the Henty Highway. The Silo Art Trail is a collection of painted silos and water towers across rural Australia. Hopetoun is distinguished by a water fountain in the middle of the road: water is a central theme running through this digital story. The speaker who is off-camera, introduces three residents of Hopetoun, each showcase a key tourism feature of the township: Lake Lascelles (including lakeside accommodation), the Mallee Mural (a painted mural depicting the history of the town) and a historical building, the Corrong Homestead, a heritage listed home built in 1846.

Mobile Storytelling with Rural Australian Communities Dissemination of the Stories Following the production of the first two digital stories on location, a web-series episodic format was conceptualised. The editing process of the digital stories was based on feedback on the digital stories provided by the storytellers over a distance, e.g. via skype (for live discussions) or email. Each method for feedback depended on a range of individual

11

MOBILE AND DIGITAL STORIES …

165

and community factors. These included, digital connectivity levels (in some towns, Internet was intermittent), availability of devices (some community members did not have access to technology) and support (some community members required support from others who were more familiar with technology, in order to respond to or join skype meetings). For some, the townships were close by to other research work sites. So, for example, in Rupanyup, face-to-face meetings were held in the local hardware store, and were timed around other research work. The authors of this chapter spent ten to twelve hours editing each digital story, i.e. four hours for the rough cuts, three hours feedback from the team and four hours community feedback. One digital story, the Hopetoun story, was edited with the Yarriambiack Shire Councillor at the University, to ensure that the story correctly depicted local events. The councillor also provided the voice-over to ensure authentic local accent and terminology. Once edited according to the storytellers feedback, the stories were then reshared with the community members, via a private YouTube channel, for final editing commentary. This approach was taken to support rural community members who had limited access to digital devices or low Internet speeds, typically an issue for rural or remote Australian communities (Thomas et al., 2020). Thus, storytellers could access the link, wait for the video to load and then view. This allowed individual community members to view and provide commentary in their own time. Each digital story was co-edited two or three times using this approach. The final, publically available version of the YouTube playlist supports sequential viewing of the individual stories. The five episodes together have a running time of 30:58. All the stories are created in a similar format to introduce people and places. As a social documentary video using participatory, grassroots and community video approaches, we feel that the mobile-mentaries captured authentic accounts of elements of Yarriambiack Shire. The mobile stories were driven in content by the participants. The web series framework was flexible but coherent to develop a local flavour and particular signature. The 360 video of the Murtoa Stick Shed, as much as the mobile stories, defined a sense of place. Here the time-lapse and drone cinematography provided visual signifiers. In the pre-production and editing/montage process the feedback from communities set the foundation for the stories. As a result, we can present an authentic engagement using an iterative method and hope to provide some inspiration for people to visit the Yarriambiack Shire communities.

166

H. DAVIS AND M. SCHLESER

Conclusion The rise of mobile digital technologies such as smart phones and tablets, which include high-resolution photography and video tools, adopted by keen amateur and professional filmmakers, has enabled a more flexible, innovative and potentially non-hierarchical approach to digital storytelling. This is because mobile digital storytellers or mobile storymakers (Schleser, 2018) may now create and select their content, create their digital stories and even edit their digital stories on mobile devices. Mobile digital storytellers may use mobile technology and freely available editing tools such as Adobe Clip16 and/or Adobe Spark Video17 to craft their stories. Furthermore, many digital storytellers are making use of social media platforms to upload, share their experiences and elicit feedback. Indeed, we see a proliferation of such stories, on social media platforms (YouTube, Facebook, Viemo and Instagram), as well as on local community platforms or displays (see, for example, Davis et al., 2015) to share their experiences. In Yarriambiack Shire Tales decision-making about the storytelling approach was based on community conversations and discussions, rough cuts were shared for feedback, and in some cases, collaborative editing sessions took place. This process was open ended and reflexive, and all participants had to manage and work-around technology and Wi-Fi challenges. Rather than developing a singular narrative we view the digital stories as conversation starters. Their online dissemination means that their presentation can take multiple forms and formats across screenings and platforms. The digital stories have been viewed many times on YouTube. In the year since becoming publically available to view on You Tube, the Stick Shed 360 Story has been viewed 580 times, the Flying Doctor’s Story has been viewed 854 times, the Hopetoun: A Lakeside Haven story 670 times, the Woods Farming and Heritage Museum Story, has 487 views; and Horsing around Beulah, was viewed 633 times. Further, some communities have shared the digital stories on community and personal Facebook pages, and one community showcases it via DVD on a television screen based at the attraction site, for information for tourists. The Royal Flying Doctors episode was re-edited into a shorter and more storydriven format and screened as a finalist at the St Kilda Film Festival in Melbourne (2020), which is Australia’s longest running short film festival. The Horsing around Beulah episode was featured in the New York University Cellphone Cinema Showcase in June 2020.18 Thus, the

11

MOBILE AND DIGITAL STORIES …

167

reach of the digital stories, as intended, has extended far beyond the five rural Victorian townships.

Notes 1. https://www.digitalstorytellers.com.au/the-story-canvas/. 2. https://www.digitalstorytellers.com.au/. 3. https://insightshare.org/. 4. https://video4change.org/. 5. http://filmforchange.org.nz/. 6. https://www.engagemedia.org/. 7. Schleser (2011) Mobile-Mentary. Mobile Documentaries in the mediascape. Saarbrücken, Germany. 8. Schleser (2018) “#24Frames 24 h: An emerging documentary form: Workshop-generated videos” in Miles, A. Digital Media and Documentary. Antipodean Approaches. Palgrave: London. 9. http://www.viewfinders.gallery/ and Schleser, M. & Cammaer, G. (2018) “Viewfinders: A Collaborative Travel Film Project: Seeing the World through the Lens of the Pocket Camera” In Braisier, H. (eds. et al.) Docuverse: Approaches to Expanding Documentary (Version 1). Zenodo. http://doi.org/10.5281/zenodo.1211520. 10. https://www.goethe.de/prj/nmf/en/index.html?wt_sc=nucleus. 11. http://siloarttrail.com/home/#about. 12. https://www.digitalstorytellers.com.au/. 13. https://www.imdb.com/title/tt0088519/. 14. https://www.imdb.com/title/tt2910904/?ref_=nv_sr_1?ref_= nv_sr_1. 15. https://www.imdb.com/title/tt5144174/. 16. https://www.adobe.com/au/products/premiere-clip.html. 17. https://spark.adobe.com/about/video. 18. https://wp.nyu.edu/iccs/australia/.

References Aston, J., & Odorico, S. (2018). The poetics and politics of polyphony: Towards a research method for interactive documentary. Alphaville: Journal of Film and Screen Media, 15, 63–93.

168

H. DAVIS AND M. SCHLESER

Australian Bureau of Statistics. (2019). 3218.0 – Regional Population Growth, Australia, 2017–18: Population Estimates by Local Government Area (ASGS 2018), 2017 to 2018. 27 March 2019. Retrieved 25 October 2019. Estimated resident population, 30 June 2018. Davis, H., Fisher, D. C., & Randjelovic, I. (2020). “I Understand, Mate”. A Codesigned Comic-based Digital Story from ‘Down Under’. In Proceedings of the 2020 ACM Designing Interactive Systems Conference (DIS’20) (pp. 243–254). Eindhoven, Netherlands. Davis, H., McCosker, A., Bossio, D., & Schleser, M. (2018). 60+ Online: Enhancing digital inclusion of seniors via mobile digital stories and social media participation. The Journal of Community Informatics, 14(1), 38–59. Davis, H., Waycott, J., & Schleser, M. (2019). Digital storytelling: Designing, developing and delivering with diverse communities. In S. Mlettinen, and M. Sarantou (Eds.), Managing complexity and creating innovation through design, chapter 1, pp. 131-140. Taylor and Francis Limited, Oxford, UK. Davis, H., Waycott, J., & Zhou, S. (2015). Beyond YouTube: Sharing personal digital stories on a community display. In OzCHI, Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction (pp. 579–587). ACM. Edmonds, F. (2014). Digital storytelling and Aboriginal young people: An exploration of digital technology to support contemporary Koori culture. In M. Berry, and M. Schleser (Eds.), Mobile media making in an age of smartphones (pp. 92–103). Pal- grave MacMillan. Lenette, C., Cox, L., & Brough, M. (2015). Digital storytelling as a social work tool: Learning from ethnographic research with women from refugee backgrounds. The British Journal of Social Work, 45(3), 988–1005. McCosker, A., Bossio, D., Holcombe-James, I., Davis, H., Schleser, M., & Gleeson, J. (2018). 60+ online: Engaging seniors through social media & digital stories. Social Innovation Research Institute. https://doi.org/10. 4225/50/5acac04852868 Nichols, B. (2010). How can we describe the observational, participatory, reflexive, and performative modes of documentary film?. In Introduction to Documentary (2nd ed.). Indiana University Press. Nichols, B. (2017). Introduction to documentary. United States: Indiana University Press. Potter, M. (2019). Local content producers: Co-creating communal stories and community in the big stories, small town participatory documentary project. In C. Batty, M. Berry, K. Dooley, B. Frankham, and S. Kerrigan (Eds.), The Palgrave Handbook of Screen Production. Springer. Schleser, M. (2011). Mobile-Mentary. Mobile Documentaries in the mediascape. Saarbrücken, Germany.

11

MOBILE AND DIGITAL STORIES …

169

Schleser, M. (2012). Collaborative mobile phone film making. In E. Milne, C. Mitchell, and N. de Lange, N. (Eds.), Handbook of Participatory Video. AltaMira Press. Schleser, M. (2018). Smart Smart (Phone) Filmmakers >> Smart (Political) Actions. In M. Bohr, and B. Sliwinska (Eds.), The evolution of the image: Political action and the digital self . London. Schleser, M. (2019). Mobile Cinematic VR. In L. Hjorth, A. de Souza e Silvia, and K. Lanson (Eds.), The Routledge Companion to Mobile Media Art. London. Thomas, J., Barraket, J., Wilson, C. K., Holcombe-James, I., Kennedy, J., Rennie, E., et al. (2020). Measuring Australia’s Digital Divide: The Australian Digital Inclusion Index 2020, RMIT and Swinburne University of Technology, Melbourne, for Telstra. Zimmermann, P. (2019). Documentary across platforms: Reverse engineering media, place, and politics. Indiana University Press.

PART IV

Creative Practice Research

Creative Practice Research (CPR) as a research method can make key contributions to media, creative arts, design, and transdisciplinary research investigations. As filmmakers, musicians, scholars, and researchers, we approach CPR by means of exploring the opportunities for innovation in storytelling forms and formats and the continuous aesthetic refinement of mobile storytelling, including mobile music making and smartphone filmmaking. CPR research allows to push the boundaries within the research field of film, television, and digital media and also can make significant contributions to research projects from other disciplines. Within a constantly changing mediascape, mobile storytelling research explores new platforms and new micro-story formats as seen in Part II: Mobile News Storytelling. In an interdisciplinary context storytelling processes can contribute to drive social change, community engagement, the development of twenty-first century (or digital) literacies and present research findings in visual methods that go beyond texts and policy reports. Within this context, creative innovation is examined as proposing novel production approaches, processes, and projects, such as “Non Traditional Research Outputs” (NTROs) more generally. In order to provide an overview of this dynamic field Mobile Story Making in an Age of Smartphones (Schleser & Berry, 2018) and Mobile Media Making in an Age of Smartphones (Berry & Schleser, 2014) introduced discussions related to aesthetics, space and place, knowledge and stories, and the self. Story-Making further defined this creative practice relating to

172

PART IV:

CREATIVE PRACTICE RESEARCH

the field of screen media. “By means of shifting the conversation from the horizontal to a vertical plane of co-producers and co-creators, a conceptual shift takes place.” (Schleser, 2018a). When engaging in screen production research the linearity of pre-production, production, and postproduction as much as the linearity in production, dissemination, and exhibition is challenged by new models that are inspired by networks and more rhizomatic structures. In order to demonstrate new directions and developments, CPR provides a framework for scholarly investigation. The chapters presented in this part on Performative Storytelling: The Model of Setting-Based Mobile Music Creation and Cine-Writing the Everyday: Walking with Agnes demonstrate that CPR research can lead this space and interact with industry developments. Research into immersive media, opens up gateways for the exploration of Creative Arts 4.0 (which Kim Vincs defined at the Smart Storytelling Day) (Schleser 2018b). In addition CPR research can make key contributions beyond the Creative Arts. Subsequently the analysis of multisensory and embodied viewing requires further exploration; Embodied Interaction and Immersive Film Experiences on Smartphones engages with this analysis of smartphone spectatorship. Mobile stories will play a key role in a continuous changing and dynamic media environment.

References Berry, M., & Schleser, M. (2014). Mobile media making in an age of smartphones. Palgrave Macmillan. Schleser, M. (2018a). Creative mobile media II—Making a difference. In M. Schleser & M. Berry (Eds.), Mobile story making in an age of smartphones. Springer International Publishing: Imprint: Palgrave Pivot. Schleser, M. (2018b). Smart storytelling day. https://www.swinburne.edu.au/ events/departments/research/2018/11/mina2018-towards-a-theory--pra ctice-of-smart-storytelling.php and https://youtu.be/4uZ7vIAihtY. Schleser, M., & Berry, M. (2018). Mobile story making in an age of smartphones. Springer International Publishing: Imprint: Palgrave Pivot.

CHAPTER 12

Performative Storytelling: The Model of Setting-Based Mobile Music Creation Martin K. Koszolko

In this chapter, I outline the model of setting-based music creation with the use of handheld mobile equipment. This is a model of performative music making where several elements of the creative process are executed in real time. Based on my own creative practice, as well as participation in multiple communities of users1 and collaborative work with other mobile musicians, I have identified six elements of setting-based music storytelling. I provide analysis of these elements and discuss them in the context of creating mobile music. Various other non-musical aspects of mobile storytelling have been explored in previous academic research (Berry & Schleser, 2014, 2018). In my past research, I have described affordances of iOS music apps (2019) and in this text, I am examining a specific aspect of working with these affordances, which can be empirically observed. I draw on the research by Norman who applied the concept of affordances to the field of human–computer interaction and defined affordances as “a relationship

M. K. Koszolko (B) School of Music Collaboration, Melbourne, Australia

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_12

173

174

M. K. KOSZOLKO

between the properties of an object and the capabilities of the agent that determine just how the object could possibly be used” (2013, p. 11). In this chapter, I am focussing on mobile music creation tools and developments over the last nine years in the battery-powered hardware as well as the ever-growing collection of apps. These tools constitute some of the most significant advances in music production technologies in recent years. These advances are characterised by increased mobility, new creative workflows, changes to music performances tools and the acquisition of knowledge afforded by social networking within communities of practice as well as the use of apps that enhance music theory learning2 (Koszolko, 2019). When I first started making music on an iPad 3 in 2012, it was obvious that the technology, although fairly limited at that time, offered a new perspective on interacting with instruments and generating music. While the iPad was expensive, the apps were sometimes free and if not free, then often quite cheap. At the same time, I discovered a portable music workstation called OP-1 by Teenage Engineering.3 These two devices introduced me to new musical interfaces and new ways of making music, often on the go or, at least, outside the confines of a traditional music studio. Currently, these types of devices still represent benchmarks of battery-powered, mobile music making technologies. Mobile tools that I investigate in this chapter represent portable mini studios that allow musicians to perform, record, compose, sequence, edit and mix sound as well as enhance their compositions with visuals. Mobile music tools provide a unique possibility for implementing advanced production techniques in musical pieces that communicate local settings.

Methodology I draw on two primary research methods: creative practice-led enquiry with related case studies and ethnography. My investigation is aligned with Smith and Dean’s view that creative works can lead to demonstrable research outputs (2009). Practice-led research is focussed on advancing knowledge about and within the practice (Candy, 2006). This type of research investigates artefacts created in the creative process (Candy & Edmonds, 2011) and in my case, the artefacts are represented by music produced with mobile technologies. As noted by Hjorth and Richardson “Mobile media shifts between being the researcher’s and participant’s tool, medium and subject matter” (2018, p. 76), which is evidenced in

12

PERFORMATIVE STORYTELLING: THE MODEL …

175

the way field recordings can be used to connect the performer and the audience as critical agents responsible for the creation of the musical piece in the context of setting-based live sampling. Field recordings and settingbased live sampling are some of the techniques that I discuss throughout this text. I reflect on several years of my own creative practice incorporating various mobile music tools. I refer to examples of my music production methods tested during sessions that presented restrictions concerning equipment, space and time available in the creative process. My ethnographic work involves active participation in large online communities of practice using Facebook and Discord groups (see endnote 1) to discuss making music with multiple types of mobile equipment. In my analysis of mobile music production, I refer to the work of scholars from the field of the art of record production. Moreover, in the discussion on elements of structure and composition, I build upon the research on electronic music by Curtis Roads (2015). While Roads did not specifically discuss mobile technologies, many aspects of his examination of music composition are relevant to composing on mobile devices. Mobile music tools are sometimes used in the creation of nonelectronic styles of music by connecting or sampling electric or acoustic instruments. However, the setting-based music production tends to favour various electronic music styles that often lack vocals. Apart from individual preferences of music makers, this seems to be largely driven by improved portability and smaller footprint of setups that are possible when focussing on mobile electronic music production. I have analysed findings from a year-long project, where I curated and produced a compilation album titled Mobile Strategies: Battery-Powered Sonics (Various Artists, 2019), which showcases key trends in contemporary mobile music. This music release includes 25 compositions from a group of international music creators and showcases mobile technologies used standalone as well as alongside other gear. Methodologies of musicians featured on the Mobile Strategies album4 exemplify the various affordances of mobile music technologies in the field of amateur as well as commercial electronic music production. The album showcases various mobile strategies ranging from a track made on a bus, one made in the mountains to those made in fully equipped studio spaces. Accompanying this music compilation was a zine where each of the featured artists discusses their influences, characteristics

176

M. K. KOSZOLKO

of the production process and the mobile tools they used. Some tracks were recorded live, in one take, and some involved painstaking layering and editing. A second music compilation that showcases the developments in appbased mobile music making is “Then & Now” (Various Artists, 2020). This release presents an overview of over a decade of mobile music making with iOS apps. I have contributed two pieces to this album and have discussed the setting-based production process with several other musicians featured there as we have participated in the same online community of users. The above two compilation albums present an in-depth survey of the contemporary landscape of mobile music making. Examples of mobile music equipment concerning this research include tablets and phones (iOS and Android powered), Nintendo game consoles, Raspberry Pi5 as well as various battery-powered instruments, sequencers and workstations by brands such as Teenage Engineering, Korg and Roland.

The Model of Setting-Based Mobile Music The setting-based model of mobile music creation contains six elements that represent various aspects of musical storytelling with mobile tools (Fig. 12.1).

Fig. 12.1 Elements of setting-based mobile music creation model

12

PERFORMATIVE STORYTELLING: THE MODEL …

177

This model is predominantly performative, which means that many of the elements are performed live in settings chosen by the music creator. When talking about the setting, I consider the location as well as the time when the creative process took place. Mobile music technologies play a vital part in this process because they allow music making within various limitations, such as working with small portable gear in environments not always conducive to music making, such as trains or forests. Working within the elements of this model portrays the artistic influence of the setting on the musical outcomes and the resulting performative storytelling. One of the factors driving the setting-based model is immediacy. As noted by Baker, Molga and Schleser, the aspect of immediacy allows to create real-time mobile media art in private and personal spaces (2009). It allows for a quick response to changing dynamics of the environment that might include space available to the performer, sounds that can be captured live and time available for the creative process. This model includes six elements that I observed in music created by various artists as well as in my own creative process. In Fig. 12.1, the elements coloured in orange represent performative actions executed and captured live. Not all elements are required when creating music in this way, and it is up to the artists to choose which building blocks they want to work with. The stage of pre-production can include both performative, live actions as well as tasks completed ahead of the performance, while post-production is non-performative and executed after the live performance. The performative elements are created live during one session, while non-performative actions can be done at any time and spread across multiple stages. It could be argued that some other technological solutions can allow us to implement any of the six elements described in Fig. 12.1. These might include a combination of portable field recorders and laptop-based Digital Audio Workstations (DAWs) and plugins. However, this approach would mean reducing the mobility and playability afforded by mobile tools considered by this research. If contemplating laptop alternatives, we also need to factor in a different way of interfacing via a trackpad or mouse. Laptop or, more broadly speaking, computer DAW-based mixing and ways of interfacing with the software have been criticised by mixing engineers as being frustrating to use and less hands-on than hardware mixing consoles (Bell et al., 2015).

178

M. K. KOSZOLKO

Pre-Production Pre-production can include a series of initial actions such as choosing the gear and apps to use, connecting various hardware components, field recording and selecting instruments and their various settings. While many of these actions are quite commonly executed in various music production situations, the one that requires more elaboration in the context of performative mobile music storytelling is field recording. This process offers a significant enhancement of storytelling with mobile tools. Field recording practices incorporate various sound production approaches, however, “place is a fundamental concept in pretty much every type of field recording” (Western, 2018). Field recordings can create soundmaps, which can “chart personal and collective, imagined and remembered, and invisible and physical relationships between sound, the world, and ourselves” (Anderson, 2016). Field recordings can capture natural soundscapes that represent the living environment which some composers also include in music (Krause, 2015). While some mobile composers can choose to leave the recordings unprocessed, with signal processing, non-musical field recordings can morph into the building blocks of musical textures. Depending on the recorded material, field recordings can be either performative, created on the spot as part of the live event, or non-performative, created ahead of the live performance. A technique that I frequently incorporate in my live performances with tablets is live sampling of short sounds recorded with the help of the audience. I invite the audience to participate in the creative process by allowing me to record their voices or sounds created through interaction with available objects. After recording these samples on the fly with iPad’s built-in microphone, I use sampling apps that also incorporate sequencing. As stated by Hugill: “the word “sampling” has come to refer to the process of digitizing recorded sound (we may sample an instrument, or the sound of a river, for example)” (2012, p. 94). In electronic music, “sampling technology refers to recording pitches under different techniques” (Mazzola et al., 2020, p. 82) often obtained from pre-recorded tracks. However, in the setting-based approach, the concert location and audience become sound sources. I incorporate various sampling production techniques live, while performing. These include audio editing, granular synthesis, layering and sequencing. I also utilise field recordings outside of the live performance context, an example of a composition featuring recordings created on an iPad during

12

PERFORMATIVE STORYTELLING: THE MODEL …

179

pre-production is my work as KOshowKO titled “Neaty+Tidy” (Various Artists, 2019). In this piece, the samples of a conversation between adults and children at a kitchen table influenced the title of the work and allowed me to create the main sonic motif, which was chopped and processed in the Egoist app. Here, the recording of my settings has allowed me to highlight the personal significance of various sounds.

Instrumental Performance In the world of mobile music interfaces, there is a large number of instruments utilising a keyboard interface, although many app creators go beyond this paradigm. Various app keyboard interfaces can be changed to represent a selected scale or to allow various forms of musical expression through parameters such as velocity or vibrato being determined by sliding the finger across the x–y field. However, the true innovation is visible in instrument interfaces designed specifically for the touch screen environment, which necessitates learning new ways of playing achieved through various gestures (Koszolko, 2019). In setting-based approach, instrumental performance often takes place with a backdrop of other real-time elements, which in Fig. 12.1 are coloured in orange. As mobile musicians are liberated from the confines of traditional studio or performance spaces, instrumental performances take place in a wide range of locations. I have played iOS instruments on trains but also while walking on streets or train platforms (Fig. 12.2). It is a safe practice as long as finger gestures can be performed without having to look at the screen. This is easily achievable with apps such as ThumbJam that allow the performer to play with a pre-defined musical scale and a desired octave span.

Fig. 12.2 Setting-based model in action. Composing and performing during train commutes

180

M. K. KOSZOLKO

Mixing and Effect Processing Live mixing in non-mobile studio settings is an established form of music production. This approach implements the use of signal processing during the tracking process (Moore, 2019), which is comparable to the process of live mixing in a setting-based approach of mobile music making. For example, when working with the AUM—Audio Mixer app on iOS, dynamic effect processing is frequently part of the live mixing process. Examples of used techniques include various types of audio granulation, tempo manipulation, reversing the order of sonic events, distortion and manipulation of time-based effects. These processes can be performed on MIDI and audio data with apps or effects built into pocket-sized hardware mobile workstations and grooveboxes. Even in non-mobile settings, mixing can be seen as a performative process that contributes to the development of music composition (Anthony, 2017). When approaching mixing in that way, mobile musicians using iOS apps often implement algorithmic compositional tools and sequencers built into apps to influence the composition while mixing in real time. AUM—Audio Mixer is possibly one of the most commonly used apps in the settings-based approach. Mobile musicians, such as Perplex On,7 frequently post videos on Facebook and YouTube showcasing mobile compositions that feature this app. The user interface is uncluttered and easy to navigate when implementing the performative approach, which includes tracking and processing as you go. A particular challenge here is that it can be difficult to control parameters of more than two apps due to the limited screen size of phones and tablets. This limitation impacts on the music production and compositional processes and can lead to heavier reliance on generative strategies.

Algorithmic Composition Algorithmic and generative composition is one of the distinguishing aspects of setting-based music making with iOS apps. The workflow of making setting-based music with iPads and iPhones often involves AUM and Audiobus mixing environments, which lack built-in sequencing or note editing functionality. For that reason, users often resolve to implement this functionality via plugin devices available in either AUv3 or IAA app formats. The performative aspect of setting-based music is

12

PERFORMATIVE STORYTELLING: THE MODEL …

181

made possible by delegating a number of sequencing or music generating actions to algorithmic composition tools implemented in various apps. These tools can help with the following elements of the production process: • Assistance with choosing the audio content. This is evidenced by auto-selecting different types of loops, for example, drums, bass, melodic and vocal. • Assistance with generating melody or chord sequences. • Assistance with choosing the song title where the software autogenerates an unlimited number of suggestions. The above features allow music makers to use generative workflows and randomisation. In addition, musicians are able to get assistance with pre-defining the musical scale and adjustment of the interface based on a chosen scale. All of the above elements significantly alter the music production process. In an overview of generative strategies, Roads cites various attractions such as offering novel musical processes and structures, going beyond the capabilities of human composers and being unpredictable (2015). Having used algorithmic composition in an iOS environment, I have seen evidence of all these facets in multiple music pieces. An example of the implementation of the above processes is a KOshowKO track titled “Rich Machine (Birthday Train Ride One Take Phone Mix)” (Various Artists, 2020) which I produced during four short train commutes that took place on the same day. This composition demonstrates that generative strategies can lead to rapid music creation— a process that is faster in comparison to where all decisions need to be trialled and executed manually by the composer (Koszolko, 2020). Using machine assistance in composition has allowed me to complete a musical piece within a limited time frame. I also see this project as an exploration of how to deal with imperfections in such aspects of music making as mixing, live triggering of parts and improvisation. My aim was to work within the time and space constraints related to my train journeys and to create a musical composition that met certain musical genre and tonal balance objectives regardless of the limitations. The time limit imposed on the process meant that editing, part triggering, live performance and audio mixing were all done on the fly in real time, with headphones on a

182

M. K. KOSZOLKO

crowded train. I have embraced the imperfections imposed by this process prioritising expressiveness over a highly nuanced mix. As stated by Omigie in her research into music-induced emotion: what could be considered imperfections (slightly irregular timing in music for instance) are often perceived by listeners as being more ‘expressive’. Empirical research shows expressivity is a major contributor to aesthetic appeal. (Nisbet, 2019)

Imperfections should also be seen as an important contextual element of the recording which I tried to convey in the subtitle added to the title of the track “Birthday Train Ride, One Take Phone Mix”. The mix would have sounded very different if completed on a desktop computer with an established DAW and a suite of plugins. Although the context of creating a song from start to finish on my birthday during a series of short train rides would be very difficult to achieve. This context provides meaning to mobile storytelling and delivers useful material for speaking about the creative process on social media or in press kits associated with a music release. In machine-assisted composition, I do not leave the final say on how to approach the storytelling to the machine. While I choose from some of the musical elements suggested by the machine, I strive to also interject a human element of improvisation and live instrument playing as is demonstrated in “Rich Machine” where I played the trumpet part on the phone while navigating the crowd at a busy train station. The small footprint of the phone allowed me to play the instrument without being burdened by the size of the device and without having to look at it during the performance.

Sequencing Sequencing is one of the key affordances of iOS music apps (Koszolko, 2019). Selected apps allow for implementation of generative sequencing strategies, which become a part of algorithmic composition. Examples of such apps include Rozeta Sequencer Suite and Egoist.8 However, on many occasions, users choose to perform the writing of musical sequences in real time. This can be accomplished with apps as well as standalone mobile devices.

12

PERFORMATIVE STORYTELLING: THE MODEL …

183

Post-Production Post-production becomes a necessary element of the process as soon as the music creator chooses to share or publish their work. At the most basic level, post-production could involve exporting an audio file and uploading it to a selected online music publishing platform. However, it is likely that before the exporting, editing might also be required. For example, even though, I recorded the mix of “Rich Machine” live using just a single stereo track, before publishing it, I decided to trim the original recording by nearly one minute. This was a simple operation which involved removing one section of the original recording and then stitching two remaining pieces of audio together in a DAW app. Some musicians, however, might decide to capture their live mixes on separate audio tracks rather than just to record a stereo mixdown as I did. If individual stems are available, it opens up more advanced mixing opportunities during post-production. A technique frequently employed in the post-production phase is the addition of video content that enhances and re-contextualises musical narratives. These videos are frequently just screen captures of apps or videos showing the settings and manipulation of portable hardware instruments. Some musicians, like Perplex On in “First Warm Sunbeams” (Various Artists, 2019), choose to showcase the location where the music was created, often mixing footage of the used mobile equipment with natural, outdoor settings. The video often becomes a critical part of the mobile music storytelling as it allows creating a visual context for the project, which in turn, also influences how the audience might perceive the narrative.

Narratives and Compositional Strategies The performative nature of the setting-based model means that the chain of sonic events is often non-linear and based on improvisation. In the electronic music realm, live mixing and signal processing contributes to non-linear narratives and impacts on the final composition. Mobile music storytelling confines the soundtrack to a place and time and can be seen as a form of sonic documentary. It can convey a specific emotion from the day of recording. The music can feature a code that the listener can decode and, in the process, look for the meaning intended by the

184

M. K. KOSZOLKO

composer. However, listeners typically construct their own interpretation of the narrative (Harden, 2019; Roads, 2015). A large proportion of setting-based mobile music is being shared within various online communities of practice that exist on social media. Sharing music via video clips allows music producers to promote their output and communicate details of the creative process, technical set up and the location where the music was made. Discussing their process, the artists featured on “Mobile Strategies” album often reflected on the influence of the immediate settings where the recording was made. When discussing his composition “Es Ist Zu Spät” (Various Artists, 2019), artist Globotom said that it “was made while I was doing some homework and tv played a horror movie. As it was foggy outside it was a very frightening atmosphere” (Clan Analogue, 2019). Another artist, BLEO, who produced the piece “Full Featured” using Android-powered phone with PPSSPP and LittleGPTracker apps,9 stated that: “I worked up this particular track over a few days while on the bus, at work, in the car in the grocery store parking lot, etc. The finished product is a render generated directly from the tracker with no additional/external mixing or mastering” (Clan Analogue, 2019). The approach to narrative also depends on the type of software chosen by the composer. Using a mobile DAW tends to help with planning the overarching structure of the composition first and then focus on the finer details, such as sound design or use of generative compositional tools. This approach is similar to what Roads referred to as a top-down strategy of organising the form of a composition (2015). However, in many cases, mobile music composers do not rely on traditional DAWs and instead choose to use mixing software that allows for more flexibility needed in the performative approach. This can involve apps like AUM or mixing and sequencing live with hardware mobile devices such as OP-1 and OP-Z. Performative strategies rely quite substantially on live sequencing, algorithmic composition and live mixing and effect processing. These methods encapsulate many elements of setting-based mobile storytelling and are chosen when no major musical post-production is required as was the case with my piece “Rich Machine”. As a result, the top-down strategy compositional approach is less common and instead, two other methods of composing music tend to guide the setting-based narrative more often. The first one is the bottom-up compositional strategy which emphasises the initial generation of sound materials as a necessary element of forming the structure of the composition (Roads, 2015). In the world of mobile

12

PERFORMATIVE STORYTELLING: THE MODEL …

185

music this material generation is achieved by all elements that I listed in Fig. 12.1, apart from post-production. A hybrid strategy encompassing both top-down and bottom-up approaches is also possible when, for example, a mobile musician would first focus on a bottom-up method and record these musical ideas as longer, individual audio stems that capture live mixing, sequencing and algorithmic composition. Later, these ideas can be re-arranged, amended and expanded during a more extensive postproduction stage that could involve a linear mobile DAW app or the four-track tape functionality of the OP-1 workstation. The musical narrative is often supported by video clips that contextualise the settings where music was created as well as the mobile equipment being used. “First Warm Sunbeams” (Various Artists, 2019) by Perplex On is a good example of this approach as the artist produced a video showcasing the natural settings in which the creative process took place. Speaking of his approach, he remarked: This track originated while hiking through the Bavarian nature in early spring. Experiencing the awakening of nature after a long winter with ice melting from the first warm vernal sunbeams inspired me to capture that feeling in a musical way. Translating the breathing nature and flowing nearby creek into waving calming pads and utilizing semi-random sequencing techniques on the OP-Z provide seemingly accidentally triggered clicks and clacks symbolizing the cracking layers of remaining thin ice around me. OP-Z and atmos were recorded on location with a Zoom H2 recorder. (Clan Analogue, 2019)

Another way a composer can communicate the setting and enhance the narrative is to include the location in the title of the work. An example of this approach is my composition titled “The Cicadas of Agnes Falls” from the forthcoming EP “Mobile Positions—Part 1” by KOshowKO. For this piece, I conducted mobile phone field recordings in Agnes Falls Scenic Reserve in the traditional Country of the Gunaikurnai Peoples in Australia. The iOS app that I used to capture the location audio, trim and catalogue it was AudioShare, which works seamlessly with AUM as both apps are made by the same developer.10 As demonstrated by researchers of mobile media using visual art, the place and locality play an important role in communicating stories (Kilby & Berry, 2018). While musical storytelling can use some techniques familiar to mobile videographers, such as capturing media on location and multimodality, it presents its own set of approaches reflected in the setting-based model.

186

M. K. KOSZOLKO

Challenges and Advantages The challenges of the setting-based model include monitoring, compositional, technical and time constraints. • Making music on location, outside of the studio, makes mixing and sound processing challenging and requires relying on intuition more than on neutral monitoring. Portable playback systems are often imperfect and are impacted by the environmental noise. • Making music rapidly within a pre-set time limit means less time for reflecting on the composition. When production time is constrained, it might be tempting to rely more heavily on looped patterns, factory instrument and effect presets and little parameter automation, which can lead to stale, predictable performances that lack movement. • While opting for portability and immediacy, it is advantageous to record location sounds with the use of microphones built into mobile equipment. However, this can mean sacrificing the recording quality as these microphones cannot match more capable standalone counterparts. • Composing with mobile tools often leads to dealing with various idiosyncrasies and limitations of the platform. It can also mean using less-flexible workflows and encountering limitations in regard to available sound processing and mixing in comparison to using established computer DAWs. Despite these challenges, there are also several advantages exemplified by the setting model, which include: • Being musically creative in spaces not designed for music making, for example, during commutes on public transport or while walking. • Being able to record location sounds almost immediately. • A sense of creative accomplishment if a full idea is formulated, recorded and mixed on the same day. Being able to execute creative ideas much more rapidly and use algorithmic composition tools. • Being able to enhance the narrative by using the music making tool, such as a phone or tablet to also film the location of the performance, use screen capture or even edit sophisticated music videos.

12

PERFORMATIVE STORYTELLING: THE MODEL …

187

Reflections and Conclusion The results of this research indicate that performative elements of mobile music production allow for immediacy and flexibility of working in various types of locations, which permits the musician to document the time and space (the setting) of their creative process. Sometimes, the production sacrifices the nuances of editing and mixing that are typically part of the studio-based music making process. Mobile tools allow for quick capturing of field recordings of various environments during the preproduction phase. This encourages creativity and exploration of sites meaningful to the musician rather than reliance on samples created by other people. Working with the model depicted in Fig. 12.1 allows to add new layers of meaning to the creative process. Live sampling in concert settings allows often accidental collaborators to gain agency as active creative participants. It adds meaning to the recording by allowing the music maker to gather unique sonic material, which often has personal significance to the audience as well as the performer. The setting often signifies time and place that the music creator can associate with personal memories. Moreover, some creators choose to translate these memories into a universal experience by adding visuals that can communicate the setting more broadly to their audiences. As the listeners construct the narrative “in response to available information within a track” (Harden, 2019), the video being a source of a substantial amount of the additional, visual information, plays a significant role in shaping this perception. The process of audio storytelling has been significantly altered by affordances of mobile tools. Mobile music often relies heavily on the technological context and the contextual information feeds into how music can be communicated to various audiences. Innovative and portable music making processes allow us to leverage the latest technological developments and enhance mobile audio storytelling. Mobile affordances expand opportunities available to music producers and enable music creation at a rapid pace. iOS apps, in particular, offer music composers a range of options enabling automation and streamlining of various compositional and arranging tasks. Contributors to the “Mobile Strategies” album often said that their process of music making was quite rapid and several artists used live recordings and minimal sequencing and post-production regardless of the used instruments.

188

M. K. KOSZOLKO

As I learned through the process of producing the “Mobile Strategies” album, musicians sometimes choose to work within the confines of a larger ecosystem of gear and add mobile tools to a non-mobile studio set up. On the opposite side of the spectrum, there are minimalistic set ups that utilise just a single mobile device and a pair of headphones. The former was evidenced in some of the material submitted to the “Mobile Strategies” album, while the latter has been documented by me in the process of creating the track titled “Rich Machine” on trains. While not without its challenges, the setting-based model presents the musician with multiple advantages that stem from such aspects of mobile music making as small footprint of instruments, portability, multifunctionality and immediacy. When assessing mobile compositional and production processes in comparison to utilising larger, desktop-based systems, it becomes evident that mobile technologies afford a different creative method which often leads to sonic outcomes that also differ from desktop-based music making. Performative and mobile music storytelling with the use of setting-based approach is frequently more rapid in comparison to using non-mobile tools and provides a way to not only talk about the location where the creative process took place but also to make this location a central element of the musical composition.

Notes 1. Examples of online communities include iPad Musician on Facebook: https://www.facebook.com/groups/Ipadmusician, iOS Music Production on Discord: https://discord.gg/NCksebU and Audiobus web forum: https://forum.audiob.us. An example of face-to-face community as well as online community is Clan Analogue collective in Australia: https://www.clananalogue.org. 2. Examples of such apps include Tonality: https://www.tonalityapp.com and EarMaster—Music Theory: https://www.earmaster. com. 3. https://teenage.engineering/products/op-1. 4. https://www.clananalogue.org/featured/ca053-mobile-strate gies-various-artists. 5. Raspberry Pi is a portable, USB-powered single-board computer developed by Raspberry Pi Foundation with the aim of fostering computing education. 6. https://kymatica.com/apps/aum.

12

PERFORMATIVE STORYTELLING: THE MODEL …

189

7. https://perplexon.com. 8. Rozeta Sequencer Suite: https://ruismaker.com, Egoist: https:// sugar-bytes.de/egoist-ipad. 9. LittleGPTracker is a multiplatform music tracker app: https:// www.littlegptracker.com. PPSSPP is an open source, multiplatform app emulating the PlayStation Portable console: http://www.ppsspp.org/downloads. html. 10. AudioShare and AUM—Audio Mixer apps are created by Kymatica AB: https://kymatica.com.

References Anderson, I. (2016). Soundmapping beyond the grid: Alternative cartographies of sound. Journal of Sonic Studies, 11. https://www.researchcatalogue.net/ view/234645/234646/0/0. Accessed 24 September 2020. Anthony, B. (2017). Mixing as a performance: Creative approaches to the popular music mix process. Journal on the Art of Record Production, 11. https://www.arpjournal.com/asarpwp/mixing-as-a-performance-creativeapproaches-to-the-popular-music-mix-process/. Accessed 24 September 2020. Baker, C., Molga, K., & Schleser, M. (2009). Aesthetics of mobile media art. Journal of Media Practice, 10(2–3), 101–122. Bell, A., Hein, E., & Ratcliffe, J. (2015) Beyond skeuomorphism: The evolution of music production software user interface metaphors. Journal on the Art of Record Production, 9. https://www.arpjournal.com/asarpwp/beyond-ske uomorphism-the-evolution-of-music-production-software-user-interface-met aphors-2/. Accessed 24 September 2020. Berry, M., & Schleser, M. (Eds.). (2014). Mobile media making in an age of smartphones. Palgrave Macmillan. Berry, M., & Schleser, M. (Eds.). (2018). Mobile story making in an age of smartphones. Palgrave Pivot. Candy, L. (2006). Practice based research, a guide, CCS report: 2006-V1.0 November. University of Technology Sydney. https://www.creativityan dcognition.com/wp-content/uploads/2011/04/PBR-Guide-1.1-2006.pdf. Accessed 14 April 2021. Candy, L., & Edmonds, E. (2011). The role of the artefact and frameworks for practice-based research. In M. Biggs & H. Karlsson (Eds.), The Routledge companion to research in the arts (pp. 120–138). Routledge. Clan Analogue. (2019). Mobile strategies zine. Clan Analogue.

190

M. K. KOSZOLKO

Harden, A. C. (2019). Narrative and the art of record production. In R. Hepworth-Sawyer, J. Hodgson, & M. Marrington (Eds.), Producing music (pp. 263–278). Routledge. Hjorth, L., & Richardson, I. (2018). Stories from the field: Playing with mobile media. In M. Schleser & M. Berry (Eds.), Mobile story making in an age of smartphones (pp. 75–85). Palgrave Pivot. Hugill, A. (2012). The digital musician (2nd ed.). Routledge. Kilby, J., & Berry, M. (2018). Wayfaring, creating and performing with smartphones. In M. Schleser & M. Berry (Eds.), Mobile story making in an age of smartphones (pp. 51–61). Palgrave Pivot. Koszolko, M. K. (2019). The tactile evolution—Electronic music production and affordances of iOS Apps. In J.-O. Gullö (Ed.), Proceedings of the 12th art of record production conference mono: Stereo: Multi (pp. 187–204). Royal College of Music (KMH) & Art of Record Production. Koszolko, M. K. (2020). Rapid music: Machine assisted composition and portability in mobile music making. http://philosophyofsound.info/blog/arc hives/1048. Accessed 24 September 2020. Krause, B. (2015). Voices of the wild. Animal songs, human din, and the call to save natural soundscapes. Yale University Press. Mazzola, G., Noer, J., Pang, Y., Yao, S., Afrisando, J., Rochester, C., & Neace, W. (2020). New concepts of musical instruments. The future of music. Springer. Moore, A. (2019). Tracking with processing and coloring as you go. In R. Hepworth-Sawyer, J. Hodgson, & M. Marrington (Eds.), Producing music (pp. 209–226). Routledge. Nisbet, G. (2019). Why imperfection is the secret to making better music. https://www.musicradar.com/news/why-imperfection-is-the-secret-to-mak ing-better-music. Accessed 24 September 2020. Norman, D. (2013). The design of everyday things: Revised and expanded. Basic Books. Roads, C. (2015). Composing electronic music. Oxford University Press. Smith, H., & Roger, T. D. (2009). Practice-led research, research-led practice in the creative arts. Edinburgh University Press. Various Artists. (2019). Mobile strategies: Battery powered sonics. Clan Analogue. https://www.clananalogue.org/featured/ca053-mobile-strategiesvarious-artists. Accessed 15 July 2021. Various Artists. (2020). Then & now. Apptronica. https://apptronica.bandcamp. com/album/then-now. Accessed 15 July 2021. Western, T. (2018). Field recording and the production of place. In S. Bennett & E. Bates (Eds.), Critical approaches to the production of music and sound (pp. 23–40). Bloomsbury.

CHAPTER 13

Cine-Writing the Everyday: Walking with Agnes and Zelda Liz Burke

In an interview from 2017, Agnes Varda talked about her filmmaking practice as a form of cinécriture, or writing with the screen. She stated: For me, a film is not written by the screenplay or the dialogue, it’s written by the way of the filming. The choices that you have to make between still shot or traveling shot, color or black-and-white, speedy way of acting or slow-motion or whatever, all these choices, and the lens you choose, and the camera you choose, and then the editing, and then the music or not, and the mixing—all these choices all the way through the film, all through the making of the film, that’s what cine-writing is. (Sharf, 2017)

When considering my mobilephone filmmaking practice, I have found this quote crucial in considering how I use my device as a way of writing with the screen, and how that concept ties to the notion of being a woman with a mobile cameraphone.

L. Burke (B) Swinburne University of Technology, Melbourne, Australia

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_13

191

192

L. BURKE

I have found her exemplary essay film, “The Gleaners and I” (2000), a useful jumping off point for considering these matters. In this film, Varda travels through France recording the activities of “gleaners”; people who follow after the rural harvests picking up the leftover fruit and vegetables. This gleaning functions as a practical activity; supplying food for those who don’t have enough to eat, and a political activity; a refusal to waste edible food in a disposable consumer culture. One scene, in particular, illustrates the playfulness and digressiveness of her film, and introduces one of her key production devices; a small prosumer camera. Varda is standing in front of a famous painting; “The Gleaners”, by Jean-Francois Millet (1857). She is carrying a sheaf of wheat in imitation of it, and then drops the sheaf and picks up her digital camera, stating in voiceover, “I’m happy to drop my ears of wheat and pick up my camera. These new small cameras, they are digital, fantastic. Their effects are stroboscopic, (as we see an image of her break up into pixels), narcissistic and even hyper-realistic”. This gesture links the act of gleaning food to Varda’s filmmaking practice. She has stated in an interview; I felt that although I am not a gleaner—I am not poor, I have enough to eat—there’s another kind of gleaning, which is artistic gleaning. You pick ideas, you pick images, you pick emotions from other people, and then you make it into a film. (Anderson, 2001, p. 24)

What is valuable for me, as a researcher into the affordances of the mobile cameraphone as a tool for filmmaking, when I ponder Varda’s film, is to consider what might have been if Varda had been able to shoot it with a mobile cameraphone; a device that didn’t have sufficient pixel power to be used as a conventional filmmaking tool in 2000. By “affordance”, I mean the relationship between the mobile cameraphone, and its user in order to ask; what are the specific properties that enable this object to be a useful filmmaking tool? (Norman, p. 9). However, Varda did film parts of “The Gleaners and I” with what was considered to be a prosumer video camera, and found it a liberating experience. She was very conscious that her use of a prosumer (what she calls an amateur camera) changed her practice when making “The Gleaners and I”. She stated:

13

CINE-WRITING THE EVERYDAY: WALKING WITH AGNES ...

193

The third reason—which pushed me to begin and continue this film—was the discovery of the digital camera. I picked the more sophisticated of the amateur models [the Sony DV CAMDSR 300]. I had the feeling that this was the camera that would bring me back to the early short films I had made in 1957 and 1958. I felt free at that time. With the new digital camera, I felt I could film myself, get involved as a filmmaker. It ended up that I did film myself more. (Anderson, 2001, p. 24)

Timothy Corrigan describes Varda’s camera as “a small digital camerastylo that intensifies the subjective fragments of this contemporary woman with digital recordings of a fleeting world” (Corrigan, 2011, p. 219). Of course, the word camera-stylo quite consciously recalls the idea of Alexandre Astruc’s camera-stylo, with its utopian promise of a tool that makes filmmaking as personal and “flexible and subtle” as writing. Astruc wrote “The Birth of the New Avant-Garde: La Camera-Stylo” in 1948. He writes in relation to this form of cinema: After having been successively a fairground attraction, an amusement analogous to boulevard theatre, or a means of preserving the images of an era, it is gradually becoming a language. By language, I mean a form in which and by which an artist can express his thoughts, however abstract they may be, or translate his obsessions exactly as he does in the contemporary essay or novel.

Astruc’s writing is a useful way for me to think about what the “linguistic and discursive registers” of the mobile phone-based film are. Filming with a mobile phone can be seen as an innovation within a history that stretches from the development of 16 mm cameras, to the video portapak of the 1970s and 1980s, the lightweight prosumer cameras developed in the 1990s, to the development of the mobile cameraphone of the twentyfirst century. I’m not proposing this lineage to suggest that this history is building, and travelling in a straight line, with one technology inherently superior to the previous. However, I think the mobilephone as a camera can be placed within this history. Agnes Varda may well have ended up filming, at least some of the time, with her mobile phone to take advantage of its lightness and flexibility. However, where I think the technology of mobile phone cameras differs from 16 mm, video portapaks, or lightweight prosumer cameras, is in their ubiquity and smallness; the fact that they live in your pocket or your handbag and that they are such a common item woven into the daily

194

L. BURKE

lives of so many people. This makes this technology qualitatively different than that of preceding ones. This ubiquity and ease of use are qualities commented on in multiple articles, by theorists of the mobile phone, and one that animates my use of my mobile phone camera. Leo Berkeley looks to this ubiquity and pervasiveness to argue that the development of the mobile cameraphone “has opened a range of new creative possibilities for documentary filmmaking, taking advantage of these mobile devices’ extreme portability to move closer to Astruc’s dream of the ‘camera-stylo” (Berkeley, p. 25). Berkeley uses his Iphone4S to document “what could be described as the ordinary rather than the extraordinary” (Berkeley, p. 26). In his film “The 57” he focuses on the daily life of work and travel as he documents his commute on the number 57 tram. Indeed, Berkeley also argues that shooting with a mobile cameraphone is part of a technological progression, and uses “Sherman’s March” (Ross McElwee, 1986) which was shot on 16 mm, and the “The Gleaners and I” as historical precursors to his practice. Berkeley describes the ubiquity of the cameraphone as its “hyper-accessibility” and notes that that is a key affordance of the technology. Where I differ with Berkeley’s analysis is that he states that the cameraphone isn’t a radical break with existing screen production practice, but rather a significant development with strong historical precedents. There are continuities with the history of increasingly portable screen production technology and the aesthetic movements aligned to it since at least the mid 20th century. (Berkeley, pp. 28–29)

However, I argue that although there are historical continuities with previous technologies, this “hyper-accessibility” represents a significant break with past production practices because at no other time has the means of production been available so widely to professional and amateur filmmakers alike. It’s the only moving image camera that we can carry easily in our pocket, or our handbag. It also differs from previous moving image technology in that the content can be immediately uploaded and shared, thus interweaving the recording of everyday life into the way we live and share our lives in the digital realm. Bettina Frankham has taken Varda’s idea of cinécriture and applied it to her practice of filmmaking with a mobile cameraphone. She uses a direct

13

CINE-WRITING THE EVERYDAY: WALKING WITH AGNES ...

195

translation of the word cinècriture to mean screen writing, in the literal sense of “writing with the screen”. She states: There is no doubt that this practice of writing with the screen has been greatly assisted by the presence of a smartphone in my pocket. The phone is my notebook, my mood board, the memory folder where I gather video, sound, still images and jottings of ideas, all to be worked further at a later date. The fact that it is mostly always at hand, that its ubiquity renders it relatively inconspicuous, permits me to capture fragments from daily life that are worthy of further consideration. I use it to capture moments of value, exceptionality, whimsy and things that quicken my heart. (Frankham, 2016, p. 50)

Over several years I have filmed my two dogs, Agnes and Zelda, using a variety of cameraphones, as part of a meandering, quotidian filmmaking practice. (And yes, Agnes was named after Madame Varda). In the spirit of Varda’s work, I glean my footage, using these cameras, in a casual, haphazard way to build a diary of my life with my dogs, which is invariably intertwined with my other quotidian activities. But, I use different technological means to record my life than Varda did. Means, however, that I think she would approve of, and which I could imagine her using in the future, if she were still alive. I filmed my dogs as part of my Ph.D. research. What I chose to film, at least at first, was driven by curiosity, and a sense of play to see what would happen if I filmed a certain event, or in a certain location. My first dog Agnes was elderly at the time, our life together was very quiet and routine. I filmed daily outings at a local café, and weekly outings at a local beach. I filmed her sleeping in my apartment, and at times wandering disconsolately around at sunset, when she tended towards confusion. I filmed us trotting around my local suburban streets. At Christmas, I filmed us visiting my family in Warrnambool, a small coastal country town. I rarely spoke any direct address to camera in this diary-based filming, although the camera phone recorded any conversation I happened to be having at the time. This kind of casual audio recording is an example of the way filming with a camera phone, and creating an “evocative documentary”, as formulated by Dean Keep, is different than a conventional professional setup. Keep argues that;

196

L. BURKE

Making digital media with smartphones may be understood as a spontaneous process, whereby individuals are engaged in the act of capturing fragments of lived experiences. With this in mind I embrace the term ‘evocative-documentary’ as a way of describing non-fiction narratives that do not easily fit within the parameters of the documentary genre but involve aspects of auto- ethnographic practices. (Keep, 2015)

During this filming, the mobile cameraphone was a handheld apparatus; something which raised several production issues. I noticed that no matter how steadily I held it, it always moved, subtlety and gently, in rhythm to my body’s breathing. This feeling of being connected directly to my camera isn’t one that I’ve experienced while working with other types of film and video cameras. Obviously, this feeling is in part due to its lightness. But, I think it’s also due to the ubiquity of cameraphones; the way in which we carry them everywhere in almost an unthinking way, and as an extension of our bodies. Kossoff has described the cameraphones as a “miniature machine at the end of an arm” (Kossoff, p. 37), and it, indeed feels as though the cameraphone functions as an extension of the body, that creates an embodied mode of filming. The camera phone gently rises and falls in time with my breath, and in this way, I feel a strong connection between my body’s breath and the bodies of the dogs that I’m filming. Because of Agnes’ age when I started filming, our walks together were gentle, meandering affairs. This changed after she died, and I acquired a new pup, Zelda. When Zelda became old enough to safely walk outside, we spent time at the local dog park, and being an extroverted pup, she quickly gloried in running and playing with the other dogs. This meant that what I filmed, and how I filmed, changed. Zelda’s play tended to be wild and chaotic, so I often tried to film this by keeping the camera low at her level. In thinking about my diary-based work, a useful concept for me is that of the “digital wayfarer”. Sarah Pink and Larissa Hjorth write about this figure as a person who interweaves their physical and digital lives together via their mobile phone, as they travel through their day. They use this following definition by Tim Ingold, who defines “wayfaring”, as their starting place:

13

CINE-WRITING THE EVERYDAY: WALKING WITH AGNES ...

197

… The path of the wayfarer wends hither and thither, and may even pause here and there before moving on. But it has no beginning or end. While in the trail the wayfarer is always somewhere, yet every “somewhere” is on the way to somewhere else. The inhabited world is a reticulated meshwork of such trails which is continually being woven as life goes on along them. (p. 45)

Pink and Hjorth quote one of their research subjects, Michelle, who says in relation to her daily practice of filming and sharing: “I used to keep a diary, but now I keep my diary in public, and in a cloud” (p. 49). Michelle’s statement resonates with Frankham’s idea of writing with the screen, in which your mobile cameraphone is notebook, mood board and memory book; using it to gather sounds and images, which she will work with at a later time. In thinking about how I work with the images gathered by my mobile cameraphone, I realise that my filmmaking practice has changed during my research from my habitual practice, in that I now use the images I’ve gathered as a way of musing over my daily life, and this then feeds into decisions I make about what and when to film. Another way for me to conceptualise this process is to return to Varda’s notion of “gleaning”; an appropriate image to use as I wayfare my way through my day, gleaning images with my mobile cameraphone that I find intriguing, affecting or compelling. Marsha Berry argues that mobile filmmaking can be placed in the confluence of “three concepts drawn from digital ethnography, namely wayfaring, co-presence and mobility placed within mobile media ecologies to see how emerging everyday creative practices and evolving aesthetics manifest” (Berry, 2017, p. 132). Pink and Hjorth have supplied us with a rich explication of wayfaring. Co-presence suggests the contemporary way in which we move through the physical world with the virtual world always co-present with us at the same time, as we film, upload and share, while mobility suggests our personal physical mobility, and the mobile pervasiveness of our cameraphones. These three concepts animate my mobile filmmaking practice, and are central to how this practice is linked to notions of everyday and Keep’s idea of the “evocative documentary”. Berry writes that: an evocative moment as being something that can occur in the midst of the flow of everyday life. The etymology of the word evocative indicates that it is from the Latin evocare, meaning to call forth, and evocativus, which pertains to summoning. (Berry, 2016)

198

L. BURKE

This everydayness, in which evocative moments can be experienced and recorded via an embodied mobile device, is a key affordance of filming with the mobile cameraphone, and enables its practice to dovetail neatly with the notion of the “evocative documentary”, which focuses on the poetry of the everyday fragments of life. Reading Pink and Hjorth made me think about how I walk my dogs, and how that walking may function as a form of digital wayfaring. There are two different ways in which I walk my dogs. One is when I have a destination; perhaps a trip to the shops for which I bring a dog along with me. These walks are purposeful, and don’t fit in with the idea of wayfaring as a meander through daily life. The second, and more common way, is when I walk my dogs primarily for their pleasure. In these situations, they wander at their own pace sniffing, and greeting other dogs and people. We pause as I chat to acquaintances, and then continue on our rather haphazard way. This haphazardness creates the idea of “desire lines”; those pathways dictated by where they want to go rather than where I direct them. This resonates well with Ingold’s description of wayfaring in which I am in a present “somewhere”, while always heading “somewhere else”. Pink and Hjorth look at what this practice of wayfaring means in a world in which the quotidian is inflected by the way mobile phones are used to weave together the physical and the digital worlds. They note that; “camera phone practices provide new ways of mapping place beyond just the geographic: they partake in adding social, emotional, psychological, and aesthetic dimensions to a sense of place” (p. 42). The figure of the digital wayfarer may also be seen as a form of the flâneur, or in the case of women, a flâneuse. I can explore this figure by asking the question: what does it mean to be a woman, a flâneuse if you will, walking her dog in the post-modern city? In order to answer this question, I will briefly explore the concept of the flâneur as theorised by Walter Benjamin, and then see how I can use a gendered form of that figure to explore how I walk the city streets, and arcades, with my dogs. I am aware that scholarship in relationship to Benjamin’s work is deep and complex. I will not be engaging with all aspects of this scholarship. Instead, I will be working through what is useful and relevant to my research. The flâneur is an exemplary figure of modern life; a figure that is usually male and associated with the streets and arcades of the modern city. Bobby Seal writes that:

13

CINE-WRITING THE EVERYDAY: WALKING WITH AGNES ...

199

The concept of the flâneur, the casual wanderer, observer and reporter of street-life in the modern city, was first explored, at length, in the writings of Baudelaire. Baudelaire’s flâneur, an aesthete and dandy, wandered the streets and arcades of nineteenth-century Paris looking and listening to the kaleidoscopic manifestations of the life of the modern city. (Seal, 2013)

Benjamin used Baudelaire’s figure as a way of writing about the impact of modernity on the individual as they wend their way through the city of busy streets and arcades. Seale notes that in The Arcades Project Benjamin wrote about two different ways in which we respond to the complexity, and cacophony of city life. These are; Erlebnis which can be “characterised as the shock-induced anaesthesia brought about by the overwhelming sensory bombardment of life in a modern city, somewhat akin to the alienated subjectivity experienced by a worker bound to his regime of labour”, and an almost polar response of Erfahrung, which “refers to the mobility, wandering or cruising of the flâneur; the unmediated experience of the wealth of sights, sounds and smells the city has to offer” (Seal, 2013). These two states can be seen as a dialectic, in which our daily lives synthesise these two states into our lived experience. As I wander the streets with my dogs, my camera can potentially capture, and respond to, both these states. However, due to the nature of my digital wayfaring, the concept of Erfahrung relates more closely to my research process. Walking with dogs makes the experience of flâneusing markedly different than if I was walking alone. There is something different about travelling through your day with a member of another species. Their sensory experience of the city is different than our human one. It’s marked by a much greater capacity to smell and hear the city than we have, and a lesser capacity to see it, because their eyesight is much weaker than ours. When I walk in the city with my dogs I note their different, and sometimes disgusting responses to their sensory overload; their overwhelming desire to sniff, and then to urinate, their sensitivity to sound, and the way they may be confused and overwhelmed with new sights. For instance, a person riding a skateboard maybe a shocking sight to a dog, if they are unfamiliar with this oddly shaped, and noisy apparition. In order to experiment and research the figure of a flâneuse walking with her dog, I spent a morning walking around Melbourne’s CBD with Zelda, just before Christmas 2019. Melbourne seems an appropriate city to become a flâneuse in, given its proliferation of arcades and laneways. We caught a train to Flinders St. Station and walked through laneways

200

L. BURKE

such as Degraves Street, and nineteenth-century arcades such as the Block Arcade and Royal Arcade. Because it was just before Christmas, the city was particularly crowded with shoppers. I filmed Zelda by holding my mobile cameraphone in my hand. I was tempted to purchase a selfie- stick, but I note how using that device makes the user so conspicuous and thus can often impede the filming process. Therefore, I decided I preferred to film in a more lowkey manner. This means the camera is usually pointed downwards towards Zelda which means I captured her constant smelling of the ground, and made me even more aware of how a dog comprehends the world via their nose, in a way we don’t. It also meant that much of the footage also captured the beautiful tile and mosaic work on the floors of the arcades, rather than film the people and the shops in the way one usually would. This results in an unusual perspective on the world, which isn’t exactly from a dog’s point of view, but gives the feel of what dog’s perspective might be, albeit filtered through a human’s sensibility. One of the main differences between the way a person usually walks around the city, and how one walks around with a dog is that we were constantly stopped by people wanting to pat Zelda. Luckily, she’s a very calm dog and wasn’t worried by this, or by the noise and bustle of Christmas shopping at all. However, it means that my experience was different because Zelda’s presence meant I was constantly interacting with people in a way that doesn’t normally happen. Being a flâneuse with a dog creates many more conversations with strangers than would otherwise occur. To return to Benjamin’s figure of the flâneur; this is a modernist, male figure, and his writing universalises this figure by not considering how different it maybe for a woman to walk through the city. The male figure is the default one here. As Janet Wolff points out: “The literature of modernity describes the experience of men” (Wolff, p. 3). However, there is another theoretical, and feminist history available for me to use to describe my experience. Anke Gleber uses the essay film “Berlin, Symphony of the City” (Ruttman, 1927) to explore the concept of the flâneuse. She calls this flâneusing, “the art of taking a walk” (Gleber, 1997, p. 68). This analysis is of particular interest to me because it joins the figure of the flâneuse to the essay film. Using “Berlin, Symphony of the City” as a context situates the flâneuse in Weimar Germany, at a time of great political and social change, and which represented both a “formulation of a new feminist consciousness”

13

CINE-WRITING THE EVERYDAY: WALKING WITH AGNES ...

201

and resistance to that same consciousness (Gleber, 1997, p. 75). Gleber notes that even though the position of women was changing at that time: The street does not “belong” to women. They cannot take possession of walking it freely without also expecting to be impeded by public judgments or conventions that cover or prescribe their images, effectively rendering them objects of the gaze. The female flâneur is considered to be absent, “invisible”; she is not presumed to have a presence in the street. (Gleber, 1997, p. 72)

She contrasts this with how the male flâneur engages with the modern city: In their intense pursuit of subjectivity and perception, these flaneurs and their gazes are restricted neither by insecurity, convention, modesty, anxiety or assault nor by barriers erected through the controlling or commodifying presence of another. (Gleber, 1997, p. 68)

Although, we are now living in the twenty-first century, arguably the same, or similar, pressures apply to women as we walk along the streets, especially at night. A 2017 survey used interactive mapping to show which public transport stops women in Melbourne felt unsafe using at night (Davey, 2017). Women still struggle to have a safe presence in the street, and to belong fully in that public space. Gleber describes a scene in “Berlin, Symphony of a City” in which she argues that the flâneuse appears on celluloid for the first time. In this scene, a woman walks through the streets of Berlin and gazes at a man through a shop window (Gleber, 1997, p. 76) Gleber argues that; “[h]er positioning in public and assumption of an active gaze provides a critical turning point for the urban woman as spectator, as her behaviour can be regarded as one of the first visually recorded manifestations of the long-absent female flaneur” (Gleber, 1997, p. 76). This is a thrilling development in both the history of the essay film and of the flâneuse. But Virginia Woolf, similarly to Benjamin another great modernist artist, also wrote an essay about her experience of being a flâneuse in London, at almost the same time as the first flâneuse appeared in Ruttman’s essay film. In Street Haunting, Woolf writes about the female pleasures of walking through the streets of London;

202

L. BURKE

As we step out of the house on a fine evening between four and six, we shed the self our friends know us by and become part of that vast republican army of anonymous trampers, whose society is so agreeable after the solitude of one’s own room. (Woolf, 1930)

This gendered figure tramping the London streets is a different one than Benjamin’s modernist hero. She isn’t the cool observer who has mastery over the scene, but instead, is someone who is part of a “vast republican army” subsumed into the flow of the street and all the delights and horrors it has to offer. The flâneuse is a very different figure than her male counterpart. She becomes “an enormous eye”. This echoes Vertov’s invocation of the “kino-glaz”, the “cinema-eye” and speaks to me as a way in which I walk through the city, observing and filming as I go. I also note that in this essay, Woolf is walking through the London streets to purchase a lead pencil with which to write. This inciting detail has a rich resonance with the idea of the camera-stylo for me, an idea which in part animates my work. Keep has adapted the neologism “phoneur” to conceptualise the figure of the flâneur/flâneuse with a mobile phone (Keep, 2015, p. 2). This figure is adapted from a concept written about by Robert Luke. Luke describes the “phoneur” as a subject who has been seduced into using their mobile phones constantly, and as such is akin to people wearing a “kind of remote-controlled animal collar—like the ones scientists use to monitor the behaviour of the animals they study” (p. 185). This is obviously a negative view of who a “phoneur” might be. Although, in light of my research around walking and filming with my dogs, I find it quite an amusing one. However, Keep writes about this figure in a much richer, and more positive way. Using his concept, the phoneur is akin to the digital wayfarer. He is a figure who wayfares and films; “as we drift through private and public spaces using smartphones to capture the mediated traces of the people, places and events that shape our day-to-day lives”. (Keep, 2015, p. 2). The phoneur is conceptually a useful figure for my research, but Jess Kilby and Marsha Berry argue that the figure of the digital wayfarer is different than that of the flâneur. “We don’t follow the lineage of the modernist notion of a flaneur (Benjamin). Instead we seek less travelled paths; we are digital wayfarers (Hjorth & Pink, 2014), whose online and physical worlds are entangled” (Kilby & Berry, 2020). Kilby and

13

CINE-WRITING THE EVERYDAY: WALKING WITH AGNES ...

203

Berry seem to be invoking a figure whose way of wending through the world is less about purposeful striding through the city, than that of the flâneur, and more about loosely drifting along desire lines, and discovering meaning through the process, of filming of whatever strikes them as meaningful, compelling or intriguing. In this way, I relate their figure of the digital wayfarer back to Woolf’s figure of the woman going to buy a lead pencil, and observing her world with an “enormous eye”. Just as I wonder what sort of work Agnes Varda would have made with a mobile cameraphone, I imagine what Virginia Woolf might have filmed with one, if the technology had existed in her time.

References Anderson, M. (2001). The modest gesture of the filmmaker: An interview with Agnes Varda. Cineaste, 4(Fall), 24–27. Astruc, A. (1948). The birth of a new Avant-Garde: La Camera-Stylo. Retrieved from http://www.newwavefilm.com/about/camera-stylo-astruc.shtml Berkeley, L. (2014). Tram travels: Smartphone video production and the essay film. In M. Berry & M. Schleser (Eds.), Mobile media making in an age of smartphones (pp. 25–34). Palgrave Macmillan. Berry, M. (2017). ‘Making films and video art with smartphones’ in creating with mobile media. Palgrave Macmillan. Berry, M. (2016). Evocative moments with smartphone cameras. Proceedings of 2016 ASPERA Conference. Retrieved from http://www.aspera.org.au/res earch/evocative-moments-with-smartphone-cameras/ Corrigan, T. (2011). Of the history of the essay film: Vertov to Varda. In N. M. Alter & T. Corrigan (Eds.), Essays on the essay film (pp. 197–226). Columbia University Press. Davey, M. (2017, August 10). Melbourne survey reveals the public transport stops women least safe. The Guardian. Retrieved from https://www.thegua rdian.com/australia-news/2017/aug/10/melbourne-survey-reveals-publictransport-stops-women-find-least-safe Frankham, B. (2016). Writing with the small, smart screen: Mobile phones, automated editing and holding on to creative agency. Journal of Writing in Creative Practice, 9(1 & 2), 47–66. Gleber, A. (1997). ‘Female flânerie and the symphony of the city’ in women in the metropolis: Gender and modernity in Weimar culture. University of California Press. Hjorth, L., & Pink, S. (2014). New visualities and the digital wayfarer: Reconceptualizing cameraphone practices in an age of locative media. Mobile Media and Communications, 2(1), 40–57.

204

L. BURKE

Keep, D. (2015). New Media/New Films: Smartphones and evocative documentary practices. The Asian Conference on Film & Documentary 2015 Official Conference Proceedings. Kobe. Kilby, J., & Berry, M. (2020). Wayfaring, co-presence and mobility: Conceptualising and re-conceptualising with smartphones. In The palgrave handbook of screen production (pp. 117–128). London, New York: Palgrave MacMillan. Luke, R. (2005). The phoneur: Mobile commerce and the digital pedagogies of the wireless web. In P. Trifonas (Ed.), Communities of difference: Culture, language, technology (pp. 185–204). Palgrave Macmillan US. Norman, D. (2013). The design of everyday things. Basic Books. A member of the Perseus Book Group. Ross McElwee. (1986). Sherman’s March. Ruttmann, R. (1927). Symphony of a city. Berlin. Seal, B. (2013). Baudelaire, Benjamin and the birth of the Flâneur. Psychogeographic Review. Retrieved from http://psychogeographicreview.com/baudel aire-benjamin-and-the-birth-of-the-flaneur/ Sharf, Z. (2017, May 19). 30 ‘Essential directing tips from 30 master filmmakers’. Indiewire. Retrieved from https://www.indiewire.com/gallery/dir ecting-tips-filmmakers-christopher-nolan-wes-anderson-quentin-tarantino-adv ice/#!17/undefined/ Wolff, J. (1985). The invisible Flâneuse: Women and the literature of modernity. Theory Culture & Society, 2(3), 37–46. Woolf, V. (1930). Street haunting: A London adventure. The death of the moth and other stories. Retrieved from https://ebooks.adelaide.edu.au/w/woolf/ virginia/w91d/chapter5.html

CHAPTER 14

Embodied Interaction and Immersive Film Experiences on Smartphones Kata Szita

Introduction Smartphone use is set around a device that has almost its entire surface covered by a screen. This screen visualizes still or moving images providing a window to various functions and visual stimuli. Smartphones can fittingly be described as pocket technospaces (Richardson, 2007): they allow millions of people to go about their businesses with pocket diaries, pocket messengers, pocket computers, pocket concert halls, and, not least, pocket cinemas. Spectatorship1 on smartphones is influenced by the continuous reinterpretation of the cinematic spectacle. The immersive quality and sonic and visual fidelity that the latest smartphone models afford approximate that of cinema or television. Besides, smartphones engage users in watching familiar audiovisual content with effortless physical involvement, whether that entails a motionless seated position or active bodily engagement

K. Szita (B) Trinity Long Room Hub, Trinity College Dublin, Dublin, Ireland e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_14

205

206

K. SZITA

similar to operating a game controller. Still, the key factors that induce a smartphone-specific viewer behavior are the touch screen control and the physical properties of the device—most notably its size and weight that allows it to be at hand at any time. Regarding the fusion of familiarity and novelty in smartphone spectatorship, the following questions arise: what factors are responsible for enabling immersive experiences with fictional audiovisual narratives and how do smartphone users engage with these narratives in such a multisensory setup? To reflect on immersive and multisensory narrative experiences, in this essay, I introduce the basic components of what connects, but, at the same time, distances smartphone spectatorship from stationary screen-based viewing, such as cinema. I argue that these components primarily concern two major features smartphones hold. On one hand, the multisensory nature of spectatorship involves vision, sound, touch, and kinesthetic engagement. On the other, a viewer’s embodied involvement affords a specific form of interactive film experiences. Exploring multisensory and embodied viewing, my goal is to reflect on audiovisual storytelling and story-receiving on smartphones. This distinction is key: while in my analysis a moving-image narration may be loyal to cinematic formulas, smartphone viewing involves spontaneous customizations, where narrative experiences become defined by a viewer’s individual choices and bodily engagement. This analysis of the peculiarities of watching moving-image content on smartphones and other types of handheld devices departs from the questions that allocate attention between storytelling and receiving: hapticity and the modes in which a viewer may tamper with the presentation of a movie narrative through bodily control. Thus, interactivity here points beyond forced-choice interactions. It entails the momentary customization of viewing parameters like screen position, playback speed, or sound volume. The result is individual and customized encounters with movies; that is, tailor-made narrative experiences that are built upon a viewer’s sensory engagement, the momentary disposition of her2 body in relation to the screen, and the mental processing of narrative events. In other words, viewing involves a viewer’s illusion of ownership of the content (Atkinson, 2014). Following the presentation of this distinctive constellation of interactive viewing, I will turn to a specific case of immersive film experiences: 360-degree screenings. Watching 360-degree movies and videos on smartphones differ from other immersive technologies, such as virtual

14

EMBODIED INTERACTION AND IMMERSIVE FILM …

207

reality CAVES (where immersive content is projected on the walls of an enclosed space) or head-mounted displays (where one’s point of view defines the sensory scope). The differences are to be sought for in the dynamic peephole navigation (Mehra et al., 2006) that portable devices afford while held in one’s hands. The viewer moves the screen to gain access to visual details. Whereas this form of immersive viewing experiences is based on the same technological solutions that head-mounted displays employ (and smartphones can be attached to headsets like Google Cardboard or Samsung Gear VR to mimic standalone head-mounted displays), I will observe it through the lens of handheld use and focus on 360-degree viewing that involves manual control. Mobile 360 (or mobile cinematic VR; Schleser, 2020) encompasses 360-degree animated or live-action film and video content. It delimits other virtual reality technologies’ scope of navigation and affords access to moving-image content on two-dimensional screens. As the content is captured from a single vantage point, viewer interaction is confined to changing the segment of a spherical space momentarily visible on the screen. Interactions indicate moving the screen within three degrees of freedom (pitch, yaw, and roll) by which one can explore the depicted diegetic environment and follow characters and objects. A great deal of power is given to viewers to observe narrative events, but content makers apply a wide range of storytelling techniques so that attention is directed to the key storytelling elements rather than wandering within the space. For instance, in Back to the Moon (Goby & Leroux, 2018), the variation of dark and illuminated segments of the space indicates the points of interest, and the story is discontinued until the relevant areas are within the scope of the mobile screen. In other films, such as A Little Negro Boy’s Prayer (Adelheim, 2018), visual—the protagonist’s animated body—and Interwoven: Veganism, Ethics, and Economics (Ramirez, 2018), auditory cues direct attention within the space.3

Multisensory Encounters Audiovisual experiences are subjective: sociocultural elements, screening environments, and technologies, as well as viewers’ personalities, knowledge, and momentary state of mind affect the intake of film narratives. Yet, smartphones and other portable screening devices allow for a specific degree of personalization. Viewers can accommodate viewing experiences according to their wills or needs at any moment employing smartphones’

208

K. SZITA

handheld design and touchscreen navigation. It is a viewer’s body and its connection to the device which determines the spheres of interactions. Mobile devices are not only objects defined by their portability. The fact that their small size and weight enables handheld control is just as fundamental to their constitution. Haptic interaction plays a significant role in defining the appearance of the sensory scope that a smartphone user has access to during media experiences. For instance, changing the position of the smartphone changes the proportion of mediated content to the surrounding environment: on-screen visual information or sound will take less of the sensory scope compared to the physical surroundings when moving the device away from one’s body (Casetti & Sampietro, 2012). This embodied link between a user/viewer and the smartphone creates the possibility of sensory fluctuation that influences the perception of information originating from the physical world and the device. Smartphone use involves haptic, visual, and sonic features, which may cause dissonances between sensory input and output. As a primarily image- and sound-producing appliance, a smartphone accurately projects visual and sonic information. But when it comes to touch control, haptic information gathered by holding the device (e.g., its size, surface, or weight that can be sensed through tactile and kinesthetic means) fails to correspond to other sensory modalities communicated by the interface. That is, haptic information has little correlation with what one can see or hear. When a face is displayed on the screen, information gained through touch will not correlate with the texture, temperature, or other attributes of human faces. It follows that haptic control has other functions than only sensing. While information is gathered through vision, hearing, and haptic feedback, such as vibration, responses are primarily executed through manual interaction. Through touch and hand movements, smartphones enable bidirectional and dynamic interactions through which one can individualize the sensory and even the structural outline of moving-image contents. This turns spectatorship from the phenomenological experience of witnessing a screen into physical contact, into tactile visuality, where the authority of muscles and receptors is minor (Atkinson, 2014; Marks, 2002).

14

EMBODIED INTERACTION AND IMMERSIVE FILM …

209

Touch Interaction While they are not touchable in reality, the visual representation of bodies, objects, and surfaces on a smartphone screen may evoke haptic memories, which complete the missing information about texture, temperature, and the like. Interaction with a touchscreen evokes a particular alliance of the screen and a user’s body, involving physical touch that elicits virtual actions manifested in sensory responses, like animations, sound effects, or vibrations. As stated above, actual haptic connections to objects supply tactile and kinesthetic information, but in the case of smartphone spectatorship, this process is rather abstract. What the smartphone does is mediating: it integrates the sensations of physical touch with virtual image and sound. And although tactile sensing in spectatorship is often attributed to the function of remembering (as explained by Barker, 2009; Huhtamo, 2007; Marks, 2000; Verhoeff, 2012), in the smartphone’s case, we encounter a more complex effect that has to be analyzed through the links between physical motion and mediated representation. A touchscreen functions through registering the presence or absence of physical conjunction: response is prompted when an operational gesture (touch, swipe, etc.) activates sensors in the screen. The touch-sensitive surface meets the flexible skin of a fingertip, the mechanical deformation of which against the rigid surface activates neural processes that signal haptic action. In response to an action of a human body, the smartphone generates feedback in the form of sensory stimuli. Studies discussing usability and interactions with mediated and even computerized interfaces (some mentioned above) begin by correlating physical touch with virtual responses to make epistemological deductions and draw conclusions about user experiences. Heidi Rae Cooley (2004) follows this method while focusing on the physique, more precisely the structure of a user’s hands, to examine what she calls the mobile screenic device. Her insights are especially valuable to this discussion of the biomechanical aspects of touching and holding a mobile device: Cooley labels the intersection of image and touch as screenic screening and claims it to be the result of an automatic process during which a user memorizes the area of touch and is thus able to focus on the result, which is perhaps the most notable in the case of gaming.

210

K. SZITA

Although her focus is on palpable instruments such as buttons, knobs, and slides, Cooley’s reasoning can serve as a basis for comprehending tactile spectatorship on touch-screened smartphones. However, full comprehension requires my return to the questions of memory and cognitive processing. “Touching” moving images through commands a video player application affords introduces similar dissonances to those in video gameplay that divide physical input and virtual output. Physical gestures and tactile sensing are detached from a sensed object, which exists in a different spatial construction—in the diegetic space. This means that one touches the surface of the smartphone screen to interact with (touch, move, enlarge, etc.) a visually represented object. To resolve this ambiguity, a combination of cognitive and cultural approaches is necessary, similar to what Laura Marks (2000; see also Marks, 2002) uses in her book, The Skin of the Film, to define tactile sensations in connection to the visual imagery of movies. This solution foregrounds the correspondence of mediation and sensory modalities, notably the way textures and other palpable, haptic qualities represented on-screen stand in reference to haptic memories. Visual (or sonic) representation, accordingly, evokes memories of bodily experiences, which, in the case of smartphones, is extended by the physical sensation of touch. While being exposed to (audio)visual content, a smartphone viewer typically maintains constant physical and perceptual contact with the device, which enables touching and interacting with the virtual platform. The screen serves as a medium between the skin and diegetic objects. Marks’ (2000, 2002) multisensory understanding of cinema suggests that film narratives, however abstract their presentation is, are understood through associations and references to earlier physical encounters with corresponding objects. Marks illustrates this with the haptic qualities that artworks presented on film evoke. Functional correlations between objects and mental images (i.e., memories) refer to earlier haptic inputs and memories of objects’ physical properties. Marks (2000) claims that the sensory understanding of narratives goes well beyond the technical capacities of any screening apparatus; the process can evoke memories of touch, smell, or even taste. “The eyes themselves function like organs of touch” (Marks, 2000, p. 162), she argues.

14

EMBODIED INTERACTION AND IMMERSIVE FILM …

211

Interactive Spectatorship The hardware design of modern mobile devices diminished—and in many cases eliminated—button control and replaced it with a touchpad built into a high-resolution screen. Through touchscreen control, a user is continually involved in shaping the content on the platform upon which it is visually represented. But what prompted the popularity of watching films and videos on smartphones is perhaps the two most apparent specifications, namely, their size and weight, which enable portability and a viewer’s corporal involvement. Compared to stationary screening apparatuses, such as cinema screens or television sets, on a smartphone, a viewer can more freely revise the spatial dimensions and adjust the synthesis of film and physical stimuli and immerse herself into an interactive experience. Following the discussion of the multisensory bases of smartphone spectatorship, in the upcoming sections, I turn my attention to the ways in which interactions define individual and customized narrative film experiences. In the case of smartphones, bodily engagement characterizes spectatorship. Interactions range from changes in the position of the screen and the firmness and stability of that position to subsequent manipulations of image and sound presentation. Smartphones afford a threefold mode of interactivity (see also, Szita, 2020). First, by analog interactions, a viewer may define the proportion of moving-image stimuli in relation to the physical world by establishing the distance between the screen and her sensory organs and performing adjustments of this distance to reach the desired position. Second, she can determine stimulus intensity using the device’s built-in functions to adjust volume, luminance, or image size. Third, smartphones allow for interactions with a screened footage that change the flow of narrative presentation: one can, for instance, jump between scenes, freeze or pause the footage, or alter the speed at which it plays. Instead of the sole dominance of external control of narrative experiences by instruments such as film editing, lighting, and framing, smartphone viewing involves explorative activities. Thus, storytelling becomes secondary to story-receiving: perception, engagement, and interaction. To illustrate how receiving takes over the role of storytelling, I briefly compare a smartphone viewer’s capacities to those attached to other forms of audiovisual narratives. This comparison is based on the following factors: a viewer’s influence on the narrative presentation and storytelling

212

K. SZITA

by manual interactions, the type, time, and amount of these interactions, as well as the mechanical and contextual elements viewers interact with. Cinematic, televised, and other similar screen-based moving-image experiences involve an observant, where a story is told irrespective of the attention—or in many cases, even presence—of a viewer. Interactions are minimal in these types of experiences. A viewer is engaged with a story by immersing in a diegetic world that includes environments, characters, objects, actions, and narrative events. Subjectivity, in these cases, is based on one’s perception of and emotional reactions to these elements. A sub-genre of cinema and the fusion of cinema and video games is the interactive film, which involves forced-choice-type interactions at ˇ cera, pre-defined times. Interactive films, such as the Kinoautomat (Cinˇ 1967), the more recent Black Mirror: Bandersnatch (Slade, 2018), or Late Shift (Weber, 2016), that was specifically made for smart devices, define and direct viewers’ choices both mechanically and contextually. First of all, moments of interaction happen at clearly pronounced times following a specific narrative design. This means that these moments are set to narrative tipping points and announced by moderators, intertitles, or similar ways. Decisions are made in a forced-choice manner, where two or more alternatives are offered to viewers to decide upon a character’s next action or a situation’s outcome. Second, in connection to the forced-choice method, it is the arranger or director of screening or film who determines the range of possible outcomes during the process of constructing a “network” of narrative events. This implies that viewers have a structural influence on a narrative by choosing one of the available storylines at each point of interaction. Smartphone spectatorship is akin to video gameplay in that they both involve spontaneous interactions executed by physical (mostly manual) gestures with effects on the narrative flow. However, whereas in video games these actions induce changes within the diegetic world (e.g., by a player character’s course of actions or motion), in smartphone spectatorship, only the sensory presentation of a narrative will be impacted. Thus, while a video game player has an evident participatory role in influencing a narrative (see Dolan & Parets, 2016), a smartphone viewer’s role is more complex and less clearly defined. As in the case of cinematic spectatorship, a smartphone viewer has an observant role: a story being played unfolds irrespective of her attention. This means that the visual and auditory outline of a moving-image narrative remains unchanged. Yet, her access to visual and auditory information

14

EMBODIED INTERACTION AND IMMERSIVE FILM …

213

may be affected. This point can be illustrated by sensory adjustments enabled by smartphones. For instance, smartphones and a wide range of video player applications afford zooming in on an image even if it is in motion. This action will enlarge some visual elements of the image, while others that fall outside of the displayed area will be inaccessible to the viewer. Due to the temporal outline of moving images, narrative events unfold once without repetition. Therefore, narrative information that is momentarily inaccessible due to changes in stimulus intensity, such as image size or volume, will not be redeemed. The temporal structure of narrative films notwithstanding, however, a smartphone viewer may have a structural influence on a narration—similar to viewers of interactive films. Even in the lack of intertitles or announced moments of interaction, a viewer can execute changes that affect the overall narrative structure of the footage being played. These changes can manifest themselves in the form of jumping between scenes, playing at a different speed, or rewinding the footage. Rearranging a narrative structure in this way has cognitive consequences—something that differentiates interactive smartphone spectatorship from interactive film screenings: a viewer gains access to narrative information in a customized way, differently than what was intended by content producers. Following the discussion in the earlier sections of this essay, a viewer’s bodily involvement plays a significant role in momentary interactions during smartphone spectatorship: the manual control of the device enables a reflexive viewing experience. Reflexive viewing entails interactions with a screened footage through hand gestures and muscle movements at any time. As I explain elsewhere, these interactions are prompted either by external factors (for instance, noise from the surrounding space or a task) or internal motivations (curiosity or personal preferences) (Szita, 2020). In addition, a movie’s storytelling formulas may also catalyze interactions. Thus, based on incoming information and internal motivations, a viewer makes conscious or unconscious decisions about whether and how to shape a screening by changing sensory connection with the content. Reflexive viewing is most clearly outlined in mobile 360-degree content. Therefore, in the concluding part of this essay, I will turn to the cognitive and phenomenological grounds of engaging with 360-degree movies and videos on smartphones.

214

K. SZITA

Immersive Narrative Experiences on Smartphones The combination of narrative and bodily dimensions links a smartphone viewer’s physical and diegetic presence using memories, haptic engagement, and contextual information. The viewer contextualizes incoming information and assembles a logically coherent meaning out of the semantic system of narrative formulas. This is precisely the setting that provides the considerable plasticity of spectatorship and allows for and motivates interactions in the form of touch control or kinesthetic operations. The interactive potentials of smartphone spectatorship place narrative information into a subjective contextual frame. Immersive viewing on smartphones is based on two main levels of mediated characteristics: the immersive qualities of a movie narrative and a viewer’s embodied involvement and interactions. To demonstrate narrative immersion, I borrow the fun house analogy that Janet Murray introduces in her 1997 book, Hamlet on the Holodeck. Murray argues: “The fun house has an entrance and an exit that mark the beginning and end of the story. As the visitor progresses on a moving platform, the dramatic tension builds from small surprises and hints of danger; then there are thrills and a mounting sense of threat or terror, which culminates in a big finish such as a free fall or an attacking beast” (p. 105). Alison Griffiths (2008) follows a corresponding notion when discussing immersion, but adds corporeality as a factor of immersive experiences. She associates immersion in fictional narratives with a beholder’s (for instance, a viewer’s) identification with and sense of presence in an environment that masks sensory and cognitive access to the physical world. Combining Murray and Griffiths’s models, one can claim that a movie narrative engages viewers’ minds and senses by “walking” them through a system of fictional events. As these events unfold, viewers acquaint themselves with characters, their relationships, actions, and surroundings, which transports them into a sphere where their corporeality is replaced by sensory experiences. But although it would be suitable for the fun house analogy, narratives in these models of immersive experiences are treated as temporal constructs, rather than spatial ones, that begin with the start of the movie and close with its end. The spatial dimension is disregarded on account of the camera’s defined position of observation. Even though a movie’s visual language provides a walk-through in various fictional environments, viewers cannot freely explore all their dimensions: the point of observation is anchored to the “fourth wall” of diegetic spaces.

14

EMBODIED INTERACTION AND IMMERSIVE FILM …

215

Movies presented in 360 degrees challenge this setup. In these cases, a full 360-degree space is available, although not all at once. It is the viewer’s task to navigate and delineate her perspective. The viewing perspective is adjusted by bodily motions, that is, one has to change one’s position to gain access to the different segments of the surrounding fictional space. When using head-mounted displays, this movement and changes of perspective happen through the head or full-body motions as one turns around to observe different parts of the space. In the case of handheld smartphones, however, the changes are induced by moving the screen. This means that, rather than enabling interactions through natural-like bodily movements of turning and looking around as standalone fully immersive virtual reality headsets do, the smartphone screen serves as a window over the 360-degree diegetic space, and access to narrative information is based on the position of this window. This distinction is not least crucial because head-mounted displays are designed to obscure physical reality, as in Griffiths’s immersion theory, whereas the screen of a smartphone presents an alternative reality embedded into the physical world. The diegetic space is shown on the screen while the physical space is still sensed; the balance of these two is defined by the screen’s distance from the viewer’s eyes—as it was addressed earlier. The window analogy is pertinent in screen studies. A screen may serve as something that delineates the frame of mediation and divides physical reality from a fictional space (see, Bruno, 2014; Friedberg, 2006). It is also a tool for storytelling: it outlines the visual scope of the diegetic space and directs attention to elements that hold essential narrative information (Grodal, 1997). Yet, the window in mobile 360-degree spectatorship is tied less to storytelling instruments, such as directing or the mise-en-scène. Instead, I argue, it must be approached as a means of story-receiving. Narrative experiences through the window of the smartphone screen immerse viewers into a new type of fun house. One proceeds through the imaginary rooms with the opportunity to look around and perceive objects and characters being placed and moving around in relation to one’s body. Some of these objects and characters are within reach; their impact is perceived as stronger than that of those that are further away. For instance, characters approaching the viewer’s point of view increase in size and gain stronger relevance which evokes stronger reactions with every step. Yet, the viewer cannot touch or move them, neither can she control her distance from them. Stepping away from a stressful situation

216

K. SZITA

caused by a character’s or object’s proximity is not possible, one can only move the screen away. This proprioceptive quality of mobile 360-degree spectatorship appears in a particular setting in which physical and mediated spaces are perceivable simultaneously. While in cinema, a viewer’s body is small relative to the screened content which fills a significant proportion of the visual scope, it feels large compared to objects depicted on a smartphone screen held in one’s hands. Biologically, proprioception is responsible for adjusting the body’s position to avoid collision, for instance, when an object is flying toward the beholder. When seen on a screen or in the physical world, such objects might evoke a flight response manifested as ducking or cocking one’s head. Being aware of the kinesthetic power over a handheld screen (that its position can be adjusted by muscle movements), these reactions are less likely when a potential threat is seen on the small screen. There, a more feasible reaction would be moving the screen farther away from the body. But moving the screen holds two consequences: on one hand, it changes the sensory scope of the 360degree narrative space, on the other, it draws attention to the viewer’s bodily presence. This body awareness is the one that evokes the sensation of corporeal presence in a diegetic space that is observed through hand movements. The smartphone viewer moves the screen to define the segments of the space to be visually available and the proportion thereof in relation to the physical world. This spatial freedom is an essential component of the sensation of embodied immersion, the physical experience of motion, and hapticity that is involved in holding and touching a smartphone. But can this complex multisensory experience that involves the visual, sonic, and haptic senses and motor engagement enhance narrative immersion? Hapticity couples with mediated images and sounds, which—according to the earlier discussion of the integration of sensory modalities—would enhance a viewer’s engagement with diegetic objects, characters, and spaces. Additionally, viewing 360-degree films on smartphones promises immersion, a simulated presence: one can explore distant landscapes, planets, or even imaginary places in 360 degrees. However, this sensation operates with the mind and the body on different levels. A viewer is to unite with narrative worlds while her body is occupied by operating the means of access to the physical world. Embodiment is a crucial component of immersive film experiences. According to Vittorio Gallese and Michele Guerra’s (2012) thesis of

14

EMBODIED INTERACTION AND IMMERSIVE FILM …

217

embodied simulation, viewers can only be fully transported into a diegetic world if they lose conscious control or awareness of their bodies. In other words, truly immersive experiences with audiovisual narratives require a viewer’s bodily stillness. In the case of 360-degree viewing on smartphones, the body is engaged in touching, holding, and moving the device, while the mind is busy computing the directions of interest within the diegetic space and coordinating movement and cognitive processes.

Conclusion Perceiving a three-dimensional space projected on a two-dimensional surface that is located in another three-dimensional space induces a peculiar spatial and affective experience, caused by the illusion that one is inside and outside the projected 360-degree environment simultaneously. Being one of the most fundamental characteristics of spectatorship, this is the eternal clash that accompanies the study of narrative experiences in terms of moving images. What, however, is propelled by technological innovations of mobile screens is that engagement is no longer restricted to mental and emotional devotion but linked to tangible bodily involvement too. When operating smartphones, users interact concurrently with physical and virtual objects. Gestures are defined by the device’s properties and laws of physics, whereas interactions with user interfaces follow the principles of a constructed, programmed sphere. Users’ fingers move over a blank, smooth surface that nevertheless induces complex plots. Interaction with a smartphone’s interface goes hand in hand with the abstraction of this uncanny relationship that connects a user or viewer with content through multisensory involvement. In smartphone and mobile 360-degree viewing, two major factors define experiences: first, the malleability of sensory and narrative information, that originates from the smartphone’s interactive capacity, and second, the viewer’s bodily presence through haptic (manual) control. Thus, perception is circumscribed by the outline of visible elements, which include the composed filmic mise-en-scène, the screen, and the surrounding environment. This outline changes when the viewer adjusts the screen’s position, visual angle, or other sensory characteristics. Bodily (haptic and kinesthetic) control manifests in nonce configurations of stimulus sources and sensory organs, on one hand, and in phenomenological contact with content, on the other. These affect narrative experiences as

218

K. SZITA

viewers execute changes to the attributes of image, sound, and narrative flow that, for instance, alter engagement or immersive effect. Smartphone spectatorship involves a complex system of mental and physical processes that define interactive viewing and a viewer’s observant, structural, and reflexive engagement with moving-image narratives. This requires the embodied presence of the viewer, who, besides perceiving audiovisual content in connection to a story being told, defines both the material and contextual framework of the screening.

Notes 1. Spectatorship in this work is understood as an act of watching (animated or live-action) moving-image content and comprehending audiovisual information through narrative contexts (Bordwell, 1985; Münsterberg, 1916/2014). The specific case of smartphone spectatorship refers to multisensory experiences in which audio and visual information are extended by haptic engagement. 2. The pronoun refers to a hypothetical user or viewer throughout the text, while the author acknowledges that a user/viewer can be of any genders. 3. For more film examples and a detailed review of storytelling techniques, see Mateer (2017) and Schleser (2020).

References Atkinson, S. (2014). Beyond the screen: Emerging cinema and engaging audiences. Bloomsbury. Barker, J. M. (2009). The tactile eye: Touch and the cinematic experience. University of California Press. Bordwell, D. (1985). Narration in the fiction film. Methuen. Bruno, G. (2014). Surface: Matters of aesthetics, materiality, and media. University of Chicago Press. Casetti, F., & Sampietro, S. (2012). With eyes, with hands: The relocation of cinema into the iPhone. In P. Snickars & P. Vonderau (Eds.), Moving data: The iPhone and the future of media (pp. 19–32). Columbia University Press. Cooley, H. R. (2004). It’s all about the fit: The hand, the mobile screenic device and tactile vision. Journal of Visual Culture, 3(2), 133–155.

14

EMBODIED INTERACTION AND IMMERSIVE FILM …

219

Dolan, D., & Parets, M. (2016). Redefining the axiom of story: The VR and 360 video complex. TechCrunch. https://techcrunch.com/2016/01/14/red efining-the-axiom-of-story-the-vr-and-360-video-complex/?guce_referrer_us= aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_cs=oEhe_9cTA dZmDDujD-rT_Q&guccounter=1&guce_referrer_us=aHR0cHM6Ly9jb25 zZW50LnlhaG9vLmNvbS8&guce_referrer_cs=6g6FCiw2L_X0OcQwYtjh5g. Accessed 20 August 2020. Friedberg, A. (2006). The virtual window: From Alberti to Microsoft. MIT Press. Gallese, V., & Guerra, M. (2012). Embodying movies: Embodied simulation and film studies. Cinema, 3, 183–210. Griffiths, A. (2008). Shivers down your spine: Cinema, museums, and the immersive view. Columbia University Press. Grodal, T. (1997). Moving pictures: A new theory of film genres, feelings and cognition. Clarendon Press. Huhtamo, E. (2007). Twin-touch-test-redux: Media archaeological approach to art, interactivity and tactility. In O. Grau (Ed.), Mediaarthistories. MIT Press. Marks, L. U. (2000). The skin of the film: Intercultural cinema, embodiment, and the senses. Duke University Press. Marks, L. U. (2002). Touch: Sensuous theory and multisensory media. University of Minnesota Press. Mateer, J. (2017). Directing for cinematic virtual reality: How the traditional film director’s craft applies to immersive environments and notions of presence. Journal of Media Practice, 18(1), 14–25. Mehra, S., Werkhoven, P., & Worring, M. (2006). Navigating on handheld displays: Dynamic versus static peephole navigation. ACM Transactions on Computer-Human Interaction, 13(4), 448–457. Münsterberg, H. (1916/2014). The photoplay: A psychological study. Appleton/Createspace. Murray, J. (1997). Hamlet on the Holodeck: The future of narrative in cyberspace. The Free Press. Richardson, I. (2007). Pocket technospaces: The bodily incorporation of mobile media. Continuum: Journal of Media and Cultural Studies, 21(2), 205–215. https://doi.org/10.1080/10304310701269057 Schleser, M. (2020). Mobile cinematic VR—MCVR. In L. Hjorth, A. de Souza e Silva, & K. Lanson (Eds.), The Routledge companion to mobile media art (pp. 375–386). Routledge. Szita, K. (2020). New perspectives on an imperfect cinema: Smartphones, spectatorship, and screen culture 2.0. NECSUS: European Journal of Media Studies, 9(1):31–52. Verhoeff, N. (2012). Mobile screens: The visual regime of navigation. Amsterdam University Press.

PART V

Mobile Education and Screen Storytelling

Mobile technology with its ubiquitous accessibility anytime and anywhere is a game-changer in mobile storytelling, be it with industry or education. Any enhanced mobile storytelling, however, starts with education. To what extent and how mobile has been playing a role in mobile-assisted education in different contexts is a major concern among policy-makers, practitioners, and professors in their respective mobile-related sectors. To address that shared concern, three chapters in this part examined how mobile has been or should be leveraged in film education, middle school education, and mobile-assisted language learning. Chapter 15 investigated the factors of middle school students’ beliefs concerning smart mobile devices (SMDs) and their technology acceptance of SMDs in their lessons in the framework of Technology Acceptance Model (TAM). A series of exploratory and confirmatory factor analyses and multiple regressions have shown that students’ beliefs have been identified with the five factors of immediacy, interest, interactivity, instability, and inconvenience, which are in consistency with the performance expectancy of the educational use of SMDs. The findings of this chapter offered not only insights but also recommendations regarding how smart mobile devices should be leveraged to enhance mobile learning. As different perspectives lead to different results, Chapter 16 offered a thought-provoking case for all areas of mobile storytelling education although it researched into Mobile-Assisted Language Learning (MALL). To find out how MALL may help English learning for college students and to what extent their self-regulated learning skills may benefit their

222

PART V:

MOBILE EDUCATION AND SCREEN STORYTELLING

learning performances, both quantitative questionnaires and qualitative interviews were designed to measure the critical dimensions of the learners’ self-regulated abilities. The results show that most participants are willing to take the initiative to adopt MALL in their language study. However, their self-regulation during the learning process and reflections afterward were insufficient to meet the learning goal. The advent of mobile technology has given rise to wholly new models of creation, circulation, usage, and spreadability of moving image storytelling by challenging traditional notions of story while expanding story horizons, as examined by Chapter 17. All these shifts also see redefinitions of some old job roles involved in creating moving image content, which, in turn, necessitates a relook into how to teach students who will seek employment in the new marketplace. This chapter has posed a lot of thought-provoking questions for the existing film educators to reconsider. Although different in approaches and cases, one shared goal of these three chapters is to examine the role of mobile in enhancing mobile storytelling through investigating both educators’ and students’ perspectives as well as looking into the results of applying mobile technologies in language learning. Although case-specific, these chapters have also offered a general and shared goal, that is, how to fully leverage mobile and mobile technologies to enhance mobile storytelling education. In future mobile storytelling will not be confined to language barriers as mobile technologies provide opportunities to connect people across timezones and languages.

CHAPTER 15

Chinese Middle School Students’ Beliefs About and Acceptance of Smart Mobile Devices Xiaoxiao Zhang

Smart education has gained significance attention in recent years. Educational research projects focused on smart education have been conducted globally in recent years (e.g., Chan, 2002; Choi & Lee, 2012; Hua, 2012; Kankaanranta & Mäkelä, 2014). In 1997, Malaysia first carried out a smart education project, Malaysian Smart School Implementation Plan (Chan, 2002). Smart schools, which are supported by the government, aim to improve the educational system to achieve the National Philosophy of Education and to prepare the future work force to meet the challenges of the twenty-first century. Singapore also has implemented the Intelligent Nation Master Plan (2006), in which technology-supported education is outlined (Hua, 2012). In the plan, eight schools focus on creating

X. Zhang (B) School of International Communication, Faculty of Humanities and Social Science, University of Nottingham Ningbo China, Ningbo, China e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_15

223

224

X. ZHANG

digital-oriented learning environments. Australia collaborated with IBM and designed a smart, multi-disciplinary student-centric education system (Shoikova et al., 2017). This system links schools, tertiary institutions and workforce training. South Korea initiated the SMART education project, which is reforming the educational system and improving educational infrastructures (Choi & Lee, 2012). China also launched such an initiative in April 2018. The Ministry of Education issued the Education Informatization 2.0 Action Plan, which announced that digital learning should be introduced in schools. All teachers and students should be trained to achieve a certain level of information literacy. This marks the beginning of the Chinese Education Informatization 2.0 era. Different from the Education Informatization 1.0 era, which focuses on the development of technology and equipment, Education Informatization 2.0 pays more attention to the design and implementation of software, platforms and service systems. A series of education reforms and programs demonstrate the commitment and the long-term integration of information technology and education resources to enhance the level of digital education. However, if digital education aims to be successful in China, students need to play an important role in evaluating this latest technology. This situation is similar to the use of computers in education 20 years ago. Previous studies on the use of technology in education reported that students’ opinions and beliefs are the main factors of technology acceptance (Hew & Brush, 2007; Orsini-Jones et al., 2017). In fact, the recorded relationship between students’ beliefs and their academic performance and classroom practice (Simpson & Rush, 2003) shows that students’ opinions about technology will affect whether and how they choose to use technology in class (Ertmer & Hruskocy, 1999). If students have a positive outlook in using new technology for educational purposes, they are more likely to try to integrate this new technology into their learning process. In contrast, if students have negative views on the use of technology in classes, these perceived technical characteristics may become obstacles to their pedagogical experience. The decision on whether and how to use technology for learning ultimately depends on students and their beliefs about the attributes or characteristics of a piece of specific technology. Therefore, in order to facilitate students to use mobile technology tools in class, it is very important to understand the main educational characteristics of these mobile technology tools and the role these technologies should play in the educational environment.

15

CHINESE MIDDLE SCHOOL STUDENTS’ BELIEFS …

225

This research focuses on understanding factors for middle school students’ beliefs and adoption of SMDs. As existing evidence indicates students’ adoption mostly relates to cost and effectiveness. SMDs are seen as a cost-effective alternative to textbooks (Blomgren, 2018). This current study intends to understand middle school students’ perspectives on SMDs, and which factors influence their opinions in the learning process. Applied models and theories have been developed to interpret technology acceptance in educational settings, particularly the technology acceptance model (TAM) (Davis, 1989). TAM presents a framework for understanding how to reinforce teachers’ intention of implementing technology in secondary school settings (Grani´c & Maranguni´c, 2019). To promote SMART education in China, it may be an important issue to understand students’ opinions and their acceptance of using SMDs in daily learning process. The purpose of this study is to investigate Chinese middle school students’ opinions and belief in SMART education and to explore their acceptance of technology in the learning environment. Although the results of this study took place in the context of China, the use of SMDs-based SMART education has become more and more common all over the world, especially in developing countries. The results of this study will be useful for schools that want to integrate SMDs in their teaching and learning environments.

The Development of SMDs-Based Smart Education Many middle schools introduce mobile devices such as smartphones or tablets into their classrooms (Ertmer, & Hruskocy, 1999). Research into this development has shown that scholars have put forward many related concepts, such as online education, mobile learning and learning management system among others. These concepts are based on the application of information technology in education from different perspectives. In recent years, the development of China’s education has entered a new period. The education reform is improving digital education quality. Many universities in China have adopted SMART education to improve learning and teaching efficiency. In a similar development, SMART education is also introduced into secondary settings (Tang et al., 2020). Not only in China but also in other developing countries, there is increasing demand for SMART education based on smart mobile devices (Mehdipour & Zerehkafi, 2013).

226

X. ZHANG

Mobile learning is defined as learning facilitated by mobile devices such as mobile phones, tablets or personal media players (Herrington & Kervin, 2007; Valk et al., 2010) in both formal and informal educational settings (Traxler, 2010). The SMDs-based SMART education is the refinement and development of this conception. With the full coverage of wireless networks, SMDs (Smart Mobile Devices) are widely used in all aspects of our daily life. SMART education represents a kind of modern education based on mobile devices, which is characterized by self-directed, motivated, adaptive, resources-enriched and technologyembedded learning. Kang et al. (2012) identified the main characteristics of teaching and learning with smart devices as sociality, continuity, adaptability, authenticity and technology-based. Based on previous definition, the SMDs-based SMART education in this study meets the following conditions. (1) SMDs-based SMART education is based on the application of mobile devices. (2) SMDs-based SMART education is student focused. (3) SMDs-based SMART education expands beyond the classroom boundaries. (4) SMDs-based SMART education helps to formulate a synchronized. According to the Chinese Ministry of Education (2019), the population of online students has dramatically increased. In 2012, about 13.8 million students from middle school were enrolled in online courses. Several previous studies have addressed the provision of SMDs as a tool to provide full access to academic, social and administrative materials (Cavus, 2011). Students using SMDs in educational have access learning content from various locations at any time (Garcia-Cabot et al., 2015; Hyman et al., 2014; Jones et al., 2013) and can share learning contents with others (Woodill, 2011). For teachers, SMDs help to improve teaching effectiveness, in particular, individual learning and create diversified class management opportunities. SMDs-based SMART education has a potential to improve educational quality by influencing students’ and students’ perception of new technology. The SMART education focuses on motivating students’ noncognitive ability by means of engaging students in interesting course

15

CHINESE MIDDLE SCHOOL STUDENTS’ BELIEFS …

227

design. SMART education is defined through a process of effective interaction and dynamic interactions between students and teachers as well as students and students.

The Technology Acceptance Module Student decisions about whether and how to use new technology in the classroom are influenced by their beliefs and concerns regarding new technologies. According to the TAM proposed by Davis (1989), one’s actual use of a technology system is directly or indirectly influenced by the perceived usefulness of the system and the perceived ease of the system in relation to external variables, attitudes and behavioral intentions. Perceived usefulness is defined as the degree to which one believes that using a particular system will enhance one’s performance, and perceived ease of use refers to the degree to which one believes that using a particular technology will be effective (Davis, 1989). TAM studies of teachers’ technology use have shown that perceived usefulness and perceived ease of use are the main factors that are key factors for choosing a particular technology in the classroom (Teo, 2011). To confirm whether an emerging technology can be used more frequently and effectively in schools, the relationship between teachers’ opinions about SMDs and the perceived usefulness and perceived ease of use of SMDs must be further investigated. At the very beginning of technology entering users’ everyday life, there is a growing necessity for understanding why the technology is accepted or rejected. First theories attempting to explain and predict those decisions were grounded in the field of Psychology. The “Theory of Reasoned Action” (TRA) represents the origin of TAM, Technology Acceptance Module, emerging alongside with the “Theory of Planned Behavior” (TPB) (Ajzen, 1985). To develop a reliable model which could predict actual use of any specific technology, Fred Davis adapted the TRA and proposed the TAM (Davis, 1989). He considered that the actual use of a system is essentially a behavior and therefore the TRA would be suitable model for explanation and prediction of that behavior. TAM suggested that the users’ motivations can be explained by three factors: perceived ease of use, perceived usefulness and attitude toward using the technology. Davis hypothesized that the attitude of a user toward the system was a major determinant of whether the user will actually use or reject the technology. During later stages of research, the model was refined to

228

X. ZHANG

include other variables. New factors with significant influence on the core variables of this model are continuously revealed. Over time TAM has evolved to become the key model in understanding human behavior toward potential acceptance or rejection of new technology. The strength of this model and its many different versions (so-called “TAM++”) is emphasized in its broad applicability to various technologies and contexts, for instance, acceptance of Internet (Lee et al., 2012), digital libraries (Thong et al., 2002), electronic mail, online learning (Farahat, 2012) and electronic commerce (Pavlou, 2003), Internet banking (Nasri & Charfeddine, 2012), Wireless Internet (Lu et al., 2003) and hedonic information systems (Van der Heijden, 2004) alongside other emerging technologies. TAM emerged as a leading scientific paradigm for investigating the acceptance of learning technology by students, teachers and other stakeholders. A user’s intention toward e-learning technology can be explained through TAM. Various studies explore TAM’s applicability for different learning technologies, such as mobile learning (Sánchez-Prieto et al., 2016), Personal Learning Environments (PLEs) (del Barrio-García et al., 2015) or Learning Management Systems (LMSs) (Alharbi & Drew, 2014). While there are several research projects related to TAM in the context of e-learning adoption or m-learning environments, the field still lacks comprehensive research study addressing existing approaches and applications that investigate TAM and its different versions in educational contexts for a variety of learning domains, learning technologies and types of users.

Method The Application of TAM on SMDs-Based Smart Education Research The participants in the study were 3113 students selected from a public middle school in North China. There were 1488 males (47%) and 1678 females (53%). The mean age of the students was 16.5 years (SD = 1.24). Students were informed in advance of the research procedure. Their participation was voluntary, and all data was collected anonymously through an online survey site. We applied for the approval of appropriate parent permission and received the school’s ethics approval. Students were

15

CHINESE MIDDLE SCHOOL STUDENTS’ BELIEFS …

229

asked about their beliefs and technology acceptance concerning SMDs, regardless of their experience in using SMDs in their daily learning. Instruments To develop the instrument for our research, we adopted a semantic differential questionnaire that is focused on students’ belief concerning SMDs-based Smart education, and also students’ technology acceptance of SMDs. The semantic differential questionnaire consisted of 21 adjective pair items that matched the characteristics of SMDs that was adapted from Sung and Mayer (2012). A semantic differential scale is a measurement method to examine the connotation or emotional meaning of concepts through a series of seven-point bipolar adjective evaluation scales. At the beginning of the questionnaire, we briefly explained the purpose and significance of the research. The questionnaire also indicated the following, We collected 29 positive and negative adjectives. You can see that on the left are negative adjectives (starting with the number 1) and on the right are positive adjectives (starting with the number 7). The closer you are to the right, the more positive your comments are likely to be, and vice versa. Whether or not you have used a smart classroom, please evaluate this innovative teaching method based on your current cognition.

The 21 adjective pairs of the semantic differential questionnaire were rated on a seven-point scale between two bipolar adjectives with opposite meanings about the feelings and images of students’ regarding SMDs, such as “dynamic” (1) versus “static” (7) or “movable” (1) versus “fixable” (7). The Cronbach’s alpha was 0.912, which indicates suitable reliability. The students’ technology acceptance questionnaire consisted of 12 items concerning the characteristics of SMDs that was adapted from Davis’s (1989) technology acceptance modeling scaling. The ten items were divided into two parts. One part comprised four items of perceived usefulness (e.g., using smart devices in a lesson would enable me to improve learning efficiency) and the other part included six items of perceived ease of use (e.g., learning to operate a smart device would be easy for me) on five-point Likert scale (1 for very little to 5 for

230

X. ZHANG

very much). The Cronbach’s alpha was 0.934, which indicates suitable reliability. The questionnaire also included demographic information regarding students’ gender, age and grade. Procedure Our study was approved by the research ethics committee of Peking University. An online survey was developed and sent to secondary students’ mobile devices. The participants were asked to complete the online questionnaire. Fifty-three cases were excluded because of missing data, yielding a total of cases 3113 that were used for the data analysis. Data Analysis To identify and verify the factors behind students’ beliefs about SMDs in learning situations, two types of analysis were used, namely “Exploratory Factor Analysis” (EFA) and “Validating Factor Analysis” (VFA). EFA was analyzed by the principal component analysis method with a maximum rotation. In addition, the first-order CFA is used to examine a priori proposed factor structure. Second-order CFA is a more advanced CFA, which is used to identify the load of first-order factors on second-order factors. We used a variety of criteria to evaluate different aspects of the IT goodness model fitting index, such as chi-square goodness of fit statistics, CFI, TLI, IFI and RMSEA (Satorra & Bentler, 2001). To further explore the relationship between students’ belief factors of SMDs and technology acceptability, we conducted regression analysis based on students’ technology acceptability. Results Table 15.1 shows the means and standard deviations for each item in identifying the students’ beliefs concerning SMDs. The total mean of the ratings was 2.58 (SD = 1.33), which ranged from 1.85 to 3.64 with SMDs from 1.14 to 1.51. When the score is close to 1, the score is closer to the feeling of the left adjective.

15

CHINESE MIDDLE SCHOOL STUDENTS’ BELIEFS …

231

Table 15.1 Mean, standard deviations analysis results Dependent variable

Independent variable

R2 (adj. R2 )

Perceived usefulness

Stability

0.14 (0.14)

F

40.83**

Enjoyment interactivity Perceived ease of use

Immediacy

0.14 (0.13)

118.79**

B

SE B

b

t

−0.12

0.03

−0.17

−4.12**

−0.11

0.03

−0.17

−4.04**

−0.09 −0.23

0.03 0.02

−0.13 −0.37

−3.50** −10.90**

** p < 0.01

Exploratory Factor Analysis The exploratory factor analysis was conducted on the students’ data (n = 3113) concerning SMDs. The KMO measure was found to be 0.95, and the Bartlett’s test was found to be significant. As a result, four factors were extracted from the 20 adjective pairs regarding the students’ beliefs concerning SMDs, which explained 69.42% of the total variance. We labeled the four belief factors of enjoyment, immediacy, interactivity and stability based on the results extracted from 20 items. Enjoyment explained 43.25% of the variance with an eigenvalue = 10.81 and factor loadings that ranged from 0.68 to 0.82. Enjoyment included the adjectives smart, easy, exciting, expressible and helpful. Enjoyment explained an additional 10.03% of the variance, with an eigenvalue of 52.51 and factor loadings that ranged from 0.69 to 0.80. Immediacy included the adjectives friendly, comfortable, accessible, fast and communicable. Interactivity explained an additional 6.65% of the variance, with an eigenvalue of 51.66 and factor loadings that ranged from 0.66 to 0.85. Interactivity included the adjectives immersing, effective, vivid, focused and new. Stability explained an additional 5.08% of the variance with an eigenvalue of 51.27 and factor loadings that ranged from 0.62 to 0.78. Stability included the adjectives virtual, stable, synchronous and two-way. To measure the strength and direction of the extracted four factors of students’ beliefs concerning SMDs, a correlation analysis was conducted. There were significant positive correlations among all the factors based on two-tailed tests with p < 0.01 that ranged from r = 0.36 to r = 0.64.

232

X. ZHANG

Confirmatory Factor Analysis To determine whether the four factors of students’ beliefs were obtained in the EFA, we conducted a series of CFA through two types of CFA as the first- and second-order models. These analyses indicated that the predefined four factors of the students’ beliefs concerning SMDs that were established by the EFA were validated as having a good structural model fit through CFA for both the first- and the second-order models. To provide more evidence concerning the link between the four factors of students’ beliefs concerning SMDs and their technology acceptance toward SMDs Smart education, we conducted a regression analysis. Table 15.1 shows the means and standard deviations of students’ smart devices technology acceptance ratings and the four-factor ratings. The mean of the students’ performance expectancy was 3.56 (SD = 0.81), and the mean of their effort expectancy was 3.46 (SD = 0.77). The mean of the four factors ranged from 2.02 to 3.32, and standard deviations ranged from 0.97 to 1.20. Table 15.1 shows that instability, inconvenience and interactivity were selected as the factors that most strongly correlated with the performance expectancy of SMDs, R 2 = 0.14, adjusted R 2 = 0.14, F = 40.83, p < 0.01. Furthermore, inconvenience was selected as the factor that correlated most strongly with effort expectancy of SMDs, R 2 = 0.14, adjusted R 2 = 0.13, F = 118.79, p < 0.01. Based on these results, we provisionally conclude that if students perceive a smart device as stable, easy and interactive, they are more likely to accept smart devices technology in their teaching because they recognize the perceived usefulness and perceived ease of use of SMDs.

Conclusion The main finding of this current study is the identification of four factors of students’ beliefs concerning SMDs, namely, enjoyment, immediacy, interactivity and stability. In particular, the four-factor structure of students’ beliefs concerning SMDs was supported by rigorous statistical analyses. In addition, the factors were validated by first-order and second-order CFA, and the most appropriate model was confirmed. The major findings concerning the factor analyses with the semantic differential scaling were that primary students tended to show some expected similarities in their connotative feelings concerning using SMDs. The four

15

CHINESE MIDDLE SCHOOL STUDENTS’ BELIEFS …

233

factors of students’ beliefs concerning SMDs included enjoyment, immediacy, interactivity and stability. Regarding the relationship among the four factors of students’ beliefs concerning SMDs and students’ smart devices technology acceptance, including performance expectancy and effort expectancy. Higher stability, higher enjoyment and higher interactivity have positive effects on students’ performance expectancy of the educational use of SMDs. Higher immediacy has a positive effect on students’ effort expectancy of SMDs. Overall, this study suggests that if students feel that SMDs are stable, comfortable or have interactivity for the educational context, they can positively use SMDs for educational purposes. These results, however, are very likely to be influenced by students’ experience of using SMDs in their classes and in their private lives. This influence needs to be further studied in the future. Previous studies found that students’ beliefs concerning new technology influence their educational use of such technology in the classroom (Carver, 2016; Ertmer, 2005; Hermans et al., 2008). Thus, students’ beliefs concerning SMDs are critical factors in teaching and learning. The findings of this study are similar to the results of previous research, this study’s results suggest that students’ beliefs concerning SMDs that are unstable or uncomfortable may be a main barrier to students’ use of technology in daily learning. Therefore, we must find solutions that can alleviate students’ negative feelings about SMDs based on smart education. For this purpose, further research is needed to identify which variables relate to students’ negative feeling. If the variables that relate to students’ negative feeling are eliminated, then, students’ use of SMDs in the classroom will be facilitated. Accordingly, school support must be provided that includes not only appropriate technical support but also timely social and emotional support (Joo et al., 2016). The present study has some limitations that should be addressed in future research. First, the present study focused on identifying the factors in students’ beliefs concerning the SMDs based on smart education. Future studies are needed to determine whether the same results would be obtained in general classroom environments with students of different ages, genders, experiences of the use of SMDs in the classroom or in their private lives, levels of digital media literacy, pedagogical philosophies and school culture. Second, the present study focused on the relationship between the factors that relate to the students’ beliefs and their technology acceptance concerning SMDs smart education. Future studies

234

X. ZHANG

should examine the effect of students’ beliefs factors on the educational use of SMDs in the classroom to eliminate their negative feeling and to promote positive feeling. Finally, future research is needed to determine whether the factors involved in students’ beliefs concerning SMDs relate to learning performance and learning outcomes in educational settings. Overall, these results suggest that the research on the effectiveness, efficiency and the attractiveness of instructional design in the educational use of SMDs should focus on the features of students’ beliefs. The five factors of students’ beliefs should be elaborated and should be applied in educational environments.

References Ajzen, I. (1985). From intentions to actions: A theory of planned behavior. In J. Kual & J. Beckmann (Eds.), Action control (pp. 11–39). Springer. Alharbi, S., & Drew, S. (2014). Using the technology acceptance model in understanding academics’ behavioural intention to use learning management systems. International Journal of Advanced Computer Science and Applications, 5(1), 143–155. Blomgren, C (2018) OER awareness and use: The affinity between higher education and K-12. International Review of Research in Open and Distributed Learning, 19(2). https://doi.org/10.19173/irrodl.v19i2.3431 Carver, L. B. (2016). Teacher perception of barriers and benefits in K-12 technology usage. Turkish Online Journal of Educational Technology-TOJET, 15(1), 110–116. Cavus, N. (2011). Investigating mobile devices and LMS integration in higher education: Student perspectives. Procedia Computer Science, 3, 1469–1474. Chan, F. M. (2002). ICT in Malaysian schools: Policy and strategies. Workshop on the Promotion of ICT in Education to Narrow the Digital Divide (pp. 15–22). Choi, J., & Lee, Y. (2012). The status of SMART education in KOREA. In T. Amiel & B. Wilson (Eds.), Proceedings of EdMedia 2012—World Conference on Educational Media and Technology (pp. 175–178). Denver, Colorado, USA: Association for the Advancement of Computing in Education (AACE). Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 319–340. del Barrio-García, S., Arquero, J. L., & Romero-Frías, E. (2015). Personal learning environments acceptance model: The role of need for cognition, e-learning satisfaction and students perceptions. Educational Technology & Society, 18(3), 129–141.

15

CHINESE MIDDLE SCHOOL STUDENTS’ BELIEFS …

235

Ertmer, P. A. (2005). Teacher pedagogical beliefs: The final frontier in our quest for technology integration? Educational Technology Research and Development, 53(4), 25–39. Ertmer, P. A., & Hruskocy, C. (1999). Impacts of a university-elementary school partnership designed to support technology integration. Educational Technology Research and Development, 47 (1), 81–96. Farahat, T. (2012). Applying the technology acceptance model to online learning in the Egyptian universities. Procedia-Social and Behavioral Sciences, 64, 95– 104. Garcia-Cabot, A., de-Marcos, L., & Garcia-Lopez, E. (2015). An empirical study on m-learning adaptation: Learning performance and learning contexts. Computers & Education, 82, 450–459. Grani´c, A., & Maranguni´c, N. (2019). Technology acceptance model in educational context: A systematic literature review. British Journal of Educational Technology, 50(5), 2572–2593. Hermans, R., Tondeur, J., Van Braak, J., & Valcke, M. (2008). The impact of primary school teachers’ educational beliefs on the classroom use of computers. Computers & Education, 51(4), 1499–1509. Herrington, J., & Kervin, L. (2007). Authentic learning supported by technology: Ten suggestions and cases of integration in classrooms. Educational Media International, 44(3), 219–236. Hew, K. F., & Brush, T. (2007). Integrating technology into K-12 teaching and learning: Current knowledge gaps and recommendations for future research. Educational Technology Research and Development, 55(3), 223–252. Hua, M. (2012). Promises and threats: IN2015 masterplan to pervasive computing in Singapore. Science, Technology and Society, 17 (1), 37–56. Hyman, J. A., Moser, M. T., & Segala, L. N. (2014). Electronic reading and digital library technologies: Understanding learner expectation and usage intent for mobile learning. Educational Technology Research and Development, 62(1), 35–52. Jones, A. C., Scanlon, E., & Clough, G. (2013). Mobile learning: Two case studies of supporting inquiry learning in informal and semiformal settings. Computers & Education, 61, 21–32. Joo, Y. J., Lim, K. Y., & Kim, N. H. (2016). The effects of secondary teachers’ technostress on the intention to use technology in South Korea. Computers & Education, 95, 114–122. Kang, I., Lim, B., & Park, J. (2012). Exploring the theoretical framework and teaching & learning strategies of smart learning: Using cases of university classrooms. The Korean Journal of Educational Methodology Studies, 24(2), 283–303. Kankaanranta, M., & Mäkelä, T. (2014). Valuation of emerging learning solutions. In J. Viteli & M. Leikomaa (Eds.), Proceedings of EdMedia 2014—World

236

X. ZHANG

Conference on Educational Media and Technology (pp. 168–172). Tampere, Finland: Association for the Advancement of Computing in Education (AACE). Lee, W., Xiong, L., & Hu, C. (2012). The effect of Facebook users’ arousal and valence on intention to go to the festival: Applying an extension of the technology acceptance model. International Journal of Hospitality Management, 31(3), 819–827. Lu, J., Plataniotis, K. N., & Venetsanopoulos, A. N. (2003). Regularized discriminant analysis for the small sample size problem in face recognition. Pattern Recognition Letters, 24(16), 3079–3087. Mehdipour, Y., & Zerehkafi, H. (2013). Mobile learning for education: Benefits and challenges. International Journal of Computational Engineering Research, 3(6), 93–101. Ministry of Education of the People’s Republic of China. (2018). Educational Informationization 2.0 Plan. Retrieved from http://www.moe.gov.cn/ srcsite/A16/s3342/201804/t20180425_334188.html Nasri, W., & Charfeddine, L. (2012). Factors affecting the adoption of Internet banking in Tunisia: An integration theory of acceptance model and theory of planned behavior. The Journal of High Technology Management Research, 23(1), 1–14. Orsini-Jones, M., Conde Gafaro, B., & Altamimi, S. (2017). Integrating a MOOC into the postgraduate ELT curriculum: reflecting on students’ beliefs with a MOOC blend. In K. Qian & S. Bax (Eds.), Beyond the language classroom: Researching MOOCs and other innovations (pp. 71–83). https:// Research-publishing.net https://hal.archives-ouvertes.fr/hal-01550840/doc ument Pavlou, P. A. (2003). Consumer acceptance of electronic commerce: Integrating trust and risk with the technology acceptance model. International Journal of Electronic Commerce, 7 (3), 101–134. Sánchez-Prieto, J. C., Olmos-Migueláñez, S., & García-Peñalvo, F. J. (2016). Informal tools in formal contexts: Development of a model to assess the acceptance of mobile technologies among teachers. Computers in Human Behavior, 55, 519–528. Satorra, A., & Bentler, P. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66(4), 507–514. Shoikova, E., Nikolov, R., & Kovatcheva, E. (2017). Conceptualising of smart education. Electrotech. Electron. E+ E, 52. Simpson, M. L., & Rush, L. (2003). College students’ beliefs, strategy employment, transfer, and academic performance: An examination across three academic disciplines. Journal of College Reading and Learning, 33(2), 146– 156.

15

CHINESE MIDDLE SCHOOL STUDENTS’ BELIEFS …

237

Sung, E., & Mayer, R. E. (2012). Five facets of social presence in online distance education. Computers in Human Behavior, 28(5), 1738–1747. Tang, H., Lin, Y., & Qian, Y. (2020). Understanding K-12 teachers’ intention to adopt open educational resources: A mixed methods inquiry. British Journal of Educational Technology, 51, 2558–2572. Thong, J., Hong, W., & Tam, K. (2002). Understanding user acceptance of digital libraries: What are the roles of interface characteristics, organizational context, and individual differences? International Journal of HumanComputer Studies, 57 (3), 215–242. Teo, T. (2011). Factors influencing teachers’ intention to use technology: Model development and test. Computers & Education, 57 (4), 2432–2440. Traxler, C. (2010). Social norms and conditional cooperative taxpayers. European Journal of Political Economy, 26(1), 89–103. Valk, J., Rashid, A., & Elder, L. (2010). Using mobile phones to improve educational outcomes: An analysis of evidence from Asia. International Review of Research in Open and Distributed Learning, 11(1), 117–140. Van der Heijden, H. (2004). User acceptance of hedonic information systems. MIS Quarterly, 28(4), 695–704. Woodill, G. (2011, March 29). The ideal learning management system for multimedia learning. Knoodle White Paper.

CHAPTER 16

The Role of Learner Self-Regulation in Mobile-Assisted Language Learning Yanhui Zhang, Yuanyuan Liang, and Nancy Xiuzhi Liu

Introduction and Related Literature The ever-faster breakthroughs in internet technology and the rapid increase of the number of netizens around the world have made the role of mobile technology more than ever pertinent in language education. As of the end of 2020, there are around 989 million netizens in China, 99.7% of whom have access to the internet by mobile phones and 9.3% of whom hold a bachelor or above degree (CNNIC, 2021). University students

Y. Zhang (B) · N. X. Liu School of Education and English, The University of Nottingham Ningbo China, Ningbo, China e-mail: [email protected] N. X. Liu e-mail: [email protected] Y. Liang School of Education, University College London, London, UK e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_16

239

240

Y. ZHANG ET AL.

are one of the main driving forces of online education, especially mobileassisted education, among other reasons, to obtain professional certificates that are not included in the school curriculum and to boost academic grades. Against this backdrop, mobile-assisted language learning (MALL) has become increasingly popular among foreign language learners in China. According to surveys reported by Zou and Yan (2014), Azar (2014), and Aydin (2013), although the majority of teachers and students are positive towards MALL, a better way to apply mobile devices in learning is desired. It has been proven that the MALL model of language learning could motivate students to learn a particular language and improve their language proficiency (Shi et al., 2017; Viberg & Gronlund, 2012). The success could be attributed to the advantages of MALL. Compared with traditional desktop computers, mobile devices are smaller in size, more affordable in price, and easy to operate. Because of these characteristics, the benefits of MALL from the users’ side can be summarized as flexibility, accessibility, low cost, portability, user-friendliness, and convenience (Chinnery, 2006; Huang et al., 2012). Among all these benefits, portability is the most important, distinguishing MALL from other learning (Pea & Maldonado, 2006) and allows learners to choose time and place of study. Although the definition of mobile learning (m-learning) is mixed in existing literature, many defined the term from one or more of the three perspectives—mobility, learning, and mobile devices (Yu et al., 2015). As indicated by El-Hussein and Cronje (2010), a broad definition of mlearning should include any type of learning taking place in a learning environment with either mobility of technology, or mobility of learners, or mobility of learning. Developed mainly in the past decade, MALL is a subfield of m-learning (Viberg & Gronlund, 2012), focusing on using mobile technologies and devices to assist in language learning. Like m-learning, MALL is an emerging and developing field (Duman et al., 2014). The research studies in MALL have covered four core skills (listening, speaking, reading, and writing) of different languages, such as French, Spanish, and English, with mobile devices’ assistance and the applications that could be downloaded or installed in these devices (Burston, 2015). Most of these research results are positive, showing that MALL could improve learners’ learning efficiency and marks. However, few research studies have indicated whether the learning outcomes are the same for MALL and traditional learning.

16

THE ROLE OF LEARNER SELF-REGULATION …

241

Notwithstanding, MALL also has constraints. With a portable size, the screen is small and the graphic shown on the screen is limited (Albers & Kim, 2001). Reading on such a screen is inconvenient with some multimedia problems (Miangah & Nzarat, 2012), such as low audio-visual quality. Some other potential disadvantages include narrowed social interaction (i.e. learners may fail to join some language-learning–related online group or forum), controlled lengths of messages, deficiency of learning environment, and limited nonverbal communications (Chinnery, 2006). Salaberry (2001) held a similar view that it is doubtful whether applying mobile technologies could result in the same pedagogical outcomes as traditional language teaching methods. These constraints and features of MALL (interactivity, instantaneity, flexibility, accessibility, and portability) convey an implication that learners need to regulate themselves during the mobile language learning process. The cyclical learning model also premises such needs in practice. Notably advocated by Zimmerman (2000), the model describes the three main phases an efficient online learner may experience in their self-regulated learning (SRL). In the forethought phase, learners may set hierarchical goals before start learning, and then they need to choose proper strategies to achieve the goals. Self-control and self-observation are the main components for the second phase for learners to concentrate on their goals and dynamically improve their efforts. Last but not least, self-reflection and self-evaluation allow the learner to compare their achievement with agreed standards and goals. Reviewing the literature, research studies focusing on whether learners can regulate themselves during the MALL process is relatively scarce. Time management and environment setup during the self-regulated learning process are crucially essential factors impacting the performance of MALL (Jansen et al., 2017). Besides, the interaction between other learners and teachers on a virtual platform is different from traditional learning. Student activities in MALL are mediated behaviourally and cognitively by devices and technologies (Sha et al., 2012). In this way, a better MALL outcome is based on better SRL skills; positive learning outcomes of MALL can only be achieved by performing self-regulated learning behaviours (Wang, 2011). When learning online, there is an unknown distance between teachers and learners; thus, teachers cannot supervise learners or address their questions face-to-face. Whether and how learners can regulate themselves is of pivotal importance in the practice and success of MALL. This study aims to fill this gap by investigating

242

Y. ZHANG ET AL.

how well English-majored students self-regulate themselves in MALL and how such self-regulation may impact the effectiveness of MALL. The objective is accomplished by answering the following specific questions. 1. Why do English majors learn in a MALL environment? 2. Do English-majored students have an awareness of self-regulation in MALL? If yes, how do they self-regulate themselves? 3. How does self-regulation in MALL impact the learners’ English proficiency?

Methodology These research participants are junior and senior students majoring in English at a Chinese university. The reason to choose the senior and junior students is because they took the Test for English Majors (TEM) in the previous school year; the score of TEM can serve as an objective measurement of one’s English proficiency. Junior students who failed TEM-4 need to prepare to retake it in April. Senior students are required to pass TEM-8, which takes place in March. Also, most junior and senior students need to prepare for the Unified National Graduate Entrance Examination (UNGEE), which occurs at the end of the first semester (December) in the last year of undergraduate study. The university curriculum doesn’t explicitly cover the training for such exams, implying that students need to learn by themselves. The needs of learning motivate these students’ reliance on MALL to self-learn these contents. The research instrument design adopts both quantitative and qualitative research methods by administering questionnaires and interviews after the ethical clearance from the School’s Ethical Panel. The questionnaire encompasses three parts. The first part asks for the participant’s demographic information, including gender, grades, and standardized test scores, such as TEM-4 and 8 scores. The second part of the questionnaire focuses on the participants’ MALL status quo, such as attitudes, aims, learning frequency, learning spending, preferred learning devices, contents, and styles, and preferred help-seeking strategies. Oxford (1989), Luo (2018), and Lin (2017) are the primary references for designing the questionnaires of this part, including checklists, multiple-choice questions, Likert scale questions, and ratio scale questions. The third part of the questionnaire concentrates on learners’ self-regulation when learning on mobile devices. A comprehensive review of related studies (Bernard et al., 2009; Hu, 2018; Schraw & Dennison, 1994) leads to adopting

16

THE ROLE OF LEARNER SELF-REGULATION …

243

a five-point Likert response research questionnaire. At the end of the questionnaire, participants can leave their contact information if they are willing to be further interviewed afterward. The follow-up interviews contribute to deeper understandings of the learners’ motivations and choices that are not reflected in the questionnaires (Dornyei, 2003). Some participants may choose logistically proper answers rather than those reflecting their actual behaviour and thinking. Some questionnaires may contain unintended inaccuracies and errors when conducted in rushing. All interviews were conducted in Chinese to avoid misunderstandings and miscommunications. Sixty-nine valid questionnaires have been selected based on item analysis. Four tests were conducted: descriptive statistic test, homogeneity test, extreme group test, and reliability test. The validity test was also carried out in terms of content validity, criterion validity, and construct validity. According to Tabachnick and Fidell (2013), a minimum value of 0.6 for the Kaiser–Meyer–Olkin (KMO) index should be reached for factor analysis. The KMO value for the current study is 0.658. All questionnaires have been checked manually before entering data in SPSS. Thirteen participants voluntarily took part in the interview, including seven students (two males and five females) in the third year and six students (two males and four females) in the fourth year. The Cronbach’s Alpha coefficient of the whole scale is 0.905, which indicates that the scale is very reliable. The Cronbach’s Alpha coefficients for the five sub-categories (Goal setting, strategic planning, self-motivational beliefs, self-observation, and self-control and self-reflection) are 0.754, 0.772, 0.765, 0.773, and 0.840, respectively. The reliability of sub-categories is also high. The KMO index is 0.818. The scale construct validity is safeguarded for factor analysis. Findings A majority of the population held a positive attitude towards MALL. Seventy-six participants thought that MALL had improved their learning efficiency. Eleven participants held a neutral view of MALL. Half of the remaining participants did not believe their efficiency had been improved, leaving the other half unaware of the effect. Most interviewees firmly believed that their efficiency of English learning had been improved with cited reasons being summarized as (1) fragmented time can be collected for study; (2) Words can be learned on mobile devices in time and places

244

Y. ZHANG ET AL.

they choose; (3) Looking up words online is faster and time-saving than checking in the dictionary; and (4) MALL makes it possible to learn TEM grammar in a relatively short period. Aims and Impacts Above 80% of the questionnaire, participants appealed to MALL to help pass exams related to English majors. Three-quarters of them wanted to improve professional quality with the help of MALL. Almost all interviewees set passing exams as their primary goal. Interviewees also had other aims. However, most of them were afraid they could not sufficiently engage in MALL without an urgent purpose. More than 70 participants in the survey thought that MALL offered them rich learning materials in various presentations meeting their personalized demand. Besides, convenience is the most highly cited experience when asking interviewees about the impact of MALL. Of the participants, 80% thought they could arrange their learning time and places with ease with the devices’ assistance. MALL had made their English learning more convenient than before, affirmed by 90.43% of participants. As a carrier of ample resources, mobile devices provided much more considerable autonomy for learning. Learners were able to search for materials that suit and interest their individual needs. Of the participants, 68.09% believed that MALL, by combining audios, videos, images, and texts, had made English learning more engaging than traditional. Nearly half of the participants thought mobile gadgets are easy to operate. Despite various positive impacts, MALL also has negative user experiences: eight participants found that MALL had lowered their learning efficiency; eight reported no difference between MALL and the traditional way of learning; 18 considered the hardware of mobile devices, such as screen and space, need to be enlarged; 16 thought the design of software should be upgraded; and another 18 cited the unstable network connection and the high data fee as undesirable features of MALL. Goal setting and Strategic Planning As shown in Table 16.1, the means in this dimension ranges from 3.32 to 4.05. The average for the dimension is 3.58. The mean response to GS1 shows that learners were highly goal-driven when learning with MALL. Learners were particularly good at setting daily goals with the mean above

16

THE ROLE OF LEARNER SELF-REGULATION …

245

Table 16.1 Descriptive statistics for the goal setting and strategic planning N = 94

Mean

SD

[GS1] I have specific aims on mobile English learning [GS2] I have short-term goals (e.g., daily, weekly) for my mobile English learning [GS3] I have long-term goals (e.g., monthly, semester) for my mobile English learning [GS4] I have a high expectation of my mobile English learning [SP1] Before mobile learning, I will compare learning modes, resources, and teachers in different applications and websites [SP2] I will adopt the default mode of applications and websites to leam English vocabulary [SP3] While doing mobile English learning, I will take notes with paper and pen [SP4] I use several applications, websites, and software together to leam English [SP5] I will make my learning materials on mobile devices according to my level and demand [SP6] I will review what I have learned on mobile devices in my spare time [SP7] I will stick to mobile English learning even though I do not have a precise aim

4.05 3.59

0.908 1.010

3.36

1.076

3.32 3.54

0.930 0.958

3.46

0.912

3.57

1.011

3.70

0.948

3.64

0.982

3.48

1.002

3.38

0.985

the average. However, they somehow lacked long-term plans, as shown in the responses to GS3. The mean of GS4 is the lowest in the dimension, indicating that the learners did not have a high expectation of MALL. Similarly, most interviewees in the interview had a daily goal, but few had long-term plans. According to Table 16.1, responses to SP1 showed that participants took seriously what they have learned on MALL. SP3 suggested that although participants adopted high-tech devices to assist English learning, the traditional way of note-taking is indispensable. The adoption of notetaking or not depends on learners’ learning aspects and their aim of MALL. Those who had taken notes could provide convenience for them if they need to review learning content in the following days. It could be interpreted from SP2 that half of the participants would adopt the default mode of applications, leaving the rest half remained neutral or disagreeing with the default mode. Participants were able to choose the proper way to utilize online resources. SP6 indicated that nearly half of the participants might not focus much on reviewing after learning on mobile devices. SP7

246

Y. ZHANG ET AL.

received the lowest mean of 3.38. Of the participants, 37.23% remained neutral. Another 18.09% of them disagreed, implying that learners may have learned English via MALL; however, the learning might not be as regular and frequent as an optimal outcome may entail. In addition to the questionnaire’s strategies, interviewees also planned environmental strategy and time management strategies. Most interviewees would like to do MALL in quiet places promoting the focus of learning such as university library and self-study rooms. As for time management, many interviewees preferred to use the fragmented time to learn words while the blocks of time were planned for online courses. Self-Observation and Self-Control According to Table 16.2, the average for this phase of the learning cycle is the lowest 3.16, suggesting participants’ through-MALL performances could be improved. The mean for SOC5 is 2.89, lower than three, indicating that participants had been reluctant to tackle problems encountered during the MALL process. Had a problem become unavoidable, they would prefer to ask friends, offline teachers, and the internet for Table 16.2 Descriptive statistics for the self-observation and control and selfreflection N = 94

Mean

SD

[SOC1] I use a specific time for mobile English learning [SOC2] I will check if I have mastered the contents of self-regulated mobile English learning [SOC3] I will follow my plan strictly while doing mobile English learning [SOC4] If my attention distracts during mobile English learning, I will learn the part I have missed again [SOC5] I always solve problems I have encountered during the mobile English learning process actively [SRI] I will compare my present learning outcome with the previous one after a period of mobile English learning [SR2] I will attribute the outcome of my mobile English learning after a period of learning [SR3] I will summarize my English learning process on mobile devices [SR4] I will adjust my learning strategies according to the problems I have encountered in mobile English learning

3.24 3.03

0.888 0.921

3.35

0.864

3.27

0.941

2.89

0.944

3.29

0.850

3.28

0.873

3.23 3.56

0.955 0.862

16

THE ROLE OF LEARNER SELF-REGULATION …

247

help. The learning outcomes had not been timely and regularly checked neither. Although the participants had both time and learning content plans, they had tried to make up for what they had missed in the MALL process. The interview and questionnaire demonstrated that the exercises and tests provided by mobile applications and online courses were not fully used for various reasons. All interviewees’ attention has been distracted when they were doing MALL. Distraction can be grouped into two kinds, online distraction and offline distraction. Online distractions refer to interruptions from mobile devices, such as instant messages, advertisement notifications, and phone calls. Offline distractions refer to disruption from others. Interviewees had creative ways to prevent distractions. Some of them would cut off their internet connection, turn into airplane mode, and log out of the public social account while doing MALL. If the internet is needed while learning, the interviewee will ignore unimportant messages. Some choose to use a study-only mobile device with only learning applications and resources. Besides, most of them tended to study in the teaching area to avoid interactions with roommates. However, learners’ attention might still not be concentrated on learning content with all such measures in place. The main reasons are: (1) the learning and teaching styles do not suit learners; (2) learners are not in a learning mood; and (3) the learning content makes learners think of something else. Self-Reflection According to Table 16.2, participants did not do well in post-MALL. SR4 is the only item that scores above the average. Participants would like to adjust learning strategies if the previous one did not work well. On the other hand, they did not pay much attention to how they could improve much with MALL’s assistance because they had a relatively low interest in comparing, summarizing, and reflecting on their knowledge. The results from interviews were similar to those of questionnaires. First, most interviewees did not do this because TEM and final exams cover a wide range of English, so it is hard to tell whether the knowledge acquired in MALL helped or knowledge gained in traditional classrooms helped. Interviewees seldom made horizontal comparisons neither. Materials learned on mobile devices are very personalized, so there is not a standard for comparison. Furthermore, learners appeared less likely

248

Y. ZHANG ET AL.

to formally share their mobile learning progress, with reasons being of interest for further studies. Further Discussion As mentioned in the methodology section, TEM scores are used metrics to measure students’ English proficiency. Students who achieved higher TEM scores are rated with higher English proficiency. It is a genuine interest to inquire whether there is a statistical correlation between the self-regulated MALL and TEM scores. Pearson correlation is chosen to test the hypothesis. Two correlations have been calculated. The first one is the correlation between TEM-4 scores and self-regulated MALL abilities. The second one is the correlation between the scores of each sub-category and TEM scores. The total score of the third part of the questionnaire has been calculated to account for the SLR. The higher score, the response received in the third part; the better self-regulated MALL ability is assumed for the participant. Surprisingly somehow, there is no statistically significant correlation between TEM scores and self-regulated MALL abilities: TEM-4 (N = 94), r = 0.050, p = 0.632; TEM-8 (N = 38), r = 0.051, p = 0.762. In a sense, the result is consistent with the notion held by Salaberry (2001), arguing that applying mobile technologies could result in the same pedagogical outcomes as traditional language teaching methods. Among others, the reason might be that MALL had not played a dominant role in learning. It worked as a supplement to the conventional way of learning. Different MALL users have different aims, which largely determined how self-regulated users would perform in the MALL. Correspondingly and as a parallel reason, learners had not yet sufficiently self-regulated in the MALL learning process. Mainly speaking, learners had adopted a mixed method of learning while exploring MALL’s benefit, adding ambiguity in attributions of learning outcomes. A more strictly conditioned experiment is desired to make a more definitive conclusion.

Conclusion Using quantitative and qualitative approaches, this study has provided a thorough multi-dimensional factor analysis of MALL and how each factor may impact the learning results following the optimal learning

16

THE ROLE OF LEARNER SELF-REGULATION …

249

cycles and learning strategies. A comprehensive and representative work demonstrating how English-majored students self-regulate their MALL is obtained via scientifically designed questionnaires and interviews at the Chinese university. The emerging format of learning, MALL, has been vastly accepted by its English-majored students. The results show that students are willing to spend time and money in MALL in that MALL has provided many unique learning-enhancing features, particularly autonomy and convenience. In turn, these features attract students to use mobile devices to explore more updated learning materials and styles. On the other hand, MALL also has drawbacks. The physical and technological constraints may affect the learning experience. A wealth of information that is unrelated to learning may distract learners’ attention. The new learning model needs to be more timely adapted and updated. As a result, learners must have strong self-regulated abilities for effective learning purposes. As demonstrated, the SRL abilities of the English-majored students under study are at a moderate level, with potential improvements in multiple aspects. Learners did well in the forethought phase with proper goals, planning, strategies, and high motivation. However, higher SRL abilities are desirable for the performance and self-reflection phases to explore MALL’s effectiveness fully. Furthermore, there is no correlation between students’ self-regulated MALL learning skills and their TEM-4 scores. This is a small-scale study that involved only 121 participants. Given that this is a relatively new area with few previous studies, more extensive research is desirable and expected in similar universities with students from a broader background in undergraduate majors. Conservations are recommended in interpreting the non-significance correlation between students’ self-regulated MALL learning skills and their TEM-4 scores, as reported in the current study. As mentioned in the Result and Analysis section, the participants had taken mainly somewhat a combined approach with both self-regulated MALL and offline studies involved in their English learning. Further decomposition and variable control are needed in future research, preferably experimental, to shed insights into this complex phenomenon.

250

Y. ZHANG ET AL.

References Albers, M., & Kim, L. (2001). Information design for the small-screen interface: An overview of web design issues for personal digital assistants. Technical Communications, 49(1), 45–60. Aydin, S. (2013). Teachers’ perceptions about the use of computers in EFL and learning: The case of Turkey. Computer-Assisted Language Learning, 26(3), 214–233. Azar, A. S. (2014). Learners’ attitudes toward the effectiveness of mobile-assisted language learning (MALL) in L2 listening comprehension. Procedia-Social and Behavioural Science, 98(6), 1836–1843. Bernard, L., Lan, W. Y., To, Y. M., Paton, V., & Lai, S. L. (2009). Measuring self-regulation online and blended learning environments. Internet and Higher Education, 12(1), 1–6. Burston, J. (2015). Twenty years of MALL project implementation: A metaanalysis of learning outcomes. ReCALL, 27 (1), 4–20. Chinnery, G. M. (2006). Emerging technologies going to the MALL: Mobileassisted language learning. Language Learning & Technology, 10(1), 9–16. CNNIC. (2021). China Internet Network Information Center. www.cnnic.net.cn Dornyei, Z. (2003). Questionnaires in second language research construction, administration and processing. Lawrence Erlbaum Associates. Duman, G., Orhon, G., & Gedtk, N. (2014). Research trends in mobile-assisted language learning from 2000 to 2012. European Association for ComputerAssisted Language Learning, 27 (2), 197–216. El-Hussein, M. O. M., & Cronje, J. C. (2010). Defining mobile learning in the higher education landscape. Journal of Educational Technology & Society, 13(3), 12–21. Hu, J. Y. (2018). A survey on non-English major college students’ self-regulated ability in M-learning (Unpublished dissertation). Jiangxi Normal University. Huang, Y. M., Huang, Y. M., Huang, S. H., & Lin, Y. T. (2012). A ubiquitous English vocabulary learning system: Evidence of active/passive attitudes vs. usefulness/ease-of-use. Computers and Education, 58(1), 273–282. Jansen, R., Leeuwen, A., Janssen, J., Kester, L., & Kalz, M. (2017). Validation of the self-regulated online learning questionnaire. Journal of Computing in Higher Education, 29(1), 6–27. Lin, Z. B. (2017). A research on the status of college students’ English mobile selfaccess learning in local colleges and universities (Unpublished dissertation). Guangxi Normal University. Luo, X. (2018). A study on the application of metacognitive strategies in college students’ English mobile learning (Unpublished dissertation), Hunan University.

16

THE ROLE OF LEARNER SELF-REGULATION …

251

Miangah, T. M., & Nzarat, A. (2012). Mobile-assisted language learning. International Journal of Parallel Emergent and Distributed Systems, 3(1), 309–319. Oxford, R. L. (1989). Strategy inventory for language learning (version 7.0 ESL/EFL). Pea, R., & Maldonado, H. (2006). WILD for learning: Interacting through new computing devices anytime, anywhere. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 427–441). Cambridge University Press. Salaberry, M. R. (2001). The use of technology for second language learning and teaching: A retrospective. The Modern Language Journal, 85(1), 39–56. Schraw, G., & Dennison, R. S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19(4), 460–475. Sha, L., Looi, C. K., & ChenW, Z. B. H. (2012). Understanding mobile learning from the perspective of self-regulated learning. Journal of Computer Assisted Learning, 28(4), 366–378. Shi, Z., Luo, G., & He, L. (2017). Mobile assisted language learning using WeChat instant messaging. International Journal of Emerging Technologies in Learning, 12(2), 16–25. Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Pearson education. Viberg, O., & Gronlund, A. (2012). Mobile assisted language learning: A literature review. Paper presented at 11th World Conference on Mobile and Contextual Learning, in Helsinki, Finland. Retrieved from http://ceur-ws. org/Vol-955/papers/paper8.pdf Wang, T. (2011). Developing web-based assessment strategies for facilitating junior high school students to perform self-regulated learning in an e-learning environment. Computer & Education, 57 (2), 1801–1812. Yu, C., Lee, S. J., Ewing, C. (2015). Mobile learning: Trends, issues, and challenges in teaching and learning. In J. Keengwe, & M. B. Maxfield (Eds.). Advancing higher education with mobile learning technologies: Cases, trends, and inquiry-based methods (pp. 60–87). IGI global. Zimmerman, B. J. (2000). Attaining self-regulation: A social cognitive perspective. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 13–39). Elsevier Academic Press. Zou, B., & Yan, X. (2014). Chinese students’ perceptions of using mobile devices for English learning. International Journal of Computer-Assisted Language Learning and Teaching, 4(3), 20–33.

CHAPTER 17

The Multi-Specialist Moving-Image Practitioner: Reframing Film School for the Twenty-First Century Daniel Wagner

Accelerating change has vaulted humanity into a state of VUCA: volatility, uncertainty, complexity, and ambiguity (Codreanu, 2016). Among the many sectors that have been upended by this new state is the motionpicture industry, where ongoing technological and resulting methodological evolution is continually disrupting a practice that, until recently, had sat on a foundation of tradition and convention. Moving image today has moved well beyond the cinema; the digital revolution has provided new tools and methods for content design, acquisition, manipulation, and delivery, disrupting the value chain of the motion-picture industry (Salvador et al., 2019). As traditional models of creation and distribution are redefined, technological evolution has also ushered in the growth of newer types of content. As media and audiences continue to shift,

D. Wagner (B) Unitec Institute of Technology, Auckland, New Zealand e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2_17

253

254

D. WAGNER

humanity is undergoing a sea change in the ways we engage with this content. User–makers are bringing a cornucopia of both traditional and twenty-first-century skills and knowledge to bear on the new media landscape. This has led to a variety of new production business models on different scales, which, in turn, has given rise to a new and continually transforming employment landscape (Ryan & Hearn, 2010). Against this backdrop of perpetual evolution, many film and media graduates find themselves unprepared for the morphing realities of the digital industries (Bridgstock, 2016). While screen programmes at institutions internationally have been endeavouring to address the evolving landscape from a variety of angles (Nincarean et al., 2013; Perry, 2018; Stewens, 2017), few if any embrace in their teaching the very nature of change itself in the context of moving-image media. Many programmes are still teaching legacy filmmaking skills (old ways with new tools) with little focus on the rapid developments in context, content, methods, and tools that are taking place just outside their doorsteps (Stoneman, 2014). This research explores one possible approach to contemporary screen education within the context of continually evolving developments in the digital content landscape. It could be argued that screen education (most of which is still in film-school mode) currently addresses two main paths of career focus which might be described as creative ownership (above-theline skills) and craft application (below-the-line skills). I maintain that it is now imperative to develop and deliver a third path, that of multi-specialist content creation. This path interweaves key areas of the first two and then combines them into a basket of complementary knowledge areas that, together, focus on the expanding and morphing requirements of today’s (and tomorrow’s) moving-image content creator.

Film Work Today Many people enter film school with the desire to become a director, writer, or producer. These roles, which exercise the major creative influence on a film, are known as “above-the-line” roles. Others come to film school with a passion for camera, sound, editing, or other more creative/technical specialisations. These roles involve the integration of technical skill and creative craftsmanship to execute the intricate web of tasks that, when brought together, add up to a motion picture. It is these jobs that people refer to when speaking of “below-the-line” roles.

17

THE MULTI-SPECIALIST MOVING-IMAGE PRACTITIONER …

255

In contrast with earlier days, when film companies kept skilled craftspeople on staff, a full-time job with a studio is no longer an option for today’s film workers. Employment in film now is centred around individual projects and today’s film workers are all freelancers (Blair et al., 2001), jumping from project to project, their livelihoods solidly lodged in the gig economy (Parnham, 2019). The history of film schools is almost as old as the film industry itself. But with a foundational shift occurring across the moving-image industries, there is cause to surmise that the majority of film schools have yet to keep up with the levels of complexity and change occurring in the moving-image sectors. So where do we go from here? Is there a new collective graduate profile emerging for tomorrow’s screen grads? How can screen students best learn to position themselves in this new landscape? How do we adapt to change as a constant? What strategies can we employ to remain in a state of flexibility, able to continually adapt and adjust to ever-evolving iterations of what was once familiar and fixed? How do we bottle these strategies and impart them to others? And how do we teach them to students, as they forge, fresh-faced, into a mutable future? And how can we curricularise change itself? These are the questions that face screen educators as we struggle to reposition ourselves, our brand, and our mission. Let us discuss ways in which screen education might reframe itself to serve an industry (now several industries) in upheaval. We will consider this from two angles. The first angle explores the challenge of teaching the art of agility itself to tomorrow’s moving-image craftspeople. With change now the norm across the spectra of technology, among creation/delivery methods, and through audience/end-user engagement, how do students locate themselves as creative practitioners in such a way that they are both grounded in their practice and flexible enough to continually morph with the times? We maintain that the first step is to ensure that they are armed with a basket of related skills to enhance their multi-disciplinary practice and deepen the pool of resources and references from which they can draw to inform their practice. The second angle will open the discussion around methods by which moving-image educators might reframe our content, curricula, and pedagogy in such a way as to allow the tides of change to continually and

256

D. WAGNER

steadily flow through our teaching studios. This means a transformation of screen curricula from being principally grounded in conventional film methods to curricula that are mouldable and continuously responsive to ever-morphing trends in the entire pipeline, from maker to user. What’s in a Name? At this point, it would be worthwhile to clarify our terms. In mercurial times such as these, processes that were at one time fixed have now become fluid. Conventions that once were familiar are now morphing into newer forms of expression. Still, it is convenient to attach the familiar names of the outgoing form until consensus cultivates newer, more appropriate names. Such is the case with the term “film”, which can refer to: • • • • •

an industry; a capture and delivery medium; a unit of programme content; a category for a form of expression; a verb describing a particular creative endeavour.

Despite these distinctions, the word “film” is, today, a legacy term that is still used frequently despite the fact that a variety of media are employed to capture and deliver moving-image content (although “sensoring” or “carding” doesn’t roll off the tongue quite as readily as “filming”). A more all-encompassing term to describe all moving-image media could simply be... “moving-image media”. People involved in its creation ply their crafts in a variety of ways beyond the cinema production model. Perhaps we should refer to them all as “moving-image craftspeople”. Returning to the educational context, employing the broader term moving-image media will allow a wider area of skills application to be defined, which in turn will serve to enable screen education to envision a more inclusive graduate profile that might be better suited to prepare students for portfolio careers in all moving-image media. A portfolio career is one in which a livelihood is created by bringing together multiple streams of income, a model that has been gaining momentum for creative practitioners over the last decade (Wilson, 2015).

17

THE MULTI-SPECIALIST MOVING-IMAGE PRACTITIONER …

257

The crux of this chapter is to make a case for the education of multispecialist practitioners who are able to thrive in a variety of moving-image scenarios. They might be called in to work on a film shoot for a couple of days as a lighting assistant, then, with that job done, they might complete post-production on a training video they had shot earlier in the week, under the banner of their own business, for a local company’s website, all the while building, in their spare time, an interactive art piece for an upcoming exhibition. Portfolio Career Layers Tomorrow’s moving-image craftspeople should be able to move with facility between film-crew work, contracted work in a more holistic maker role, and personal artistic work. These three moving-image contexts vary in the degree of change to which they are subject, from the least (film-crew work) to the most (personal artistic work). Film-crew work. This involves filling one of a number of singlespecialisation roles on a hierarchal team, shooting productions bound for cinema, broadcast/cable TV, and streaming services. Entry-level positions include production assistant, lighting assistant, boom operator, and camera PA. “Entry level” here refers to roles in which a skilled crew member can begin work on film sets. Virtually all traditional film-crew roles are skilled positions and require training that often begins in school and continues out on the set. Change Factors The roles and responsibilities of a conventional film crew have changed very little in the past 80 years. While methods have evolved, most core crew roles are still very similar to what they were in the early days of filmmaking. The most significant change, the digitisation of moving image, has had reverberations throughout the entire workflow—from the technical specifics of image acquisition to the multiple methods for delivering programme content. And many new tools have been developed to light a set, to move a camera, to capture and process data, and to manipulate footage. For example, some visual effects, heretofore solidly in the domain of post-production, have gone live and become in-camera effects as massive high-resolution LED panels surrounding a set replace blue and green screen for shooting set backgrounds. New jobs are continuing to be created in film production. However, the core roles on every film set (camera, lighting, grip, sound, AD, etc.) are, for all intents and purposes, the same as they were “back in the day”.

258

D. WAGNER

Mounting production costs in an unstable world economy, a more fickle market for long-form narrative, the decrease of international travel, rising energy costs, and an uncertain public-health picture have put feature and episodic production on less solid ground. There are thousands of skilled craftspeople around the world who work crew, but for most, it does not provide income steady enough to be relied upon as a sole source. Many people trained in a single film specialisation find only occasional opportunities to practice their craft because, for the majority, it is not steady work. These craftspeople need additional income streams in order to live. If they were trained in an array of crafts to enable them to generate digital content in their own time, they would be using their moving-image skills in a variety of ways.

Contracted work in a more holistic maker role, either as a sole practitioner or as part of a small collaborative team. In this form, the craftsperson creates content for a variety of situations from corporate communications to internet ads for SMEs; from directing and shooting music videos to shooting and editing wedding videos. It can include content creation with emerging technologies or working with an app developer or web designer to construct new interactive communications assets for future-thinking clients. Change Factors The digital revolution has given rise to many new platforms and uses for moving image. There is a growing need for digital content (Bottomley, 2019; Pophal, 2018) and with this growth comes much opportunity. Over the past 30 years, new information/communication platforms have been developed and employed at an increasing rate (Davis, 2020). This has fostered new contexts for the use of moving image. Video has become the go-to choice for communications among individuals, groups, and businesses, with applications ranging from instructional and corporate communications to promotional and short-form narratives. Increasingly, extended-reality platforms are finding their place within this landscape. Location awareness and context awareness are beginning to enter into the mix. Eye tracking has joined the fray and will be increasingly employed in concert with moving-image assets to gather feedback on content effectiveness. There is no reason that eye tracking cannot be utilised as a creative tool, as well as one to measure engagement. A multi-specialist moving-image craftsperson might be called upon to design a website or an app to contain a group of videos. Or they might

17

THE MULTI-SPECIALIST MOVING-IMAGE PRACTITIONER …

259

be asked to create video content for an AR app or to incorporate into any of the numerous other emerging moving-image-related technologies. So change is present in this arena as well.

Personal artistic work. Here, the moving-image craftsperson carves out space and time for themselves to explore moving-image media and employ experimental techniques to express their inner selves. This work may or may not be shared immediately upon creation. But after time, the artist may find a conduit to put their work out. They might start by giving it away, freely sharing under a creative commons license in the spirit of the sharing economy. As they develop a fanbase, they may then begin charging for the premium stuff. Change Factors Individual artistic expression harbours the greatest opportunity for change. With no briefs or deadlines from external clients to satisfy, makers are free to experiment. They are more light-footed and able to respond quickly to emerging tools and forms of expression, so creative work can shape-shift with each iteration. This is the space where innovation occurs unbridled as artists can explore oblique relationships between concepts and forge new ground synthesising platforms and methods into new amalgamations.

More Tools, Makers, Uses, and Usages So what are some of the expanded avenues for moving-image practice? Augmented Reality, which is yet to hit its stride as a storytelling platform, offers rich performative possibility in the prospect of, for example, live performer interacting with virtual performer, both of whom are interacting with the user. There is much to be mined and new languages to construct with AR. Meanwhile, the platform has already found its way into the moving-image creation process, from production design pre-visualisation to real-time monitoring of composited images (Escobar, 2017). The trend toward increased audience creative engagement in postdelivery content is also evidenced with Virtual Reality, where the viewer controls not only which direction they’re looking but even, sometimes, where they want to go next in the story. In the evolving conventions around VR narrative storytelling, there is little to no inner-scene editing; this would be jarring to the user, who is virtually in the scene. So it is not possible to either vary the scene’s pacing or to change the size of any

260

D. WAGNER

given shot except within the action of the scene itself. With conventional narrative filmmaking, lens size and editing choices are tools for creatively managing audience attention. In VR filmmaking, audience attention must be managed on set. Other technologies that show promise for expanding and enhancing the audiovisual experience are entering the fray as well. Emerging affordances—such as a skin-integrated wireless haptic interface for AR and VR (Yu et al., 2019); Deep Scent technology (Hariri et al., 2016); Deep Dream software (Spratt, 2018); Brain-Computer interfaces (Tiwari et al., 2018); Computer Vision, which involves the ability of computers to achieve understanding from analysing digital images or videos (Huang, 1996); Ambient Intelligence, in which various computational devices work together in an environment that is responsive to the people within it (Cook et al., 2009)—and tools already in their early stages of employment, such as Facial Recognition; Gesture Recognition; E-Textiles; Location Awareness; Deep Fake; and various other manifestations of AI—hold great potential as instruments of creative expression. What new vocabularies will these tools engender? Might a reimagined and repurposed screen education serve as a digital Rosetta Stone, a key to unlock new families of cinematic language? Developments in moving-image storytelling technologies and affordances are not limited to creation tools and methods. How we receive content, as well as what we are doing with it, is also transforming and is transforming us. McLuhan theorised that the development of each significantly new information platform (in modern terms) is accompanied by a development in our cognitive processes. Content has carved multiple delivery pathways into our lives, and we’re learning to engage with it in new ways. It’s been speculated that the transformations across our lives brought on by digitisation are forging new neural pathways for our communication capacities. Mobile in the Mix Sitting alongside these emerging tools—both of them and apart from them—is the cornucopia of opportunities afforded by the handheld mobile device. Mobile provides an enabling platform for other technologies and cuts across such boundaries as geography and skills training. Mobile cameras are excellent tools for teaching and learning moving-image craft, useful

17

THE MULTI-SPECIALIST MOVING-IMAGE PRACTITIONER …

261

for visual diaries, for practising framing and composition, as a shot previz sketch pad, as a director’s finder, and even as an image capture device (a.k.a. camera). With mobile, anyone can make a movie anytime, anywhere. Its power as a collaborative tool enables people across the planet to make movies together instantly. Whether they realise it or not, every smartphone user carries an international community in their pocket. There is little need to convince readers of this volume about the many potentialities offered by the mobile device and by the ever-growing ocean of apps that facilitate a multitude of affordances. Suffice to say, though, that just as the craft of moving image is now in an ever-developing state, so is mobile as a platform, continually evolving to both accommodate, and to lead, an army of creators on all levels of resource and output. Mobile has, of course, become an important tool in moving-image acquisition, but in that it is also a fine image manipulation and post-production tool, as well as being a tool for delivering and for viewing moving-image work, it is, in effect a complete sub-system in and of itself. This is not an insignificant element in the larger scope of moving-image evolution. There will always be stories to tell, spun through evolving tools that will, in turn, continue to evolve the visual sophistication of audiences. And so cinematic storytelling (in whatever form) is likely not going away. However … In what other un-thought-of ways might today’s students, and subsequent generations—born connected and spreading content as a way of life—engage with visual media? And what types of careers will arise around what will surely be transforming uses for visual media?

Career Perspectives In the process of mapping employment in the creative workforce, Higgs and Cunningham (admittedly 13 years ago) identified three distinct “occupational situations” within which cultural sector jobs occur. These are: 1. Creative occupations within the core creative industries, which they call “Specialist Creatives”; 2. Non-creative occupations within the creative industries, which they refer to as “Support Workers” (business and managerial occupations, for example, such as running a small arts festival); and

262

D. WAGNER

3. Creative occupations outside of the creative industries, which they call “Embedded Creatives” (one example could involve creating instructional videos for a large manufacturer). Based on analysis of the job distribution data from among these three segments within the Australian economy at the time of the study, between the Specialist and the Embedded creative jobs, the majority of work falls in the Embedded category, which indicates that there are more people working in creative occupations outside of specifically creative industries than there are people working in creative occupations inside the creative industries (Higgs & Cunningham, 2008). In that most work within the creative industries is built largely around providing what can be classed as “entertainment product”, it is reasonable to speculate that those businesses outside of the creative industries, engaged in other sectors of the economy, use creatives to generate material that is more informational in nature. This signals great possibilities for moving-image makers to find and/or create opportunities for themselves as the need for visually presented information increases. Consider, for a moment, the notion of post-literacy, the idea that literacy is doomed but will be replaced with something more powerful. Michael Ridley, who defines post-literacy as “the state in which reading and writing are no longer a dominant means of communication”, explains that “while the idea of a post-literate world is shocking and disturbing to some, perhaps many, it is presented here not as a regression into some new Dark Age but rather as a powerful way forward which will dramatically expand our capabilities” (Ridley, 2009). This begs speculation around the degree to which a post-literate culture will increasingly rely on moving image as a platform by which to receive essential information that once had been solely the province of the written word. There is active research, for example, around the potential of video games to enhance cognitive ability (Martinˇcevi´c & Vrani´c, 2019).

Where to with Screen Education? Studies spanning the last 22 years suggest that creative careers increasingly involve working across multiple sectors and disciplines. As discussed above, workers in the creative industries often derive their livelihood from multiple income streams, each of which represents a slightly different skill set (Hopson & Ledger, 2010; Mallon, 1999; Wilson, 2015).

17

THE MULTI-SPECIALIST MOVING-IMAGE PRACTITIONER …

263

The rapidly evolving arena of digital content creation is operating under new paradigms, and the successful fledgling creative practitioner needs to demonstrate a far greater range of capabilities than the twentiethcentury creative or technician. Not only do students bound for careers in the creative industries need to learn to harness their creativity with deep disciplinary knowledge and technical skills but they also must be taught how to manage their own careers as well as how to grow those careers through enterprise and entrepreneurship (Bridgstock, 2013; Daniel & Daniel, 2015). Change is occurring at a rate that precludes the possibility of emerging fully formed from any post-secondary education or training. So what is our new role as moving-image educators? Perhaps it is to broaden students’ knowledge and skill base to include disciplines that both inform moving image and are informed by moving image. These areas include art—both contemporary art as a discipline and immersion in the historical marriage between art and technology; design—both as a set of disciplines and as a way to think; ICT skills to empower the creative technologist within; and business and entrepreneurship, which includes a bit of advertising, marketing, and promotion. Perhaps it is time to lift the borders of discipline to allow more flow between them. In the new landscape, there are many connecting lines between disciplines that tie together previously discrete fields. These connections call for a lateral approach toward integrating the once-separate studies into new synergistic areas. In order to be armed with capabilities in a range of areas, the multi-specialist moving-image creator will need theoretical and hands-on exposure to these domains. If they are to enjoy dynamic and sustainable careers, moving-image students will need to continually create work that is vital and original in order to keep a competitive edge. Immersion in a blend of historical context and forward-thinking vision will enable the fledgling moving-image craftsperson to experience themselves as being at the wavefront of unfolding cultural history. Art Moving-image work invokes a synthesis of the technical and the artistic. Technical aspects are well covered in most existing moving-image programmes. But for a moving-image creator to exercise full creative command over their work, they will need to learn what it is to be an

264

D. WAGNER

artist by immersing in a contemporary arts worldview. Folding contemporary art into a moving-image programme would integrate not only theory and technique but also the iterative, incremental process of art making. Conventional film methodologies are linear by nature. How can we add a cross-current of iterative making into the world view of the movingimage craftsperson? How might this colour the ways media makers look at their making process and therefore their output? What strategies might we employ to nurture the right brain of the maker? A layer of art education and experience will go a long way toward this end. Technology If tomorrow’s graduates are to contribute to digital culture, they will need conversancy in computing basics. This should include some elementary coding (in languages such as HTML, HTML5, javascript, OpenGL, and C++); working knowledge of key development platforms (such as Velocity and Unity); and proficiency in current basic creative software (all Adobe CC applications, Resolve, LogicPro, etc.). It would of course behove the moving-image content creator to know something about app design and development, which would sit on top of their foundational knowledge of these basic computing principles. Art + Technology To further contextualise the above two strands, moving-image students will benefit from an exploration of the rich synergistic history that is the blending of art and technology, a convergence that has never been more widespread than it is today. There are hundreds of examples of the use of both mechanical and emerging digital platforms as media for creative expression. This module would focus on art movements, collectives, and practitioners in the twentieth and twenty-first centuries—movements such as futurism, Dada, surrealism, kinetic art, and conceptual art; and artists such as Marcel Duchamp, Man Ray, Len Lye, Andy Warhol, Christo, Nam June Paik, the Fluxus movement, Chris Burden, Stelarc, Mark Pauline and, more recently, Aaron Koblin, Chris Milk, and Ken Rinaldo, among others—who, in one way or another, have used technology (including the body as technology) as an artistic vehicle. Students would then be given a provocation to venture into uncharted waters with their own explorations in artistic–technological synthesis.

17

THE MULTI-SPECIALIST MOVING-IMAGE PRACTITIONER …

265

Design Not enough can be said about the benefits that can result from bringing design into the world of the moving-image maker. This includes both the elements and principles of design and also the virtues of the analytic and creative process in which designers engage. A progressive moving-image curriculum would introduce design both as a set of disciplines (especially graphic design, product design, and UX design) and as a process (the design thinking framework). In order to remain adaptable to morphing media and economic landscapes, graduates should be empowered with the ability to identify, frame, and resolve issues that might arise through change. The elements and various facets of design thinking are well documented, and involve an iterative cycle of ideation, prototyping, testing, and evaluating. But suffice to say that design thinking encompasses a set of skills vital to remaining agile in today’s world. Business Were business to be integrated into a moving-image programme, it should be done from different angles. As an employee living from project to project, survival in the gig economy is a skill set that needs to be emphasised to prepare graduates for the unfamiliar world of independent contracting and freelance work (Mulcahy, 2019). Those who seek to derive income from their own independent creative practice need to know the basics of being and running a business (Haine, 2019). This encompasses both the standard conventions of business operation and a look at new and emerging, particularly disruptive, business models. To this end, moving-image students need to learn the ins and outs of forging their own entrepreneurial path through the assessment of risk and opportunity (Damásio & Bicacro, 2017; Neff et al., 2005). Failure to address these concerns is to usher the graduating moving-image craftsperson out into the world to fend for themselves with little-to-no capabilities in the art of economic survival. Discipline skills are only one part of the picture. Advertising, Marketing, and Promotion The skills of outreach and promotion are vital not only for the practitioner themselves to maintain a continuous inflow of clients but also to inform the production of ad content that they might find themselves developing

266

D. WAGNER

for their clients. To this end, it is suggested that along with a module related to business and entrepreneurship, a unit covering advertising, marketing, and promotion be woven throughout a new moving-image curriculum. Agility It is not only the skills and knowledge of these discrete areas that this research suggests imparting—it’s the world views that drive these disciplines that we see as so important to weave into an integrated creative practice. On one hand, what I’m talking about here is arming tomorrow’s content architects with a cornucopia of skills; a full set of brushes and a palette of many colours with which to address the changing creative needs before them. And also, we’re opening the topic of a new way of looking at curriculum for moving-image content creation. In order to teach multispecialism and agility, this approach to curriculum must transcend current institutional silos, and it also must be able to itself remain agile, able to adapt quickly to changes in the landscape. But this is, of course, uncertain territory. As Scott Dierdorf points out in his book Becoming Agile, “There’s very little predictability anymore in change. And you can’t deal with constant change using the old tools. Innovation means doing something you’ve never done before, which means you’re learning as you go” (Dierdorf, 2019). We must develop new ways of looking at the teaching-and-learning of moving-image content creation in our quest to most faithfully prepare graduates for a morphing future. One possible starting point could be to take a page from the agile development movement, which is an iterative set of guiding methods for building a complex system for release into a dynamic, changing environment. Existing research has looked at a variety of marriages between agile and education (Bojorque & Pesántez, 2019; Ciupe et al., 2018; Fontdevila, 2017; Palmer et al., 2016; Parkin et al., 2020; Sharp & Lang, 2018) but its application to a dynamic moving-image curriculum has yet to be explored. Agile development is based on a set of ideas and principles that value human interaction over technology for its own sake and is designed to respond quickly to external changes. It is structured around self-organising teams rather than a rigid hierarchy and, as such, seems

17

THE MULTI-SPECIALIST MOVING-IMAGE PRACTITIONER …

267

a good fit for continual development and redevelopment of a dynamic curriculum that facilitates teaching and learning a continually changing base of knowledge and skills. How useful might the agile development model be as a framework for rolling screen-curriculum development?

Challenges It is acknowledged that this solution, on the face of it, is problematic within the current structure of most institutions of higher learning. Resistance to change, programme funding structures, regulations, and prerequisites tied to qualifications, and the problem of encapsulating such a programme to clearly convey its essence and make it attractive to potential enrolees are some of the challenges facing implementation. But what is clear is that screen education as it currently exists is, for the most part, not adequately serving the needs of today’s changing world. The literature is replete with discussion around the need to impart “twenty-first-century skills” in order to arm students with capabilities to tackle increasingly complex challenges (Boyles, 2012; Brookhouser, 2016; National Research Council, 2012; Pheeraphan, 2013). This has, in turn, prompted calls to reimagine pedagogy to effectively teach these newer skills (Bridgstock, 2013; National Research Council, 1997; Perry, 2018; Peters & Besley, 2013; Scott, 2015). Further, implicit in the discussion of the twenty-first-century skill set is graduates’ ability to continually adjust to, and eventually drive, change. Each year, film schools will be sending moving-image content creators out into an evolving landscape. We need to prepare them for change, which, by necessity, will require them to innovate. For this, they will need a foundational storehouse of inspirations, references, skills and capabilities. They will need to aspire to become multi-specialist visual experience designers with a big basket of tools. And in order to begin thinking about incubating innovators, we ourselves need to promote innovation within the education system that is meant to facilitate their development.

268

D. WAGNER

References Blair, H., Grey, S., & Randle, K. (2001). Working in film—Employment in a project based industry. Personnel Review. https://doi.org/10.1108/004834 80110380334 Bojorque, R., & Pesántez, F. (2019). Curriculum design based on agile methodologies. In Advances in human factors in training, education, and learning sciences—Proceedings of the AHFE 2018 International Conference on Human Factors in Training, Education, and Learning Sciences, 2018 (Vol. 785). Springer Verlag. Bottomley, A. (2019). Giant pools of content: Theorizing aggregation in online media distribution. JCMS: Journal of Cinema and Media Studies, 59(1), 149– 156. https://doi.org/10.1353/cj.2019.0070 Boyles, T. (2012). 21st century knowledge, skills, and abilities and entrepreneurial competencies: A model for undergraduate entrepreneurship education. Journal of Entrepreneurship Education. Retrieved September 29, 2019, from http://link.galegroup.com/apps/doc/A289621049/AONE? sid=lms Bridgstock, R. (2013). Professional capabilities for twenty-first century creative careers: Lessons from outstandingly successful Australian artists and designers. International Journal of Art & Design Education, 32(2), 176–189. https:// doi.org/10.1111/j.1476-8070.2013.01756.x Bridgstock, R. (2016). Educating for digital futures: What the learning strategies of digital media professionals can teach higher education. Innovations in Education & Teaching International, 53(3), 306–315. https://doi.org/10. 1080/14703297.2014.956779 Brookhouser, K. (2016). Teaching future-ready students. LinkedIn Learning. Ciupe, A., Ionescu, R., Meza, S., & Orza, B. (2018). Towards agile integration within higher education: A systematic assessment. BRAIN: Broad Research in Artificial Intelligence & Neuroscience, 9(3), 69. Codreanu, A. (2016). A VUCA action framework for a VUCA environment—Leadership challenges and solutions. Journal of Defense Resources Management, 7 (2), 31–38. Cook, D., Augusto, J., & Jakkula, V. (2009). Ambient intelligence: Technologies, applications, and opportunities. Pervasive and Mobile Computing, 5(4), 277– 298. https://doi.org/10.1016/j.pmcj.2009.04.001 Damásio, M., & Bicacro, J. (2017). Entrepreneurship education for film and media arts: How can we teach entrepreneurship to students in the creative disciplines? Industry and Higher Education, 31(4), 253–266. https://doi. org/10.1177/0950422217713110 Daniel, R., & Daniel, L. (2015). Success in the creative industries: The push for enterprising and entrepreneurial skills. Journal of Australian Studies, 39(3), 411–424. https://doi.org/10.1080/14443058.2015.1046896

17

THE MULTI-SPECIALIST MOVING-IMAGE PRACTITIONER …

269

Davis, C. (2020). The rise of technology in filmmaking. Raindance. Retrieved October 19, 2020, from https://www.raindance.org/the-rise-of-technologyin-filmmaking/ Dierdorf, S. (2019). Becoming agile. Lulu.com. Escobar, E. (2017). Hacking film: How augmented reality tech is revolutionizing film production. Film Independent. Retrieved October 8, 2018, from https://www.filmindependent.org/blog/hacking-film-how-augmentedreality-tech-is-revolutionizing-film-production/ Fontdevila, D. (2017). Tales from an agile journey: Designing curricula for millennials in industry and academia. 2017 IEEE/ACM 1st International Workshop on Software Engineering Curricula for Millennials (SECM), 2–2. https://doi.org/10.1109/SECM.2017.11 Haine, C. (2019). Business and entrepreneurship for filmmakers: Making a living as a creative artist in the film industry. Routledge. Hariri, S., Mustafa, N. et al. (2016). Electrical stimulation of olfactory receptors for digitizing smell. In Proceedings of the 2016 workshop on Multimodal Virtual and Augmented Reality, MVAR ’16 (pp. 1–4). Association for Computing Machinery. Higgs, P., & Cunningham, S. (2008). Creative industries mapping: Where have we come from and where are we going? Creative Industries Journal, 1(1), 7–30. https://doi.org/10.1386/cij.1.1.7/1 Hopson, B., & Ledger, K. (2010). And what do you do? 10 steps to creating a portfolio career. A&C Black. Huang, T. (1996). Computer vision: Evolution and promise. CERN School of Computng, 1996, 21–25. https://doi.org/10.5170/CERN-1996-008.21 Mallon, M. (1999). Going “portfolio”: Making sense of changing careers. Career Development International, 4(7), 358–370. https://doi.org/10.1108/136 20439910295727 Martinˇcevi´c, M., & Vrani´c, A. (2019). Can video-games enhance cognitive ability? Psychological Topics, 28(3), 507–528. Mulcahy, D. (2019, October 3). Universities Should Be Preparing Students for the Gig Economy. Harvard Business Review. National Research Council. (1997). Preparing for the 21st century: The education imperative. 9780309584678. National Academies Press. National Research Council. (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. Neff, F., Wissinger, E., & Zukin, S. (2005). Entrepreneurial labor among cultural producers: ‘Cool’ jobs in ‘hot’ industries. Social Semiotics, 15(3), 307–334. https://doi.org/10.1080/10350330500310111 Nincarean, D., Alia, M. B., et al. (2013). Mobile augmented reality: The potential for education. Procedia - Social and Behavioral Sciences, 103, 657–664. https://doi.org/10.1016/j.sbspro.2013.10.385

270

D. WAGNER

Palmer, I., Ralley, J., & Davenport, D. (2016). Agile curriculum design for the creative industries. https://diglib.eg.org:443/xmlui/handle/10.2312/ eged20161026 Parkin, I., Cezayirli, E., et al. (2020). Agile development of an anatomy curriculum in a new graduate-entry medicine degree. Journal of Anatomy, 236, 255–255. Parnham, P. (2019). New Zealand cinematographers society—Changing the rules: The devil in the detail. New Zealand Cinematographer’s Society. https:// www.nzcine.com/7786628 Perry, M. (2018). 21st century skills through film production in tertiary education: A transformative assessment in a literature and media course. 3L: Language, Linguistics, Literature ® 24(4). https://doi.org/10.17576/3L2018-2404-16 Peters, M., & Besley, T. (2013). Re-imagining the creative university for the 21st century. Brill | Sense. Pheeraphan, N. (2013). Enhancement of the 21st century skills for Thai higher education by integration of ICT in classroom. Procedia—Social and Behavioral Sciences, 103, 365–373. https://doi.org/10.1016/j.sbspro.2013.10.346 Pophal, L. (2018). Video popularity still on the rise: opportunities for publishers. Econtent, 41(4), 4–8. Ridley, M. (2009). Beyond literacy: are reading and writing doomed? In Pushing the edge: Explore, engage, extend (pp. 210–213). Association of College and Research Libraries, American Library Association. Ryan, M. D., & Hearn, G. (2010). Next generation ‘filmmaking’: New markets, new methods and new business models—C33169.pdf. Media International Australia, 136. http://eprints.qut.edu.au/33169/1/c33169.pdf Salvador, E., Simon, J. P., & Benghozi, P. J. (2019). Facing disruption: The cinema value chain in the digital age. International Journal of Arts Management, 22(1), 25–40. Scott, C. L. (2015). The futures of learning 3: What kind of pedagogies for the 21st century? Sharp, J. H., & Lang, G. (2018). Agile in teaching and learning: Conceptual framework and research agenda. Journal of Information Systems Education, 29(2), 45–51. Spratt, E. L. (2018). Dream formulations and deep neural networks: humanistic themes in the iconology of the machine-learned image. ArXiv:1802.01274 [Cs]. Stewens, S. (2017). The importance of digital filmmaking and how it affects education in filmschools. International Journal of Film and Media Arts. https://revistas.ulusofona.pt/index.php/ijfma/article/view/6156 Stoneman, R. (2014). The future of film training. Policy & Practice: A Development Education Review, 18(Spring), 30–38.

17

THE MULTI-SPECIALIST MOVING-IMAGE PRACTITIONER …

271

Tiwari, N., Edla, D. R., et al. (2018). Brain computer interface: A comprehensive survey. Biologically Inspired Cognitive Architectures, 26, 118–129. https:// doi.org/10.1016/j.bica.2018.10.005 Wilson, C. (2015). The rise of the portfolio career in the creative industries [Information Portal for Asia-Europe Foundation]. Culture 360. http://culture360. asef.org/magazine/the-rise-of-the-portfolio-career-in-the-creative-industries/ Yu, X., Xie, Z., et al. (2019). Skin-integrated wireless haptic interfaces for virtual and augmented reality. Nature, 575(7783), 473–479. https://doi.org/10. 1038/s41586-019-1687-0

Index

A Algorithmic composition, 16, 180–182, 184–186 App, 3, 16, 28, 32, 48, 69, 100, 106, 110, 111, 139–143, 145–147, 179, 180, 183, 185, 258, 264 Artificial Intelligence (AI), 20, 37, 44–47, 50, 260 C Camera-stylo, 193, 194, 202 Challenges, 2, 10, 60, 100, 105, 108, 137, 166, 186, 188, 223, 267 Change Factors, 257–259 China image, 85–87, 89 Citizen journalism, 51, 99–101 Citizen news, 99–104, 107–112 Co-creation, 30, 43, 120, 122, 131, 132, 155 Co-design, 120–123, 126–132 Community arts, 60 Community engagement, 120, 121, 132 Compositional Strategies, 183

COVID-19, 4, 50, 56, 63, 69–71, 74, 80, 109, 124 Creative arts, 10, 16 Creative Practice Research (CPR), 171, 172

D Digital stories, 21, 121–123, 125, 126, 129–132, 152, 154–158, 160, 161, 163, 164, 166, 167 Digital wayfarer, 196, 198, 202, 203 Douyin, 5, 14, 69, 71, 73, 74, 79, 106–109

E Effect processing, 180, 184 Embodiment, 216 Emerging Media, 5, 43, 45, 227 Empowered, 29–31, 40, 126, 156, 265 Empowering, 30, 132 Empowerment, 25, 34, 64

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 M. Schleser and X. Xu (eds.), Mobile Storytelling in an Age of Smartphones, https://doi.org/10.1007/978-3-030-87247-2

273

274

INDEX

Engaged, 29–31, 74, 119, 126, 127, 141, 158, 159, 196, 217, 262 Engagement, 11, 13, 15, 17, 34, 41, 42, 50, 71, 73, 77, 122, 125, 130, 152, 153, 156, 206, 211, 214, 216–218, 255, 258, 259 Engaging, 10, 30, 103, 120, 126, 131, 198, 226, 244 Enhanced, 29, 30, 32–34, 79 Enhancement, 26, 33, 34, 178 Enhancing, 27, 30, 32, 34, 56, 60, 260 Enlightened, 29–31 Enlightening, 30 Enlightenment, 34, 73, 79 Entertained, 29–31 Entertaining, 30, 60, 109 Entertainment, 4, 11, 20, 30, 34, 41, 43, 70, 75, 78, 89, 102, 112, 122 Enticed, 29, 31 Enticement, 34 Enticing, 30 Experience, 11, 14, 17, 19, 28–34, 47, 56, 64, 79, 122, 125–127, 131, 136, 138, 141, 143, 152, 156, 161, 164, 196, 199, 201, 206, 207, 211, 213, 214, 216, 217, 224, 229, 233, 244, 263, 264, 267

F Framing, 71–73, 75, 79, 90, 94, 261 Framing devices, 72–75, 78, 86, 94

G Going mobile, 107, 109, 112 Google maps, 37–44, 47, 49, 50

I Immersive Narrative Experience, 47, 79, 206, 214 Immigration, 60, 64 Instrumental performance, 179 Interactive Spectatorship, 211 Interdisciplinary Studies, 3 International Mobile Storytelling Congress, 1 iPhone, 3, 14, 16, 38–40, 43, 47, 49, 129, 160, 180, 194

M Mapping, 16, 33, 38–43, 45, 48, 50, 201 Media convergence, 99, 104–110, 112 mHealth services, 137, 139–141, 146 Mixing, 61, 177, 180, 181, 183–187, 191 Mobile apps, 139, 145–147 Mobile Assisted Language Learning (MALL), 221, 222, 240–249 Mobile cinema, 60, 64 Mobile education, 3, 239 Mobile experience, 29–34 Mobile health, 136, 137 Mobile Innovation Network & Association (MINA), 4, 133 Mobile media, 1–3, 5, 9, 10, 12, 14, 17–20, 29, 73, 177, 185, 197 Mobile montage, 16 Mobile music, 5, 16, 174–180, 183–185, 187, 188 Mobile Screen Stories, 10 Mobile Short Video, 4, 69, 76, 77, 79, 80 Mobile stories, 1, 5, 9–21, 29, 30, 34, 132, 158, 161, 165 Mobile storytelling, 3, 5, 9–11, 20, 26, 27, 29, 30, 32–34, 69, 119,

INDEX

123, 125, 129, 132, 152, 153, 156, 184 Mobile Studies Congress, 4, 10, 20 Mobile Studies International (MSI), 4 Mobile technology, 108, 166, 224 Moving Image Media, 205, 210 Moving Image Practitioner, 253 Multimodal framing, 73 Multisensory Encounter, 207 Multi-specialist, 257, 258, 263, 267

N Narratives, 11, 16, 19, 21, 156, 183, 196, 206, 207, 210, 211, 214, 217, 218 Nation branding, 91, 94 News APP, 100, 106, 108, 110–112 News framing, 69, 71–73, 79 News media, 69–71, 75, 79, 101, 106

O Omnimedia, 28, 34, 106

P Performative storytelling, 177 Post-production, 62, 161, 177, 183–185, 187, 257, 261 Pre-production, 158, 165, 177–179, 187

R Rural communities, 152, 153, 156, 157, 159 Rural Experience, 159

275

S Screen education, 254–256, 260, 267 Self-regulation, 242 Sentiment, 86–89, 92–95 Sequencing, 17, 178, 180, 182, 184, 185, 187 Smart mobile devices (SMDs), 221, 225–234 Smartphone filmmaking, 10, 11, 17, 18, 121, 125, 132, 153, 156 Smartphone production, 120 Spectatorship, 206, 208, 209, 211–218 Story structure, 10, 12, 19, 132, 155, 158 Storytelling elements, 26, 27, 33, 34 Storytelling evolution, 27 Storytelling Experience, 29 T Technology acceptance, 224, 225, 227, 229, 232, 233 Touch Interaction, 209 Tweets, 86–93, 95 Twitter, 49, 70, 85–95, 105 V Vulnerable communities, 3, 120, 122, 130 W Web series, 156, 165 WeChat, 99, 106–108 Weibo, 79, 99, 106–109 Women mHealth, 136, 140, 147