Multimodal Im/politeness: Signed, spoken, written 9027213437, 9789027213433

Table of contents :
Multimodal Im/politeness
Editorial page
Title page
Copyright page
Table of contents
Preface
Chapter 1. Multimodal im/politenessIntroduction
1.Why multimodal im/politeness?
2.What is “im/politeness”?
3.Co-construction of im/politeness through multiple modalities
4.What does multimodality contribute to the study of im/politeness?
5.Overview of the book
6.Looking ahead
References
Part I. Signed
Chapter 2. In your face. Im/politeness in signed languages, with examples from Swiss German Sign Language (DSGS)
1.Introduction
2.The factor of visibility in im/polite signing
2.1Always face-to-face
2.2Getting the addressee/s’ attention
2.3The addressee’s eye gaze
2.4The addressee’s backchannelling behaviors
2.5The signer’s eye gaze
2.6The onlooker’s eye gaze
3.Other factors affecting im/politeness in signing
3.1Setting and register
3.2Speech act and the addressee’s hearing or signing status
3.3Cultural factors in im/polite signing
3.3.1Differing connotations of a sign in other cultures or historical periods
3.3.2Deaf signers living in both a deaf-signing and a hearing-speaking culture
4.Examples of im/politeness perceived especially by DSGS signers
4.1Methods of collecting these DSGS examples
4.2Examples of signing perceived as impolite “bragging”
4.2.1Bragging and the use of “mouthings”
4.2.2Bragging by volunteering too much personal information
4.2.3Bragging by modifying phonological components
4.2.4Bragging by using too many “pedagogical” signs
4.3Discussion: Self praise perceived as a form of im/polite bragging
4.3.1Self-praise perceived as bragging in different cultures
4.3.2Swiss modesty and a heightened aversion to bragging
5.Summary and suggestions for further research
Acknowledgements
References
Appendix A.The complex sub-groups of signers according to modality of language and when it was learned
Appendix B.A historical example of the low opinion about signing held by hearing persons
Chapter 3. Politeness in Catalan Sign Language (LSC). Mitigation of criticisms in spontaneous discourse in LSC
1.Introduction
2.Background
2.1Politeness, face, and face-work
2.2Mitigation strategies: Attenuation and opinion expression
2.3Previous research on politeness in sign languages
2.4Non-manual markers, doubling and vertical palm sign
3.Methodology
4.Results
4.1Expressions for restricting opinions with lexical units
4.2Expressions for restricting opinions with pointing signs
4.3Expressions for restricting opinions with vertical-palm sign
4.4Summary
5.Discussion and conclusion
Funding
Acknowledgements
References
Part II. Spoken
Chapter 4. Multimodality in refusals in English as a lingua franca
1.Introduction
2.Background
2.1Approach to politeness
2.2Empirical studies on refusals
2.3Refusals and English as a lingua franca (ELF)
2.4Multimodal politeness
3.Methodology
3.1Participants
3.2Instruments
Scenario 1
Scenario 2
3.3Research procedures
3.4Data analysis method
4.Findings
4.1Body position
4.2Smiling voices/smiling facial expressions
4.3Long gaze aversion
5.Discussion
5.1Multimodal politeness in refusals
5.2Refusals and English as a lingua franca
6.Conclusion
References
Appendix.Transcription conventions (adapted from Jefferson 2004; Mondada 2019)
Chapter 5. Indexing social distance through bodily visual practices in two languages
1.Introduction
2.Background
2.1Universality versus specificity in multimodal politeness
2.2Bodily visual practices analysed in the current study
3.Methodology
3.1Data collection
3.2Procedure
3.3Participants
3.4Coding
3.5Statistical analysis
4.Results
4.1Head nodding
4.2Head shaking
4.3Adaptors
4.4Haptics
4.5Gaze aversion
5.Discussion
6.Conclusion
Funding
Acknowledgments
References
Chapter 6. Multimodal manifestation of ta’ârof in Persian
1.Introduction
2.Linguistic and behavioral ta’ârof
3.Data and method
4.Results
5.Conclusion
Bibliography
Chapter 7. Within-speaker accommodation behavior in apology-centered interactions. The role of socio-pragmatic factors
1.Introduction
2.Background
2.1Vocal accommodation behavior
2.2Function of accommodation behavior
2.3Factors affecting accommodation behavior
2.4Accommodation effects
3.Methods
3.1Participants
3.2Stimuli
3.3Annotation
3.4Measuring accommodation
3.5Statistical analysis
4.Results
4.1F0 variation depending on social distance between the interlocutors
4.2F0 accommodation behavior as a function of social distance
4.3F0 accommodation behavior as a function of speaker gender
4.4Within-speaker analysis of accommodation
5.Discussion and conclusion
Funding
Acknowledgments
References
Chapter 8. Perceptual changes between adults and children for multimodal im/politeness in Japanese
1.Introduction
2.Methods
2.1Listeners
2.2Stimuli
2.3Acoustic and facial analysis
2.3.1Acoustic and facial measurements
2.3.2Acoustic parameters variation
2.3.3Action Units
2.3.4Head movements
2.3.5Gaze direction
2.4Experimental paradigm
2.4.1Experiment 1
2.4.2Experiment 2
2.5Statistical processing
2.5.1Experiment 1
2.5.2Experiment 2
3.Results
3.1Experiment 1
3.1.1Question 1
3.1.2Question 2
3.2Experiment 2
3.2.1Spread of stimuli across the perceptual space
3.2.2Shape of the perceptual space
3.2.3Proportion of shared knowledge
4.Discussion and conclusions
Funding
Acknowledgments
References
Chapter 9. Multimodal markers of irony in televised discourse. A corpus-based approach
1.Introduction
2.The functions of irony-in-interaction and im/politeness
3.Ironic constructions
3.1Tell me about it
3.2Syntactically independent as if clauses
4.Method
4.1Multimodal archive
4.2Procedure
4.3Analysis of multimodal features
4.4Statistical analysis
5.Results and discussion
5.1Tell me about it
5.2Syntactically independent as if clauses
5.3Formulaic as if clauses
6.Conclusions
References
Part III. Written
Chapter 10. Multimodality and subtitles. A focus on im/politeness in Japanese films with French subtitles through the example of some rituals
1.Introduction
2.Multimodality and expression of im/politeness in Japanese
2.1Ojigi (お辞儀) and aizuchi (相槌)
2.2A brief historical review
3.Subtitling
4.Corpus and methodology
5.Analysis
5.1A meaningful contextual environment
5.2Highly ritualized situations
5.3Impact of clothing and behaviors
6.Conclusion
References
Chapter 11. Customer support agents in Spanish live chats. Affective communication and multimodal politeness
1.Introduction
2.Theoretical framework
2.1Politeness and multimodality in digital discourse
2.2Emoticons, other non-verbal modes and emotions
2.3Online professional-layperson communication
3.Methodology
3.1Research data
3.2Approach
4.Analysis
4.1Communication style type with low affective content
4.2Communication style type with medium affective content
4.3Communication style with high affective content
5.Discussion and conclusion
Acknowledgements
References
Chapter 12. “He offered an apologetic smile”The politeness of apologetic gestures
1.Introduction
2.Previous literature
3.Data and method
4.Analysis
4.1Register profiles of apologies
4.2Collocational patterns of apologies and gestures
4.3The politeness of apology gestures
5.Discussion and conclusion
Acknowledgements
Corpora
References
Subject index
Name index
Index of languages and cultures

Citation preview

Multimodal Im/politeness Signed, spoken, written

edi t ed by Andreas H. Jucker Iris Hübscher Lucien Brown

John Benjamins Publishing Company

Multimodal Im/politeness

Pragmatics & Beyond New Series (P&bns) issn 0922-842X Pragmatics & Beyond New Series is a continuation of Pragmatics & Beyond and its Companion Series. The New Series offers a selection of high quality work covering the full richness of Pragmatics as an interdisciplinary field, within language sciences. For an overview of all books published in this series, please see benjamins.com/catalog/pbns

Editor

Associate Editor

Anita Fetzer

Andreas H. Jucker

University of Augsburg

University of Zurich

Founding Editors Herman Parret

Jef Verschueren

Robyn Carston

Sachiko Ide

Sandra A. Thompson

Thorstein Fretheim

Kuniyoshi Kataoka

John C. Heritage

Miriam A. Locher

Jacob L. Mey

University of Southern Denmark

Belgian National Science Foundation, Universities of Louvain and Antwerp

Belgian National Science Foundation, University of Antwerp

Editorial Board University College London University of Trondheim University of California at Los Angeles

Susan C. Herring

Indiana University

Masako K. Hiraga

St. Paul’s (Rikkyo) University

Japan Women’s University Aichi University

Universität Basel

Sophia S.A. Marmaridou University of Athens

Srikant Sarangi

Aalborg University

Marina Sbisà

University of California at Santa Barbara

Teun A. van Dijk

Universitat Pompeu Fabra, Barcelona

Chaoqun Xie

Zhejiang International Studies University

Yunxia Zhu

The University of Queensland

University of Trieste

Volume 333 Multimodal Im/politeness. Signed, spoken, written Edited by Andreas H. Jucker, Iris Hübscher and Lucien Brown

Multimodal Im/politeness Signed, spoken, written

Edited by

Andreas H. Jucker University of Zurich

Iris Hübscher University of Zurich, URPP Language and Space; Zurich University of Applied Sciences

Lucien Brown Monash University

John Benjamins Publishing Company Amsterdam / Philadelphia

8

TM

The paper used in this publication meets the minimum requirements of the American National Standard for Information Sciences – Permanence of Paper for Printed Library Materials, ansi z39.48-1984.

doi 10.1075/pbns.333 Cataloging-in-Publication Data available from Library of Congress: lccn 2022056126 (print) / 2022056127 (e-book) isbn 978 90 272 1343 3 (Hb) isbn 978 90 272 5445 0 (e-book)

© 2023 – John Benjamins B.V. No part of this book may be reproduced in any form, by print, photoprint, microfilm, or any other means, without written permission from the publisher. John Benjamins Publishing Company · https://benjamins.com

Table of contents Preface chapter 1. Multimodal im/politeness: Introduction Lucien Brown, Iris Hübscher and Andreas H. Jucker

vii 1

part i. Signed chapter 2. In your face: Im/politeness in signed languages, with examples from Swiss German Sign Language (DSGS) 27 Penny Boyes Braem and Katja Tissi chapter 3. Politeness in Catalan Sign Language (LSC): Mitigation of criticisms in spontaneous discourse in LSC 65 Berta Moya-Avilés, Gemma Barberà and Carme Bach part ii. Spoken chapter 4. Multimodality in refusals in English as a lingua franca Xianming Fang chapter 5. Indexing social distance through bodily visual practices in two languages Lucien Brown, Iris Hübscher, Hyunji Kim and Bodo Winter chapter 6. Multimodal manifestation of ta’ârof in Persian Farbod Farahandouz and Shima Moallemi chapter 7. Within-speaker accommodation behavior in apology-centered interactions: The role of socio-pragmatic factors Omnia Ibrahim and Iris Hübscher

101

131 163

185

chapter 8. Perceptual changes between adults and children for multimodal im/ politeness in Japanese 213 Takaaki Shochi, Albert Rilliard and Donna Erickson chapter 9. Multimodal markers of irony in televised discourse: A corpus-based approach 251 Claudia Lehmann

vi

Multimodal Im/politeness

part iii. Written chapter 10. Multimodality and subtitles: A focus on im/politeness in Japanese films with French subtitles through the example of some rituals 275 Chantal Claudel chapter 11. Customer support agents in Spanish live chats: Affective communication and multimodal politeness Ester Iyanga-Mambo

297

chapter 12. “He offered an apologetic smile”: The politeness of apologetic gestures Andreas H. Jucker

327

Subject index

353

Name index

356

Index of languages and cultures

359

Preface Research on politeness and impoliteness has so far focused almost exclusively on the linguistic aspects of polite and impolite behaviour. This volume extends the scope and brings together a range of papers that explore the multimodal nature of im/politeness. They are based on the conviction that people communicate polite and impolite attitudes towards each other not only through the words that they exchange but also – and in many cases perhaps even more importantly – through their intonation, tone of voice, their facial expressions, their gestures, the positioning of their bodies towards each other, and so on. The papers in this volume go back to a workshop and a conference panel on the topic of multimodal im/ politeness. The first was the one-day “Symposium on Multimodal Im/politeness: Gesture, Sign and Spatial Configurations” hosted by the University Research Priority Program “Language and Space” at the University of Zurich in October 2020. The second was a panel entitled “Multimodal im/politeness in spoken, sign and written language” at the seventeenth conference of the International Pragmatics Association organised by the University of Applied Sciences in Winterthur in June 2021. Because of the Covid pandemic, both events had to be carried out via online conferencing tools, which added some unexpected new insights to our understanding of the importance of spatiality in communicative exchanges. The current volume brings together a selection of what we believe to be the most significant contributions at these events. Our thanks go to the contributors of this volume for their patience and for their co-operation with our many requests for modifications and revisions, and to the two anonymous reviewers of the book series for their very careful reading of our manuscript and the numerous important suggestions for improvements. We also wish to thank the University Research Priority Program “Language and Space” at the University of Zurich for their support. Andreas H. Jucker, Iris Hübscher and Lucien Brown, December 2022

chapter 1

Multimodal im/politeness Introduction Lucien Brown,1 Iris Hübscher2, 3 & Andreas H. Jucker2 1

Monash University | 2 University of Zurich | 3 Zurich University of Applied Sciences

1.

Why multimodal im/politeness?

Im/politeness is evidently not something that resides only in referential language. Even the most ostensibly polite utterance (e.g., “Thank you very much”) can be rendered in a way that makes it open to evaluation for impoliteness if delivered with specific prosody (e.g., elongation of “very”) or accompanied by certain body movements or gestures (e.g., turning your back, averting gaze and giving the middle finger gesture). This observation applies not just to spoken languages, but also to sign languages where various facial gestures and bodily movements are used for im/politeness-related meanings. Similarly, in writing, we also use punctuation, font and emoji to signal im/politeness, and fictional texts often refer to the multiple modalities of im/politeness (e.g., “She gave a polite smile”). In short, politeness is never just “doing things with words”. The fact that im/politeness is fundamentally multimodal has not gone unnoticed in politeness research. Indeed, P. Brown and Levinson’s (1978, 1987) fielddefining work on politeness universals mentions both prosody and gesture (or “kinesics”) at various junctures, albeit in a cursory way. In addition, the iconic orange book cover of P. Brown and Levinson (1987) features a photograph depicting a multimodal politeness practice, specifically South Indian villagers performing the namaste greeting (pressing palms together with fingers pointed upwards in a gassho gesture, whilst bowing and smiling) to bid farewell to a departing government official. Despite this, detailed analysis of multimodal features has been difficult to find in the im/politeness literature until the last few years. Culpeper (2011a: 151) notes that “non-verbal cues … [receive] relatively little attention in communication in pragmatic studies”. There may be a number of reasons for the way that multimodality has been sidelined. Fundamentally, the focus on the verbal content of spoken languages at the expense of multimodality (and the omission of sign lanhttps://doi.org/10.1075/pbns.333.01bro © 2023 John Benjamins Publishing Company

2

Lucien Brown, Iris Hübscher & Andreas H. Jucker

guages) in politeness research seems to relate to the fact that early work on politeness was led by philosophical work on the nature of spoken language (P. Brown and Levinson 1978, 1987; Leech 1983). In such research, the focus was on explaining the reasons why speakers deviate from Gricean maxims (and, in particular, why speakers choose to use indirect language) rather than on describing the way that politeness emerges in situ. Mapson (2014) furthermore points out that conducting multimodal research can be very challenging for researchers given that it requires many different elements to be brought together into the analysis. The trend for im/politeness research to focus only on verbal politeness is being reversed, however. Indeed, recent years have seen the emergence of a vibrant interest in prosodic, embodied and visual aspects of im/politeness, spawning a range of paper publications in journals such as Journal of Politeness Research, Journal of Pragmatics and Intercultural Pragmatics, amongst other venues. Furthermore, the first handbook in the field of im/politeness research (Culpeper, Haugh and Kádár 2017) contained a chapter on “(Im)politeness: Prosody and Gesture” (L. Brown and Prieto 2017). At the same time, im/politeness in sign languages has emerged as an important research target (e.g., George 2011; Mapson 2014). This volume represents the first attempt to create a book-length treatment of multimodal politeness via an edited collection of papers on multimodality and im/politeness ranging across spoken, signed and written languages. In this introduction, we provide an overview of what we already know about multimodal politeness (Section 3), and argue for the importance of multimodality as a research target in im/politeness research (Section 4). Finally, we provide an overview of the papers included in this volume (Section 5) and a few thoughts about possible future developments of multimodal im/politeness research (Section 6). But, before all of that, let’s pause briefly to introduce the concept of “im/politeness” (Section 2).

2.

What is “im/politeness”?

Politeness first attracted serious attention in the pragmatics literature in the 1970s with the seminal work of Lakoff (1973), Leech (1977, 1983), and P. Brown and Levinson (1978, 1987). This work has come to be known as the first wave of politeness theory (see, for instance, Grainger 2011; Culpeper 2011b; Culpeper and Hardaker 2017; Jucker 2020: Chapter 1). It focused almost exclusively on polite behaviour and tried to map specific linguistic forms to specific politeness functions which explained speakers’ preferences in certain situations for indirect and linguistically more complex utterances (e.g. “Could you pass the salt, please?”

Multimodal im/politeness

instead of the more direct “Pass the salt!”). Some studies, notably P. Brown and Levinson (1978, 1987), focused on what they considered to be universal aspects of politeness behaviour. But it did not take long for a broad range of studies to appear that shifted the focus to the culture-specific aspects of politeness. Ide (1989), for instance, pointed out that in Japanese culture the focus is on discernment, i.e., the speakers’ recognition of their social position and the choice of appropriate linguistic expressions, rather than on the strategic use of politeness to achieve interactional goals. And Wierzbicka (1985) showed that for Polish speakers in many situations direct formulations appear to be more appropriate than indirect ones. In the early 2000s, politeness theory underwent a discursive turn which led to the second wave of politeness theories. Politeness was no longer seen as residing in specific linguistic forms but as resulting from discursive negotiations between speakers and addressees (Grainger 2011; Watts 2003; Locher and Watts 2005). Theoretical definitions of politeness (referred to as “politeness2”) were rejected in favour of emic, culture-specific understandings of politeness (referred to as “politeness1”) (Watts, Ide and Ehlich 1992). At the same time, the scope of research was extended to include impoliteness (Culpeper 2011a) and the neutral ground between polite and impolite behaviour, the so-called politic behaviour (Watts 2003). As a result, this broader perspective on politeness became part of interpersonal pragmatics (see, for instance, the handbook edited by Locher and Graham 2010). The third wave of more recent politeness theoretic work can be seen as a consolidation of the first and the second waves. It allows for frame-based default values of specific linguistic expressions and constructions but recognises that such values must always be contextually verified. There has also been a shift from the largely qualitative methods used by discursive politeness studies to corpus-based, quantitative approaches complementing the qualitative data (see Kádár 2019 for an overview). In the current volume, as we will argue in more detail in the following section, we see the analysis of multimodal resources as essential for an understanding of the discursive and context-specific nature of politeness in different speech communities.

3.

Co-construction of im/politeness through multiple modalities

Until recently, research on im/politeness was almost entirely focussed on verbal politeness. However, an emergent and growing body of research is showing that im/politeness-related social meanings are indexed in complex, robust and crossmodal ways.

3

4

Lucien Brown, Iris Hübscher & Andreas H. Jucker

Work on multimodality in politeness research (and pragmatics more broadly) assumes that context-based interpretations of language use rely not only on situated linguistic choices, but also other choices across different modalities. As noted by O’Halloran, Tan and Marissa (2014), this approach has its origins in the ideas of Halliday’s social semiotic theory (e.g., Halliday 1978), as exemplified by Kress and van Leeuwen (2006) and O’Toole (2011). The fundamental premise is that language and other semiotic systems are systematically organized around discrete choices that can be used to signal what Halliday calls “metafunctions” which construe various layers of meaning, including social relations and interpersonal meanings such as politeness. From this perspective, the functions of verbal language can be compared to meanings which arise from acoustic, gestural, visual and other modalities. Just as the use of an honorific title (e.g., Madam or Sir) is a linguistic choice that indexes a politeness-related meaning, the same can be said for the speaker choosing to bow, curtsy, employ a “polite” voice, or even to wear more formal attire. Although research on multimodality does not tend to place a “limit” on what might count as a potential semiotic cue, it is evident that different modalities can work at different levels. Some aspects of multimodality are closely linked to verbal speech, and/or appear to be more ostensive markers of im/politeness. Prosodic features such as pitch and loudness are integral parts of verbal languages, and many gestures and other body movements are closely synchronized with the production of speech, and/or are explicit markers of politeness (e.g., bowing) or impoliteness (e.g., obscene hand gestures). Similarly, in sign languages nonmanual movements (or “visual prosody”, see Moya-Avilés et al., this volume) are closely tied to speech and to politeness-related meanings (see Mapson 2014), and in written language the use of emoticons and non-standard punctuations are tightly linked to verbal content, and to the explicit communication of im/politeness. At the other end of the spectrum, we can think of factors such as bodily position, bodily attire, and adornment, or even the physical arrangement of a room, as semiotic resources that can signal politeness-related meanings. Although some of these may be considered ostensive markers of politeness (e.g., having an honoured guest sit in the most prominent or comfortable seat), evidently none of these factors are synchronized with speech. They may also be constrained by or used to index completely different factors (e.g., bodily attire may be constrained by financial factors and may be used to express style or fashion). However, as pointed out by Sperber (2019: 19), interactants can potentially manipulate these resources in ostensive ways such as changing your posture or your attire in a very deliberate and attention-grabbing way (e.g., suddenly adopting an upright body position or suddenly turning up in a suit).

Multimodal im/politeness

Prosodic features of spoken languages are perhaps the most widely researched semiotic cues for politeness, particularly voice pitch. P. Brown and Levinson (1987: 267) followed by Ohala’s (1994) “frequency code” proposed that high pitch is universally associated with politeness due, respectively, to associations with the voice of a child, and with small body size. In short, people are assumed to associate a squeaky voice with a “small” and non-threatening person, and therefore with a subservient and submissive demeanour. A number of studies have shown support for this hypothesis, including Caballero et al.’s (2018) study of indirect requests in English, Orozco’s (2010) study of Mexican Spanish requests, and Loveday’s (1981) study of Japanese women’s production of formulaic politeness expressions. However, some recent studies have shown the opposite pattern: lower pitch is associated with at least some im/politeness related meanings in Catalan (Hübscher, Borràs-Comes and Prieto 2017), Chinese (Oh and Cui 2020), Japanese (Idemaru, Winter and Brown 2019), and Korean (Winter and Grawunder 2012). Recent studies also show that further prosodic features beyond pitch can contribute to im/politeness. These features include using decreased intensity (i.e., a quieter voice) to sound polite (Idemaru, Winter, Brown and Oh 2020), or decreasing speech rate (i.e., speaking slower) (Hübscher, Borràs-Comes and Prieto 2017; Lin, Kwock-Ping and Fon 2006; Ofuka, McKeown, Waterman and Roach 2000; Winter and Grawunder 2012). In addition, speakers can manipulate dimensions of voice quality for politeness-related meanings, including breathiness (Campbell 2004 for English; Ito 2004 for Japanese; Winter and Grawunder 2012 for Korean). For example, Ito (2004) showed that aspiration noise, which could be a reflection of breathiness, is associated with polite speech in Japanese. Next, im/politeness-related meanings can be communicated via alterations in the way that people position their bodies during interaction. In some situations and some cultures, it may be polite to stand further away from the interlocutor, or closer to them (Beaulieu 2004). It also appears in general to be more polite to interact on the same physical plane as the interlocutor (i.e., both sitting, both standing). However, when there is a power imbalance between the speakers, there may be more onus on the junior party to assume a standing position, whereas the senior party enjoys the privilege of remaining seated (Andersen, Guerrero and Jones 2006; L. Brown and Winter 2019). In formal events (ceremonies, meetings, etc.), those with more powerful roles tend to be positioned in central locations in the room (Burgoon and Dunbar 2006; see also Farahandouz and Moallemi, this volume). Various body movements are associated with im/politeness, and these often show cultural specificity. In some cultures, bowing the head or performing the gassho gesture (pressing the hands together in front of the chest as if praying)

5

6

Lucien Brown, Iris Hübscher & Andreas H. Jucker

are conventionalised markers of politeness (see Fang, this volume), including in namaste greetings in the Indian subcontinent and wai greetings in Thailand. Other conventionalized markers of politeness include the use of two-handed gestures for giving and receiving objects in many East Asian cultures (L. Brown and Winter 2019; Dennison and Bergen 2010), nodding in order to show an attentive attitude in Japanese (Kita and Ide 2007), and smiling (Nadeu and Prieto 2011). Speakers may also avoid rude or taboo gestures in situations where politeness is required. In Ghana, pointing with the left hand (Kita and Essegbey 2001) is a gesture that is avoided, whereas, in Yoruba, the same can be said for index-finger pointing towards an elder (Orie 2009). In many cultures, open hand pointing is used to assign speaking rights (e.g., in a classroom or official meeting), whereas index-finger pointing for the same purpose may be judged as highly impolite. The specifics of gesture production may also be sensitive to im/politenessrelated factors. In a recent study, L. Brown et al. (forthcoming) found that Catalan and Korean speakers produced gestures with less animacy when interacting with a status superior compared to when interacting with a friend. Korean speakers produced fewer gestures overall when addressing the superior, and speakers of both languages reduced their gesture space, decreased the encoding of manner in motion events, and reduced the use of character-viewpoint gestures. In sign languages, conversationalists use non-manual body movements for politeness. When performing sensitive speech acts, signers may apply raised eyebrows, tight lips, grimaces, tilts of the head (or head and upper body) to the side and a “polite duck” (lowering the head while hunching the shoulders) (Mapson 2014 for British Sign Language). In Japanese Sign Language, George (2011: 113) observed that signers use a smaller signing space, and a lowered and forwardleaning chin and head position were associated with heightened speech register (comparable to honorific speech in spoken Japanese). These behaviours “signal a non-threatening act display and in turn have an association with a more polite register” (George 2011: 113; see Cokely and Baker-Shenk 1980; Liddell and Johnson 1989 for similar findings for American Sign Language). Conversely, when performing impoliteness via taboo gestures, Loos et al. (2020) showed how in German Sign Language a variety of linguistic means are used to introduce and enhance offense. For example, the degree of visual explicitness of a sign increases its potential to offend, and modality-enhanced “vicarious embarrassment” in the person observing can be created, which then leads to specific signs being considered offensive. In contrast to face-to-face communication with its multiple modalities, written communication may appear to be more or less monomodal with an almost total reliance on the strictly textual modality. However, written texts can also be a fertile context for studying multimodal im/politeness. In computer-mediated

Multimodal im/politeness

communication, for instance, conversationalists regularly make use of CMC cues (Vandergriff 2013) such as emoticons, nonstandard punctuation and lexical surrogates to mark affective meanings, including im/politeness-related meanings. For instance, the use of three or more periods in succession marks ellipsis or “silence”, but also becomes a cue for irony (Hancock 2004) or in other contexts a way of delaying and thereby mitigating rapport-threatening or dispreferred actions (Keng Wee Ong 2011; Vandergriff 2013). Speakers also use lexical surrogates (e.g. hmmm) to mitigate their statements of disagreement or scepticism (Vandergriff 2013). Finally, smiley emoticons and emoji display orientation to dispreferred action (Golato 2003; Vandergriff 2013), and package such action as occurring with friendly or non-threatening intentions (Darics 2010). In addition, written texts are also a useful resource for the study of multimodal im/politeness in that they may contain relevant descriptions, i.e., firstorder accounts, of multimodal im/politeness. Such accounts can be particularly useful for historical periods, which are not accessible for direct investigations. Hübler (2007), for instance, uses prescriptive evidence from the sixteenth and seventeenth century courtesy books to argue for a prosodic turn in Early Modern English conversations. These texts take great pains to characterise the features of a “good voice”, such as appropriate loudness, pitch and tempo (Hübler 2007: 148–158). Kukorelly (2020) uses conduct books written for young women in eighteenth-century Britain to show how much politeness, conversation and self-control in body and mind were connected. These books admonish young women against anger because of the effects this is supposed to have on their self-control: “the angry woman’s thoughts become uncontrolled, unreasonable, passionate. These thoughts, under the impulse of passion, overflow into public manifestations: her body becomes distorted, her face ugly; her voice becomes strident, urgent; her words become hurtful, thoughtless” (Kukorelly 2020: 129). But descriptions of the multimodality of conversations and their relation to im/politeness also occur outside of courtesy manuals and conduct books, as Jucker (this volume) shows. He uses the Corpus of Contemporary American English to investigate descriptions of facial expressions and gestures that are used to apologise and argues that they are an important part of the politeness of apologies. Ultimately, the concept of im/politeness may reach into domains that have little connection with language, or with ostensive communication for that matter. This seems to be particularly the case when we consider politeness in terms of demeanour (originally Goffman 1956), in other words as a projection of the inner qualities of the speaker (see Dunn 2013: 239). Withey (2015) looked at how not just posture, but also shaving, personal grooming and use of eyewear became semiotic resources for constructing a polite bodily image in eighteenth-century Britain. “The eighteenth-century body,” writes Withey (2015: 8) “was a mannequin

7

8

Lucien Brown, Iris Hübscher & Andreas H. Jucker

upon which were hung conventions of fashion, taste and politeness.” Meanwhile, Stobart, Hann and Morgan (2013) investigate the ways in which the English town became “a site of polite consumption” that represented “the individual and collective embodiment of politeness”. For Japanese, Dunn (2013: 232) notes that business etiquette training courses typically feature physical appearance, such as the need for clean and neat hair, ironed clothes, and polished shoes. So far in this section we have largely conceived of im/politeness as something that arises due to the use of certain im/polite cues, be that particular sounds of the voice, movements of the body, uses of emoticons, or even ways of dressing. However, rather than being achieved through a certain “polite voice” or “polite behaviour”, im/politeness may also depend on the extent to which speakers accommodate during interaction (for discussion of accommodation see Coupland, Coupland and Giles 1991; for discussion of other related terms and concepts see Rasenberg, Özyürek and Dingemanse 2020; see also Ibrahim and Hübscher, this volume). When speakers are interacting compliantly, they may repeat the words of the other party at the same relative pitch in order to sound supportive (Couper-Kuhlen 1996), although absolute pitch-matching (or “hyperaccommodation”) runs the risk of being perceived as impolite mimicry (Culpeper, Bousfield and Wichmann 2003: 1574). Although failure to accommodate can become a feature of non-compliant behaviour (Culpeper, Bousfield and Wichmann 2003: 1574), choosing not to match the multimodal behaviours of the other party may at times be the more appropriate choice. Paxton, Brown and Winter (2018) found that Korean speakers do not synchronize their body movements in status-asymmetrical interactions (i.e., in interactions between superiors and inferiors), whereas synchronization can be observed in symmetrical interactions (i.e., interactions between status equals). Particularly in cultures that are very hierarchical, it may not be appropriate for inferiors and superiors to mirror each other’s behaviour. In sum, the current section has shown that various different modalities come together as social semiotics to mark im/politeness-related social meanings. In order to be im/polite, it is not enough for speakers to simply use im/polite words. They also have to make sure that the sound of their voice, their bodily comportment, their use of emoji and punctuation and so forth match their im/polite intentions. Im/politeness does not necessarily reside “in” any of these behaviours in isolation, but rather occurs through the confluence of various modalities as they are used in context.

Multimodal im/politeness

4.

What does multimodality contribute to the study of im/politeness?

The overview of literature in the previous section has shown that different modalities come together to signal im/politeness-related meanings. Although this might be seen as a strong mandate for the widespread application of multimodal analysis in im/politeness research, some may question how important these multimodal features really are. One could argue that they are just incidental things that happen to occur alongside im/politeness, but which are secondary to the other cues. How important are multimodal features of politeness, and what does studying them contribute to the field of im/politeness research? There is a distinct lack of studies that directly compare on the one hand the importance of different multimodal im/politeness cues in sign vs. spoken languages and on the other hand compare the relative importance of multimodal im/politeness cues specifically in spoken languages. However, the evidence that we have seems to show that although multimodal cues for im/politeness may be weaker or less precise than verbal cues in spoken languages, they nevertheless play important roles in the perception of impoliteness. L. Brown et al. (2014) followed by Idemaru et al. (2019) showed that Korean and Japanese participants, respectively, could identify which of two identical sentence fragments was taken from a deferential utterance spoken to a status superior at accuracy rates of 70% (Korean) and 56% (Japanese). Although both of these results are reliably above chance, the authors conclude that they do not appear high enough to suggest that prosody would be the primary means for identifying honorific speech. One reason for this is that the social and affective meanings of prosodic patterns are under defined. Slower speech rate, for instance, can mark deference or politeness, but it can also take on a range of other meanings including maturity and intelligence, as well as associations with the speech of a teacher or counsellor (see Schnoebelen 2009). Likewise, the wai gesture in Thai culture marks a polite greeting, but also meanings related to utilitarianism, status, nationalism, personal enhancement, and religion (Powell, Amsbary and Hickson 2014). In sign languages as well, visual prosodic markers can also have multiple functions. For instance, a palm–up gesture can mark various stances including acceptance, apology, and that an utterance is merely a personal opinion (see Moya-Avilés et al., this volume). As noted above, modalities that are less involved in ostensive communication such as posture, personal attire and grooming may be constrained by various contextual and social factors that have little or nothing to do with politeness. In contrast, some verbal aspects such as honorifics and indirect request formulae or swear words and slurs are heavily biased towards polite and impolite interpretations, respectively. As Culpeper (2005: 41) colourfully explains, “one has to work quite hard to imagine contexts in which ‘you fucking cunt’ would not be considered impolite.”

9

10

Lucien Brown, Iris Hübscher & Andreas H. Jucker

Although the relative importance of multimodal politeness markers may be difficult to assess, it seems that access to visual modalities of communication help in identifying politeness-related meanings. Some experimental studies have shown that the provision of video stimuli helps people to perceive im/politenessrelated meanings, compared to when they only have access to the sound file. In a study of the perception of genuine impoliteness versus mock impoliteness, McKinnon and Prieto (2014) found that participants displayed more certainty in identifying mock politeness when a video showing gestural behaviour was provided (rather than just sound). Meanwhile, in a paper that compared the relative importance of prosodic versus gestural cues, Nadeu and Prieto (2011) found that a smiling facial expression resulted in increased pitch height being perceived as polite, whereas this prosodic pattern was perceived as less polite when the facial gesture was not supplied. In the current volume, Claudel shows that politenessrelated meanings in subtitled versions of Japanese films for overseas consumption sometimes rely entirely on visual cues to signal politeness. Furthermore, when looking at the acquisition of multimodal politeness, it has been found that 3-year-old children are sensitive to politeness based on prosodic and facial cues at an age when they are still acquiring different verbal markers to politeness (Hübscher, Wagner and Prieto 2020). Similarly, already pre-schoolers are adapting their prosodic and facial gestural cues when requesting an object from a teacher compared to a peer (Hübscher, Garufi and Prieto 2019). So, from a developmental perspective, different prosodic, facial and body cues seem to develop in parallel and might act as precursors to the verbal encoding of politeness. It therefore appears that multimodal cues help in the processing of social meanings such as im/politeness, and also in the way that they are learned. Rather than thinking of these cues as competing with each other, it may be more fruitful to conceive of them as occurring together in order to make the communication of im/politeness-related meanings more robust. Given the social importance of being perceived as polite (or potentially as impolite), having multiple cues for politeness is essential to ensure that the speaker’s intentions are successfully conveyed (Mason, Winter and Grignolio 2015). Likewise, the multiplicity of politeness cues is expected to aid in their acquisition by children. The perspective that speakers use a layering of multiple cues to ensure that their im/polite intentions are conveyed is consistent with explanations in im/ politeness theory regarding how im/politeness is communicated, including approaches that adopt Relevance Theory (RT) (Christie 2007; Jary 1998). From this perspective, im/politeness is achieved by the listener assigning relevance to social stimuli in the utterance (Chen 2014), which activate listener assumptions and knowledge regarding norms of im/politeness and forms of politeness real-

Multimodal im/politeness

ization (Escandell Vidal 1998). Speakers may use a variety of “ostensive stimuli” to make their im/polite intentions more or less salient (or, more or less fuzzy), including multimodal stimuli, which are given equal weighting in RT (Sasamoto and Jackson 2016: 41; Wilson and Sperber 2002: 254). However, as noted above, some multimodal features such as posture and personal attire may be questionable in terms of their status as ostensive markers. The way that interactants use multimodal cues to assign social meanings should not be seen as a static or individual process. Rather it is one that is social, dynamic, and context specific. The dynamic way that interactants assign social meanings is evinced by the variability of this process, and also by the agency that people are shown to exert when choosing to evaluate a social action as being im/polite. Mitchell and Haugh (2015) showed that interactants exert a degree of choice as to whether to take offense at rapport-threatening utterances. Although Mitchell and Haugh (2015) focussed on verbal politeness, we expect that people also assign im/politeness-related meanings to multimodal aspects of politeness in a way that is dynamic and agentive. Multimodal analyses can therefore be integrated into discursive approaches to politeness, and indeed may have much to offer to such approaches. Although there are many different ways to discursively analyse politeness, the main assumption of the discursive movement is that im/politeness revolves around evaluations of social actions that take place in a dynamic and context-dependent way (see papers in Linguistic Politeness Research Group 2011). Rather than being random and isolated, these evaluations tend to be recursive – the same social behaviour occurring in similar contexts tends to get evaluated in similar ways over time and across social spaces (Kádár and Haugh 2013: 185). Including multimodal analysis in a discursive approach to politeness provides the analyst with an additional window for identifying evaluations of politeness, since listener evaluations of politeness may occur in an embodied fashion that features limited or no verbal content. For example, a nod or a smile may be taken as a positive evaluation of a social action. Moreover, including multimodal analysis of the behaviour of the speaker can provide important information about why a particular social action is being assigned to an utterance. For instance, if an utterance that appears ostensibly polite is evaluated as impolite, the reason may lie not in the spoken content of the utterance (or necessarily in contextual factors), but in the speaker’s acoustic and embodied delivery of the utterance. The inclusion of multimodal cues may be particularly important in research that looks at utterances that contain mixed messages regarding politeness, such as polite and impolite language in the same utterance (see Culpeper, Haugh and Sinkeviciute 2017). Multimodality can also contribute to the discussion of universality versus culture specificity in politeness research. As noted above, P. Brown and Levinson’s

11

12

Lucien Brown, Iris Hübscher & Andreas H. Jucker

(1987) observed universal connections between high pitch and politeness via the assumption that high pitch sounds childlike and thus less threatening and more submissive. Around the same time, this idea that high pitch universally associates with politeness was included by phonetician John Ohala in his theory of sound symbolism known as the “frequency code” (Ohala 1994). The theoretical link between politeness and high pitch has only met with partial support in subsequent empirical studies (see above) seemingly because, at least in part not all aspects of politeness depend on embodying a submissive self-image (see Winter et al. 2021). However, studies that have looked at more tightly defined areas of politeness have indeed shown cross-linguistic patterns, even if these do not necessarily pattern with the claims of the frequency code. Hübscher, Borràs-Comes and Prieto (2017) followed by Hübscher et al. (under revision) suggested that showing politeness towards a social superior may involve the strategies of “prosodic mitigation” and “gestural mitigation”, respectively, whereby speakers curtail and suppress the animacy of vocal and visual aspects of politeness (see L. Brown et al., this volume). Indeed, this suppression of gesture when politeness is required features in the results of several studies in this volume (including L. Brown et al; Farahandouz and Moallemi; Claudel). Although there may be some cross-cultural patterns that can be found in multimodal politeness that are “universal” to some extent, it is evident that multimodality also involves culture-specific factors. In particular, the conventional im/politeness values of certain iconic gestures and body movements can be culture specific to some extent. For example, bowing and/or gassho gestures are strong markers of politeness in some Asian cultures, whereas their usage is not as conventionalised in Western cultures. Likewise, socially taboo gestures also display culture-specificity, such as the avoidance of left-handed gestures in Ghana mentioned above (Kita and Essegbey 2001). Cross-cultural variation can also be observed in CMC cues. For example, Korean (and some other Asian cultures) use tildes (~) to express an elongated friendly voice (Chun 2014) and smiley eyes to expressive positive affect (^^) (Pasfield-Neofitou 2007). Embodied manifestations of politeness furthermore appear to be closely linked to culture-specific metapragmatic conceptualizations of politeness. In the current volume, Farahandouz and Moallemi discuss the concept of ta’ârof in Iranian culture, which they describe as a ritualized display of politeness that encompasses both linguistic and behavioural components. Meanwhile, Shochi, Rilliard and Erickson (current volume) investigate five different Japanese concepts of politeness (teinei, seii, kyoshuku, heijo, and zonzai), each of which is associated with distinctive prosodic realizations. Indeed, it seems to us that the mono-modal way that politeness has been researched within pragmatics is inconsistent with layman conceptualizations of politeness (i.e., what is known as first-order politeness or

Multimodal im/politeness

“politeness1”), which tend to be inherently multimodal. Even in English, im/politeness related concepts such as etiquette, manners, friendliness or sarcasm all appear to be concepts that are inherently embodied (see also the studies by Hübler 2007 and Kukorelly 2020 on Early Modern English courtesy manuals and eighteenthcentury conduct books with their advice on voice and body comportment). Ultimately, multimodality may help us to understand some of the most fundamental questions raised by im/politeness research. These include questions about what politeness really is, and why it is needed in human interaction. To take an example, the argument of P. Brown and Levinson (1987) and of Ohala (1994) that high pitch signals politeness via universal associations with child-like voices and small body size would suggest that politeness is all about submission. Indeed, in an evolutionary account of politeness, Bax (2011: 260) proposes that negative politeness evolved from ritual submission displays, the counterpart to superiority displays, which have their origins in animal bluff displays. However, as mentioned above, empirical research has shown that at least some forms of politeness involve using lower pitch, as well as a quieter and more monotonous voice (Hübscher, Borràs-Comes and Prieto 2017; Idemaru et al. 2020). These findings show that politeness is not necessarily about submission, but rather linked to notions that may include calmness, dullness, or appearing mature. This opens up interesting questions also for the development of politeness in children’s language acquisition process. Looking at multimodal im/politeness through the lens of children who are in the process of becoming competent members of a specific language/cultural background could give us some insight about the importance of different multimodal cues such as prosody, gesture and also lexical/grammatical items to signal social relationships and how they are being integrated into a complex whole over time. Furthermore, still little is known about how children from different cultural and language backgrounds develop their perception and production of multimodal politeness and whether there are certain commonalities in the acquisition process (see e.g. Hübscher, Garufi and Prieto 2019). Multimodality may be particularly important for politeness research due to its propensity to bring sign languages into the remit of our field. The addition of sign languages does not only ensure that the field of im/politeness research becomes more inclusive and comprehensive, but it also has the potential to help answer the kind of fundamental questions about the nature of politeness mentioned above. As noted by Pfau et al. (2012: 1) sign languages provide linguistics with a special path for exploring the most fundamental questions about human language, including “the role of modality in shaping language, the nature of linguistic universals approached cross-modally, the functions of iconicity and arbitrariness in language, and the relationship of language and gesture.”

13

14

Lucien Brown, Iris Hübscher & Andreas H. Jucker

Ultimately, the inclusion of multiple modalities and sign languages may allow us to imagine a very different field of im/politeness research. Vigliocco et al. (2014) point out how the accepted understanding of language within linguistics and related fields has been founded on the study of speech and text, and the assumption that language is an arbitrary system for transmitting meaning. The authors then pose the question: “What if the study of language had started from signed language rather than spoken language?” Since sign languages are inherently multimodal and feature highly salient non-arbitrary features, Vigliocco et al. (2014) followed by Perniss (2018) argue that we would end up with a very different understanding of language, where multimodality would be the norm and where mono-modal speech or text would be seen as atypical and impoverished. Likewise, we can imagine that im/politeness research would be a very different field if it had begun with the study of sign languages and multimodality, rather than with the study of indirect speech, hedges, honorifics and address terms. Now is the time for the field to embrace the multimodal and embodied nature of communication and work towards a more holistic understanding of im/politeness.

5.

Overview of the book

The eleven chapters in this volume all provide insights into the various modalities through which politeness is enacted, evaluated and negotiated. The chapters are arranged into three sections to reflect whether they cover sign languages, spoken languages or written languages, although some papers could potentially fit into more than one section. They cover a range of languages and cultures, including (in order of appearance) Swiss German Sign Language, Catalan Sign Language, English (both as a native language and as a lingua franca), Korean, Catalan, Persian, Japanese and Spanish. We also see a variety of different research methodologies, including the use of conversational data, media data, roleplays, written corpora, and experimental techniques. The section on sign language includes two papers, both of which provide coverage of languages yet to be examined for their politeness features. In the first of these, Penny Boyes Braem and Katja Tissi introduce politeness features in Swiss German Sign Language (Deutschschweizerische Gebärdensprache, DSGS), whereas in the second paper Berta Moya-Avilés, Gemma Barberà, and Carme Bach look at Catalan sign language (LSC), focussing on attenuation of criticism. Both of these papers demonstrate how sign language users apply non-manual markers alongside manual sign language for im/politeness related meanings, and point to culture and language specificity in how these are applied. Boyes Braem and Tissi demonstrate how signers in Swiss German culture navigate cultural

Multimodal im/politeness

preferences for the avoidance of bragging, and Moya-Avilés, Barberà, and Bach demonstrate how Catalan sign language users make use of pronoun doubling when emphasizing that something is a personal opinion. These papers bring into relief the problematic nature of assumptions about im/politeness in sign languages, particularly the “Deaf-as-direct” stereotype mentioned in the second paper. The section on spoken languages is by far the largest in the volume, featuring a total of seven papers. In the first of these, Xianming Fang explores the functions of body position (standing/sitting), smiles, and gaze aversion during refusal roleplays. Although role-play methodology has been extensively used in studies of speech acts, previous papers using this kind of methodology have rarely if ever applied multimodal analysis. The fact that this paper focusses on English as a Lingua Franca is also meaningful, given that this is an important emerging field in pragmatics (see Taguchi and Ishihara 2018). The next three papers share a common thread in that all of them are concerned with multimodal politeness in terms of indexing meanings such as distance, respect or deference towards status superiors. Lucien Brown, Iris Hübscher, Hyunji Kim and Bodo Winter explore how speakers of Korean and Catalan modulate their use of different multimodal cues (or “bodily visual practices”) depending on whether they are addressing a superior or an intimate friend. Next, Farbod Farahandouz and Shima Moallemi investigate the multimodal manifestations of ta’ârof, a politeness ritual in Persian, particularly in Iranian culture, whereby socially superiors are elevated, whereas the self is lowered or humbled. Finally, the paper by Omnia Ibrahim and Iris Hübscher returns to Catalan and looks at how speakers apply vocal accommodation when making apologies to superiors and to intimate friends. Taken together, these papers show a number of crosslinguistic similarities in how distance, respect or deference are indexed. But we also see culture-specific practices, such as custom in Persian-speaking communities to invite a guest to sit in the position furthest from the door. The next two papers explore a number of complex im/politeness-related phenomena. Takaaki Sochi, Albert Rilliard and Donna Erickson look at how children come to recognize the multimodal expression of several im/polite stances between 6- to 10-year-olds, with coherence in responses increasing with age. Children find it particularly difficult to learn kyoshuku ‘shrink fear’: a mode of politeness whereby the speaker displays their ashamedness for being unable to act in accordance with the expectations placed on them by social superiors, and which is typically performed with a frown and a bow. Next, Claudia Lehmann investigates irony, focussing on English Tell me about it and syntactically independent as if clauses. Whereas the former co-occurs with slower tempo, gaze aversion, raised eyebrows, smiles and head movements, the latter comes with pauses and head

15

16

Lucien Brown, Iris Hübscher & Andreas H. Jucker

tilts. These two papers show that multimodal features play important roles in disambiguating complex politeness-related stances. The third and final section of the book is on written language and contains three papers that address quite different manifestations of multimodal im/politeness. The first two of them address areas of written language that have direct and very important practical applications. Chantal Claudel investigates the way that culture-specific manifestations of politeness in Japanese films are represented (or not) when they are subtitled in French. We see in Claudel’s analysis that the written modality (i.e., the subtitles) does not need to represent politeness-related meanings to the French viewers due to the abundance of visual cues denoting politeness and/or rank, such as kneeling and bowing to the ground or personal attire (e.g. wearing a suit). Although we classify this paper as “written language” due to the author’s focus on subtitles, the analysis clearly shows that politeness is communicated visually rather than verbally. In the next paper, Ester IyangaMambo explores the multimodality of mitigation in online live chats between customers and service providers. The analysis shows, crucially, that the use of CMC cues such as emoticons, exclamation marks, laughing (e.g., hahaha) and repetition of punctuation tends to lead to higher levels of engagement from the customers, and potentially to higher rates of satisfaction. While the results of these papers speak to the importance of multimodality in real-world contexts, the last paper in this volume by Andreas H. Jucker looks at descriptions of multimodal ways of performing apologies and im/politeness in fictional texts. Based on data from the Corpus of Contemporary American English, he shows that appropriate facial expressions and gestures are regular parts of apologetic behaviour, and indeed in many cases, narrators in fictional texts describe the apologetic behaviour only in terms of an appropriate smile or shrug without any words being uttered by the fictional characters. The multimodality of the speech act, therefore, is an essential ingredient of the discursive negotiation of face and politeness. What all the contributions of this volume have in common is their understanding of im/politeness as the result of multimodal interaction. A focus on the verbal modality alone is clearly inadequate. Signs, gestures, tone of voice and so on are not just embellishments of the spoken word, but they are essential elements of the overall message. Indeed, in many cases, above all in the case of sign languages, they are the only carrier of whatever is polite or impolite in an interaction.

Multimodal im/politeness

6.

Looking ahead

With the publication of this volume, we hope to take a further step towards establishing multimodality as a key concern in im/politeness research, and towards reimaging the way that language is treated within the field. Rather than seeing im/ politeness as spoken or textual phenomenon, this volume challenges scholars to treat it as embodied and multimodal entity. We also call for scholars to give equal treatment to sign languages, alongside spoken languages, in order to reach a more complete understanding of the workings and meanings of im/politeness. Going forward, further research will be needed to investigate the multimodal properties of politeness in more depth and across a greater range of contexts. Most importantly, although the current volume has included Japanese, Korean and Persian alongside several European languages, multimodality will need to be studied across a far greater range of languages and cultures. These should surely include African languages as well as the indigenous languages of Central and South America, Austronesia and Australasia, which, given the findings of ethnographic (e.g., Dixon 1989; Gay 1986) and CA scholarship (e.g., Dingemanse 2015) are expected to be treasure troves for multimodal politeness research. In addition, we see a particular need for multimodal politeness research to be carried out across professional contexts, such as the workplace, service encounters, health settings and legal settings, and political exchanges. These are all important contexts for politeness research (as evinced by dedicated chapters in Culpeper, Haugh and Kádár 2017), but which do not feature in multimodal im/ politeness research to date. Specific populations where multimodal im/politeness may be of particular importance could include people with autism, as well as minority populations and second language learners. Finally, whereas current research in multimodal politeness tends to separate different modalities (i.e., some papers study prosody, whereas others study gesture) and keep spoken, signed and written language separate, there is a need for more studies that combine different modalities and/or that span across spoken, signed and written forms. We believe that research across these boundaries of different modalities will ultimately shed the most light on the nature of politeness.

References Andersen, Peter A., Laura K. Guerrero, and Susanne M. Jones. 2006. “Nonverbal Behavior in Intimate Interactions and Intimate Relationships.” In The SAGE Handbook of Nonverbal Communication, ed. by Valerie Lynn Manusov and Miles L. Patterson, 259–278. Thousand Oaks, CA: Sage. https://doi.org/10.4135/9781412976152.n14

17

18

Lucien Brown, Iris Hübscher & Andreas H. Jucker

Bax, Marcel. 2011. “An Evolutionary Take on (Im)politeness: Three Broad Developments in the Marking out of Socio-proxemic Space.” Journal of Historical Pragmatics 12 (1–2): 255–282. https://doi.org/10.1075/jhp.12.1-2.11bax Beaulieu, Catherine. 2004. “Intercultural Study of Personal Space: A Case Study.” Journal of Applied Social Psychology 34 (4): 794–805. https://doi.org/10.1111/j.1559-1816.2004.tb02571.x Brown, Lucien, Hyunji Kim, Iris Hübscher and Bodo Winter. Forthcoming. “Gestures Are Modulated by Social Context: A study of Multimodal Politeness in two Cultures.” Gesture. Brown, Lucien, and Pilar Prieto. 2017. “(Im)politeness: Prosody and Gesture.” In The Palgrave Handbook of Linguistic (Im)politeness, ed. by Jonathan Culpeper, Michael Haugh and Daniel Z. Kádár, 357–379. London: Palgrave Macmillan. https://doi.org/10.1057/978-1-137-37508-7_14

Brown, Lucien, and Bodo Winter. 2019. “Multimodal Indexicality in Korean: ‘Doing Deference’ and ‘Performing Intimacy’ through Nonverbal Behavior.” Journal of Politeness Research 15 (1): 25–54. https://doi.org/10.1515/pr-2016-0042 Brown, Lucien, Bodo Winter, Kaori Idemaru, and Sven Grawunder. 2014. Phonetics and Politeness: Perceiving Korean Honorific and Non-honorific Speech through Phonetic Cues. Journal of Pragmatics 66: 45–60. Brown, Penelope, and Stephen C. Levinson. 1978. “Universals in Language Usage: Politeness Phenomena.” In Questions and Politeness: Strategies in Social Interaction, ed. by Esther N. Goody, 56–310. Cambridge: Cambridge University Press. Brown, Penelope, and Stephen C. Levinson. 1987. Politeness: Some Universals in Language Usage. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511813085 Burgoon, Judee K., and Norah E. Dunbar. 2006. “Nonverbal Expressions of Dominance and Power in Human Relationships.” In The SAGE Handbook of Nonverbal Communication, ed. by Valerie Lynn Manusov and Miles L. Patterson, 279–297. Thousand Oaks, CA: Sage. https://doi.org/10.4135/9781412976152.n15

Caballero, Jonathan, Nikos Vergis, Xiaoming Jiang and Marc Pell. 2018. “The Sound of Im/politeness.” Speech Communication 102: 39–53. https://doi.org/10.1016/j.specom.2018.06.004

Campbell, Nick. 2004. “Listening Between the Lines: A Study of Paralinguistic Information Carried by Tone-of-Voice.” International Symposium on Tonal Aspects of Languages: With Emphasis on Tone Languages, Beijing, China, March 28–31, 2004, ISCA Archive, http:// www.isca-speech.org/archive/tal2004 Chen, Xinren. 2014. “Politeness Processing as Situated Social Cognition: A RT-Theoretic Account.” Journal of Pragmatics 71: 117–131. https://doi.org/10.1016/j.pragma.2014.07.010 Christie, Chris. 2007. “Relevance Theory and Politeness.” Journal of Politeness Research 3 (2): 269–294. https://doi.org/10.1515/PR.2007.012 Chun, Sun-young. 2014. “Features and Strategies of Korean EFL Elementary School Children’s Email Discourse.” STEM Journal 15 (1): 197–217. https://doi.org/10.16875/stem.2014.15.1.197 Cokely, Dennis, and Charlotte Lee Baker-Shenk. 1980. American Sign Language: A Teacher’s Resource Text on Curriculum, Methods, and Evaluation. Silver Spring, MD: TJ Publishers. Couper-Kuhlen, Elizabeth. 1996. “The Prosody of Repetition: On Quoting and Mimicry.” In Prosody in Conversation: Interactional Studies, ed. by Elizabeth Couper-Kuhlen and Margret Selting, 366–405. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511597862.011

Multimodal im/politeness

Coupland, Justine, Nikolas Coupland, and Howard Giles. 1991. “Accommodation Theory: Communication, Context, and Consequence.” In Contexts of Accommodation: Developments in Applied Sociolinguistics, ed. by Howard Giles, Justine Coupland and Nikolas Coupland, 1–68. Cambridge: Cambridge University Press. Culpeper, Jonathan. 2005. “Impoliteness and Entertainment in the Television Quiz Show: The Weakest Link.” Journal of Politeness Research 1 (1): 35–72. https://doi.org/10.1515/jplr.2005.1.1.35

Culpeper, Jonathan. 2011a. Impoliteness: Using Language to Cause Offence. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511975752 Culpeper, Jonathan. 2011b. “Politeness and Impoliteness.” In Pragmatics of Society. Handbooks of Pragmatics 5, ed. by Gisle Andersen and Karin Aijmer, 393–438. Berlin: de Gruyter. https://doi.org/10.1515/9783110214420.393

Culpeper, Jonathan, Derek Bousfield, and Anne Wichmann. 2003. “Impoliteness Revisited: With Special Reference to Dynamic and Prosodic Aspects.” Journal of Pragmatics 35 (10–11): 1545–1579. https://doi.org/10.1016/S0378-2166(02)00118-2 Culpeper, Jonathan, and Claire Hardaker. 2017. “Impoliteness.” In The Palgrave Handbook of Linguistic (Im)Politeness, ed. by Jonathan Culpeper, Michael Haugh and Dániel Z. Kádár, 199–225. London: Palgrave. https://doi.org/10.1057/978-1-137-37508-7_9 Culpeper, Jonathan, Michael Haugh, and Dániel Z. Kádár. 2017. The Palgrave Handbook of Linguistic (Im)politeness. London: Palgrave Macmillan. https://doi.org/10.1057/978-1-137-37508-7

Culpeper, Jonathan, Michael Haugh, and Valeria Sinkeviciute. 2017. “(Im)politeness and Mixed Messages.” In The Palgrave Handbook of Linguistic (Im)politeness, ed. by Jonathan Culpeper, Michael Haugh and Dániel Z. Kádár, 323–355. London: Palgrave Macmillan. https://doi.org/10.1057/978-1-137-37508-7_13 Darics, Erika. 2010. “Politeness in Computer-mediated Discourse of a Virtual Team.” Journal of Politeness Research 6 (1): 129–150. https://doi.org/10.1515/jplr.2010.007 Dennison, Heeyon and Benjamin Bergen. 2010. Language-driven motor simulation is sensitive to social context. Proceedings of the Annual Meeting of the Cognitive Science Society, 32. Dingemanse, Mark. 2015. “Other-initiated Repair in Siwu.” Open Linguistics 1 (1): 232–255. https://doi.org/10.1515/opli-2015-0001

Dixon, Robert. 1989. “The Dyirbal Kinship System.” Oceania 59 (4): 245–268. https://doi.org/10.1002/j.1834-4461.1989.tb02335.x

Dunn, Cynthia. 2013. “Speaking Politely, Kindly, and Beautifully: Ideologies of Politeness in Japanese Business Etiquette Training.” Multilingua: Journal of Cross-Cultural and Interlanguage Communication 32 (2): 225. https://doi.org/10.1515/multi-2013-0011 Escandell Vidal, Victoria. 1998. “Politeness: A Relevant Issue for Relevance Theory.” Revista Alicantina de Estudios Ingleses 11: 45–57. https://doi.org/10.14198/raei.1998.11.05 Gay, Judith. 1986. “‘Mummies and Babies’ and Friends and Lovers in Lesotho.” Journal of Homosexuality 11 (3–4): 97–116. https://doi.org/10.1300/J082v11n03_07 George, Johnny. 2011. “Politeness in Japanese Sign Language (JSL): Polite JSL Expression as Evidence for Intermodal Language Contact Influence.” Berkeley, CA: University of California dissertation. Goffman, Erving. 1956. “The Nature of Deference and Demeanor.” American Anthropologist 58 (3): 473–502. https://doi.org/10.1525/aa.1956.58.3.02a00070

19

20

Lucien Brown, Iris Hübscher & Andreas H. Jucker

Golato, Andrea. 2003. “Studying Compliment Responses: A Comparison of DCTs and Recordings of Naturally Occurring Talk.” Applied Linguistics 24 (1): 90–121. https://doi.org/10.1093/applin/24.1.90

Grainger, Karen. 2011. “‘First Order’ and ‘Second Order’ Politeness: Institutional and Intercultural Contexts.” In Discursive Approaches to Politeness. Mouton Series in Pragmatics 8, ed. by Linguistic Politeness Research Group, 167–188. Berlin: De Gruyter Mouton. https://doi.org/10.1515/9783110238679.167 Halliday, Michael. 1978. Language as a Social Semiotic: The Social Interpretation of Language and Meaning. London: Edward Arnold. Hancock, Jeffrey. 2004. “Verbal Irony Use in Face-to-Face and Computer-mediated Conversations.” Journal of Language and Social Psychology 23 (4): 447–463. https://doi.org/10.1177/0261927X04269587

Hübler, Axel. 2007. The Nonverbal Shift in Early Modern English Conversation. Pragmatics & Beyond New Series 154. Amsterdam/Philadelphia: John Benjamins. https://doi.org/10.1075/pbns.154

Hübscher, Iris, Joan Borràs-Comes, and Pilar Prieto. 2017. “Prosodic Mitigation Characterizes Catalan Formal Speech: The Frequency Code Reassessed.” Journal of Phonetics 65: 145–159. https://doi.org/10.1016/j.wocn.2017.07.001 Hübscher, Iris, Martina Garufi, and Pilar Prieto. 2019. “The Development of Polite Stance in Preschoolers: How Prosody, Gesture and Body Cues Pave the Way.” Journal of Child Language 46 (5), 825–862. https://doi.org/10.1017/S0305000919000126 Hübscher, Iris, Cristina Sánchez-Conde, Joan Borràs-Comes, Laura Vincze, and Pilar Prieto. Under revision. “Multimodal Mitigation: How Facial and Body Cues Index Social Meaning in Catalan Requests.” Hübscher, Iris, Laura Wagner, and Pilar Prieto. 2020. “Three-Year-Olds Infer Polite Stance from Intonation and Facial Cues.” Journal of Politeness Research 16 (1): 85–110. https://doi.org/10.1515/pr-2017-0047

Ide, Sachiko. 1989. “Formal Forms and Discernment: Two Neglected Aspects of Universals of Linguistic Politeness.” Multilingua 8 (2/3): 223–248. https://doi.org/10.1515/mult.1989.8.2-3.223

Idemaru, Kaori, Bodo Winter, and Lucien Brown. 2019. “Cross-cultural Multimodal Politeness: The Phonetics of Japanese Deferential Speech in Comparison to Korean.” Intercultural Pragmatics 16 (5): 517–555. https://doi.org/10.1515/ip-2019-0027 Idemaru, Kaori, Bodo Winter, Lucien Brown, and Grace Oh. 2020. “Loudness Trumps Pitch in Politeness Judgments: Evidence from Korean Deferential Speech.” Language and Speech 63 (1): 123–148. https://doi.org/10.1177/0023830918824344 Ito, Mika. 2004. “Politeness and Voice Quality: The Alternative Method to Measure Aspiration Noise.” Speech Prosody 2004, International Conference; Nara, Japan, March 23–26, 2004, ed. by Bernard Bel and Isabelle Marlien, ISCA Archive, http://www.isca-speech.org /archive/sp2004 Jary, Mark. 1998. “Relevance Theory and the Communication of Politeness.” Journal of Pragmatics 30 (1): 1–19. https://doi.org/10.1016/S0378-2166(98)80005-2 Jucker, Andreas H. 2020. Politeness in the History of English. From the Middle Ages to the Present Day. Cambridge: Cambridge University Press. https://doi.org/10.1017/9781108589147

Multimodal im/politeness

Kádár, Dániel Z. 2019. “Introduction: Advancing Linguistic Politeness Theory by Using Chinese Data.” Acta Linguistica Academica 66 (2): 149–164. https://doi.org/10.1556/2062.2019.66.2.1

Kádár, Dániel Z., and Michael Haugh. 2013. Understanding Politeness. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9781139382717 Keng Wee Ong, Kenneth. 2011. “Disagreement, Confusion, Disapproval, Turn Elicitation and Floor Holding: Actions as Accomplished by Ellipsis Marks-only Turns and Blank Turns in Quasisynchronous Chats.” Discourse Studies 13 (2): 211–234. https://doi.org/10.1177/1461445610392138

Kita, Sotaro, and James Essegbey. 2001. “Pointing Left in Ghana: How a Taboo on the Use of the Left Hand Influences Gestural Practice.” Gesture 1 (1): 73–95. https://doi.org/10.1075/gest.1.1.06kit

Kita, Sotaro, and Sachiko Ide. 2007. “Nodding, Aizuchi, and Final Particles in Japanese Conversation: How Conversation Reflects the Ideology of Communication and Social Relationships.” Journal of Pragmatics 39 (7): 1242–1254. https://doi.org/10.1016/j.pragma.2007.02.009

Kress, Gunther, and Theo van Leeuwen. 2006. Reading Images: The Grammar of Visual Design. London: Routledge. https://doi.org/10.4324/9780203619728 Kukorelly, Erzsi. 2020. “‘This Demon Anger’: Politeness, Conversation and Control in Eighteenth-century Conduct Books for Young Women.” In Manners, Norms and Transgressions in the History of English. Literary and Linguistic Approaches. Pragmatics & Beyond New Series 312, ed. by Andreas H. Jucker and Irma Taavitsainen, 121–140. Amsterdam: John Benjamins. https://doi.org/10.1075/pbns.312.06kuk Lakoff, Robin. 1973. “The Logic of Politeness: Or, Minding your p’s and q’s.” In Proceedings from the Annual Meeting of the Chicago Linguistic Society 9 (1): 292–305. Chicago: Chicago Linguistic Society. Leech, Geoffrey. 1977. Language and Tact, Series A, Paper No. 46. Trier: Linguistic Agency, University of Trier. Leech, Geoffrey. 1983. Principles of Pragmatics. London: Longman. Liddell, Scott, and Robert Johnson. 1989. American Sign Language: The Phonological Base. Washington DC: Gallaudet University Press. Lin, Hsin-Yi, John Tse Kwock-Ping, and Janice Fon. 2006. “An Acoustic Study on the Paralinguistic Prosody in the Politeness Talk in Taiwan Mandarin.” ISCA Tutorial and Research Workshop on Experimental Linguistics. Presented at Athens, Greece. Linguistic Politeness Research Group (eds.). 2011. Discursive Approaches to Politeness. Mouton Series in Pragmatics 8. Berlin: De Gruyter Mouton. Locher, Miriam A., and Sage L. Graham (eds.). 2010. Interpersonal Pragmatics. Handbooks of Pragmatics 6. Berlin/New York: De Gruyter Mouton. https://doi.org/10.1515/9783110214338 Locher, Miriam A., and Richard J. Watts. 2005. “Politeness Theory and Relational Work.” Journal of Politeness Research 1 (1): 9–33. https://doi.org/10.1515/jplr.2005.1.1.9 Loos, Cornelia, Jens-Michael Cramer, and Donna Jo Napoli. 2020. “The Linguistic Sources of Offense of Taboo Terms in German Sign Language.” Cognitive Linguistics 31 (1): 73–112. https://doi.org/10.1515/cog-2018-0077

Loveday, Leo. 1981. “Pitch, Politeness and Sexual Role: An Exploratory Investigation into the Pitch Correlates of English and Japanese Politeness Formulae.” Language and Speech 24 (1): 71–89. https://doi.org/10.1177/002383098102400105

21

22

Lucien Brown, Iris Hübscher & Andreas H. Jucker

Mapson, Rachel. 2014. “Polite Appearances: How Non-manual Features Convey Politeness in British Sign Language.” Journal of Politeness Research 10 (2): 157–184. https://doi.org/10.1515/pr-2014-0008

Mason, Paul, Bodo Winter, and Andrea Grignolio. 2015. “Hidden in Plain View: Degeneracy in Complex Systems.” Biosystems 128: 1–8. https://doi.org/10.1016/j.biosystems.2014.12.003 McKinnon, Sean, and Pilar Prieto. 2014. “The Role of Prosody and Gesture in the Perception of Mock Impoliteness.” Journal of Politeness Research 10 (2): 185–219. https://doi.org/10.1515/pr-2014-0009

Mitchell, Nathaniel, and Michael Haugh. 2015. “Agency, Accountability and Evaluations of Impoliteness.” Journal of Politeness Research 11 (2): 207–238. https://doi.org/10.1515/pr-2015-0009

Nadeu, Marianna, and Pilar Prieto. 2011. “Pitch Range, Gestural Information, and Perceived Politeness in Catalan.” Journal of Pragmatics 43 (3): 841–854. https://doi.org/10.1016/j.pragma.2010.09.015

Ofuka, Etsuko, J. Denis McKeown, Mitch Waterman, and Peter Roach. 2000. “Prosodic Cues for Rated Politeness in Japanese Speech.” Speech Communication 32 (3): 199–217. https://doi.org/10.1016/S0167-6393(00)00009-1

Oh, Eunhae, and Mao Cui. 2020. “The Acquisition of Acoustic Correlates of Politeness by Native Chinese Speakers.” Linguistic Research 37: 113–134. Ohala, John J. 1994. “The Frequency Code Underlies the Sound-symbolic Use of Voice Pitch.” In Sound Symbolism, ed. by Johanna Nichols, John J. Ohala, and Leanne Hinton, 325–347. Cambridge: Cambridge University Press. O’Halloran, Kay L., Sabine Tan, and Marissa, K. L. E. 2014. “Multimodal pragmatics.” In Pragmatics of Discourse. Handbooks of Pragmatics 3, ed. by Klaus P. Schneider and Anne Barron, 239–268. Berlin: De Gruyter Mouton. https://doi.org/10.1515/9783110214406-010

O’Toole, Michael. 2011. The Language of Displayed Art. London: Routledge. Orie, Ọlanikẹ Ọla. 2009. “Pointing the Yoruba Way.” Gesture 9 (2): 237–261. https://doi.org/10.1075/gest.9.2.04ori

Orozco, Leonor. 2010. Estudio sociolingüístico de la cortesía en tratamientos y peticiones: Datos de Guadalajara [Sociolinguistic study of courtesy in treatments and requests: The case of Guadalajara]. PhD thesis, El Colegio de México. Pasfield-Neofitou, Sarah. 2007. “Textual Features of Intercultural Internet Chat Between Learners of Japanese and English.” Call-EJ Online 9 (1): 9–1. Paxton, Alexandra, Lucien Brown, and Bodo Winter. 2018. “Complex Coordination: How Power Dynamics and Task Demands Shape Interpersonal Motor Synchrony.” In CogSci Proceedings. Changing / Minds, 2934. Perniss, Pamela. 2018. “Why We Should Study Multimodal Language.” Frontiers in Psychology 9: 1109. https://doi.org/10.3389/fpsyg.2018.01109 Pfau, Roland, Markus Steinbach, and Bencie Woll. 2012. Sign Language. Berlin: De Gruyter Mouton. https://doi.org/10.1515/9783110261325 Powell, Lewis, Jonathan Amsbary, and Mark Hickson. 2014. “The Wai in Thai Culture: Greeting, Status-marking and National Identity Functions.” Journal of Intercultural Communication 34 (1): 7. Rasenberg, Marlou, Aslı Özyürek, and Mark Dingemanse. 2020. “Alignment in Multimodal Interaction: An Integrative Framework.” Cognitive Science 44 (11). https://doi.org/10.1111/cogs.12911

Multimodal im/politeness

Sasamoto, Ryoko, and Rebecca Jackson. 2016. “Onomatopoeia–showing-word or Sayingword? Relevance Theory, Lexis, and the Communication of Impressions.” Lingua 175: 36–53. https://doi.org/10.1016/j.lingua.2015.11.003 Schnoebelen, Tyler. 2009. “The Social Meaning of Tempo.” Retrieved from http://www .stanford.edu/~tylers/notes/socioling/Social_meaning_tempo_Schnoebelen_3-23-09.pdf Sperber, Dan. 2019. “Personal Notes on a Shared Trajectory.” In Relevance, Pragmatics and Interpretation, ed. by Kate Scott, Billy Clark and Robyn Carston, 13–20. Cambridge: Cambridge University Press. https://doi.org/10.1017/9781108290593.002 Stobart, Jon, Andrew Hann, and Victoria Morgan. 2013. Spaces of Consumption: Leisure and Shopping in the English Town, c. 1680–1830. London: Routledge. https://doi.org/10.4324/9780203608128

Taguchi, N., and N. Ishihara. 2018. “The Pragmatics of English as a Lingua Franca: Research and Pedagogy in the Era of Globalization.” Annual Review of Applied Linguistics 38: 80–101. https://doi.org/10.1017/S0267190518000028 Vandergriff, Iliona. 2013. “Emotive Communication Online: A Contextual Analysis of Computer-mediated Communication (CMC) Cues.” Journal of Pragmatics 51: 1–12. https://doi.org/10.1016/j.pragma.2013.02.008

Vigliocco, Gabriella, Pamela Perniss, and David Vinson. 2014. “Language as a Multimodal Phenomenon: Implications for Language Learning, Processing and Evolution.” In Philosophical Transactions of the Royal Society B: Biological Sciences, 369(1651), 20130292. https://doi.org/10.1098/rstb.2013.0292

Watts, Richard J. 2003. Politeness. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511615184

Watts, Richard J., Sachiko Ide, and Konrad Ehlich. 1992. “Introduction.” In Politeness: Studies in its History, Theory and Practice, ed. by Richard J. Watts, Sachiko Ide, and Konrad Ehlich, 1–19. Berlin: Mouton de Gruyter. Wierzbicka, Anna. 1985. “Different Cultures, Different Languages, Different Speech Acts: Polish vs. English.” Journal of Pragmatics 9 (2–3): 145–178. https://doi.org/10.1016/0378-2166(85)90023-2

Wilson, Deirdre, and Dan Sperber. 2002. Relevance Theory. Malden, MA; Blackwell. Winter, Bodo, and Sven Grawunder. 2012. “The Phonetic Profile of Korean Formal and Informal Speech Registers.” Journal of Phonetics 40 (6): 808–815. https://doi.org/10.1016/j.wocn.2012.08.006

Winter, Bodo, Grace Oh, Iris Hübscher, Kaori Idemaru, Lucien Brown, Pilar Prieto, and Sven Grawunder. 2021. “Rethinking the Frequency Code: A Meta-analytic Review of the Role of Acoustic Body Size in Communicative Phenomena.” Philosophical Transactions B. https://doi.org/10.1098/rstb.2020.0400

Withey, Alun. 2015. Technology, Self-Fashioning and Politeness in Eighteenth-Century Britain: Refined Bodies. London: Palgrave Macmillan.

23

part i

Signed

chapter 2

In your face Im/politeness in signed languages, with examples from Swiss German Sign Language (DSGS) Penny Boyes Braem & Katja Tissi

Center for Sign Language Research | University of Teacher Education in Special Needs (HFH)

This chapter begins by describing impolite forms of signing reported for many sign languages that seem to be based on the visual modality of this form of communication. Other im/polite conventions are influenced by the setting, register, type of speech act, or the perceived signing ability of the addressee. Some signs are perceived as impolite by signers of other cultures because their handshape or location components have impolite connotations in their culture; others reflect historically changing norms of the society. A cultural aversion to what is perceived as bragging seems to be shared by hearing German/Swiss German speakers and deaf signers of Swiss German Sign Language (DSGS). Keywords: Swiss German Sign Language, DSGS, politeness, eye gaze, bragging, deaf culture, swiss culture

1.

Introduction

There are several ways to be im/polite in situations in which persons communicate with each other in a signed language. In this chapter, we will give a brief overview of some commonly reported conventions for im/politeness, grouping them according to whether they seem to be primarily motivated by the visual modality of signed languages, or whether the type of setting, register, speech act, addressee or shared cultural conventions are also important factors. We will then present some examples from a first consideration of politeness in Swiss German Sign Language (Deutschschwerizische Gebärdensprache, DSGS). These discussions are prefaced with a short description of the major findings from the field of signed language linguistics, with additional information about DSGS.

https://doi.org/10.1075/pbns.333.02boy © 2023 John Benjamins Publishing Company

28

Penny Boyes Braem & Katja Tissi

The linguistic research of signed languages is a relatively new academic field. The first linguistic studies were done on American Sign Language (ASL) in the 1960’s and 1970’s (Stokoe 1960; Klima and Bellugi 1979), followed in the 1980’s by studies of several other sign languages, including Swiss German Sign Language (DSGS) used in the German-speaking Swiss cantons. (For an international review of the history of signed language linguistics, cf. McBurney 2012.) DSGS is one of three sign languages used in Switzerland, the other two being the Langue des Signes Suisse Romande (LSF-SR) used by signers in the Frenchspeaking cantons and the Lingua dei Segni Italiana (LIS-Ticino) used in the Swiss Italian regions. These sign languages are not only clearly different from each other, they are also different from the languages spoken in their respective geographical regions. There are no official census numbers for users of these sign languages in Switzerland, but DSGS has been estimated to be the primary language of approximately 5,500 Deaf users and a second language to approximately 13,000 hearing persons (Boyes Braem et al. 2012). The community of practice of DSGS signers is very complex, as it involves many diverse subcommunities, some of which overlap (Appendix A gives an overview of the different kinds of sign language bilingual/ bimodal users). Unlike in many other countries, none of these sign languages has been recognized as an official language in Switzerland.1 The basic linguistic structures of DSGS are like those that have been reported for other signed languages. At the phonological level, these linguistic structures and processes involve not only manual components of the arms and hands (the hand’s shape and orientation, location, and movement) but also linguistically important non-manual components of the face (mouth, cheeks, eyes, nose, eyebrows, eye gaze) and positions and movements of the head and upper torso (Boyes Braem 1995). The non-manual components are used linguistically to mark, for example, different kinds of sentences (interrogative, negative, topicalized,

1. A legal recognition of sign languages has been a concern of the Swiss Federation of the Deaf (SGB-FSS) for over 40 years. However, these languages are still not recognized as “official” languages, on the ground that their users do not reside in a definable geographical area, as is the case of the speakers of the four official Swiss spoken national languages of Rhaeto-Romance, Italian, French and German/Swiss German (Boyes Braem et al. 2012). In 1994, the Swiss Parliament did adopt a postulate which requests the use of sign language for the integration of deaf and hearing-impaired persons by promoting it alongside spoken language in education, training, research and conveyance of information. However, a recent request for official recognition of these sign languages was again rejected in 2021, although in the same year the Swiss German schools for the deaf and the Swiss Deaf Association (SGB-FSS) published an official statement apologizing for the years of suppression of signed language in schools (https://www.sgb-fss .ch/wp-content/uploads/2022/06/Faltblatt-Unterschrift-Zusammenarbeit-SGB-FSS-Sonos-d .pdf).

In your face: Im/politeness in signed languages

etc.), as well to indicate agent, patient, and other semantic roles. A small sample of the extensive literature on the linguistic functions of non-manual behaviors includes Wilbur (2000); Lepic and Occhino (2018) and Perniss (2018) for ASL; Aristodemo and Geraci (2017) for Italian Sign Language; Dotter (2018) for Austrian Sign Language; Sandler (2018) for Israeli Sign Language, as well as Boyes Braem and Sutton-Spence (2001) for mouth movements in several sign languages. Non-manual components are also important indicators of im/politeness in signed languages (cf. e.g., for ASL, Roush 2007 and Hoza 2007a, for Catalan Sign Language, Moya-Avilés 2020 and Chapter 3 in this volume). A special aspect of signed languages is that the visual/corporal modality of their production and perception makes possible the simultaneous production and perception of many linguistically important components at the phonological, morphosyntactic and discourse levels. (For an overview of different forms of simultaneity in signed languages, see Vermeerbergen et al. 2007). The use of the two hands, for example, allows the production of two different lexical units simultaneously. This is often the case in what have been termed “productive” structures. For example, in locative constructions, one hand can refer to a ‘table’ and the other to a small animal previously identified as a cat sitting on, under or beside it, depending upon the relative position of the two hands. An example of this is the final sign in Figure 1.

Figure 1. DSGS sentence meaning ‘The cat is on the table’

Another linguistically special feature of signed languages is their use of the 3-dimensional “signing space” in front and to the sides of the signer’s upper body as locations for articulating signs (Figure 2a). The normal signing space can be enlarged – for example, to sign on a stage or to sign to someone in the distance. It can also be made smaller if one does not want to attract attention, as in whispering. For signing in digital social media (Skype, Zoom, etc.), the signing space is usually more restricted than usual to fit the size of the computer screen. The location of lexical signs can be in this signing space around the body or on the signer’s upper body. The grammar also utilizes this 3-dimensional space. For example, grammatical referents can be tracked by indicating directions or

29

30

Penny Boyes Braem & Katja Tissi

locations (or “spatial loci”) in the signing space that the signer has linked to these referents previously in the discourse (Figure 2b). Once a locus has been defined and identified, one can establish a spatial reference later in the discourse by directing a point, the movement of a sign, or only the eye gaze to this locus. Signing space is also used for a variety of “timelines” (Figure 2c), on which signs are placed or moved to indicate tense and temporal relations (e.g., before, after, from, until). (For more information on timelines, see e.g., Engberg-Pedersen 1993).

a.

b.

c.

Figure 2. (a) The ‘signing space’; (b) Spatial loci for reference tracking; (c) Spatial timelines for marking tense and temporal relations

The linguistic use of simultaneous components and of the 3-dimensional signing space results in many aspects of the grammars of signed languages looking quite different from grammars of spoken languages for which the sequence of the components is a more prominent factor in their linguistic structuring.2 These modality-linked factors have also made the written documentation of signed languages especially challenging, although recent computer technologies that allow the linking of tiers of annotations to videos of the raw data have been of great help (cf. the overview of written methods of documenting signed languages in Boyes Braem 2012). As sign language linguistics is a relatively new field, it is not too surprising that the first goals of researchers have usually been to describe the lexica and gram2. Examples of different kinds of sign language components can be seen in videos of signed examples of DSGS grammar in a pilot online Handbook of DSGS (www.dsgs-handbuch.ch).The description of this grammar was originally conceived and documented in videotaped DSGS, for which written German translations were made (translations into English forthcoming). Online examples in a descriptive grammar of the Sign Language of the Netherlands have also been posted at https://www.lotpublications.nl/a-descriptive-grammar-of-sign-language-of-thenetherlands.

In your face: Im/politeness in signed languages

mars of these previously undocumented languages. Consequently, the pragmatics of these languages has been largely neglected in the past and has only recently come into the research focus of some signed language linguists. The attention given to pragmatic factors, however, has often been limited to conversational regulators such as opening, turn-taking, and turn-keeping techniques (e.g., Baker 1977 and Baker et al. 2016: 76–78) or text linguistics (cf. for example Hansen 2012). An overview of studies of pragmatics in signed languages, is given in Section IV of the Pfau et al. (2012) handbook on sign language research. Although often not included in formal linguistic descriptions of various signed languages around the world, tips abound concerning what is considered “rude” and “polite” by deaf signers in books for teaching signed languages (cf. e.g., examples given in Cokely and Baker 1980; Chambers 1998), as well as in descriptions of deaf culture (e.g., Hall 1989; Moore and Levitan 1993) and in books for sign language interpreters (e.g., Mindess 1999). Some more recent linguistic studies have focused on topics that are especially taboo in specific deaf communities, especially signs related to sex or body parts (cf. e.g., Fisher et al. 2019, Mirus et al. 2012, and Napoli et al. 2013 for taboos in American Sign Language; Loos et al. 2020 in German Sign language; Sze et al. 2017 in Hong Kong, Jakarta, Sri Lankan and Japanese Sign Languages, as well as euphemisms used in Italian Sign Language in Volterra et al. 2019 and 2022, and in British Sign Language in SuttonSpence and Woll 1999; and signs indicating body parts by Pyers 2006 for ASL). Other studies have looked at more general issues of impoliteness in specific signed languages (e.g., for ASL see Hall 1989; Mindess 1999; Hoza 2007a and Roush 2011; for British Sign Language, see Mapson 2013, 2014, 2015; for Brazilian Sign Language, see Ferreira Brito 1995; and for Irish Sign Language, see Mohr and Renkwitz 2020). Like for spoken languages, there are often differences in politeness forms for signed languages in different cultures. Hoza (2007a) looks at the difference in politeness forms used by speakers and signers as being rooted in different kinds of culture-bases. ASL signers are rooted in a Deaf community that highly values showing ‘involvement’ with the community members, whereas the culture that most English speakers in America share is one which allows or even highly values showing an individualistic desire to ‘impress’. It is interesting that the more extensive comments on im/politeness and the pioneering larger studies of this topic have often been made by persons who have worked in the field of signed language interpreting. In her early 1999 book entitled Reading between the signs. Intercultural communication for sign language interpreters, the ASL interpreter Mindess has an entire section on “Rude behaviors”. Also, Hoza, who has written extensively about im/politeness in ASL requests (2001, 2007a, 2007b, 2008, 2011) and Daniel Roush (1999, 2007, 2011), who has

31

32

Penny Boyes Braem & Katja Tissi

written early studies about im/politeness in ASL, are professional sign language interpreters. Mapson, a professional British Sign Language interpreter, has written explicitly about the sensitivity of interpreters to im/politeness in her 2020 article “Intercultural (im)politeness: Influences on the way professional British Sign Language/English interpreters mediate im/polite language” as well as addressing this in several of her other publications on politeness (Mapson 2013, 2014, 2015, 2019, 2020, 2021). Cultural differences in conversational rules have been characterized as often being “generally unquestioned and out of awareness” (Holtgraves and Dulin 1994: 282). It is perhaps partly due to this, that interpreters have been at the forefront of the issue of im/politeness in signing. Interpreters are constantly working in inter-linguistic, inter-cultural situations in which breaking a politeness norm can be immediately evident to themselves and at least one set of their addressees. Radanovic Felberg (2016: 15) reports in her article “Impoliteness: A challenge to interpreters’ professionalism” that she found that spoken language interpreters’ awareness of impoliteness is more common than expected, as it involves “the interpreter’s self-esteem being under attack, a sense of humiliation, loss of credibility and trust issues”. This awareness would especially affect sign language interpreters for whom not only their faces but their whole bodies are always visible to – in the faces of – their clients. This contrasts with the situation of most spoken language interpreters, who often sit unseen in booths or with their backs to an audience and primarily present only their voices to their addressees. The second principle of the Code of Ethics of the American Registry of the Interpreters for the Deaf (RID) requires that the interpreter convey both the factual content of a speaker’s message and the attitudes of the speaker. As Caccamise et al. (1980) point out, The skillful use of facial expression, pauses, and natural body movements are all important to the interpreter in serving as a model of the speaker. In brief, the interpreter has a responsibility to use these in a manner that reflects the speaker, neither adding to nor deleting behavioural characteristics of the speaker that would change either the factual information or attitudes the speaker her/himself is conveying. (Caccamise et al. 1980: 14)

Signed language interpreters therefore operate in settings that especially sensitize them not only to the inter-cultural problems involving im/politeness, but also to the physical signals beyond the words or manual signs themselves which can make an utterance im/polite. Sign language linguists, on the other hand, have historically been slower to recognize and devote research to the pragmatic features of signed languages.

In your face: Im/politeness in signed languages

In Section 2, several politeness norms are identified that are motivated by the face-to face modality of signing, which makes the factor of visibility a fundamental factor of im/polite signing behavior. All of these “visibility-based” im/politeness norms have been reported by users of many signed languages. In Section 3, examples of politeness norms will be described, which seem to be influenced by other overlapping factors of interaction, setting, register, type of speech act, and hearing or signing status of addressee/s as well as cultural and historical factors. In Section 4, examples of how signs from their own as well as other signed languages are perceived as “bragging” by the DSGS signers consulted in this preliminary study. The discussion in this section then widens to include the perception of bragging as being impolite in the larger, hearing Swiss society. Section 5 presents a summary of the main points presented in this chapter and suggests ideas for further research on im/politeness in signed languages.

2.

The factor of visibility in im/polite signing

2.1 Always face-to-face Many of the most widely reported norms for im/politeness in signed languages stem from the visual nature of these languages. The current concept in pragmatics that “politeness is perceived, not produced” used for spoken languages has a double meaning when used for signed languages. The original meaning of this phrase is that im/politeness is how something being communicated is perceived as opposed to a specific form of the communication being somehow innately “im/ polite”. The implications of this phrase, for both spoken and signed languages, is that the perception of what is im/polite can differ between persons, settings, and cultures (Mills 2003; Locher and Watts 2005). For signed languages, however, the notion of “perceived” has the additional meaning of ‘being visually perceptible’. A form of communication that is always “face-to-face” has the potential for being “in your face”. Basically, a visible form of communication needs to be seen to be perceived and must be looked at to be understood.3 Something that impedes the visibility of linguistically important

3. Making communication as visible as possible often involves considering the physical setting of the communication. For example, both authors of this chapter have observed the following: Avoiding signing when there is no light on oneself; avoiding standing in front of a bright background (such as a sunny window) which backlights one, putting one in shadow; not placing a huge flower bouquet in the middle of a dining table as it would inconsiderately block persons from seeing what persons across the table are signing; designing ones living spaces to make

33

34

Penny Boyes Braem & Katja Tissi

features can be perceived as a form of impoliteness as it violates a requirement for clarity. This might be due to the signer being ignorant of this factor (for example, a beginning signer), or the signer knows this, but for some reason is disregarding it. The following issues will be addressed in this section on the importance of visibility: Getting the addressee/s attention, the addressee/s eye gaze and backchannelling behaviors, the eye gaze of the signer and that of any onlookers.

2.2 Getting the addressee/s’ attention Signers use several techniques for getting their deaf addressee/s’ attention that are based on the necessity of being seen. These techniques have not only been reported by Tissi to be used by DSGS signers, but also by users of other signed languages (cf., for example, Mindess 1999 and Baker 1977 for ASL, or Fischer and Jürgensen 2000 for German Sign Language). Before beginning to sign, it is considered polite to be assured of the addressee/s’ visual attention. In some group situations (for example in a classroom), it is appropriate to blink the overhead room lights to get the attention of everyone; however, this light blinking is not considered polite when used only for one person. Creating vibrations by patting a table near the addressee/s, or even stomping on the floor is also acceptable in some informal situations but would be seen as inappropriate in more formal situations (for example, in a church or large public lecture), especially if many hearing persons are also present. The signer often signals a readiness to begin by raising the hands into the signing space, sometimes holding the first sign while scanning the room to be sure all eyes are on oneself. This is sometimes accompanied by a nodding of the head or a slight flapping of the hand across the audience to get attention. The signer can also lightly tap the intended addressee at the edge of the shoulder (or ask someone closer to that person to do that for them). However, if the tap is heavier, or goes beyond the edge of the shoulder, it could be considered inappropriate and hence impolite. Table 1 shows, in the opinion of the co-author Tissi, the changes in different types of physical contact used by signers to get the attention of an addressee, which vary on a scale of increasing impoliteness.

communication visibility more possible – for example, by putting a mirror on the wall behind the sink so that one can follow the conversation in the room behind, even when washing the dishes. In a classroom with participation of several signers, chairs are arranged so that people do not have their backs to others. The importance of visibility for communication has been incorporated into entire building projects (cf., for example, Edwards and Harold 2014).

In your face: Im/politeness in signed languages

Table 1. Four gestures that could be used by a signer to get the attention of an addressee, with increasing degrees of im/politeness. (The more impolite components are in bold print.) (Gesture a) Polite

(Gesture b) Slightly rude

↓

↓

(Gesture c) More rude ↓

(Gesture d) Taboo ↓

Handshape

Fully outstretched hand

Outstretched index finger

Fully outstretched hand

Cupped grasping hand

Location

At top of outer side of addressee’s shoulder

At top of addressee’s arm

At top of addressee’s shoulder near neck

On addressee’s face/chin

Movement

Making a couple of small light taps

Making one sharp poke

Resting one’s hand on the addressee’s shoulder

Moving the addressee’s head toward oneself

2.3 The addressee’s eye gaze The addressees of signed discourse are expected to maintain uninterrupted gaze on the person signing. Closing one’s eyes, looking down or turning away from the person signing is considered very impolite. This would seem quite logical to hearing speakers, who could easily imagine the analogous rudeness of their addressees plugging their ears or walking out of hearing distance when one is speaking to them. Tissi has observed a variation of this form of impoliteness when, for example, hearing L2 learners of a sign language tend to look at the hands of the signer rather than towards the signer’s face. This is somewhat acceptable if the addressees are rank beginners, who are primarily trying to capture what the hands are communicating and perhaps are not yet aware of (or don’t yet have processing time for) the linguistically important non-manual behaviors occurring on the face. However, if looking at the hands rather than the face is done by more advanced learners, who have been explicitly told not to track the hands, this is seen as especially annoying to the point of being perceived as impolite. Relevant here is a very early study by Siple (1978), who looked at where deaf addressees normally focus when watching a signer. She reports that eye movements of these addressees (which can occur as often as every 200 milliseconds) probably occur between two fixation points – one at the level of the signer’s eyes and the other at a point slightly below the chin, as shown in Figure 3. As research in human vision has reported that accuracy is best near the point fixated upon and becomes less distinct in areas further away from this spot, the zones of acuity for viewing signing encompasses the face, neck, and upper body.

35

36

Penny Boyes Braem & Katja Tissi

Figure 3. Zones of visual relative acuity around two different fixation points: (A) between the eyes of the signer, including direct eye contact and (B) a slightly downward cast of the viewer’s eyes. Numbers in the zones are reductions in acuity from that of the focal point (i.e., 50% > 25% > 10%) [Figure from Siple 1978: 99]

Based on this, Siple made the following prediction: “We should expect the limits of the visual system for perception of fine detail to affect the degree to which these aspects of signs themselves are finely or grossly different” (Siple 1978: 96). In other words, the closer to the focal point, the smaller the differences could be in the handshapes, movements, and locations of the signs and still be accurately perceived. Minimal pairs of signs have been found to differ in meaning by locations two centimeters apart on the cheek, whereas locations differing over several centimeters on the upper arm, a zone of minimal acuity, result in no differences in meaning. Siple confirmed her predictions that the form components of signs differ in amount of detail or complexity (linguistic “markedness”) according to the zone of acuity of its location, through an examination of the complexity of signs made in different locations as documented in a dictionary of American Sign Language (Stokoe et al. 1976). These fixation points of the addressee are significant because, as pointed out earlier, a great deal of semantic, syntactic and discourse information is conveyed by facial expressions and the postures and movements of the mouth and head.

In your face: Im/politeness in signed languages

These focal points also seem to be important for some im/politeness aspects of signing. For example, Tissi reports that if DSGS signs that are normally made in a more central location of high acuity, are made in the outer zones of lower acuity but have more marked handshapes or complex movements, they are perceived as impolite, as they place additional work for comprehension of the message on the addressee.4 If an addressee wishes to look away from the signer to briefly sign something to a person outside the conversation, the signs of both hands and the changing non-manual behaviors of the face, head and eye gaze are highly synchronized. Table 2 shows an annotation of how, as modelled by Tissi, such a small aside is politely done in a DSGS conversation. She first interrupts the current Signer 1 with signs that mean “You – wait, stop, I’m sorry I must quickly pay a little attention to that person”. Then, to the outside Signer-2, she signs “Good…stop…good”. Her attention then returns to Signer-1 when her hands go down, out of the signing space, indicating a relinquishing of the floor back to Signer-1. Throughout this example, the direction of eye gaze and head indicate whom is being addressed. When addressing Signer-2, the holding of the wait sign in the nondominant hand is directed to Signer-1. This is a way of indicating one intends to return attention to that person after a moment. The two-handed sign apologize is accompanied by a “polite grimace”. Throughout the the entire apologetic explanation, the eyebrows are furrowed, and the eye shape is squinty, behaviors which cease when she begins the aside to Signer-2. The signs to the Signer-2 (good, stop!, good) are forms of backchannels with stop additionally being accompanied by tight lips.

2.4 The addressee’s backchannelling behaviors Addressees signal that they are paying attention, are understanding or want to interrupt by using backchanneling forms of signs or gestures. These are normally at a lower height in the signing space (Mesch 2016). This also includes regularly making smaller than normal versions of single signs, such as those meaning ‘right’, ‘wow’, or ‘true’, sometimes accompanied by nodding or shaking the head or making small differences in facial expressions. The failure of the addressee to provide manual and non-manual backchanneling at fairly regular intervals can be perceived by signers to be a display of impoliteness. These behaviors noted by 4. The signing space and zones of acuity discussed here are very similar to the use of space by orchestral conductors for gestures intended to convey information beyond that of temporal beating to the players seated in front of them, who are only able to glance intermittently at the conductor (cf. Boyes Braem & Bräm 2000, “A pilot study of the expressive gestures used by classical orchestra conductors”).

37

38

Penny Boyes Braem & Katja Tissi

Table 2. Polite behaviors of the addressee to sign a wish to look away to a person signing outside the conversation, with annotations of the signs made on both the dominant and non-dominant hand and the synchronized signals from the direction of eye gaze and head, as well as the shape of the eyebrows, eyes, and mouth actions

Tissi for DSGS signers have also been documented for several other signed languages, including e.g., by Cokely and Baker (1980) for ASL, by George (2010, 2011) for Japanese Sign Language, by Mapson (2014) for British Sign Language, and by Mesch (2016) for Swedish Sign Language.

2.5 The signer’s eye gaze The use of direction of eye gaze by the signer is complicated and is influenced by many different factors. These include the use of gaze as a regulator of the conversation, in which gaze is usually not directed towards the addressee when beginning to sign, but turns to the addressee when asking a question, or to signal willingness to yield a turn, which is often accompanied by a decrease in the speed of signing or moving the hands to the resting position (Baker 1977; Van Herreweghe 2002). At the same time, gaze also has several important functions in the grammar of the language (for example, to establish and indicate referents that have been established in the signing space, or when using a “constructed action” style, to identify the referent whose role one has taken on). Nevertheless, although the signer does not usually look uninterruptedly at the addressee/s, frequent glances in their direction are usually made. The purpose of these glances is to see if the addressee is continuing to pay attention and/or giving backchanneling signals or wants to interrupt. In triad conversations, the signer is expected to switch eye gaze between both addressees. It is generally considered impolite in informal conversations to continue to sign for long periods of time. Constantly signing beyond the length of time expected in that situation for a conversational turn can be an indication that one is arguing, being pedantic, or indicating higher status or more power than the addressee. The polite signer usually checks often for feedback/back-channeling

In your face: Im/politeness in signed languages

for permission to continue. However, the signer can hold the floor by simply looking away, sometimes accompanied by a slight finger-wiggling gesture that is analogous to “um” in spoken conversations (Johnston and Schembri 2007). If the signer fails to check at regular intervals for feedback indicating understanding and permission to continue signing, or simply doesn’t yield the floor, this can be viewed as not only being annoyingly pedantic but also impolite. For descriptions of several other kinds of eye gaze used by the signer and addressee during DSGS conversations, see, Groeber and Pochon-Berger (2014) and Girard-Groeber (2015).

2.6 The onlooker’s eye gaze Many signers find it impolite when persons outside a signed conversation – for example passersby in a public place – stare at their signed conversation. A quick glance is acceptable, but it would be felt to be impolite if the onlooker stared for a longer period, especially if the signers do not know whether or not the onlooker can understand sign language. These perceptions of impoliteness have been reported not only by Tissi for DSGS but also by users of other signed languages (cf. e.g., the discussion of onlookers in Moore and Levitan 1993 for ASL).

3.

Other factors affecting im/politeness in signing

3.1 Setting and register Studies have been done on the differences between more formal/distancing and more informal/proximal styles of signing. For example, in distancing language styles, there is a tendency to make the production of signs larger and slower, to switch more often between using the dominant and non-dominant hand, and to indicate the intensity or importance of something with lexical items rather than non-manual components (Roy 1989; Zimmer 1989). In Italian Sign Language, the more formal register has been reported to be characterized by less use of highly iconic structures (such as productive signs and constructed speech or action), a more frequent use of sequential units (i.e., more like spoken language), and a more consistent use of “mouthings” (mouth movements like those of spoken language words). “Mouth gestures” (mouth movements unrelated to spoken language) are often modified to be less noticeable, and facial expressions are used that are considered to be more polite in hearing society (Volterra et al. 2022). Tissi reports that strong facial and bodily behaviors are not used for signing DSGS to large audiences, although they would be acceptable in informal settings, such as with small groups of friends.

39

40

Penny Boyes Braem & Katja Tissi

Of the features reported in formal and more informal types of signing, inappropriate forms of facial expression and bodily behaviors are probably the ones that are the most likely candidates for making some signed productions to be perceived as less polite. This will be further discussed in Section 4 on impolite “bragging”, for which the location and size of the manual signs also come into play.

3.2 Speech act and the addressee’s hearing or signing status In smaller groups or one-on-one situations, the im/polite modifications of signing can depend more on the kind of speech act involved. Roush (2007) has documented several indirectness strategies used in ASL requests and refusals. In a study of the types of behavior modifications in requests made by ASL signers, Hoza (2007a) found that the use or strength of these signals seem to depend on the type of request. A “light” request (for example, asking the boss to please pass over a pencil) could be accompanied only by a simple single submissive facial expression. A more “difficult” request (such as asking for a week’s vacation) requires adding more signed phrases prefacing the request, together with more and repeated submissive facial expressions. Hoza also reported that another factor influencing these markers is whether the addressee is hearing or deaf. A hearing addressee usually requires more politeness markers, but with deaf addressees who are members of the American deaf community, there seems to be less need for suppression of strong facial expressions, unless perhaps the addressee is a higherranking member of that community (Moore and Levitan 1993: 271). In contrast, Tissi reports that in German Switzerland, signing deaf persons do not feel such a need for showing politeness to “higher classes” in their small signing communities. There are not such clear or sharp social distinctions between “average” and “elite” signers analogous to the situation in the much larger ASL communities. Nevertheless, she reports that there are some common polite requesting behaviors within her Deaf community. One is the adding of a kind of “coda gesture” at the end of a very polite signed request. This gesture is produced with one or both hands with upturned palms, which are held for a moment outward toward the person receiving the request, sometimes accompanied by raised eyebrows and a slight tilting of head and body to one side (see Figure 4a). If this “coda gesture” were to be omitted – for example, if the signer simply abruptly drops the hands after making the request (as in Figure 4b) – this would be perceived by DSGS signers as indicating that the signer is not expecting a response, and is therefore demeaning, and impolite. A similar palms-up, full hand or index finger gesture is also used for pointing in more formal or polite situations, replacing the normal pointing with the index finger with palm facing down. This is the

In your face: Im/politeness in signed languages

only manual modification of a pronoun for politeness in DSGS (i.e., there is no distinction comparable to du and Sie in German).

a.

b.

Figure 4. (a)Polite added gesture vs (b) impolite lack of this gesture following a request

3.3 Cultural factors in im/polite signing 3.3.1 Differing connotations of a sign in other cultures or historical periods Some signs which are acceptable in some signed languages are considered impolite in others. Name signs for countries are a treasure trove for such forms. Often name signs are based on some stereotype associated with the country, such as an image from the national flag or national emblem. Name signs for the same country may be based on different images. For example, the ASL sign for Germany is an allusion to the wings of an eagle, whereas the DSGS sign depicts the pointy emblem on an old German military helmet. A name sign can also depict a behavior associated with that country (e.g., making the motions of playing a guitar in the ASL sign for Argentina). However, some of the older signs for another country can over time come to be perceived as being impolite. For example, the older ASL signs for China involved indications of slanting eye shapes, but this has been replaced in modern ASL by a sign borrowed from Chinese Sign Language which involves an L-shaped movement on the upper body. Such adoption of the nation’s own sign for itself is an attempt to show respect for that country. Among the different signs for Switzerland, Tissi reports there is one from another sign language that is not liked by Swiss signers. The DSGS name sign for Switzerland is made by tracing the form of a cross (as on the Swiss flag) on the side of the chest with a small-cupped L-handshape (Figure 5a). However, in the sign language of Germany (DGS), although the sign for Switzerland also depicts the form of the Swiss cross, it is traced in neutral space in front of the signer with a bent two-finger V-handshape (Figure 5b). Swiss German signers find this

41

42

Penny Boyes Braem & Katja Tissi

sign for their country impolite, as for them the handshape used evokes several other DSGS signs which have negative connotations, such as the signs meaning ‘vampire’, ‘argue’ or ‘cochlear implantation’ (Figure 6). (Cochlear implantation is troublesome as some members of the DSGS deaf community view this medical procedure negatively, seeing it as an attempt to make deaf people more like hearing people.)

a. Switzerland (in DSGS)

b. Switzerland (in DGS)

Figure 5. Name signs for Switzerland in (a) Swiss German (DSGS) and and (b) German (DGS)

Figure 6. DSGS Sign for cochlear implantation

Different connotations clinging to signs in different cultures are not limited to names of countries. In Japanese Sign Language, a sign involving tapping the side of the fist to the nose means ‘good’. However, for Swiss and American deaf persons, the nose is generally associated with more negative concepts, so the Japanese sign can be perceived as somewhat puzzlingly impolite by these signers. Opinions within the community about their own signs can also change over time. An older DSGS sign for ‘woman’ indicates the shape of a breast. As this became increasingly perceived as a form of sexism and hence impolite, it has been replaced by a new, more politeness-neutral DSGS sign for ‘woman’ in which the forefinger and thumb lightly pinch the earlobe, evoking an earring. A recent example of changing a sign to conform to evolving cultural norms is a new ASL sign for ‘parent’. The older sign involves a handshape with all digits fully extended, and the tip of the thumb touching the forehead (which is the location for many

In your face: Im/politeness in signed languages

ASL signs connected with males, such as ‘father’ or ‘husband’) and moving down to touch the chin (the location for several signs connected with females, such as ‘wife’ or ‘woman’). The newer form of the sign avoids these gender connotations by simply touching the cheek, midway between the older locations, and which is also used by the ASL sign for ‘home’. (For other examples of signs being “polite in yours – but not/or no longer in my sign language”, see Mirus et al. 2012; Mapson, 2014; Mirus et al. 2019; Fisher et al. 2019.) 3.3.2 Deaf signers living in both a deaf-signing and a hearing-speaking culture Many deaf persons, especially those living in societies where formal education has been available to them, are not only bimodal-bilingual in their local signed and spoken languages, but also bi-cultural in the majority hearing and minority deaf communities. Historically, deaf signing persons in many countries have tended to form local communities in which they can share information and activities by using the visible form of language most accessible to them for expressing themselves and understanding others. Even signers with varying degrees of measurable residual hearing often choose, when allowed, to spend their private time with other persons using a language that is more accessible, efficient, and expressive for them. There is a long history of these deaf signing cultural communities, documented in several books on Deaf Culture (a small sample of which are Mottez 1981 for French signers, Lane et al. 1996; Brag 2001; Padden and Humphries 2005 for ASL signers, and Hesse et al. 2020 for DSGS signers). Deaf signers in most countries also have daily encounters and needs for communicating effectively with hearing speakers. This includes hearing members of their own families, their neighbors, their teachers, their co-workers, doctors, as well as officials in political and governmental settings. Living simultaneously in both cultures, deaf signers are exposed to pressures to conform to expected norms of each culture in order to “fit in”. For over 100 years, most of the education in schools for the deaf in both the US and Europe has focused on enabling deaf persons to “fit in” to the majority hearing culture by prioritizing the teaching of speaking, lipreading and writing the spoken language, often to the exclusion of other subjects taught in schools for hearing children. Signing was thus generally discouraged and often banned in the classrooms of these schools. “Fitting in” also involves learning proscriptive norms to avoid behaviors which makes one “stick out”. Most signing deaf persons have almost always had to overcome negative attitudes of the majority hearing culture to their use of visual bodily language, not only in schools. (cf. the documentations in Baynton’s 1996 book, Forbidden Signs. American Culture and the Campaign against Sign Language.)

43

44

Penny Boyes Braem & Katja Tissi

At the beginning of the nineteenth century, when the first schools for the deaf were being established, many educators initially assumed that sign languages were more primitive languages, but being earlier languages, also closer to God. However, at a decisive conference of hearing educators of the deaf in 1880 in Milan, these assumptions changed. In the opening talk for this conference, the Italian host stated: “…remember that living speech is the privilege of man, the sole and certain vehicle of thought, the gift of God, of which it has been truly said: ‘Speech is the expression of the soul/ As the soul is the expression of divine thought’” (quoted in Lane 1992: 114). After this conference, “fitting in” meant learning the spoken/written language as well as possible, and “not sticking out” meant using signed language as little as possible not only in the schools but also in public. (See Appendix B for a historical example of this suppression of signing in favor of speaking in the life of Alexander Graham Bell.) This widespread low opinion of signing was countered in later decades by the “Deaf Pride” movement, which began in the US in the 1980s, following the examples of the 1960s civil rights movements of African-Americans in the US (Rittenhouse et al. 1991). This movement, which had at its core an expression of pride in one’s own sign language, spread in the following years through European countries. A strong identity with the deaf community can be seen in the examples of proficient signers who, although they may also have good skills in communicating with the spoken language – which often wins them a higher regard by speakers – nevertheless continue to consider themselves to have a dominant bicultural-deaf identity. Goldblat and Most (2018), for example, found that many Israeli deaf who have had a cochlear implant that increased their spoken language skills, but have also had opportunities to develop their signing skills, maintained a dominant deaf (or, for some, bicultural-hearing) identity. In the past few decades, hearing persons in many countries have become more aware of signed languages though their increased presence in public places. Especially influential has been sign language interpreting on television, from which it is evident that even complex subject matters can somehow be expressed in this form of communication. Although these experiences have increased many speakers’ curiosity about this language (reflected in an increasing number of hearing persons signing up for courses to learn a signed language), there is little documentation as to what extent it has raised the general public’s previous low opinion of this form of communication to the point of recognizing it as a true language. To make matters of acceptance more complicated, there is some evidence that it is not always easy for hearing L2 learners to be accepted as members of signing communities. The authors of the Loos et al. (2020) study of taboo signs make an explicit link between this and the issue of im/politeness. Their conclusions contain the observation that it is difficult, (or perhaps inappropriate?) for outsiders to

In your face: Im/politeness in signed languages

study such delicate matters as taboos in signing and recommend that there should be guidelines for how outsiders can study impoliteness in different languages/cultures such as those of signing communities.

4.

Examples of im/politeness perceived especially by DSGS signers

4.1 Methods of collecting these DSGS examples The examples reported in this section are from a small pilot study of im/politeness in the signing of DSGS. As this study was conducted during a period in 2020 when the Covid epidemic had imposed a lockdown on the Swiss, it would have been difficult if not impossible to employ a traditional methodological approach with controlled variables and a large group of participants. A qualitative research method was chosen as being appropriate for this situation, but also for being a first step into researching this topic in DSGS. In this collection (as in some observations by Tissi in earlier sections on norms of im/politeness found in several sign languages), we used an autoethnography approach. This allows a researcher to draw upon one’s own experiences as a tool to begin to understand a new phenomenon (Marshall and Rossman 2016). The specific kind of autoethnography used here is what Anderson (2006) has termed “analytic” (as opposed to “evocative”) and in which “the researcher is (1) a full member in the research group or setting, (2) visible as such a member in published texts, and (3) committed to developing theoretical understandings of broader social phenomena” (Anderson 2006: 373). In accordance with this autoethnographic method, the examples of DSGS signing reported here were gathered from the user’s perspective of the co-author, Tissi, who is a L1 deaf signer of DSGS. In addition to being a member of the DSGS deaf community, Tissi has also had many contacts with several other sign languages, including the two other Swiss signed languages, as well as the sign languages of the USA, France, and Germany. Because of these experiences in several communities of signing practice, she brings first-hand observations of what she, as a DSGS signer, views as im/politeness in some of these other signed languages that she knows. As an experienced DSGS researcher and trainer of hearing DSGS interpreters, she has also become especially aware of forms of signing used by hearing L2 learners that she finds to be of varying degrees of “impoliteness”. The other co-author, Boyes Braem, is a hearing L2 learner of ASL and DSGS. Her first linguistic research was on American Sign Language (ASL), but for over the past 40 years she has focused primarily on DSGS. The two co-authors began looking at politeness in DSGS together through several lengthy discussions conducted via Zoom. Some of the guiding questions

45

46

Penny Boyes Braem & Katja Tissi

that were addressed in these discussion were similar to those used by Roush (2011) – for example, the following: “What are the characteristics of a rude person; Do you know someone who seems rude or unpleasant and if so, what makes this person unpleasant?”; “What are rude ways in DSGS to open, break away from or close a conversation?”; “Are there different ways to pose questions or make requests that are appropriate for different partners?”; “In what situations do you feel you need to sign extra politely and how do you do that?”, “Do you ever find signers of other signed languages rude?”; “What are the DSGS signs for that indicate ‘rudeness’?”. These discussions were conducted in DSGS for which Boyes Braem made notes. Tissi recorded videotapes of herself signing examples of im/polite signing behaviors that had been discussed in these conversations. The screenshots in this chapter showing DSGS examples are from these videos. Tissi followed up these discussions with the co-author by a consultation with a large Facebook group of DSGS signers who are professional teachers of this language. More details of how this was done are described in Section 4.2.4.

4.2 Examples of signing perceived as impolite “bragging” Several of the DSGS examples of im/politeness collected in this pilot study evoke for the DSGS signer a perception that the signer is somehow bragging. This section will therefore focus on what seems to be a special sensitivity to boasting being seen as a form of impoliteness. DSGS has several colorful signs relevant to “bragging behavior”, including those shown in Figure 7 which could be translated as ‘bragging’, ‘snotty/snob’, and two different signs for ‘boastful’.

a. brag

b. snob

c. boastful_1

d. boastful_2

Figure 7. DSGS signs used for describing bragging behavior: (a) brag (with German word “bluff ” as mouthing), (b) snob (“nose in the air”, (c) boastful_1, (d) boastful_2

In your face: Im/politeness in signed languages

4.2.1 Bragging and the use of “mouthings” In the past, DSGS signers have not typically used fingerspelling with their signing. This is different from signers of some other languages, such as American Sign Language (ASL), who have long been very fluent practitioners of incorporating fingerspelled English words into their signed sentences. The fingerspelled words mixed into the signing stream are often used for referents for which there are no established signs, such as names of persons, places, technical terms or new objects and devices. The fact that DSGS signers have never used much fingerspelling could be an artifact of the decades of “oral schooling” in the Swiss schools for the deaf, where the curriculum focused on learning to speak and lipread and the use of any form of manual communication was banned (Caramore 1998; Hesse et al. 2020). If there is no existing sign for a concept, rather than using fingerspelling, DSGS signers use a “mouthing”, which are mouth movements that are not identical with but visually closely related to pronunciations of a spoken language word for the concept. Mouthings usually are not accompanied by any voicing. They can be produced alone or simultaneously with one or more manual signs, in which case they are an instance of “simultaneous language-mixing”. Although the function of mouthings as a factor in im/politeness has not been extensively studied, Tissi has observed that in her Deaf community, it is viewed as polite to adapt the amount and kind of mouthing one produces to the addressee. For example, producing a mouthing much slower and more precisely than normal (as one often does to talk to someone who one thinks doesn’t know the language very well) is deemed to be very unpleasant. Also, the use of a great deal of mouthing of German technical terms for which there is no DSGS sign is not considered polite if one is aware that the addressee probably does not know those terms. In this situation, the addition of a fingerspelled version of the term, or simply giving a short description of the meaning of the term is more acceptable. If modifications like these are not done, particularly the addition of fingerspelled versions of mouthed technical terms, the mouthings would be perceived in this community of signers as a kind of arrogant and impolite “bragging”. There was a brief period in the 1980s, during which many deaf communities in different countries were going through their own version of the international “Deaf pride” movement when some DSGS signers tried to reduce the amount of mouthing in their signing. This was a time when the deaf were no longer embarrassed to use signing in public. It was a time of coming out, of communicating with this non-vocal language brazenly “in your face” with the majority hearing population. Some signers thought that showing pride in the independence of their sign language from the spoken language could be expressed by dropping all links to the spoken language, which meant omitting all mouthings. Although DSGS

47

48

Penny Boyes Braem & Katja Tissi

signers continued throughout this period to use their normal mouthings in their signing, some signers of other languages, for example German Sign Language, did argue for signing with no mouthings. However, as years went by and signing received more public attention through newspaper articles and more sign language interpreters and deaf signers on television, the advocacy for omitting all mouthings with sign languages lessened. At the same time, linguists were beginning to recognize and write about the many linguistic functions which these oral components can have, even if they might have their origins in spoken language words. It was becoming clearer that signed language production profits from the efficiency of simultaneously using for linguistic purposes signals from the different visible channels available. Mouthings produced simultaneously with manual signs can, for example, not only be lexically important (to communicate the meaning of uncommon or new signs more clearly) but are also very efficient for adding stress to a lexical item. The holding (or “stretching”) of the mouthing through the production of several manual signs can function as a prosodic binder of a succession of signs into phrases or clauses (cf. e.g., Boyes Braem 2001). Today, Tissi reports that in her DSGS deaf community it is considered impolite if someone switches from their normal signing which is accompanied by mouthings to a style of signing with absolutely no mouthings at all. This, she reports, is perceived as bragging in the DSGS community, as it gives the impression that “I’m sooo good at this sign language!”. 4.2.2 Bragging by volunteering too much personal information Swiss German signers are much less likely than, for example, many signers in the larger ASL communities to volunteer personal information, unless expressly asked. Giving information about one’s level of education can be tricky in this Swiss minority community where historically – and still today – very few deaf persons have access to higher education, resulting in few deaf persons having BAs or other higher degrees when compared to deaf persons in other countries. This educational factor has no doubt contributed to the feeling within the DSGS community that overtly showing one has had a lot of higher education is considered impolite, unless one is specifically asked about it. For example, spontaneously coming out with the information “I have an M.A.” is impolite. Some persons in the community who have higher degrees are known to have actively tried to hide this information when conversing with other DSGS signers. However, if specifically asked, “do you have an MA. degree?”, the polite response is given with a neutral facial expression, avoiding an assertive or proud facial expression, and with no more information added (such as “and I also have a PhD”).

In your face: Im/politeness in signed languages

4.2.3 Bragging by modifying phonological components There are some ways of signing the names of signed languages that are seen as bragging by the DSGS signers consulted for this study. This was particularly true for a variant of the ASL name sign for itself, as shown in Figure 8a. Unlike the normal form which resembles the fingerspelling letters A, S and L, the modified form is produced with an enlarged, fast, sharply whipped movement (Figure 8b). Tissi felt that a similar kind of change to the movement that some signers of German Sign Language /Deutsche Gebärdensprache add to the name of their language (DGS) would also be perceived by DSGS signers as a form of bragging.

a. ASL (‘normal’ production as A-S-L)

b. ASL (with ‘bragging’ movement) Figure 8. Variants of the name sign for ASL: (a) normal form and (b) form with a movement variation that makes it seen by DSGS signers to be boasting

Tissi perceived this movement modification as a form of bragging. She had the same opinion of another ASL name sign for an American deaf signing Facebook group, “ASL-that”. In this sign, the that component of the name is formed with a palm-down Y-handshape together with a large, swiftly slammed down, forward movement (Figure 9b), in contrast to the normal production of that shown in Figure 9a. Tissi showed this sign, as well as the “bragging” versions of the signs for ASL shown in Figure 8 as well as the similar DGS name sign, to a large private Facebook group of professional DSGS teachers who meet regularly online to propose and evaluate new DSGS signs and suggest revisions of existing signs. Approximately 25 members of this Facebook group (both male and female) were active

49

50

Penny Boyes Braem & Katja Tissi

a. ASL-that (normal form)

b. ASL-that! (bragging form)

Figure 9. The ASL sign that in (a) normal form and (b) as modified in the name sign of an American Facebook group

participants in this session. As a part of their normal procedure for evaluating signs, these participants afterwards wrote their evaluations of the signs that were discussed. For the two signs from other sign languages which Tissi showed them, all the participants said that, although some found the modified name signs for other languages amusing (often including the ‘strong laughing’ emoji), they would reject a proposal for modifying their existing sign for DSGS in the same way. Many commented that these were modifications that a DSGS signer simply would not use for the name of their language. As one DSGS commentator put it, “We Swiss are modest, we don’t need to brag”. Clearly, further research with a larger group and other examples needs to be done to determine whether it is not only the larger and faster movements of these signs which helps trigger the perceptions of im/politeness, but perhaps also the resulting extended forward movement towards the addressee, moving beyond the forward boundary of the normal signing space. Bragging is not only perceived in larger versions of a sign. As mentioned earlier, if signs normally produced in the areas of high visual acuity (as shown in Figure 3) are instead produced with smaller than normal movements in the periphery of the signing space, this can also be perceived as being an impolite style of signing. According to Tissi, this conveys the message that “It’s up to you to catch what I’m intending to communicate here, I’m not going to bother to make it optimally perceptible.” 4.2.4 Bragging by using too many “pedagogical” signs An overuse of a DSGS phrase meaning ‘you know’ made with special facial and body behaviors is reported by Tissi to be commonly interpreted as meaning ‘I need to explain to you because I know more than you do’ (Figure 10a). This connotation is reinforced if the phrase is combined with the upright “pedagogical finger” gesture, a gesture which signers report having often seen their teachers using in their school classrooms (Figure 10b).

In your face: Im/politeness in signed languages

a. you-know?

b. “Pedagogical-finger” gestures Figure 10. The sign you know perceived as bragging if used repeatedly: (a) you-know and (b) various forms of the “pedagogical-finger” gesture

Tissi furthermore reports that in situations in which someone constantly produces this pedagogical forefinger, some DSGS signers, in order to express the extent to which this has gotten on their nerves, turn aside and sign privately to another person a sign meaning ‘cut-off-finger’ (Figure 11).

Figure 11. Sign construction meaning ‘cut-off-finger’ used as a private comment on someone else’s excessive use of you-know? or the pedagogical-finger gesture

51

52

Penny Boyes Braem & Katja Tissi

4.3 Discussion: Self praise perceived as a form of im/polite bragging 4.3.1 Self-praise perceived as bragging in different cultures As several perceptions by DSGS signers of im/politeness reported here involve a perception of bragging, it is helpful to see how this topic has been dealt with in different cultures. Boasting is a form of self-praise, which Dayter (2021: 1) describes as “a speech act that gives credit to the speaker for some attribute or possession that is positively valued by the speaker and the potential audience”. (Related to the point made earlier of the prominence of interpreters as early identifiers of much im/polite behavior, the title of Dayter’s 2021 article is “Dealing with interactionally risky speech acts in simultaneous interpreting”.) In the special case of bragging, however, although self-praise might be viewed positively by the speaker, it is perceived by the addressee as annoying or more strongly as facethreatening. Leech (1983), in describing his six “maxims of politeness”, notes that “…the first submaxim of Modesty [maxim] is not to commit the social transgression of boasting” (Leech 1993: 136). How boasting or bragging is perceived can vary according to factors such as gender (Miller et al. 1992) and, most relevant here, culture. Holtgraves and Dulin (1994), for example, looked at to what extent the same written passages from three different types of persons (identified as truthful, non-truthful or neutral persons) were interpreted negatively as bragging by members of two different subcultures: African American students and European American students. The latter negatively evaluated all sorts of positive self-pride statements, or boasting, by both truthful and untruthful boasters, which the authors interpreted as violating Leech’s Modesty maxim. The African Americans, on the other hand, were more accepting of the boasting if it was perceived as coming from a truthful bragger. The authors mention that this is reflected in the example of African Americans generally accepting positively the boxer Muhammad Ali’s many boastful statements. They make the following conclusions: Overall, our results demonstrate how people’s perceptions are filtered through the (often unacknowledged) rules for conversational interaction. Different rules can result in the same behavior having very different meanings, and hence produce cross-cultural misunderstandings… Thus, a behavior that appears to some to be natural, spontaneous, and a cause for good feelings, might be viewed by others as conceited and insensitive. (Holtgraves and Dulin 1994: 283)

4.3.2 Swiss modesty and a heightened aversion to bragging At the time of writing, the topic of Swiss Germans’ modesty and related sensitivity to bragging does not seem to have been the specific topic of any published academic research. There has been some socio-psychological research on social and

In your face: Im/politeness in signed languages

personal values that might be relevant to the topic of bragging. Bennett (2005), for example, looked at the dimension of “collectivism” versus “individualism” in data from 1,000 hearing Swiss citizens aged 20 and older for which he reported that the older generations of Swiss Germans tended more frequently to favor collectivistic and rightist values (stressing national security, conformity and tradition). Bennett (in personal communication) suggests that this dimension of collectivism would explain to a certain extent a reluctance to bragging by these Swiss speakers. In other kinds of non-academic publications, comments on this Swiss aversion to bragging are bountiful. The following are typical examples (all in translations to English from German by the authors): Swiss modesty is legendary. In this country, people like to hide their light under a bushel, acknowledge praise and thanks with a bashful “that’s right” – but are then offended when no one notices how good they are. And that’s what we Swiss really are. (Editorial by Heinrich, 2015 in Werbewoche (Advertising Week) Switzerland wants to shine above all with modesty. That starts at school…Swiss students don’t want to be overachievers. After all, anyone who is one is excluded. (Schreier and Zanni, 2017, in the boulevard newspaper 20 min) The modesty that was Switzerland’s trademark for a long time did not come about by chance. Small countries usually have no airs of great power, or they are quickly exorcised. Moreover, Switzerland was a poor country for a long time, without ores, oil or colonies. Thus, arrogance could never really develop. Modest prosperity was possible but had to be worked hard for. “Being” was therefore more important to the Swiss than “appearances” for a long time. The display of wealth was frowned upon. (Schwarz, 2015, in the blog of Avenir Suisse, a Swiss think tank for economic and social issues)

This characterization of the Swiss as being modest and sensitive to high self-praise is also found in historical writing. For example, Hofe (1943), reports that Gottfried Keller (one of the most famous Swiss authors of the 19th century), viewed Switzerland as being characterized by “complete reliance upon the desirability of democracy for Switzerland, censure of attempts at conversion abroad, and reprobation of vain display” (Hofe 1943: 80). A modest person is one who doesn’t show off, who “doesn’t stick out too much”. An effective way of insulting someone in a Swiss German dialect is to say they “have a big mouth” (hän e grossi Schnuure). The comment of Brown and Levinson (1987: 39) that the raising of the self can be seen as implying a lowering of the other seems especially relevant for a country where there has never been an aristocracy, and where the head of the federal government is not a single political leader, such as a president or prime minister, but a committee of seven

53

54

Penny Boyes Braem & Katja Tissi

persons. Furthermore, as Switzerland is composed of three distinct geographical regions each with its own spoken language (French, Italian, German) or a combination of languages (such as German, Italian and Rhaeto-Romance in the canton of Grisons), the federal government has been found to run most efficiently by consensus and compromise. At the present time, the situation concerning bragging in Switzerland could be summed up as follows. Although there still seems to be no published academic studies on the specific topic of a Swiss cultural aversion to what is perceived as “bragging”, the recognition of a sensitivity to bragging can nevertheless be attested in numerous older and recent non-academic publications, of which the examples presented here are only a small sample. In Switzerland, the speaking culture and the signing culture certainly have many differences. However, the adoption by DSGS signers of the aversion of Swiss German hearing culture to bragging is an indication of how deeply rooted this politeness norm seems to be in both Swiss cultures. This could possibly be due to the value of ‘inclusiveness’ (Hoza 2007a) or ‘collectivism’ (Bennett 2005) shared by both cultures. (For a further discussion of other issues of politeness in languages in contact, see Mapson 2013.)

5.

Summary and suggestions for further research

Several aspects of im/politeness have been described that have been reported in the literature for different signed languages. This includes im/polite behaviors of both the signer and the addressee/s which are based on the necessary visibility of the signer, as well as modifications of manual and non-manual behaviors based on the setting, register, type of speech act, and the signer’s judgement of the addressee/s’ sign language competencies. Also discussed were differences in politeness norms between users of different signed languages that are due to differing cultural experiences of the signers, sometimes during different historical periods. Examples were presented of a sensitivity to bragging in pilot data from DSGS signers, who also regard some forms of signs that are acceptable in other signed languages as an impolite form of bragging. A dislike of bragging has been described here as a trait shared by members of both the dominant speaking and the minority signing Swiss German cultures, which have historically shared the same national governmental, political, and legal situations. We have proposed that this shared cultural setting is a factor that has nourished a general heightened sensitivity to and negative attitude toward boasting that involves an egalitarian wish

In your face: Im/politeness in signed languages

“not to put others down” combined with an individual tendency to try “not to stick out”. Clearly, more research needs to be carried out on all facets of im/politeness in signed languages. Not only does more data need to be collected from different types of signing adults (cf. Appendix A), but also from signing children to see how they learn these politeness norms (such as reported by Baker and Bogaerde, 2020). Systematic studies of what is normally done in different settings and registers of signing (reviewed by Hansen 2012) could be complemented by studies of the perception of im/politeness when the normal conventions in these settings are violated by the signer or addressee. Attention should also be made to im/politeness norms that might be undergoing change. This would especially apply to the norms of the younger generation of both speakers and signers, who through extensive new social media possibilities are exposed to im/politeness norms that are different from those of the cultures in which they grew up. Finally, we hope that the information that has been presented here on im/ politeness in signing can be a contribution toward addressing the broader research goal of identifying the significant similarities and the differences between languages in spoken and signed modalities.

Acknowledgements The illustrations in Figures 1 and 2 are by Katja Tissi. The screenshots of signed examples in Figures 4, 5, 6, 7, 8, 9, 10 and 11 have been permitted by the signer, also Katja Tissi, to be published here.

References Anderson, Leon. 2006. “Analytic Autoethnography.” Journal of Contemporary Ethnography 35 (4): 373–395. https://doi.org/10.1177/0891241605280449 Aristodemo, Valentina, and Carlo Geraci. 2017. “Visible Degrees in Italian Sign Language.” Natural Language & Linguist Theory. https://doi.org/10.1007/s11049-017-9389-5 (Accessed online 14 November 2017). Baker, Charlotte. 1977. “Regulators and Turn-taking in American Sign Language.” In On the Other Hand: New Perspectives on American Sign Language, ed. by Lynn Friedman, 215–236. New York: Academic Press. Baker, Anne, Beppie van den Bogaerde, Roland Pfau, and Trude Schermer (eds.). 2016. The Linguistics of Sign Languages: An Introduction. Amsterdam: John Benjamins. https://doi.org/10.1075/z.199

55

56

Penny Boyes Braem & Katja Tissi

Baker, Anne, and Beppie van den Bogaerde. 2020. “Overlap in Turn-taking in Signed MotherChild Dyadic and Triadic Interactions.” In Understanding Deafness, Language, and Cognitive Development: Essays in Honour of Bencie Woll, ed. by Gary Morgan, 33–52. Amsterdam: John Benjamins. https://doi.org/10.1075/tilar.25.03bak Baynton, Douglas C. 1996. Forbidden Signs: American Culture and the Campaign against Sign Language. Chicago: University of Chicago Press. https://doi.org/10.7208/chicago/9780226039688.001.0001

Bennett, Johnathan. 2005. Fitting Security into the Swiss Value Landscape: Personal and Social Security Concerns in Switzerland. Bern: Peter Lang. Booth, Katie. 2021. The Invention of Miracles: Language, Power, and Alexander Graham Bell’s Quest to End Deafness. New York: Simon & Schuster. Boyes Braem, Penny. 1995. Einführung in die Gebärdensprache und ihre Erforschung [An introduction to sign language and its research]. Hamburg: Signum. Boyes Braem, Penny, and Thüring Bräm. 2000. “A Pilot Study of the Expressive Gestures Used by Classical Orchestra Conductors.” In The Signs of Language Revisited, ed. by Karen Emmorey, and Harlan Lane, 143–167. Mahwah, New Jersey: Lawrence Erlbaum Associates. Boyes Braem, Penny. 2001. “Functions of the Mouthings in the Signing of Deaf Early and Late Learners of Swiss German Sign Language (DSGS).” In The Hands Are the Head of the Mouth: The Mouth as Articulator in Sign Languages, ed. by Penny Boyes Braem, and Rachel Sutton-Spence, 99–132. Hamburg: Signum. Boyes Braem, Penny, and Rachel Sutton-Spence (eds.). 2001. The Hands Are the Head of the Mouth: The Mouth as Articulator in Sign Languages. Hamburg: Signum. Boyes Braem, Penny, Tobias Haug, und Patty Shores. 2012. “Gebärdenspracharbeit in der Schweiz: Rückblick und Ausblick.” Das Zeichen 90: 58–74. Boyes Braem, Penny. 2012. “Evolving Methods for Written Representations of Signed Languages of the Deaf.” In Methods in Contemporary Linguistics: Trends in Linguistics Series, ed. by Andrea Ender, Adrian Leemann, and Bernard Wälchli, 411–438. Berlin: de Gruyter Mouton. https://doi.org/10.1515/9783110275681.411 Brag, Louise (ed.). 2001. Deaf World: A Historical Reader and Primary Sourcebook. New York: New York University Press. Brown, Penelope, and Stephen C. Levinson. 1987. Politeness. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511813085 Caccamise, Frank, Richard Dirst, James Stangarone, and Marilyn Mitchell-Caccamise. 1980. “General Factors to Consider in Interpreting Assignments.” In Introduction to Interpreting for Interpreters/Transliterators, Hearing-impaired Consumers, Hearing Consumers, ed. by Frank Caccamise, Richard Dirst, Rita Dominque DeVries, Joy Heil, Carl Kirchner, Suzie Kirchner, Anna Maria Finaldi, and James Stangarone, 15–31. Silver Spring, Md: Registry of the Interpreters for the Deaf (RID). Caramore, Benno. 1998. Die Gebärdensprache in der schweizerischen Gehörlosenpädagogik des 19. Jahrhunderts [Sign language in 19th-century Swiss education of the deaf]. Hamburg: Verlag hörgeschädigte kinder. Chambers, Diane P. 1998. Communicating in Sign: Creative Ways to Learn American Sign Language. New York: Fireside. Chen Pichler, Deborah, and Helen Koulidobrova. 2015. “Acquisition of Sign Language as a Second Language (L2).” In The Oxford Handbook of Deaf Studies in Language, ed. by Marc Marschark and Patricia Elizabeth Spenser. Oxford: Oxford University Press. https://doi.org/10.1093/oxfordhb/9780190241414.013.14

In your face: Im/politeness in signed languages

Cokely, Dennis, and Charlotte Baker. 1980. American Sign Language: A Teacher’s Resource Text on Curriculum, Methods, and Evaluation. Silver Spring MD: T.J. Publishers. Dayter, Daria. 2021. “Dealing with Interactionally Risky Speech Acts in Simultaneous Interpreting: The Case of Self-praise.” Journal of Pragmatics 174: 28–42. https://doi.org/10.1016/j.pragma.2020.12.010

Dotter, Franz. 2018. “Most Characteristic Elements of Sign Language Texts are Intricate Mixtures of Linguistic and Non-linguistic Parts, Aren’t They?”. Colloquium New Philologies 3 (1). https://doi.org/10.23963/cnp.2018.3.1.1 Edwards, Claire, and Gill Harold. 2014. “Deaf Space and the Principles of Universal Design.” Disability and Rehabilitation 36 (16): 1350–1359. https://doi.org/10.3109/09638288.2014.913710

Engberg-Pedersen, Elisabeth. 1993. Space in Danish Sign Language: The Semantics and Morphosyntax of the Use of Space in a Visual Language. Hamburg: Signum. Ferreira Brito, Lucinda. 1995. Por uma Gramática de Línguas de Sinais [Towards a grammar of sign language].” Rio de Janeiro: Tempo Brasileiro. Fischer, Renata, and Cathrin Jürgensen. 2000. “Gesprächsrollenwechsel (Turn Taking) in Dialogen gehörloser Seniorinnen im Altenheim.” Das Zeichen: Zeitschrift für Sprache und Kultur Gehörloser 51:110–125. Fisher, Jami, Gene Mirus, and Donna Jo Napoli. 2019. “Sticky: Taboo Topics in Deaf Communities.” In The Oxford Handbook of Taboo Words and Language, ed. by Keith Allan. Oxford: Oxford Press. https://doi.org/10.1093/oxfordhb/9780198808190.013.8 George, Johnny Earl. 2010. “Universals in the Visual-kinesthetic Modality: Politeness Marking Features in Japanese Sign Language (JSL).” Proceedings of the Annual Meeting of the Berkeley Linguistics Society 36: 129–143. https://doi.org/10.3765/bls.v36i1.3907 George, Johnny Earl. 2011. Politeness in Japanese Sign Language (JSL): Polite JSL Expression as Evidence for Intermodal Language Contact Influence. PhD. Dissertation, University of California at Berkeley. (https://escholarship.org/uc/item/4jq1v247). Girard-Groeber, Simone. 2015. “The Management of Turn Transition in Signed Interaction Through the Lens of Overlaps.” Frontiers in Psychology 6: article 741. https://doi.org/10.3389/fpsyg.2015.00741

Goldblat, Ester, and Tova Most. 2018. “Cultural Identity of Young Deaf Adults with Cochlear Implants in Comparison to Deaf without Cochlear Implants and Hard-of-hearing Young Adults.” Journal of Deaf Studies and Deaf Education 23 (3): 228–239. https://doi.org/10.1093/deafed/eny007

Groeber, Simone, and E. Pochon-Berger. 2014. “Turns and Turn-taking in Sign Language Interaction: A Study of Turn-final Holds.” Journal of Pragmatics 65: 121–136. https://doi.org/10.1016/j.pragma.2013.08.012

Hall, Stephanie. 1989. “Train-Gone-Sorry: The Etiquette of Social Conversations in American Sign Language.” In American Deaf Culture: An Anthology, ed. by Sherman Wilcox, 89–102. Burtonsville, MD: Linstok Press. Hansen, Martje. 2012. “Textlinguistik: Gebärdensprache im Kontext.” In Handbuch Deutsche Gebärdensprache: Sprachwissenschaftliche und anwendungsbezogene Perspektiven, ed. by Hanna Eichmann, Martje Hansen, and Jens Hessmann, 199–224. Hamburg: Signum. Hesse, Rebecca, Alan Canonica, Mirjam Janett, Martin Lengwiler, and Florian Rudin. 2020. Aus erster Hand: Gehörlose, Gebärdensprache und Gehörlosenpädagogik in der Schweiz im 19. und 20. Jahrhundert. Zürich: Chronos Verlag.

57

58

Penny Boyes Braem & Katja Tissi

Hofe, Harold von. 1943. “Gottfried Keller’s Conception of the Unique Character of Swiss Democracy.” Monatshefte für Deutschen Unterricht 35(2): 73–80. Holtgraves, Thomas, and Jeffrey Dulin. 1994. “The Muhammad Ali Effect: Differences between African Americans and European Americans in Their Perceptions of a Truthful Bragger.” Language and Communication 14 (3): 275–285. https://doi.org/10.1016/0271-5309(94)90005-1 Hoza, Jack. 2001. The Mitigation of Face Threatening Acts in Interpreted Interaction: Requests and Rejections in American Sign Language and English. Boston University doctoral dissertation. Hoza, Jack. 2007a. It’s Not What You Sign, It’s How You Sign it: Politeness in American Sign Language. Washington DC: Gallaudet University Press. Hoza, Jack. 2007b. “How Interpreters Convey Social Meaning: Implications for Interpreted Interaction.” Journal of Interpretation: 39–68. Hoza, Jack. 2008. “Five Non-manual Modifiers that Mitigate Requests and Rejections in American Sign Language.” Sign Language Studies 8 (3): 264–288. https://doi.org/10.1353/sls.2008.0006

Hoza, Jack. 2011. “The Discourse and Politeness Functions of HEY and WELL in American Sign Language.” In Discourse in Signed Languages, ed. by Cynthia Roy, 69–95. Washington, DC: Gallaudet University Press. https://doi.org/10.2307/j.ctv2rh28s4.9 Johnston, Trevor, and Adam Schembri. 2007. Australian Sign Language: An Introduction to Sign Language Linguistics. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511607479

Klima, Edward, and Ursula Bellugi. 1979. The Signs of Language. Harvard: Harvard University Press. Lane, Harlan. 1992. The Mask of Benevolence: Disabling the Deaf Community. New York: Alfred A. Knopf. Lane, Harlan, Robert Hoffmeister, and Ben Bahan. 1996. A Journey into the Deaf-World. San Diego: Dawn Sign Press. Leech, Geoffrey. 1983. Principles of Pragmatics. Longman, London. Lepic, Ryan, and Corinne Occhino. 2018. “A Construction Morphology Approach to Sign Language Analysis.” In The Construction of Words, ed. by Ryan Lepic and Corinne Occhino, 141–172. Berlin: Springer. https://doi.org/10.1007/978-3-319-74394-3_6 Locher, Miriam A., and Richard J. Watts. 2005. “Politeness Theory and Relational Work.” Journal of Politeness Research 1 (1): 9–34. https://doi.org/10.1515/jplr.2005.1.1.9 Loos, Cornelia, Jens-Michael Cramer, and Donna Jo Napoli. 2020. “The Linguistic Sources of Offense of Taboo Terms in German Sign Language.” Cognitive Linguistics. 31(1): 73–112. https://doi.org/10.1515/cog-2018-0077

Mapson, Rachel. 2013. “Politeness in British Sign Language: The Effects of Language Contact.” In Multilingual Theory and Practice in Applied Linguistics: Proceedings of the 45th Annual Meeting of the British Association for Applied Linguistics, ed. by Alasdair N. Archibald, 167–170. London: Scitsiugnil Press. Mapson, Rachel. 2014. “Polite appearances: How Non-manual Features Convey Politeness in British Sign Language.” Journal of Politeness Research 10 (2): 157–184. https://doi.org/10.1515/pr-2014-0008

Mapson, Rachel. 2015. Interpreting Linguistic Politeness from British Sign Language to English. PhD Dissertation, University of Bristol.

In your face: Im/politeness in signed languages

Mapson, Rachel. 2019. “Im/politeness and Interpreting.” In The Routledge Handbook of Translation and Pragmatics, ed. by Rebecca Tipton, and Louisa Desilla, 15–21. London: Routledge. https://doi.org/10.4324/9781315205564-3 Mapson, Rachel. 2020. “Intercultural (Im)politeness: Influences on the Way Professional British Sign Language/English Interpreters Mediate Im/polite Language.” In Politeness in Professional Contexts, ed. by Dawn Archer, Karen Grainger, and Piotr Jagodzinski, 151–178. Amsterdam: John Benjamins. https://doi.org/10.1075/pbns.311.07map Mapson, Rachel. 2021. “Interpreters, Rapport, and the Role of Familiarity.” Journal of Pragmatics 176: 63–75. https://doi.org/10.1016/j.pragma.2021.01.020 Marshall, Catherine, and Gretchen B. Rossman. 2016. Designing Qualitative Research. Thousand Oaks, CA: Sage Publications. McBurney, Susan. 2012. “History of Sign Languages and Sign Language Linguistics.” In Sign Language: An International Handbook, ed. by Roland Pfau, Markus Steinbach, and Bencie Woll, 909–948. Berlin/Boston: Walter de Gruyter GmbH & Co. https://doi.org/10.1515/9783110261325.909

Mesch, Johanna. 2016. “Manual Backchannel Responses in Signers’ Conversations in Swedish Sign Language.” Language & Communication 50: 22–42. https://doi.org/10.1016/j.langcom.2016.08.011

Miller, Lynn, Linda Cooke, Jennifer Tsang, and Faith Morgan. 1992. “Should I Brag?: Nature and Impact of Positive and Boastful Self-disclosures for Women and Men.” Human Communication Research 18 (3): 364–399. https://doi.org/10.1111/j.1468-2958.1992.tb00557.x Mills, Sara. 2003. Gender and Politeness. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511615238

Mindess, Anna. 1999. Reading between the Signs. Intercultural Communication for Sign Language Interpreters. Yarmouth, Maine/Boston MA: Intercultural Press. Mirus, Gene, Jami Fisher, and Donna Jo Napoli. 2012. “Taboo Expressions in American Sign Language.” Lingua 122: 1004–1020. https://doi.org/10.1016/j.lingua.2012.04.001 Mirus, Gene, Jami Fischer, and Donna Jo Napoli. 2019. “(Sub)lexical Changes in Iconic Signs to Realign with Community Sensibilities and Experiences.” Language in Society 49 (2): 1–27. https://doi.org/10.1017/S0047404519000745 Mohr, Susanne, and Katrin Renkwitz. 2020. “Multimodal Politeness in Ireland: A Crosslinguistic Comparison of Irish Sign Language and Irish English.” Presentation at the Symposium on Multimodal Im/politeness. Zurich, 23 October 2020. Moore, Mathew S., and Linda Levitan. 1993. For Hearing People Only: Answers to Some of the Most Commonly Asked Questions about the Deaf Community, its Culture, and the ‘Deaf Reality’. Rochester: Deaf Life Press. Mottez, Bernard. 1981. La Surdité dans la Vie de Tous les Jours. Paris: Centre technique national d’études et de recherches sur les handicaps et les inadaptations, C.T.N.E.R.H.I. Moya-Avilés, Berta. 2020. “Pragmatics.” In A Grammar of Catalan Sign Language (LSC), ed. by Josep Quer, and Gemma Barberà. Online open access https://www.sign-hub.eu /grammardetail/UUID-GRMM-d7eb3bcf-3877-4d92-b15c-edc0c3143604 Napoli, Donna, Jami Fischer, and Gene Mirus. 2013. “Bleached Taboo-term Predicates in American Sign Language.” Lingua 123: 148–167. https://doi.org/10.1016/j.lingua.2012.11.001 Padden, Carol, and Tom Humphries. 2005. Inside Deaf Culture. Cambridge: Harvard University Press. https://doi.org/10.4159/9780674041752

59

60

Penny Boyes Braem & Katja Tissi

Perniss, Pamela. 2018. “Why We Should Study Multimodal Language.” In Visual Language, ed. by Wendy Sandler, Marianne Gullberg, and Carol Padden. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2018.01109

Pfau, Roland, Markus Steinbach, and Bencie Woll. 2012. Sign Language: An International Handbook. Berlin/Boston: Walter de Gruyter. https://doi.org/10.1515/9783110261325 Pyers, Jennie. 2006. “Indicating the Body: Expression of Body Part Terminology in American Sign Language.” Language Sciences 28 (2–3): 280–303. https://doi.org/10.1016/j.langsci.2005.11.010

Radanovic Felberg, Tatjana. 2016. “Impoliteness: A Challenge to Interpreters’ Professionalism.” Ambivalence 3 (1): 1–20. Rittenhouse, Robert K., Calvin Johnson, Betty Overton, Shirley Freeman, and Kyle Jaussi. 1991. “The Black and Deaf Movements in America since 1960: Parallelism and an Agenda for the Future.” American Annals of the Deaf 136 (5): 392–400. https://doi.org/10.1353/aad.2012.0403

Roush, Daniel. 1999. Indirectness Strategies in American Sign Language: Requests and Refusals. Master’s thesis. Gallaudet University, Washington, D.C. Roush, Daniel. 2007. “Indirectness Strategies in American Sign Language Requests and Refusals: Deconstructing the Deaf-as-Direct Stereotype.” In Translation, Sociolinguistic, and Consumer Issues in Interpreting, ed. by Melanie Metzger, and Earl Fleetwood, 103–156. Washington DC: Gallaudet University Press. Roush, Daniel. 2011. “Language Between Bodies: A Cognitive Approach to Understanding Linguistic Politeness in American Sign Language.” Sign Language Studies 11 (3): 329–373. https://doi.org/10.1353/sls.2011.0000

Roy, Cynthia. 1989. “Features of Discourse in an American Sign Language Lecture.” In The Sociolinguistics of the Deaf Community, ed. by Ceil Lucas, 443–457. San Diego, TX: Academic Press. https://doi.org/10.1016/B978-0-12-458045-9.50017-3 Sandler, Wendy. 2018. “The body as Evidence for the Nature of Language.” Frontiers in Psychology 9. https://doi.org/10.3389/fpsyg.2018.01782 Schreier, S. / Zanni B. 2017. “Fluch der Bescheidenheit: Schweizer nehmen sich Mittelmass zum Vorbild” [The curse of modesty: The Swiss take mediocrity as their model]. Blog in 20min.ch, 7. Nov 2017. Schwarz, Gerhard. 2015. “Bescheidenheit ist mehr als eine Zier” [Modesty is more than an adornment]. Blog www.avenir-suisse.ch/bescheidenheit-ist-mehr-als-einezier/. Feb. 17, 2015. Retrieved June 10, 2022. Siple, Patricia. 1978. “Visual Constraints for Sign Language Communication.” Sign Language Studies 19: 95–110. https://doi.org/10.1353/sls.1978.0010 Stokoe, William C. 1960. Studies in Linguistics – Sign Language Structure: An Outline of the Visual Communication Systems of the American Deaf. Buffalo, NY: University of Buffalo. Stokoe, William C., D. C. Casterline, and C. G. Croneberg. 1976. A Dictionary of American Sign Language on Linguistic Principles. Silver Spring, MD: Linstok Press. Sutton-Spence, Rachel, and Bencie Woll. 1999. The Linguistics of British Sign Language: An Introduction. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9781139167048

Sze, Felix A. B., Monica Xiao Wei, and Aaron Yiu Leung Wong. 2017. “Taboos and Euphemisms in Sex-related Signs in Asian Sign Languages.” Linguistics 55 (1): 153–205. https://doi.org/10.1515/ling-2016-0034

In your face: Im/politeness in signed languages

Van Herreweghe, Mieke. 2002. “Turn-taking Mechanisms and Active Participation in Meetings with Deaf and Hearing Participants in Flanders.” In Turn-taking, Fingerspelling, and Contact in Signed Languages, ed. by Ceil Lucas, 73–103. Washington, D.C.: Gallaudet University Press. Vermeerbergen, Myriam, Lorraine Leeson, and Onno Crasborn (eds.). 2007. Simultaneity in Signed Languages: Form and Function. Amsterdam: John Benjamins Publishing Company. https://doi.org/10.1075/cilt.281 Volterra, Virginia, Maria Roccaforte, Alessio Di Renzo, and Sabina Fontana. 2019. Descrivere la Lingua die Segni Italiana: Una Prospettiva Cognitiva e Sociosemiotica [Describing Italian Sign Language: A Cognitive and Sociosemiotic Perspective]. Bologna: Mulino. Volterra, Virginia, Maria Roccaforte, Alessio di Renzo, and Sabina Fontana. 2022. Italian Sign Language from a Cognitive and Socio-semiotic Perspective: Implications for a General Language Theory. Amsterdam: John Benjamins. https://doi.org/10.1075/gs.9 Wilbur, Ronnie B. 2000. “Phonological and Prosodic Layering of Nonmanuals in American Sign Language.” In The Signs of Language Revisited, ed. by Karen Emmorey, and Harlan Lane, 215–243. New Jersey: Lawrence Erlbaum Associates. Zimmer, June. 1989. “Toward a Description of Register Variation in American Sign Language.” In The Sociolinguistics of the Deaf community, ed. by Ceil Lucas, 253–272. San Diego, TX: Academic Press. https://doi.org/10.1016/B978-0-12-458045-9.50018-5

Appendix A. The complex sub-groups of signers according to modality of language and when it was learned Researchers studying sign language acquisition or assessment are confronted with many different types of signers. Chen Pichler and Koulidobrova (2015) have pointed out that for some of these signers there might be effects of learning a language in a new modality in addition to the usual second language learning effects. A convention has therefore been developed to specify whether the language is not only learned as a first language (L1) or a second language (L2) but also whether the language is in the same or a different modality (first modality =M1, second modality =M2). For example, a L1 signer is one who has learned sign language as a first language, and this would also be a first modality. These signers are sometimes referred to as “native” signers. M2L2 refers to a person learning a second language in a second modality. This would designate a speaking person who has learned a sign language as a second language. For some studies, it is further necessary to identify whether the signers involved are themselves deaf or hearing. A deaf person can be described as a ‘deaf L1’ signer, and a hearing person growing up in a deaf family could be identified as a ‘hearing L1’ or a ‘CODA’ (Child of Deaf Adults). A signer, hearing or deaf, could also learn another sign language. For example, in addition to having learned DSGS as a first language, one could then learn ASL, LSF-Suisse or DGS as a second sign language. The designation for the user of these other additional sign languages would be M1L2. If the deaf person has grown up in a hearing family (as do 90% of persons who are born deaf or have been early deafened), there are a variety of different kinds of exposures to a sign language that they may have had. If their hearing parents used sign language early while the child was growing up, then the signer would be a M1L1 signer. Some studies further differentiate as to whether the hearing parents themselves were M1L1 or M2L2 signers). Some researchers

61

62

Penny Boyes Braem & Katja Tissi

would use the designation of ‘later M1L1’ if the deaf child had learned a sign language in a school setting in early childhood. A deaf person who had had no exposure to signing either in the home or in early schooling but learned a sign language later in life would be designed a ‘Deaf M2L2’ signer. In Figure 3, the different types of signers shown are grouped according to both when they learned the language (L1/L2) and the modality of the language (M1/M2) and whether they are themselves deaf or hearing. However, one must keep in mind that the issues of modality and when it was learned don’t alone predict good signers in adult life. Also, when one learned a language (as L1 or L2) does not always overlap with the language preferred by the adult signer. Not included in this figure are other possibly relevant factors such as the nature and amount of exposure to signing in the school setting (e.g., constant exposure to signing in the classroom or only a few hours a week), whether the deaf signer uses a cochlear implantation device or if the signer is atypical in some way (e.g., has any impairments due to aging factors, ASD, etc.).

Figure 12. Designations for different types of signers The most typical late learners of a sign language are hearing M2L2 learners. There are also many professions which require some level of competence in a sign language. For example, both deaf and hearing persons who are L1 and M2L2 signers are active in many countries in the professions of sign language research, interpretation, teaching, as well as a variety of social services.

Appendix B. A historical example of the low opinion about signing held by hearing persons One example of the assumptions that seemed to nurture the low opinions of signing can be found in the life of Alexander Graham Bell (1847–1922), who is most widely known for his invention of the telephone. Bell had a Scottish mother and an American wife who both had become deaf as children. After immigrating to the US from Scotland, he made many contacts with deaf educators and leaders within the American deaf communities, from whom he also learned to communicate in American Sign Language. Booth’s (2021) biography of Bell, entitled “The Invention of Miracles”, has the subtitle: “Language Power, and Alexander Graham

In your face: Im/politeness in signed languages

Bell’s Quest to End Deafness”. The author describes how, despite his close contact with a great many deaf persons, Bell always saw the major aim of his life not as helping hearing persons to communicate long distance through a tele-device, but rather as helping deaf persons to fit into the majority culture of speakers. From his youth, he had been interested in research on speech sounds, an interest that ran in his family from his grandfather through to his father. His grandfather had stated, “Mere voice is common to the brutes as man; Articulation marks the nobler race…” (Booth 2021: 19). Another influential factor for Bell was the necessity for him to learn how to fit into the quite young United States. At the time (the late nineteenth and early twentieth centuries), the nation was still striving to meld together a viable nation of the many immigrants coming from a diversity of nations. Especially pertinent for the topic of politeness is how, as Booth documents, Bell became over the years increasingly influenced by a dominant norm of the hearing culture of his era, the necessity of trimming one’s behavior so as not to seem “different”. This was considered the most likely path to succeed in this growing country. Many Americans (including the coauthor Boyes Braem) remember stories of how their grandparents or great-grandparents never used their native language in the family after they immigrated to the US. Bell thought that not only for his deaf relatives and friends, but for all deaf Americans to succeed, one must prioritize improving their speech. As he grew older, this increasingly fervent endeavor made him into one of the public persons most despised by members of the American deaf community. This cultural attitude requiring “fitting in” was not unique to the United States at the time but was prevalent in many European countries where it has influenced generations of deaf signers to not stick out by signing in public.

63

chapter 3

Politeness in Catalan Sign Language (LSC) Mitigation of criticisms in spontaneous discourse in LSC Berta Moya-Avilés, Gemma Barberà & Carme Bach Universitat Pompeu Fabra

This paper focuses on politeness in Catalan Sign Language (LSC), concretely on criticisms. The main goal is to analyze linguistic mitigation, specifically through the discourse strategy that consists of expressions for restricting opinions. The data comes from a TV show in LSC, which features real and spontaneous productions of Deaf adult signers. The results show the existence of three different strategies: lexical units that work as hedges of subjectivization, the use of the first-person pronoun to emphasize the personal point of view, and the use of the vertical palm sign with a mitigating function. Moreover, particular non-manual markers are also involved, namely, shrugs, head tilts or raised eyebrows, which work at the prosodic and pragmatic level for mitigation purposes. Keywords: Catalan Sign Language (LSC), politeness, criticisms, linguistic attenuation, discourse analysis, mitigation, hedges, non-manual markers

1.

Introduction

Research on sign languages began in the mid-twentieth century because its social, legal and academic recognition came late. The first studies focused on describing the phonological and syntactic basis of these visual-gestural languages (Stokoe 1960), such as its syntax, phonology or morphology. Although in recent years some research has started to delve into the semantics and pragmatics of Sign languages, it is still an emerging area. This paper focuses on the pragmatics of discourse and more specifically on politeness and linguistic mitigation in Catalan Sign Language (llengua de signes catalana, LSC). Studies on politeness began to develop in the 1970s, and different academic views of politeness have proliferated since then (Culpeper et al. 2017). It is widely accepted that politeness is not exactly an individualistic and utterance-based https://doi.org/10.1075/pbns.333.03moy © 2023 John Benjamins Publishing Company

66

Berta Moya-Avilés, Gemma Barberà & Carme Bach

phenomenon, as the traditional approach represented by the classic work of Brown and Levinson (1987) defined. In this sense, Spencer-Oatey (2002: 533) states that “we cannot sensibly divorce linguistic politeness from the social context in which it occurs.” Therefore, politeness phenomena should be addressed by focusing on the interpersonal and social dimensions in human interaction. Subsequent social and discourse approaches on politeness, from studies such as Watts (2003); Arundale (2006) and Terkourafi (2007), support this general claim and emphasize the importance of contextual and interactional analysis. In this paper, we focus on the concept of face, which has been traditionally addressed in politeness research. It has been shown that face is a politeness motivator and that it has a crucial role in interactions in general (Spencer-Oatey 2000). Goffman (1967) initially stated that we all have a public image (i.e., face) that is always exposed in society, and diverse studies have addressed and discussed the relationship between this concept and politeness (Brown and Levinson 1987; Spencer-Oatey 2002, 2005; Arundale 2006; O’Driscoll 2007, 2017; Terkourafi 2007). Broadly, it is generally assumed that face can be threatened in social interactions by face-threatening acts (FTAs), and face-work may therefore be needed to balance the threat. We understand an FTA as an act that may be offensive to face in a given context (Terkourafi 2005; O’Driscoll 2007). These FTAs can affect both the sender’s and the addressee’s image, and strategies can either prevent or repair the damage. The main goal of this paper is to study mitigation (also known as attenuation), which is related to politeness since “it is one of the most frequent linguistic operations to protect or redress a face-threatening act” (Albelda and Estellés 2021: 72). Caffi (1999) argues that attenuation is a strategy that reduces the risks of FTAs for participants in conversations at various levels. Concretely, we address an attenuation discourse strategy consisting of restricting one’s opinion (see Section 2.2). This strategy appears in a sentence such as in my opinion this soup is salty, in which the structure in my opinion allows the sender to express the opinion the soup is salty subjectively, not as an absolute truth, with mitigating purposes. The main motivation of the present study arises from an important issue that has been discussed in the literature, namely the universality of politeness. Kerbrat-Orecchioni (2004) claimed that the desire to maintain image “constitutes a universal framework for politeness phenomena” (Kerbrat-Orecchioni 2004: 47). However, the performance may experience cultural variation. This means that the absence of a particular act may be considered polite in one culture and discourteous in another. According to this assumption, misunderstandings between senders from different cultures may arise due to intercultural communication difficulties.

Politeness in Catalan Sign Language (LSC)

Cultural variation in politeness applies to Deaf1 culture as well (Roush 2007; Mapson 2013). The Deaf community is a social group that has their own social characteristics and behaviors. They have a language, an identity and share some cultural traditions which are different from the majority hearing society in which they live. Also, there are many sociolinguistic factors that differ between the hearing and the Deaf community. This is the reason why, as Roush (2007) and Mapson (2013) showed, there might be some misunderstandings about politeness between hearing and Deaf people. Therefore, it is important to deeply investigate the requirement of face work in Deaf interactions and to analyze the characteristics of politeness in this culture.2 The so-called Deaf-as-direct stereotype is pervasive among the hearing community (Roush 2007; Hoza 2007, 2008) as well as the Deaf community (Hoza 2007). This stereotype is based on the idea that Deaf people are too direct when they sign, and this can be perceived as rude manners for the hearing counterpart. However, it is important to note that this perception is based on the politeness conceptions of hearing people. For instance, Deaf people in Catalonia – as individuals that are part of a particular culture, which are the focus of this paper – have their own way to show politeness and mitigate discourse, but as yet there is no research about it. The present research has two main goals. The first goal is to describe the mitigating behaviors in LSC, concretely through opinion expressions, in spontaneous discourse and in criticisms. The second goal is to establish correlations between different politeness strategies or non-manual markers and mitigation discourse strategies. To achieve these goals, we mainly focus on three different strategies in LSC involving (1) lexical units, (2) first-person pronominal signs, and (3) the vertical palm sign. As will be shown, non-manual markers (NMMs) co-occurring in the three constructions play a crucial role in the mitigation strategy. The rest of the paper is organized as follows. Section 2 addresses the theoretical background and the state of the art in current research. Section 3 describes the methodology used, both to collect and to analyze data. Section 4 and 5 show and analyze the results, respectively. Finally, Section 6 highlights the main conclusions of the present research.

1. In this paper the term Deaf is used in capital letters to refer to individuals who are part of a community with its own identity and culture, i.e., Deaf community. It is used in lowercase when we refer only to the non-hearing condition. 2. Note that politeness not only changes depending on whether it is a hearing or Deaf culture. In the same way that there are different hearing cultures that have their own politeness behavior, politeness may also change depending on the Deaf culture. For instance, white signers of American Sign Language (ASL) perceive as impolite the Black ASL community. See Bayley et al. (2017) for more information.

67

68

Berta Moya-Avilés, Gemma Barberà & Carme Bach

2.

Background

2.1 Politeness, face, and face-work Regarding politeness theory, Holmes et al. (2012: 1064) state that “politeness theory is one of the most rapidly developing areas of pragmatics with a constant refinement of theoretical approaches and re-definition of core concepts.” In fact, Culpeper et al. (2017: 2) affirm that “there is no one-size-fits-all definition of politeness or impoliteness for the simple reason that the research questions that drive the field are diverse, encompassing a whole range of different theoretical and methodological stances.” In general terms, Leech (2014: 3) holds that politeness is “to speak or behave in such a way as to (appear to) give benefit or value not to yourself but to the other person(s), especially the person(s) you are conversing with.” Among other characteristics, it is important to mention that politeness (1) is not obligatory, (2) depends on the situation (certain activity types are per se more polite or more impolite), (3) there are varying gradations of polite and impolite behavior, and (4) members of society can recognize norms of how to be polite for a particular occasion (Leech 2014: 4–5). To take other examples, Mills (2003) addresses the contextual and social factors of politeness, such as gender, age or ethnicity, and Spencer-Oatey (2000, 2002) focuses on the relational and interactional behavior involved. One prominent approach to politeness is the relational work of Locher and Watts (2005). The authors define it as “the ‘work’ individuals invest in negotiating relationships with others” (Locher and Watts 2005: 10) and it includes everything from the most cooperative to the most aggressive attitudes, all conceptualized on a continuum from the politest to the most impolite. Thus, a key concept of Locher and Watts’s (2005) approach is “appropriateness”: in a given communicative situation a behavior can be perceived as marked (i.e., it goes beyond the expected) either in a positive or negative way, or as non-marked (unnoticed), which will be politic/appropriate. The participants evaluate behaviors and perceive their appropriateness in relation to politeness. In sum, according to Locher and Watts (2005: 17) relational work “posits that social behavior which is appropriate to the social context of the interactional situation only warrants potential evaluation by the participants (or others) as polite or impolite if it is perceived to be salient or marked behavior.” Thus, Locher and Watts (2005) contend that a discourse approach is needed to understand politeness as a behavior inherent to human relations, and they state that politeness is just a part of this relational work. As introduced in Section 1, this discourse and interactional turn in approaches to politeness define the phenomenon taking into account not only lin-

Politeness in Catalan Sign Language (LSC)

guistic expressions, but also social and interpersonal factors, as well as the context of interaction, as crucial elements to understand it in a more comprehensive way. Hence due to the discursive nature of politeness, it should be addressed from a discourse analysis perspective (Garcés-Conejos Blitvich and Sifianou 2019). Another central aspect of our study is the concept of face, which has been widely studied and debated in politeness studies. Goffman (1967: 5) defined face as “the positive social value a person effectively claims for himself by the line others assume he has taken during a particular contact”. By implication, Goffman (1967) argued that face-work is used to counteract events that threaten face. An interesting perspective on the concept of face and face-work is presented by Haugh (2005), since this study highlights the value of culture-specific features of face. Even though face-work was initially theorized as a synonym of politeness by Brown and Levinson (1987), much subsequent research in the area showed that face can be one motivator for politeness, but it is not the only one (Spencer-Oatey 2002; Locher and Watts 2005; Arundale 2006; O’Driscoll 2007; Culpeper et al. 2017). O’Driscoll (2007: 467) argues that face-work “is widely accepted as a useful term to denote actions which have a bearing on face”. It is also clearly recognized that face exists in all human beings: “face is universal simply because interaction is universal” (O’Driscoll 2017: 93). The universality of face and politeness have been widely discussed. Scholars such as Gu (1990) or Matsumoto (2003) state that the first theory of politeness by Brown and Levinson (1987) was biased towards Anglo-Saxon culture, as mentioned above, which is more individualistic than collective, the opposite of Asian cultures. Therefore, “a universal theory of linguistic politeness must take into account at a more fundamental level the cultural variability in the constituents of ‘face’” (Matsumoto 1988: 403, 2003: 1516). For instance, Haugh (2005) proposes that politeness in Japanese is not motivated by the notion of face, but rather by the concept of “place”. In western societies, politeness strategies reflect the importance of individual rights, while in other cultures group duties are more important. Thus, face needs to be addressed from a cultural perspective “as an alternative and possibly more fruitful way of studying the relevance and dynamics of ‘face’ and ‘facework’ in interpersonal contacts” (Bargiela-Chiappini 2003: 1463). In the present study, focused on politeness in LSC, face-work is an essential aspect since we focus on criticisms in a TV show in this culture. Leech (2014: 191) defines criticism as a “speech event in which S [speaker] says something that is (either directly or by implication) uncomplimentary or derogatory about O [other person]”. Leech (2014) also states that different tactics to mitigate criticisms exist, such as the use of what he called downtowners, such as a bit, a little, or the use of comparative forms, among others. Due to the specific context of our corpus (see Section 3), correct management of the offense that the criticisms entail in this

69

70

Berta Moya-Avilés, Gemma Barberà & Carme Bach

particular context is needed. That is, face-work is crucial for the participants to achieve balanced interactions, and mitigation is the central issue we address in this regard.

2.2 Mitigation strategies: Attenuation and opinion expression The present study focuses concretely on linguistic mitigation, also known as attenuation. Albelda and Estellés (2021) review the different approaches in this field, starting with the semantic study of Lakoff (1975) about hedges, and the later pragmatic perspectives focused on the strategic possibilities of mitigation, i.e., the role of hedges in interaction. Mitigation has also been linked to politeness by the literature in this field, and according to Albelda and Estellés (2021) most of the authors agree that hedges can be used with mitigating purposes to reduce risks in communication. Even though proposals about mitigation differ according to perspective, i.e., socio-relational (related to verbal politeness only regarding FTAs) or rhetoricallinguistics (communicative functions and effects of mitigation, among others), Albelda and Estellés (2021: 72) argue that some of the research has offered comprehensive proposals, e.g., Caffi (1999, 2005). Moreover, other approaches take into account multimodality, such as Hübscher et al. (under revision). This study focuses on how facial and body cues are used in requests in spoken Catalan and shows differences in the frequency of the use of facial and body movements depending on the polite or non-polite condition of a given context. Hübscher et al. (under revision) argue that “the expression of politeness-related meanings is accomplished through a variety of articulators and modalities, involving lexical hedges and morphosyntactic strategies, as well as different prosodic markers and different multimodal strategies.” Caffi (1999) defines mitigation as a synonym of attenuation that consists of weakening one of the interactional parameters, and it is used to reduce the risks of an FTA. This is because it reduces obligations for the interlocutors and permits “smooth management of the interactions and reduces the risks (refusals, conflict, losing face, etc.)” (Caffi 2005: 40). Thus, attenuation can be an attempt to mitigate a real or a potential FTA. Caffi (1999) argues that mitigation can affect epistemic modality (which is related to the degree of certainty) and that hedges are used for that purpose. Hedges have been extensively addressed in studies on mitigation and politeness. For instance, Brown and Levinson (1978: 145) defined a hedge as “a particle, word or phrase that modifies the degree of membership of a predicate or a noun phrase in a set; it says of that membership that it is partial or true only in certain respects […].” We are particularly interested in the classification of Namsaraev (1997), who identifies a hedging strategy consisting of subjectiviza-

Politeness in Catalan Sign Language (LSC)

tion. That is, “using I + think/suppose, assume and other verbs of thinking with the purpose of signaling the subjectivity of what is said, as a personal view instead of the absolute truth” (Namsaraev 1997: 67). Related to hedges and epistemicity, Boncea (2014: 11) states that “when used epistemically as hedging elements these verbs express […] the speaker’s unwillingness to vouch for understanding the utterance as more than a personal opinion.” In the Spanish literature about mitigation in general, Albelda and Cestero (2011: 5) define attenuation as a pragmatic strategy in communication used to minimize the effect of what we say or do. In this respect, Briz and Albelda (2013) argue that attenuation is a strategic activity to reduce the illocutive force, i.e., it is used intentionally to reduce the intention of the sender. Briz and Albelda (2013) also claim that, as a linguistic strategy, attenuation must be addressed from a pragmatic perspective, since it can only be defined by the specific context of interaction. Accordingly, Briz (2004) argues that real interaction (rather than face-work, see Section 2.1) is what ultimately activates attenuation. This is why it can not only be defined systematically by face-work because it is a phenomenon defined mainly by real conversation. Briz and Albelda (2013: 13) go one step further and state that to study attenuation it is not enough to define the general context of communication, but rather the specific situation of the interaction is needed, what they call “interactional context”. This context includes (1) the affected speech act, (2) the preceding and following sequences of the sender and the addressee, and (3) the position of the attenuated element in the sentence. In addition, other factors are considered, such as the age and gender of the interlocutors, the level of education, the place where they talk, etc. We have based our analysis to a large extent according to the Spanish literature due to (1) the proximity of the territory and cultures between spoken Spanish and LSC, and (2) the detailed and useful taxonomy of the previous studies in this area. Focusing especially on attenuation procedures, Albelda et al. (2014) present a taxonomy of twenty-two attenuation strategies that may be used in spoken Spanish. We focus here on a strategy related to the subjective expression of opinions – which, in fact, is similar to Namsaraev’s (1997: 67) classification of hedges. According to this classification, the sender voluntarily restricts the expression of his/her opinion to his/her own viewpoint and does not state it as a general or absolute truth. Expressions like in my opinion, I think that, in my view, for me are such examples in English. In our corpus, the expressions for restricting opinions are used by the participants to mitigate criticisms they are expressing out loud. As it will be shown in Section 4, a sentence such as In my opinion the soup is salty is used instead of The soup is salty, in which the structure In my opinion helps to mitigate the criticism. The information is presented from the sender’s perception, not as a general truth, similar to what Hyland (2005) identifies as the roles of hedges

71

72

Berta Moya-Avilés, Gemma Barberà & Carme Bach

in specialized discourse. This strategy is used to reduce the strength of the FTA that a criticism clearly entails. To the best of our knowledge, this strategy has not been the focus of attention in any research on sign language to date.

2.3 Previous research on politeness in sign languages Research about politeness in sign languages started at the beginning of this century. So far, few sign languages have been the focus of attention, namely American Sign Language (ASL, Roush 2007, 2011; and Hoza 2007, 2008), Japanese Sign Language (JSL, George 2011), British Sign Language (BSL, Mapson 2013, 2014), and Sign Language of Venezuela (LSV, Pietrosemoli 2001).3 Also, new research in this area has been presented lately based on Swiss German Sign Language (DSGS, see Boyes Braem and Tissi, this volume). Previous studies define for the first time which non-manual markers are used in different sign languages to mitigate certain types of speech acts, taking into account the three social variables presented by Brown and Levinson (1987). For instance, Hoza (2008) identified that tight lips are used to mitigate requests. Roush (2007) also found different linguistic strategies that mitigate at the discourse level, such as justifications or promises to minimize requests. George (2011) was the first to argue that polite non-manual markers – he found eight different ones – are prosodic markers. Mapson (2014) also stated that when politeness appears in discourse, non-manual markers need to be understood at the prosodic level. That is, they can work at different levels of language (phonological, morphological, syntactic, etc.), but visual prosody conveyed by the upper body and the face has a crucial role in politeness (see Section 2.4). Mapson (2014) argued that previous studies lack focus on prosody partially because of the methodology they use for collecting and analyzing data. She also explained that there are problems with the categorization of politeness strategies in sign language when using frameworks that were developed for spoken languages, as done in some other previous studies. Moreover, apart from the three social variables of Brown and Levinson (1987), Mapson (2014) also included the influence of the Deaf or hearing condition of the addressee in order to understand politeness in terms of multiculturality. Mapson (2014) also focused not only on directive speech acts but on apologies as well, which are considered expressive speech acts. This work described for the first time that tight lips are used in apologies too. Even though previous research laid the foundations of politeness studies in Sign languages, it is also true that our study differs from them in some respects 3. The LSV study has a very different approach from our research, hence its account is not considered in the present paper.

Politeness in Catalan Sign Language (LSC)

that we review below. This may be the reason why the methodologies and especially the results may differ from ours. On the one hand, none of the previous research presented above addressed criticisms. Except for Mapson (2014), who focused on apologies, all the research available on other sign languages is about requests and rejections. On the other hand, concerning the methodology, the data collected – apart from Roush (2007), who analyzed videos previously recorded for other purposes – was obtained via elicitation methods, which is not the case in our study. Therefore, our methodology differs from previous research presented above, as explained in detail in Section 3. Before presenting the methodology, we introduce in the next section some linguistic elements that are relevant for the present analysis.

2.4 Non-manual markers, doubling and vertical palm sign The vast evidence from more than sixty years of research supports the claim that sign language grammar is structured with the hands but also with non-manual markers. Numerous studies show that they are relevant to all grammatical components and that they have a role at the phonological, morphological, syntactic, semantic, pragmatic and prosodic levels (Sandler and Lillo-Martin 2006; Pfau and Quer 2010; Wilbur 2021). Grammatical non-manual markers include movements of the head and the torso, movements and changes in facial expression (namely eye blinks, head tilts, establishing or avoiding eye contact, tight lips, raised and furrowed brows), movements of the shoulders, and movements of the cheeks and lips, to name just a few. In this paper, we go beyond the discussion of whether non-manual markers come from gestural and affective facial expressions (see Benítez-Quiroz et al. 2016 for discussion), and we focus on grammaticalized non-manual markers that contribute a pragmatic role and convey mitigation strategies in LSC. Non-manual markers include a number of independent channels whose production is simultaneously expressed with manual signs: torso and shoulder position, head position, forehead and eyebrow position, eye gaze, and nose, mouth, tongue and cheek position. Wilbur (2000: 224) points out that, while the lower portion of the face is used to provide adverbial and adjectival information, such as with tight lips, the upper half of the head and the body have clear prosodic and syntactic functions, such as with blinks, eyebrows, gaze, head tilts, head affirmations or trunk inclinations, among others, to mark questions, relative clauses, conditionals, topics, etc. For instance, Sandler (2012) states that head tilts can be used for focalizations and also convey negation. It is important to note that the same non-manual markers may convey different meanings and functions, e.g., head tilts may express changes of meaning, conditional clauses or negation. Other

73

74

Berta Moya-Avilés, Gemma Barberà & Carme Bach

interesting examples are wrinkled noses used to convey an evaluative meaning (Wilbur 2000) or furrowed eyebrows, which are also relevant to epistemic interpretation (Pfau et al. 2012: 199), among many other functions. More recently, Bross and Hole (2017) have identified a clear correlation with functional projections in cartographic syntax. In particular, the highest categories (speech acts, evaluation, epistemic) are mapped to the upper face (eyebrows and eyes) in German Sign Language (DGS, Bross and Hole 2017) and Turkish Sign Language (TİD, Karabüklü et al. 2018). Slightly lower categories (scalarity, gradience) are expressed on the lower part of the face in DGS (Bross and Hole 2017) and ASL (American Sign Language, Wilbur 2021). Wilbur (2000) also claims that, regardless of place of articulation or articulators, the onset and offset provide a clear indicator of function: abrupt for syntactic functions and gradual otherwise. As we already presented in Section 2.3, when politeness arises in discourse, non-manual markers seem to work at the prosodic level (Mapson 2014). In sign languages, we talk about visual prosody, expressed both with the upper body – for stress – and the face – for intonation – (Dachkovsky and Sandler 2009). Sandler and Lillo-Martin (2006: 259) argue that “grammatical facial expressions in sign languages are best understood as intonational tunes”. Thus, we talk about facial intonation. For instance, we can understand raised eyebrows in sign languages having the same function as high pitch in spoken languages. Also, hands can be used to convey rhythm and speed, which involves prosody as well (Dachkovsky and Sandler 2009; Sandler 2012). Eye blinks, eyebrows, gaze, head tilts, and body leans are examples of non-manual markers that function at the prosodic level (Wilbur 2000). Barberà (2016) also claims that in LSC other non-manual markers, such as raised shoulders and shrugs, are found in indefiniteness and non-specificity contexts conveying uncertainty. After a detailed annotation of a small-scale corpus in LSC, Barberà (2016: 27) proposes an analysis of shrugs in terms of semantic epistemicity: the referential expression is uttered with a shrug when the discourse referent is unknown. Concerning epistemicity, Lackner (2017) shows that in ÖGS (Austrian Sign Language) a side-to-side movement is a speculative marker used to indicate that there are alternatives, and also that the head tilt is a non-manual marker to indicate possibility. As will be shown in Section 4.3, these non-manual markers mentioned above are relevant also in our findings regarding the expression of politeness, since epistemicity is involved in opinion expressions as well. In the rest of this section, we focus on some aspects related to the features of pronominal signs and doubling in LSC. Since the sender needs to be explicitly present when using subjective expressions for restricting opinions it is important to know in broad terms how the first-person personal pronoun and possessive are expressed in LSC. On the one hand, the first-person personal pronoun is per-

Politeness in Catalan Sign Language (LSC)

formed with a pointing sign directed towards the body of the signer (glossed as ix1 in Figure 1). On the other hand, the first-person possessive pronoun my (meaning ‘my’ or ‘mine’) can be expressed with the palm of the dominant hand directed towards the chest of the signer (Figure 2). Quer and GRIN (2008) argue that in LSC the possessor may be expressed with a personal pronoun if it belongs to the group of discourse participants. This means that the sign ix1 may adopt a possessive value meaning ‘me’, ‘my’ or ‘mine’, as well as ‘I’. Furthermore, non-manual markers also play a crucial role in the expression of pronouns. Veiga-Busto (2021) in her study of the pronominal system in LSC shows that the eye gaze expressed with pronouns referring to the signer may be either directed to the addressee or to other locations. This means that when the sender performs ix1 they can look either to the addressee or to other points in space avoiding eye contact.4

Figure 1. Personal pronoun ix1

Figure 2. Possessive pronoun mine

Pronoun doubling has been found in our data. Kimmelman (2013: 100), on a study based in Russian Sign Language (RSL) and Sign Language of the Netherlands (NGT), claims that “doubling, whereby some constituent occurs twice referring to the same object or action, is commonly attested in many signed and

4. All participants have signed the informed consent for the use of personal data. The images of the figures in the paper are screenshots from the video we analyze, which are in the public domain, available on the Webvisual TV channel. © 2019 Webvisual. Accessible at https://www .webvisual.tv/ This applies to all figures in the paper. See Section 3 for detailed information.

75

76

Berta Moya-Avilés, Gemma Barberà & Carme Bach

spoken languages.” The elements doubled can be clause-internal and involve reduplication of nouns, wh-words, modal verbs, or adjectives. Also, Kimmelman (2013) found that in NGT pronoun doubling conveys a specific function called “topic copying”, and it can be used to emphasize the topic of the sentence. In general, both in NGT and in RSL doubling conveys emphatic functions. Last but not least, a particular sign that we gloss as vertical-palm sign is also relevant in the findings of our data. Kendon (2004) analyzes a similar gesture that he calls Vertical Palm gesture, which pertains to the subfamily of the Open Hand Prone family (or palm down family, either vertical or horizontal) and it is performed with the hand facing directly away from the sender. Kendon (2004) argues that the Vertical Palm gesture, which is sometimes performed together with a head tilt, is used to stop a line of action, e.g., a communicative action or mental activity; that is, “the speaker uses the flat surface of the hand to establish a barrier, […] to push something away, or to keep something from rising up before him” (Kendon 2004: 262). Moreover, if the hand is placed close to the sender’s body, it stops its own line of action. From the perspective of signed languages, GabarróLópez (2020) analyzed the palm-up sign – a bimanual sign executed by raising the palms of the hand looking up – as a discourse marker of the family Open Hand Supine (or palm up family, either forwards or upwards), which is included in the bigger category of the same family as the Open Hand gestures proposed by Kendon (2004). Both the vertical-palm and the palm-up pertain to the bigger family of Open Hand gestures from Kendon’s (2004: 248) taxonomy, and they are both characterized by the open hand shape “held with all digits extended and more or less adducted (they are not ‘spread’)”. Although Gabarró-López (2020) explains that palm-up is a multi-functional discourse marker, she did not find a mitigation function in her corpus in LSC. According to Gabarró-López (2020) the functions assigned to palm-up were agreement, monitoring, opening, or punctuation, among others. In this regard, Engberg-Pedersen (2021) found that in DSL (Danish Sign Language) the palm-up sign together with a side-to-side movement at the end of the sentence is used to mark uncertainty. As will be shown in the results (see Section 4.3), in our case the vertical-palm sign (which pertains to the same family as palm-up, namely Open Hand family gesture) is related to epistemic modality as well, and it is also crucial in mitigation contexts. Since there is no previous research about vertical-palm in any other signed languages, we consider research based on a similar sign from the same family (i.e., palm-up sign) as a point of departure. Note that Özyürek (2012) explains that in signed languages, gestures may become lexicalized when they acquire a grammatical meaning, and therefore they may acquire additional grammatical functions. This is the case for the palm-up sign: “the palm-up presentation gesture, for example, has

Politeness in Catalan Sign Language (LSC)

taken on the function of a discourse marker in several sign languages” (Özyürek 2012: 641), as is the case in LSC.

3. Methodology As argued in Section 2, linguistic attenuation is a pragmatic phenomenon that when possible should be studied in spontaneous communicative contexts. We consider that the data studied needs to be as naturalistic as possible in order to obtain results close to the linguistic reality. However, “although participant observation may yield the most natural ethnographic data […] it may take years to collect the data in question” and “the use of video equipment, which is necessary in signed language research, poses even more problems with regard to naturalness” (Roush 2007: 120–121). Based on Roush’s (2007) formula, we therefore analyzed LSC videos of interactions with a fairly high degree of spontaneity that were previously recorded for other purposes, but susceptible to include varied forms of linguistic attenuation. The data analyzed in this paper comes from two episodes of a television program entitled Mans a la massa5 from the online LSC television channel called Webvisual.tv.6 Specifically, we focus on the first episode of the first season, which lasts 33 minutes and 24 seconds, and on the first episode of the second season, which lasts 53 minutes and 32 seconds. The program is a culinary competition involving five Deaf people in each episode: four different participants in each episode, and the same presenter in both episodes (see Table 1). The range of ages coincides with the middle group of signers of the LSC Corpus Projects (Barberà et al. 2015). All participants have signed informed consent forms for personal data protection. Following the standards of sign language research, codes are used to maintain the anonymity of the participants. All the participants were aware of the cameras, the goals of the TV show related to food criticism and the general procedure. However, it is worth mentioning that the interventions were not previously prepared. That is, the planning of their interventions is quite similar to a real context in which the participants can of course think before intervening, but without preparing the text beforehand. In order to contextualize the data presented, we briefly describe how the TV show works. In each episode, the participants visit the cook’s house, and the cook has to prepare a meal for the others. In the first episode of the first season, the participant being evaluated is S2, who cooks for the rest of the participants, including 5. Available in https://www.webvisual.tv/?s=mans+a+la+massa 6. Available in https://www.webvisual.tv/

77

78

Berta Moya-Avilés, Gemma Barberà & Carme Bach

Table 1. Information about the participants Signer

Role

S1

Sex

Age

Season

Participant Female

Between 45–65

One

S2

Participant Female

Between 45–65

One

S3

Participant Female

Between 45–65

One

S4

Participant Male

Between 45–65

One

S5

Participant Female

Between 45–65

Two

S6

Participant Female

Between 45–65

Two

S7

Participant Male

Between 45–65

Two

S8

Participant Female

Between 45–65

Two

SP

Presenter

Between 45–65

One and two

Male

the presenter. In the second season, the cook is S6. All the Deaf signing participants are gathered at the table except for S2 and S6 respectively, who are only in charge of cooking and serving the dishes. The function of the presenter, SP, is to guide and ask questions to the rest (S1, S3 and S4 in the first season and S5, S7 and S8 in the second season) about their opinions on the meal. The participants respond, talk to each other spontaneously and rate the experience. The conversations are mainly focused on evaluating and criticizing the food and the service protocol, and this is why the data is very well suited for our study. The methodology used in the present study is qualitative. The episodes were annotated with the software ELAN,7 a multimodal tool that allows synchronization, annotation and transcription of videos. In our case, seventeen annotation tiers were created for each signer i.e., a total of one hundred seventy analysis tiers were annotated. Moreover, inside some tiers we created what is called a “controlled vocabulary”, i.e., specific predefined settings to facilitate and accelerate the annotation task. In order to simplify and clarify the methodological process and the results here, only the more relevant annotation tiers and some of the items of the controlled vocabulary are listed in the table below. Some of these tiers were based on the taxonomy and methodology of data analysis of Albelda et al. (2014) (see Section 2.2), while others are based on our proposal for the annotation of non-manual markers in the criticisms we analyze. As shown in Table 2, some of the tiers required a deeper analysis than others, so transcription and a first analysis were done simultaneously. Note that as we

7. ELAN (Version 6.3) [Computer software]. 2022. Nijmegen: Max Planck Institute for Psycholinguistics, The Language Archive. Retrieved from https://archive.mpi.nl/tla/elan

Politeness in Catalan Sign Language (LSC)

Table 2. Selection of the main annotation tiers in ELAN Name of the tier

Content

1. Gloss

Transcription of all signs of the intervention

2. Face affected

Annotation of which face, among all participants, is attacked by an FTA in the intervention: – Controlled vocabulary: neg-face present addressee; neg-face nonpresent addressee; self-face (only the sender’s face is threatened).

3. Position

Annotation of the position of the strategy in relation to the attenuated segment: – Controlled vocabulary: integrated; middle-parenthesis; middlereformulation; initial; final; complete intervention

4. Number

Annotation of the total number of strategies expressed in the same intervention

5. Eyebrows

Eyebrow movement annotation: – Controlled vocabulary: raised; furrowed

6. Eyes

Eyes movement annotation: – Controlled vocabulary: open; squint; closed; blink; eye contact; no eye contact

7. Lips

Lip movement annotation: – Controlled vocabulary: tight; kiss; mm; smile; U; side; whistle; sh.; stretched

8. Head

Head movement annotation: – Controlled vocabulary: head shake; head tilt; head nod; negation; head forward; head backward; right-left movement; head-ear; chin up; others

9. Shoulders

Shoulder movement annotation: – Controlled vocabulary: shrug; shake; forward; backward; side-to-side

10. Other Non‑manual markers

Annotation of other non-manual markers not covered

have developed in Section 2, we address criticisms, so we decided to transcribe selectively and annotate only those interventions – taking into account, of course, the previous or subsequent intervention when appropriate. The transcription was done by the first author of the paper, and some of the issues were discussed with two Deaf native researchers. Once the transcription was completed, all the annotations were transferred to an Excel table. Other relevant information was added a posteriori in a database, such as whether particular signs were bimanual or monomanual and what implications this had in terms of attenuation.

79

80

Berta Moya-Avilés, Gemma Barberà & Carme Bach

4. Results In our corpus, we found thirty-three instances of expressions for restricting opinions used to mitigate criticisms in twenty-seven interventions. The appearances per participant are as follows: S1 uses this strategy on three occasions, S3 on seven occasions, S4 on nine occasions, S5 on one occasion, S7 on eleven occasions, S8 on zero occasions, and SP on three occasions. According to our data, opinion expressions are mostly conveyed with three different strategies that may co-occur with particular non-manual markers. First, lexical units can be expressed with a shrug. Second, the pointing sign is used as a personal pronoun, sometimes doubling, and sometimes co-occurring with eye gaze, avoiding eye contact and mouthing. Finally, we found the vertical palm sign, which is co-expressed with head tilts and shrugs in final position. The following sections delve into each strategy.

4.1 Expressions for restricting opinions with lexical units Opinion expressions are conveyed from a restricted point of view through different strategies involving lexical units, either alone or in combination. The first strategy seems to follow the most prototypical structure using expressions like “In my opinion” (see Section 2.2). We use the term “lexical units” when the signers use nouns such as opinion, verbs like think, adverbs like person (i.e., ‘personally’) or expressions such as my perspective (i.e., ‘in my perspective’) that function as hedges. Specifically, we found the following ten tokens presented in (1) with approximate translations:8 (1) a.

[…] my opinion ‘… in my opinion.’ b. ix1 opinion my […] opinion. ‘My opinion is that […], in my opinion.’ c. ix1 think, see, my point-of-view […] ‘I think… what I see… From my point of view…’

8. This article follows the usual glossing conventions in sign language literature. Manual signs are represented by the capitalized word corresponding to the most general translation of the sign. The following abbreviations are used: ix refers to pointing signs; numbers are placeholders indicating 1st, 2nd or 3rd person; subscripts indicate the presence of non-manual markers and the horizontal line shows the extension of their scope: h. ti indicates a head tilt; h. ti r indicates a head tilt to the right; h. ti l indicates a head tilt to the left; sh indicates shrug; tight l indicates tight lips. cl. indicates the occurrence of classifiers; lower case letters represent the actions with classifiers. Letters separated by dashes represent the letters of the manual alphabet.

Politeness in Catalan Sign Language (LSC)

d. […] ix1 opinion ‘… in my opinion.’ e. ix1 person[…] ‘I personally…’ f. ix1 say[…] ‘What I say is…’ g. say[…] ‘I say that…’ h. say[…] ‘I say that…’ i. ix1 think[…] ‘I think that…’ j. my my […] ‘For me…’

opinion is the only noun used in the corpus, the verb think appeared twice, and then say (only used by S7) and see (by S4) on one occasion. Also, the expression my point-of-view (‘From my point of view’) is used once. The first-person possessive pronoun my, which in its citation form is monomanual, is performed bimanually in (1a). This change on the articulators – from one hand to two – contributes to emphasizing the expressions for restricting opinions, and to the understanding that it is just a personal opinion rather than an assertion. However, it only appears once, so it does not seem to be systematic in our corpus. Finally, note that the structures presented in (1) are only expressed by S4 and S7, who are the male participants. Regarding the use of non-manual markers co-occurring with these lexical units, although no strictly systematic pattern was found, it is observed that in more than half of the cases (six out of ten) it is common to shrug simultaneously with the expression of the lexical units (Figure 3).

Figure 3. Signs ix1 and opinion with a shrug in the second still

81

82

Berta Moya-Avilés, Gemma Barberà & Carme Bach

4.2 Expressions for restricting opinions with pointing signs Another strategy to restrict one’s opinion found in our corpus is the use of the first-person personal pronoun ix1 not meaning ‘I’ but ‘for me’. Eight instances were found. As previously mentioned, this personal pronoun may convey a possessive value too in LSC (see Section 2.4). (2) need more garlic ix1. ‘For me, this needs more garlic’.

In (2) the pronoun with possessive value (in bold type) at the end of the sentence restricts the claim that the sender is expressing. The result is the attenuation of the criticism since the pronoun allows the addressee to understand that the information is presented from a personal point of view, not as an assertion of real truth, and could therefore be debated. This pronoun with possessive value may also be found before a criticism, as shown in (3): (3) this good not. only good what? this from here. palm-up, good. tasty ix1 not. average. ‘This isn’t good. The only positive thing is that [the meal] is typical from here. That’s nice. But for me, it isn’t tasty. It’s average’.

In (3) the signer uses the first-person pronoun between the adjective tasty and the negation particle not. In this construction, as in (2), the pronoun is not grammatically obligatory, since she could have said tasty not in LSC, meaning ‘it isn’t tasty’. In these constructions, the first-person pronoun is not a syntactically obligatory component in the sentence, such as the sentence subject. Rather, its function is to contribute to the expressions for restricting opinions. The use of this pronoun with possessive value meaning ‘for me’ is performed by both males and females in our corpus. Moreover, six of the seven instances of first-person pronoun ix1 with possessive value have a non-manual marker in common related to eye gaze, namely avoiding eye contact with the addressee. Figures 4 and 59 represent the non-manual markers that accompany the pronouns in bold types in Examples (2) and (3) respectively. It is interesting to note that SP is the only signer who performs in one instance (of two occasions that he uses it) the first-person pronoun ix1 with possessive value looking directly at the eyes of the addressee (see Figure 6). Apart from the possessive value of ix1, we also detected in our corpus a special emphasis in ix1when it is used as a regular personal pronoun functioning as the 9. Note that although in Figure 5 the signer wears sunglasses, a detailed zoomed-in analysis in ELAN shows that she is not looking at the interlocutor.

Politeness in Catalan Sign Language (LSC)

Figure 4. Pronoun ix1 avoiding eye contact from Example (2)

Figure 5. Pronoun ix1 avoiding eye contact from Example (3)

Figure 6. sp performs the pronoun ix1 with direct eye gaze on the addressee

subject meaning ‘I’. This pronoun with this function appears six times, and three of them with an emphasis at the structural level of the sentence. The pronoun ix1is grammatically obligatory at the beginning of the sentences in (4) and (5) below. Moreover, the signers double the pronoun after a verb or a noun in order to emphasize that the content expressed is restricted to the sender. In the English translation we follow some varieties of colloquial English that allow a pronoun at the end of the sentence. This translation helps us understanding an equivalent for the LSC signed sentence, but this does not imply that the pronoun doubling in LSC results in a colloquial register. We attribute pronoun doubling more to a spontaneous use of the pronominal forms.

83

84

Berta Moya-Avilés, Gemma Barberà & Carme Bach

(4) apple baked ix1 love ix1 not apple. ‘I don’t love baked apples, me’. (5) is price is ix1head ix1full. palm-up, impossible. ‘I can’t get my head around the price, me. I mean, it’s impossible’.

There remain three cases with the personal pronoun ix1 emphasized, in which the stress is conveyed mainly by the non-manual markers. In Figures 7 and 8 the signers simultaneously raised their eyebrows and mouth yo (‘I’ in Spanish). Moreover, S3 shrugs (Figure 7), and S7 squints and leans backward (Figure 8). These non-manual markers contribute to emphasizing the prosody co-occurring with the first-person pronoun.

Figure 7. S3 emphasizing ix1 pronoun with raised eyebrows, pronoun mouthing and a shrug

Figure 8. S7 emphasizing ix1 pronoun with raised eyebrows, squint eyes, pronoun mouthing and leaning backwards

Finally, in Figure 9, S5 performs ix1 as well, but in this case, there is no emphasis on the pronoun. The non-manual markers, consisting of furrowed eyebrows, wrinkled nose, head tilt and tight lips, convey an evaluative meaning, a negative evaluation in this case, and a certain degree of certainty in the content, as already noted in previous research for other sign languages (Wilbur 2000; Sandler 2012; Pfau et al. 2012; Lackner 2017; see Section 2.4). Regarding the particular eye gaze mentioned above with the use of ix1, in five of the six cases of the first-person pronoun meaning ‘I’, the sender does not avoid

Politeness in Catalan Sign Language (LSC)

Figure 9. S5 performs ix1 conveying evaluative meaning through furrowed eyebrows, a head tilt and tight lips

eye contact but looks directly into the eyes of the addressee, which is usually the front camera (see Figure 9). In sum, in these cases in our corpus, we detected that eye gaze to the addressee appears with ix1 when it is used as a personal pronoun without a possessive value.

4.3 Expressions for restricting opinions with vertical-palm sign The last strategy on which we focus in the present research involves the use of the vertical-palm sign. Previous research in signed languages has focused on the palm-up sign as a lexicalized discourse marker (Gabarró-López 2020). In the present paper, we rather focus on the vertical-palm sign, which belongs to the same family as the palm-up sign but presents some differences, namely the orientation of the hand and the meanings associated. The vertical-palm sign is related to the meaning of stopping a line of action (see Section 2.4). Based on previous research of the same sign family (such as the palm-up sign), we assume the value of vertical-palm sign may also be determined according to various factors: the function, the position in the clause and the non-manual markers that co-occur with it. The eight tokens found in our corpus show a specific and systematic performance of vertical-palm sign that clearly restricts the content of what has been said. One such example is presented in (6), where the signer S3 says to S1 that she prefers tomato soup without garlic. At the end of the intervention, she performs the vertical-palm sign – in bold type – twice, corresponding to Figure 10. Note that at the beginning of the sentence S3 also uses the pronoun ix1 with a possessive value, as presented before. Thus, Example (6) clearly shows how different strategies can be combined to restrict one’s opinion. The translation of the vertical-palm sign is approximate since there is not an exact equivalent. In

85

86

Berta Moya-Avilés, Gemma Barberà & Carme Bach

the intervention of (6), the vertical-palm sign is expressed monomanually and twice. Also, a variant with a single repetition can be found, as in (8).10 (6) ix1garlic out better to taste more this. garlic inside lose h.ti sh.

h.ti sh.

this. vertical-palm vertical-palm ‘For me, it is better without garlic to taste like this. If there’s garlic, it loses flavor. At least for me’.

Figure 10. vertical-palm sign corresponding to the performance of Example (6)

In this case, since the soup they are eating does not contain any garlic, the FTA is not addressed to the cook (S2), but to S1, who previously said that tomato soup always tastes better with garlic. S3 is disagreeing with S1, and the expression for restricting opinions is intended to reduce the conflict in this disagreement between the two participants. The vertical-palm sign is performed in a very specific way: the movement of the hand is towards the signer’s body with a head tilt and a simultaneous shrug, always at the end of the intervention. On the one hand, the specific movement of the hand towards the sender’s body helps to convey the meaning of restriction because it locates the content of what has been said closer to the body of the sender. On the other hand, the head tilt co-occurring with the vertical-palm sign together with a shrug are used to express a degree of uncertainty, as already claimed by Barberà (2016) for LSC and Lackner (2017) for ÖGS (see Section 2.4). The signer expresses a negative opinion from a personal point of view and does not want to show certainty in her opinion in order to reduce the offense. Since identical hand movements and the position of vertical-palm sign can also convey other meanings, as shown in (7), we argue that non-manual markers are crucial in determining the meaning. In (7), S3 is answering the question previously asked by SP about whether she liked the salad. In this intervention, we find two different vertical-palm signs at the end of the sentence. The last one in 10. See Footnote 8 for the abbreviations of the subscripts.

Politeness in Catalan Sign Language (LSC)

bold is the one that restricts the opinion expression. The first one (underlined in the glosses) conveys partial disapproval. Even though the two of them involve a very similar hand movement and are situated at the end of the sentence, these two meanings are conveyed through different non-manual markers. h.ti.r

h.ti.l

h. ti sh.

(7) good. only nuts too-much. vertical-palm. vertical-palm ‘It’s good, the only thing is that it has too many nuts. So… kind of. Well, for me.’

For the first underlined vertical-palm sign, S3 performs two head tilts, to the right and left respectively. This side-to-side head movement co-occurring with vertical-palm sign at the end of the sentence conveys the meaning of doubt and uncertainty, concretely it is used to mark that there are alternatives (as also argued for palm-up in DSL by Engberg-Pedersen 2021, see Section 2.4, where it is proposed that they pertain to the same family of the Open Hand gestures according to Kendon 2004). That is, the signer is not sure about whether she likes it (one option) or not (the alternative). Leaving aside whether the uncertainty may be untrue and employed strategically to reduce the offense, in Example (7) this non-manual marker adds a piece of evaluative information about the food S3 is criticizing, but there is neither subjectivization nor restriction highlighted, only indecision about the opinion expressed. On the other hand, the second verticalpalm sign in bold type is performed with a single head tilt and a shrug at the same time. As head tilts can be used to express possibility (Lackner 2017; see Section 2.4), we suggest that here the head tilt helps to state the personal opinion (restriction conveyed by the vertical-palm sign) just as a possibility. That is, the opinion that the participant holds is understood just as one possibility without an explicit alternative, but it is not an absolute truth either. Then, the second head tilt marks a higher degree of certainty than the first one, which reinforces the individual perspective. This expression in this context includes a meaning similar to ‘for me’, or even ‘at least for me’, or ‘well, that’s what I think”. Apart from that, other non-manual markers may interact simultaneously with vertical-palm sign adding new information. For instance, when the verticalpalm sign to restrict one’s opinion is expressed together with tight lips, it adds an apologetic value, as shown in Example (8). (8) ix1 opinion my first salad bowl better change this inside. salad plate normal cl. eating with a fork. NUTS cl. eating with a fork with h. ti sh. tight.l

difficulty cl. eating by spoonful don’t-know. opinion. vertical-palm.

87

88

Berta Moya-Avilés, Gemma Barberà & Carme Bach

‘In my opinion, it is better to put this [cheese] in the salad we ate in the first place. The normal salad was difficult to eat with the fork, with all those nuts, so I needed to eat it in spoonful. I don’t know. It’s my opinion. That’s how I see it. I’m sorry.’

This apology value is explicit and therefore confirmed in (9), where S4 says ‘sorry’ after his criticism, and then performs the vertical-palm sign to restrict his opinion (in bold type) with a head tilt, a shrug and tight lips. The aim is both to restrict his opinion and to apologize for the FTA at the same time. Notice that the vertical-palm sign before the apologies is performed with one hand, while with the other hand S4 is signing opinion (see Figure 11).11 (9) palm-up. good palm-up. same first second food first plate second fit fit. this contrary. food another plate better fit. h. ti sh.

h. ti sh. tight.l

this different. my opinion vertical-palm, sorry, vertical-palm. ‘Exactly. It’s good. Umm… The first and second meals, combine very well. But this one is quite the opposite. Another food would have been better. This one is different. This is my opinion. I’m sorry. That’s how I see it.’

Figure 11. vertical-palm sign to restrict sender’s opinion with tight lips

In sum, we affirm that the vertical-palm sign may convey restriction of the sender’s opinion when performed with a combination of cues that include (1) a movement of the hand towards the signer’s body, (2) a head tilt, (3) a shrug, and (4) when it appears in final position.

11. Note that here there appear two palm-up signs, but their functions have nothing to do with the mitigation function and other similar purposes related to the vertical-palm sign that we are analyzing.

Politeness in Catalan Sign Language (LSC)

4.4 Summary In our corpus, we found thirty-three tokens (in twenty-seven interventions) of the strategy called expressions for restricting opinions. The general components and metadata related to each structure previously described are summarized in Table 3. Table 3. Summary of results Strategy for restricting the opinion expressions

Total occurrences Participant Sex of participants

Role of signers in the TV show

Lexical units

11

S4, S7

Male

Participants

ix1 pronoun

14

S1, S3, S4, S5, S7, SP

Male (7 cases) and female (7 cases)

Participants and the presenter

vertical-palm sign

 8

S3, S4, SP

Male (5 cases) and female (3 cases)

Participants and the presenter

As for the linguistic strategies, our data shows the following. First when lexical units are involved, the signers use different hedges, such as the noun opinion, the adverb person, the verb think, say or see, or the expression my point-of-view. Also, the possessive my is performed once bimanually. Regarding non-manual markers, in six of ten occurrences the signers perform a shrug co-occurring simultaneously with the lexical unit. Second, the opinion expression can be conveyed through the pointing sign ix1. On the one hand, we detected that this pronoun is used with a possessive value to restrict opinion expressions. In those cases, except for SP on one occasion, all the participants avoid eye contact when performing the pronoun. On the other hand, this pronoun is used as a personal pronoun meaning ‘I’. In these cases, this pronoun can either appear reduplicated at the structural level of the sentence or prosodically emphasized with particular non-manual markers. Eye gaze towards the addressee is frequently used when the pronoun is used as a personal pronoun (five of six cases). Finally, vertical-palm sign can be used to restrict the opinion expression as well. In those cases, (1) the sign appears at the end of the speaking turn, (2) the movement of the hands goes towards the signer’s body, and it co-occurs with (3) a head tilt, and (4) a shrug.

5.

Discussion and conclusion

The main goal of this research was to study how linguistic attenuation works in LSC in spontaneous discourse of Deaf native signers. Concretely, we focused on

89

90

Berta Moya-Avilés, Gemma Barberà & Carme Bach

the expressions for restricting opinions as a mitigation strategy. We adopted a qualitative methodology to analyze spontaneous discourse and we obtained the first results about this topic in LSC, including three different strategies: lexical units that work as hedges of subjectivization, the use of the first person pronoun, and the use of vertical-palm sign. The three strategies co-occur with particular non-manual markers. Note that even though all three actually work as a strategy to perform subjectivization, we addressed them separately because of their nature. The first strategy (lexical units) needs the appearance of nouns or verbs. The second strategy appears conveyed with the pronominal system, involving grammar; and finally, the use of vertical-palm sign works at the discourse level. We now turn to the interpretation and discussion of the results, as they indicate how attenuation works in LSC in the context analyzed. First, we detected lexical units that signers use as hedges to restrict their opinion. We found the use of nouns such as opinion, or verbs like think, for instance. These lexical units, as Namsaraev (1997) and Boncea (2014) propose, allow the sender to voluntarily express their opinion as a personal view instead of as a general truth. This strategy reduces the offense of the criticisms in our corpus since the negative opinions about the food are not expressed as absolute facts. Then, it can be perceived as partially true, and as partially false as well, which softens the FTA. Regarding non-manual markers, we found that it is common (six of ten cases) for the signers to simultaneously shrug with these hedges. As Barberà (2016) shows, in LSC shrugs may convey an epistemic meaning, more concretely a degree of uncertainty with regards to the knowledge of the discourse referent being mentioned. In our corpus, this degree of uncertainty has a mitigation purpose: it reinforces the meaning of subjectivity, i.e., it helps the sender to make claims from a personal perspective, rather than expressing it as an absolute truth. In the context we analyze, the non-manual marker consisting of a shrug helps to mitigate the criticism in LSC. It intensifies the subjectivization and the result is a softened criticism. Note that all these kinds of hedges are performed only by the male participants. Further analysis of a larger set of data will help to confirm this sociocultural pattern. Second, the first-person personal pronoun ix1is found in our corpus with the use to restrict opinion expressions. On the one hand, ix1is used with a possessive value, as initially argued in Quer and GRIN (2008). In the context we address, this possessive meaning is used to convey restriction in the opinion expression, i.e., it transmits a subjective point of view. Hence, the appearance of the possessive has a mitigating purpose. On the other hand, we found the first-person personal pronoun ix1meaning ‘I’. In these cases, since the pronoun was grammatically obligatory, the signers used other strategies to emphasize that the content is expressed from a personal point of view. Pronoun doubling and particular non-

Politeness in Catalan Sign Language (LSC)

manual markers are such strategies. The former, as Kimmelman (2013) pointed out, is used to emphasize the element doubled, in this case the reference to the first person. In our context, this contributes to making it explicit that the negative opinion is only a particular point of view. We propose that the appearance of doubling has a mitigating purpose too. The latter strategy consists of raised eyebrows, mouthing or squinted eyes, among others non-manual markers, that cooccur with the ix1 pronominal sign in order to emphasize it. As Wilbur (2000) and Sandler and Lillo-Martin (2006) propose, these non-manual markers work at the prosodic level conveying emphasis. In our corpus, this intensification of the pronoun results in an attenuation of the FTA, due to the fact that subjectivization – which is responsible for mitigating the criticisms – is highlighted. Moreover, we show how other non-manual markers, such as tight lips, may add evaluative meaning. As Wilbur (2000) points out, the lower portion of the face is used to provide adverbial and adjectival information. This may co-occur with mitigating functions of the first-person pronoun ix1 in LSC. In relation to non-manual markers co-occurring with the pronoun ix1, we detected differences regarding eye gaze: avoiding eye contact appears with the possessive value of the pronoun, while eye gaze directed to the addressee is used when it functions as subject. In addition, it is interesting that SP – who has more power in the communicative context because he is the presenter – is the only one who gazes directly at the addressee (in one of two occasions that he uses possessive ix1). However, there are no substantial differences between our results and those of Veiga-Busto (2021), who found that eye gaze may be either directed to the addressee or to other locations, that allow us to interpret the eye gaze in the expressions for restricting opinions in terms of mitigation. Further contrast with complementary data is needed to go deeper into this topic, as well as the role of social power. Third, we identified that the vertical-palm sign also conveys a mitigation meaning. With a sign of the same family, namely the palm-up sign, GabarróLópez (2020) claims that it is a multi-functional discourse marker. Our data and analysis offer a new approach to a similar yet different sign: the vertical-palm sign conveys an attenuation function. In our corpus, the vertical-palm sign is used to restrict the opinion expression when it appears with (1) a movement of the hand towards the signer’s body, (2) head tilts, (3) shrugs, and (4) when it appears in the final position of the intervention. Since the vertical-palm sign can be performed with the same movement of the hands or at the final position conveying other functions, we demonstrate that the combination of the four factors just listed is crucial for this mitigation value. Kendon (2004) argues that Vertical Palm is used to stop a line of action or to establish a barrier. We interpret this in terms of mitigation in our data since the sender creates from their criticism by this sub-

91

92

Berta Moya-Avilés, Gemma Barberà & Carme Bach

jectivization strategy. Moreover, item (1) above is in line with the claim of Kendon (2004) that movement towards the signer’s body emphasizes the restriction of the line of action to oneself. Our results regarding vertical-palm sign and the uncertainty conveyed simultaneously by a head tilt or a rocking movement in LSC are in line with those for the palm-up sign in Engberg-Pedersen (2021) in DSL, and Lackner (2017) for ÖGS. Moreover, similar to what Mapson (2014) states for BSL, we propose that tight lips can add an apologetic value to the mitigating verticalpalm sign in LSC. Besides, due to the regularity of vertical-palm sign in our corpus, we propose that in LSC this sign works as a discourse marker, not as a gesture, because the regularity may signal lexicalization (Özyürek 2012). Yet further research on this regard is needed to confirm this claim with a larger set of data. In what follows we represent a summary of the discussion in the form of a table, to make it more visual. Table 4. Summary of the discussion Strategy

Characteristics Non-manual markers

Overview

Lexical units

Nouns such as opinion, or verbs like think

Simultaneously shrug

Conveying a degree of uncertainty with the non-manual markers reinforces the meaning of subjectivity of the hedges, which soften an FTA and therefore, it entails mitigation effects.

ix1 pronoun

First person pronoun (possessive meaning) Doubling first person pronoun (subject function)

Raised eyebrows, mouthing or squinted eyes co-occur with the possessive meaning of ix1

The intensification (either with the doubling or the non-manual markers) results in a reduction of the effect of an FTA, because the subjectivization – which is responsible for mitigating the criticisms – is highlighted.

verticalpalm sign

Discourse marker

Head tilt and shrug

vertical-palm sign at the end of the sentence and with the uncertainty conveyed by the non-manual markers (namely head tilt and shrug) contributes to the subjectivization and therefore to mitigation purposes.

Concerning previous research of non-manual markers in signed languages, Wilbur (2000) also claims that, regardless of place of articulation or articulator,

Politeness in Catalan Sign Language (LSC)

the onset and offset provide a clear indicator of function: abrupt for syntactic functions and gradual otherwise. Our corpus data confirms this claim. All the non-manual markers expressed in the three strategies described are articulated in a gradual manner, showing that they function beyond the syntactic level. As for the hypothesis presented by Bross and Hole (2017) for DGS about speech acts being mapped to the upper face of the signer, our claim is that in LSC the same holds true although the systematicity of the use of eye gaze and brow raising needs to be contrasted with a larger set of data. The present research has analyzed different hedges expressed in LSC previously identified in various spoken languages by Albelda et al. (2014); Namsaraev (1997) or Boncea (2014), among others. Specific LSC strategies involving pronouns and discourse markers with mitigation purposes have also been characterized for the first time. We want to emphasize the relevance of studying mitigation in context. In this particular scenario, participants are expected to criticize the food, since this is the aim of the TV show competition. However, the participants try to strike a balance between achieving this aim and minimizing the FTAs. This particular feature of our corpus could motivate the reason why expressions for restricting opinions are widely used. We saw how the main FTAs were addressed to a non-present addressee, i.e., the cook, who was not eating at the table, but due to the public nature of the program, face-work arose anyhow. As also pointed out in Albelda and Estellés (2021), the analysis of mitigation in context was considered crucial in order to understand the strategic possibilities of hedges, and their impact on communicative efficiency and interaction as well, a factor that SpencerOatey (2000, 2002) stresses for politeness phenomena. In this sense, interpreting data in context and in real interactions, as the discursive turn of politeness studies proposes (see Mills 2003, for instance), also allowed us to see the strategic use of epistemic modality with mitigation purposes in LSC. As shown, the results we obtained are undoubtedly linked to the concrete communicative situation, that is a TV cooking competition show. This clearly reinforces that the social context configures the necessity of what is appropriate or not (see Locher and Watts 2005) and we cannot define what is linguistically polite out of context. In conclusion, we detected different expressions for restricting opinions with attenuation purposes in LSC. The results are in line with some previous research (Namsaraev 1997; Wilbur 2000; Sandler and Lillo-Martin 2006; Quer and GRIN 2008; Dachkovsky and Sandler 2009; Kimmelman 2013; and Mapson 2014). Moreover, we provided new evidence on additional aspects, namely the vertical-palm sign and the mitigating purpose of pronoun doubling and the possessive ix1. As argued above, more data is needed in order to determine the role of eye gaze in attenuation, as well as the role of gender or social power of the signers. The Deaf signers in our corpus do use different strategies to mitigate their

93

94

Berta Moya-Avilés, Gemma Barberà & Carme Bach

criticisms, both with linguistic strategies and with non-manual markers, making a relevant contribution to the description and analysis of LSC, politeness in general, as well as intercultural and multimodal phenomena. The Deaf-as-direct stereotype has been called into question by virtue of analyzing spontaneous signed discourse.

Funding This research was partly made possible thanks to different institutions. Moya-Avilés acknowledges the Department of Translation and Language Sciences of Universitat Pompeu Fabra, in Barcelona. Barberà, who is a Serra Húnter fellow, acknowledges the project “Microdiachrony in endangered languages across modalities” (MICRODIAC(H)RO), financed by the Spanish Ministry of Science, Innovation and Universities / Agencia Estatal de Investigación, PID2020119041GB-I00, 2021–2025, as well as the Government of the Generalitat de Catalunya (2017 SGR 1478). Bach, who is a Serra Húnter fellow, acknowledges the Government of the Generalitat de Catalunya (2017 SGR 915).

Acknowledgements We would also like to thank Webvisual.TV and the participants of the TV cooking show Mans a la massa for making the LSC data available to be used as the corpus data for the present study. Thank you to the editors for their suggestions. All remaining mistakes are our own responsibility.

References Albelda, Marta, and Ana Mª Cestero. 2011. “De nuevo sobre los procedimientos de atenuación lingüística.” Español actual 96: 9–40. Albelda, Marta, and Maria Estellés. 2021. “Mitigation Revisited: An Operative and Integrated Definition of the Pragmatic Concept, its Strategic Values, and its Linguistic Expression.” Journal of Pragmatics 183: 71–86. https://doi.org/10.1016/j.pragma.2021.07.002 Albelda, Marta, Antonio Briz, Ana Mª Cestero, Dorota Kotwica, and Cristina Villalba. 2014. “Ficha metodológica para el análisis pragmático de la atenuación en corpus discursivos del español.” (ES. POR. ATENUACIÓN). Oralia 17: 7–62. https://doi.org/10.25115/oralia.v17i.7999

Arundale, Robert B. 2006. “Face as Relational and Interactional: A Communication Framework for Research on Face, Facework, and Politeness.” Journal of Politeness Research 2 (2): 193–216. https://doi.org/10.1515/PR.2006.011 Barberà, Gemma. 2016. “Indefiniteness and Specificity Marking in Catalan Sign Language (LSC).” Sign Language & Linguistics 19 (1): 1–36. https://doi.org/10.1075/sll.19.1.01bar

Politeness in Catalan Sign Language (LSC)

Barberà, Gemma, Josep Quer, and Santiago Frigola. 2015. “Primers passos cap a la documentació de discurs signat. El projecte pilot de constitució del corpus de la llengua de signes catalana.” Treballs de Sociolingüística Catalana 25: 287–302. Bargiela-Chiappini, Francesca. 2003. “Face and Politeness: New (Insights) for Old (Concepts).” Journal of Pragmatics 35 (10–11): 1453–1469. https://doi.org/10.1016/S0378-2166(02)00173-X

Bayley, Robert, Joseph C. Hill, Caroline McCaskill, and Ceil Lucas. 2017. “Attitudes Towards Black American Sign Language.” University of Pennsylvania Working Papers in Linguistics 23 (2): 21–30. Benitez-Quiroz, C. Fabian, Ronnie B. Wilbur, and Aleix M. Martinez. 2016. “The Not Face: A Grammaticalization of Facial Expressions of Emotion.” Cognition 150: 77–84. https://doi.org/10.1016/j.cognition.2016.02.004

Boncea, Irina Janina. 2014. “Hedging Patterns Used as Mitigation and Politeness Strategies.” Annals of the University of Craiova. Series: Philology, English 2 (1): 7–23. Briz, Antonio. 2004. “Cortesía verbal codificada y cortesía verbal interpretada en la conversación.” In Pragmática sociocultural: estudios sobre el discurso de cortesía en español, ed. by Diana Bravo, and Antonio Briz, 67–93. Barcelona: Ariel Briz, Antonio, and Marta Albelda. 2013. “Una propuesta teórica y metodológica para el análisis de la atenuación lingüística en español y portugués: La base de un proyecto en común (ES. POR. ATENUACIÓN).” Onomázein 28: 288–319. https://doi.org/10.7764/onomazein.28.21

Bross, Fabian, and Daniel Hole. 2017. “Scope-Taking Strategies and the Order of Clausal Categories in German Sign Language.” Glossa: A Journal of General Linguistics 2 (1): 1–30. https://doi.org/10.5334/gjgl.106

Brown, Penelope, and Stephen C. Levinson. 1978. “Universals in Language Usage: Politeness Phenomena.” In Questions and Politeness: Strategies in Social Interaction, ed. by Esther. N. Goody, 56–311. Cambridge: Cambridge University Press. Brown, Penelope, and Stephen C. Levinson. 1987. Politeness: Some Universals in Language Usage. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511813085 Caffi, Claudia. 1999. “On Mitigation.” Journal of Pragmatics 31 (7): 881–909. https://doi.org/10.1016/S0378-2166(98)00098-8

Caffi, Claudia. 2005. Mitigation. Leiden: Brill. https://doi.org/10.1163/9780080466224 Culpeper, Jonathan, Michael Haugh, and Dániel Z. Kádár. 2017. “Introduction.” In The Palgrave Handbook of Linguistic (Im)politeness, ed. by Jonathan Culpeper, Michael Haugh, and Dániel Z. Kádár, 1–8. London: Palgrave Macmillan. https://doi.org/10.1057/978-1-137-37508-7_1

Dachkovsky, Svetlana, and Wendy Sandler. 2009. “Visual Intonation in the Prosody of a Sign Language.” Language and Speech 52 (2/3): 287–314. https://doi.org/10.1177/0023830909103175 Engberg-Pedersen, Elisabeth. 2021. “Markers of Epistemic Modality and their Origins: Evidence from Two Unrelated Sign Languages.” Studies in Language 45 (2): 277–320. https://doi.org/10.1075/sl.19065.eng

Gabarró-López, Sílvia. 2020. “Los marcadores del dicurso PALM-UP e IGUAL y su posición: un estudio sobre la lengua de signos catalana (LSC).” Revista de Estudios de Lenguas de Signos REVLES 2: 1–27. Garcés-Conejos Blitvich, Pilar, and Maria Sifianou. 2019. “Im/politeness and Discursive Pragmatics.” Journal of Pragmatics 145: 91–101. https://doi.org/10.1016/j.pragma.2019.03.015

95

96

Berta Moya-Avilés, Gemma Barberà & Carme Bach

George, Johnny Earl. 2011. Politeness in Japanese Sign Language (JSL): Polite JSL Expression as Evidence for Intermodal Language Contact Influence. PhD Dissertation. Department of Linguistics, University of California: Berkley. Goffman, Erving. 1967. Interaction Rituals: Essays on Face-to-Face Conversation. Nueva York: Doubleday. Gu, Yueguo. 1990. “Politeness Phenomena in Modern Chinese.” Journal of Pragmatics 14 (2): 237–257. https://doi.org/10.1016/0378-2166(90)90082-O Haugh, Michael. 2005. “The Importance of ‘Place’ in Japanese Politeness: Implications for Cross-cultural and Intercultural Analyses.” Intercultural Pragmatics 2 (1): 41–68. https://doi.org/10.1515/iprg.2005.2.1.41

Holmes, Janet, Meredith Marra, and Bernadette Vine. 2012. “Politeness and Impoliteness in Ethnic Varieties of New Zealand English.” Journal of Pragmatics 44 (9): 1063–1076. https://doi.org/10.1016/j.pragma.2011.11.006

Hoza, Jack. 2007. It’s Not What You Sign, It’s How You Sign It: Politeness in American Sign Language. Washington D.C: Gallaudet University Press. Hoza, Jack. 2008. “Five Nonmanual Modifiers That Mitigate Requests and Rejections in American Sign Language.” Sign Language Studies 8 (3): 264–288. https://doi.org/10.1353/sls.2008.0006

Hübscher, Iris, Cristina Sánchez-Conde, Joan Borràs-Comes, Laura Vincze, and Pilar Prieto. Under revision. “Multimodal Mitigation: How Facial and Body Cues Index Politeness in Catalan Requests.” Journal of Politeness Research. Hyland, Ken. 2005. “Stance and Engagement: A Model of Interaction in Academic Discourse.” Discourse Studies 7 (2): 173–192. https://doi.org/10.1177/1461445605050365 Karabüklü, Serpil, Fabian Bross, Ronnie B. Wilbur, and Daniel Hole. 2018. “Modal Signs and Scope Relations in TID.” FEAST. Formal and Experimental Advances in Sign Language Theory 2: 82–92. https://doi.org/10.31009/FEAST.i2.07 Kendon, Adam. 2004. Gesture: Visible Action as Utterance. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511807572 Kerbrat-Orecchioni, Catherine. 2004. “¿Es universal la cortesía?.” In Pragmática sociocultural: estudios sobre el discurso de cortesía en español, ed. by Diana Bravo, and Antonio Briz, 39–53. Barcelona: Ariel. Kimmelman, Vadim. 2013. “Doubling in RSL and NGT: A Pragmatic Account.” Interdisciplinary Studies on Information Structure 17 (99): 118. Lackner, Andrea. 2017. Functions of Head and Body Movements in Austrian Sign Language. Berlin: Mouton De Gruyter. https://doi.org/10.1515/9781501507779 Lakoff, George. 1975. “Hedges: A Study in Meaning Criteria and the Logic of Fuzzy Concepts.” In Contemporary Research in Philosophical Logic and Linguistic Semantics, ed. by Donald J. Hockney, William Leonard Harper, and Bruce Freed, 221–271. Dordrecht: Springer. https://doi.org/10.1007/978-94-010-1756-5_9 Leech, Geoffrey N. 2014. The Pragmatics of Politeness. Oxford: Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195341386.001.0001

Locher, Miriam A., and Richard J. Watts. 2005. “Politeness Theory and Relational Work.” Journal of Politeness Research 1 (9): 9–33. https://doi.org/10.1515/jplr.2005.1.1.9 Mapson, Rachel. 2013. “Politeness in British Sign Language: The Effects of Language Contact.” In Multilingual Theory and Practice in Applied Linguistics. Proceedings of the 45th Annual Meeting of the British Association for Applied Linguistics, ed. by Alasdair N. Archibald, 167–170. Southampton: University of Southampton.

Politeness in Catalan Sign Language (LSC)

Mapson, Rachel. 2014. “Polite Appearances: How Non-manual Features Convey Politeness in British Sign Language.” Journal of Politeness Research 10 (2): 157–184. https://doi.org/10.1515/pr-2014-0008

Matsumoto, Yoshiko. 1988. “Reexamination of the Universality of Face: Politeness Phenomena in Japanese.” Journal of Pragmatics 12 (4): 403–426. https://doi.org/10.1016/0378-2166(88)90003-3

Matsumoto, Yoshiko. 2003. “Reply to Pizziconi.” Journal of Pragmatics 35 (10–11): 1515–1521. https://doi.org/10.1016/S0378-2166(03)00077-8

Mills, Sara. 2003. Gender and Politeness. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511615238

Namsaraev, Vasili. 1997. “Hedging in Russian Academic Writing in Sociological Texts.” In Hedging and Discourse: Approaches to the Analysis of a Pragmatic Phenomenon in Academic Texts, ed. by Raija Markkanen, and Hartmut Schroeder, 64–80. Berlin: Walter de Gruyter. https://doi.org/10.1515/9783110807332.64 O’Driscoll, Jim. 2007. “Brown & Levinson’s Face: How It Can – and Can’t – Help Us to Understand Interaction across Cultures.” Intercultural Pragmatics 4 (4): 463–492. https://doi.org/10.1515/IP.2007.024

O’Driscoll, Jim. 2017. “Face and (Im)politeness.” In The Palgrave Handbook of Linguistic (Im)politeness, ed. by Jonathan Culpeper, Michael Haugh, and Dániel Z. Kádár, 89–118. London: Palgrave Macmillan. https://doi.org/10.1057/978-1-137-37508-7_5 Özyürek, Aslı. 2012. “Gesture.” In Sign Language: An International Handbook, ed. by Roland Pfau, Markus Steinbach, and Bencie Woll, 626–646. Berlin: Mouton de Gruyer. https://doi.org/10.1515/9783110261325.626

Pfau, Roland, and Josep Quer. 2010. “Nonmanuals: Their Grammatical and Prosodic Roles.” In Sign Languages, ed by Diane Brentari, 381–402. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511712203.018 Pfau, Roland, Markus Steinbach, and Bencie Woll. 2012. “Tense, Aspect and Modality.” In Sign Language: An International Handbook, ed. by Roland Pfau, Markus Steinbach, and Bencie Woll, 186–203. Berlin: Walter de Gruyter. https://doi.org/10.1515/9783110261325.186 Pietrosemoli, Lourdes. 2001. “Politeness and Venezuelan Sign Language.” In Signed Languages: Discoveries from International Research, ed. by Valerie Dively, Melanie Metzger, Sara Taub, and Anne Marie Baer, 163–179. Washington D.C: Gallaudet University Press. Quer, Josep, and GRIN. 2008. “Possessive and Existential Structures in Catalan Sign Language.” In Typological Studies on Possessive and Existential Constructions in Sign Languages, ed. by Ulrike Zeshan, and Pamela Perniss, 33–53. Nijmegen: Ishara Press. Roush, Daniel. 2007. “Indirectness Strategies in American Sign Language Requests and Refusals: Deconstructing the Deaf-as-direct Stereotype.” In Translation, Sociolinguistic, and Consumer Issues in Interpreting, ed. by Melanie Metzge, and Earl Fleetwood, 103–56. Washington D.C: Gallaudet University Press. Roush, Daniel. 2011. “Language Between Bodies: A Cognitive Approach to Understanding Linguistic Politeness in American Sign Language.” Sign Language Studies 11 (3): 329–374. https://doi.org/10.1353/sls.2011.0000

Sandler, Wendy. 2012. “Visual prosody.” In Sign Language: An International Handbook, ed. by Roland Pfau, Markus Steinbach, and Bencie Woll, 55–67. Berlin: De Gruyter Mouton. https://doi.org/10.1515/9783110261325.55

Sandler, Wendy, and Diane Lillo-Martin. 2006. Sign Language and Linguistic Universals. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9781139163910

97

98

Berta Moya-Avilés, Gemma Barberà & Carme Bach

Spencer-Oatey, Helen. 2000. “Rapport Management: A Framework for Analysis.” In Culturally Speaking: Managing Rapport Through Talk Across Cultures, ed. by Helen Spencer-Oatey, 11–46. London: Continuum. https://doi.org/10.5040/9781350934085 Spencer-Oatey, Helen. 2002. “Managing Rapport in Talk: Using Rapport Sensitive Incidents to Explore the Motivational Concerns Underlying the Management of Relations.” Journal of Pragmatics 34 (5): 529–545. https://doi.org/10.1016/S0378-2166(01)00039-X Spencer-Oatey, Helen. 2005. “(Im)politeness, Face and Perceptions of Rapport: Unpackaging their Bases and Interrelationships.” Journal of Politeness Research 1 (1): 95–119. https://doi.org/10.1515/jplr.2005.1.1.95

Stokoe, William. 1960. Sign Language Structure. An Outline of the Visual Communication Systems of the American Deaf. Silver Spring, MD: Linstock Press. Terkourafi, Marina. 2005. “Beyond the Micro-level in Politeness Research.” Journal of Politeness Research 1 (2): 237–262. https://doi.org/10.1515/jplr.2005.1.2.237 Terkourafi, Marina. 2007. “Toward a Universal Notion of Face for a Universal Notion of Cooperation.” In Explorations in Pragmatics: Linguistic, Cognitive and Intercultural Aspects, ed. by Istvan Kecskes, and Laurence Horn, 307–338. Berlin: Mouton de Gruyer. Veiga-Busto, Raquel. 2021. Person and Number in Catalan Sign Language Pronouns. PhD Dissertation. Department of Translation and Language Science, Universitat Pompeu Fabra: Barcelona. Watts, Richard J. 2003. Politeness. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511615184

Wilbur, Ronnie B. 2000. “Phonological and Prosodic Layering of Nonmanuals in American Sign Language.” In The Signs of Language Revisited: An Anthology to Honour Ursula Bellugi and Edward Klima, ed. by Karen Emmorey, and Harlan L. Lane, 215–244. New Jersey: Lawrence Erlbaum Associates Publishers. Wilbur, Ronnie B. 2021. “Non-manual Markers: Theoretical and Experimental Perspectives.” In Routledge Handbook of Theoretical and Experimental Sign Language Research, ed. by Josep Quer, Roland Pfau, and Annika Herrmann, 530–565. London: Taylor & Francis Group. https://doi.org/10.4324/9781315754499-24

part ii

Spoken

chapter 4

Multimodality in refusals in English as a lingua franca Xianming Fang

Monash University

This study investigates how multimodal cues are used for rapport management in refusals in the context of English as a lingua franca (ELF). Ten Chinese and ten Indonesian speakers were put in pairs and conducted role-plays in relation to requests and refusals. After the role-plays, they had immediate interviews to reflect on their own and their partners’ performance. The results suggest that body positions (standing/sitting), smiling voices and smiling facial expressions, and the long gaze aversion are used by ELF refusers to maintain rapport: controlling power relationships, mitigating the force of refusals, and conveying a non-engagement stance. The results show that mitigation in the ELF context is a multimodal achievement which can be intentionally realized through various multimodal cues. Keywords: refusals, English as a lingua franca (ELF), multimodal conversation analysis, rapport management, gaze aversion, smiles, body positions

1.

Introduction

A refusal is usually viewed as a rapport-threatening act (Spencer-Oatey 2008). A number of studies in both L1 and L2 contexts have demonstrated that refusers use various linguistic strategies to mitigate their refusals, such as semantic formulas (i.e., refusal strategies) and lexical downgraders/upgraders. In the current study, I look at how refusals are used in English as a lingua franca (ELF) which refers to “any use of English among speakers of different first languages for whom English is the communicative medium of choice and often the only option” (Seidlhofer 2011: 7). Nowadays, more than 70% of English users are non-native speakers (Statista, 2021) and they use English as a communicative medium for education, business, politics, travelling, and so on. Different from English as a foreign lan-

https://doi.org/10.1075/pbns.333.04fan © 2023 John Benjamins Publishing Company

102

Xianming Fang

guage (EFL) whose ultimate goal for learners is to become nearly native-like when interacting with native-speakers, ELF focuses on the analysis of non-native speakers’ usage of English as legitimate speakers and criticizes the focus on nativespeaker norms in EFL and SLA research. The fact that English has become a global lingua franca presents an opportunity to study refusals in the ELF context. Yet, to my knowledge, there is still no adequate discussion on ELF refusals, and a few existing studies (e.g., House 2008) focus only on the verbal dimension. However, a complete refusal is not just about what refusers say but how refusers say it. For example, a refuser may use a co-speech gesture with his/her refusal statement, such as hand waving, a smiling facial expression, or head shaking. These multimodal cues are definitely part of their refusals and are of great significance, but they usually receive less attention than the verbal aspects. In this study, I investigate how refusals are realized in ELF interactions multimodally. Specifically, I examine what multimodal cues accompany participants’ refusals and what potential politeness functions are embedded in them. The investigation contributes to the emerging research area of multimodal politeness research and ELF pragmatics. The results suggest that multimodal cues are important politeness resources for ELF speakers to constitute mitigated refusals, and politeness is still highly valued in rapport-threatening refusals in ELF interactions. The paper is structured as follows. First, I review relevant theory and research. Then, I introduce the methodology including participants, instruments, procedures and analysis methods. After that, I present the findings of the study and the discussion. The paper concludes by summarizing its contributions and giving possible future research directions.

2.

Background

2.1 Approach to politeness This study adopts a discursive politeness view, using multimodal Conversation Analysis (CA) to investigate politeness displayed by participants in context. Multimodal CA is not only a tool for analysing verbal resources but multimodal cues in interactions, such as points (Mondada 2007), walking bodies (Mondada 2016), body posture when leaning forward (Rasmussen 2014), gestures and gaze (Matsumoto and Canagarajah 2020). Multimodal CA relies on videos and has a “careful and precise attention to temporally and sequentially organized details of actions that account for how co-participants orient to each other’s multimodal conduct, and assemble it in meaningful ways, moment by moment” (Mondada

Multimodality in refusals in English as a lingua franca

2016: 340). The current study uses role-plays to collect data (see Section 3.2). Role-plays have been examined extensively by CA in recent literature, including requests (Al-Gahtani and Roever 2012, 2013, 2015; Hassall 2020) and refusals (AlGahtani and Roever 2018) (For more detailed discussions see Taguchi and Roever 2017; Culpeper, Mackey, and Taguchi 2018.) The temporality and sequentiality of bodily movements are useful for politeness analysis, as Kádár and Haugh (2013) suggest that these contain important cues for participants, and thus for analysts, to infer or evaluate politeness. However, Culpeper (2011) reminds us that CA itself, at least its original purpose, does not concern itself with social or cultural categories such as politeness and researchers still need to use politeness theories to interpret data. In this study, I consider Spencer-Oatey’s (2008) rapport management (RM) framework suitable for examining politeness in the ELF context, as this framework particularly focuses on intercultural communication. Spencer-Oatey’s (2008) rapport management (RM) framework consists of three aspects: face (the recognition of personal positive qualities and identity), interactional goals (whether the “wants” are achieved), and sociality rights and obligations (expected behaviors in certain contexts). Rapport is likely to be threatened if these three elements are jeopardized. In lingua franca communications where less common ground is shared among participants, rapport management is particularly important because participants may have different interpretations of face and obligations, and have different interactional goals. By examining the data through a multimodal analysis and adopting a RM angle, this paper can show how refusers maintain rapport in interactions through interactional resources and multimodal cues.

2.2 Empirical studies on refusals Refusals, as a commissive speech act (Searle 1976), are one of the frequently investigated objects in pragmatics. From the perspective of CA, the refusal is a pairtype related second pair part (SPP) (Schegloff 2007), with the first pair part (FPP) being request, offer, or invitation. Previous research on refusals mainly focused on two areas: cross-cultural comparison and interlanguage pragmatics. In crosscultural refusal studies, researchers compared refusals from different cultures to find commonalities and differences, such as the comparison between Egyptian Arabic and American English (Morkus 2014; Nelson et al. 2002), Chinese and American English (Chen 1996; Liao and Bresnahan 1996), Iranian and American English (Abdolrezapour and Vahid Dastjerdi 2013), Polish and English (Bhatti and Žegarac 2012), and Korean and English (Kwon 2004). In interlanguage pragmatic refusal research, researchers focused on what influences learners’ acquisition of L2 refusals, such as proficiency (Taguchi 2013; Bella 2014; Al-Gahtani and

103

104

Xianming Fang

Roever 2018), study abroad (Barron 2007; Ren 2012), length of residence (FélixBrasdefer 2004), and pragmatic transfer (Beebe et al. 1990; Takahashi and Beebe 1987). These studies typically compared learners’ performance to that of native speakers. Research in the two areas largely focused on the verbal aspect of refusals (e.g., refusal strategies, lexical/phrasal downgraders). An often-used taxonomy for refusal strategies is Beebe et al.’s classification (1990) where refusals are divided into three categories: direct (e.g., I can’t), indirect (e.g., explanations), and adjuncts (e.g., pause fillers, statement of empathy). Direct and indirect strategies are the semantic formulas that can be used as refusals independently while adjuncts must be used with the two strategies. If a refusal is direct without any modifications, it is a raw refusal, such as no. However, a CA analysis of refusals would depart from Beebe’s taxonomy. According to Schegloff (2007), the pause filler well is a turn-initial delay, occupying the turn-initial position instead of the second pair part itself; the statement of empathy is a mitigation showing solidarity; the explanations can be anticipatory accounts, positioned early in the turns. Sequentially, these interactional resources usually help to organize refusals as dispreferred. That is to say, the second pair part (SPP) is delayed by these supportive moves and the contiguity between the first pair part (FPP) and SPP is broken. Although researchers have already recognized the importance of multimodality in refusals, few of them have explored refusals in a multimodal way. Beebe et al.’s (1990) original refusal taxonomy has a category named “Avoidance” which includes four multimodal cues: silence, hesitation, do nothing, and physical departure. But in later refusal research which adopts the taxonomy, the multimodal category is usually excluded (e.g., Bella 2014; Morkus 2014; Nelson et al. 2002; Taguchi 2013). The exclusion of multimodal cues in refusal research neglects the multimodal nature of politeness in social actions and leads to an incomplete picture of politeness behaviors in refusals.

2.3 Refusals and English as a lingua franca (ELF) Refusals are still less investigated in the ELF field. The existing literature has just suggested that raw rejections are preferred in ELF interactions. For example, Murray (2012) argues that ELF speakers tend to favor raw rejections over prefacing supportive moves via mitigating devices because ELF interaction is “highly pragmatic” so that ELF participants “[opt] for a kind of communication that enables them to ‘get business done’ without forcing them to adopt pragmatic behaviors with which they may be uncomfortable” (Murray 2012: 322). In a study examining ELF interactions among European participants, House (2008) asked

Multimodality in refusals in English as a lingua franca

participants to have group discussion on the topic of European integration. The context in her study is relatively low in terms of sensitivity (i.e., low power, social distance, and rank of imposition). She found that raw rejections are frequently used by speakers to address the interlocutor’s previous move. Despite the lack of mitigation, she argues that raw rejections do not lead to open conflicts because “the interactants’ behaviors are apparently heavily overlaid by the let-it-pass principle1” (House 2008: 355). Despite the dearth of ELF studies on refusals, ELF studies on complaints (Konakhara 2017) and disagreements (Maı́ z-Aré valo 2014; Matsumoto 2018) provide general insights about how ELF users address rapport-threatening speech acts. Results show that ELF users prefer to organize and mitigate their complaints/ disagreements in a multimodal way. For example, in a case study on complaints, Konakahara (2017) found that complainers used multimodal cues to make complaints. Verbally, they used specific accounts, self-rephrasing, prefaces, etc. Multimodally, they adopted an accusing tone, gestures (palms upward), and serious facial expressions. When responding to complaints from a disaffiliated stance, on the verbal level, the speaker avoided immediate responses and expanded the sequence by seeking for more information. Multimodally, laughter was often used “to avoid threatening positive faces of all individuals involved in the complaint sequence” (Konakahara 2017: 336). Similar to Konakahara, Matsumoto (2018) describes the multimodal nature of disagreements in ELF classes in the United States where English is used as a communicative medium between the teacher and international students from different countries. She found that students effectively deployed long silence, body orientation (leaning back), gaze aversion, smiles, and other gestures to politely disagree with their teacher. Evidence from complaints and disagreements shows that in ELF contexts, rapport-threatening acts are carefully addressed through both verbal strategies and multimodal cues.

2.4 Multimodal politeness Recent years have seen an emergence of a growing body of multimodal politeness research which demonstrates the importance of multimodal cues in perceiving and producing politeness (See Brown, Hübscher and Jucker, this volume, for a review). Consulting previous literature in rapport-threatening speech acts, this paper looks at the following modalities: body positions, smiling voices/smiling

1. The “let-it pass” principle is that “the hearer thus lets the unknown or unclear action, word, or utterance ‘pass’ on the (common-sense) assumption that it will either become clear or redundant as talk progresses” (Firth 1996: 243)

105

106

Xianming Fang

facial expressions, and gaze movements, which have potential politeness functions in refusals. The first type of multimodal cue that I look at is body position, which refers to how interactants arrange and posture their bodies (e.g., sitting or standing). Previous studies (Brown and Winter 2019) on Korean show that speakers usually stood when interacting with superiors, while superiors often slouched (i.e., sitting or standing with hunched shoulders or a leaning posture). Similarly, Burgoon and Dunbar (2006) observed that superiors adopt more open body positions compared with inferiors when sitting, with arms akimbo (i.e., having hands on the hips) and more expansive gestures. Additionally, superiors have precedence over inferiors in deciding body positions, such as where to sit and stand and the interactional distances (Burgoon and Dunbar 2006). Brown and Winter (2019: 22) pointed out that “doing deference also entailed allowing the superior to go first” and in Korean culture, inferiors usually stand unless being requested to sit or offered a seat. The second multimodal cue that I look at is smiling voices and smiling facial expressions (without vocal sound). These are common cues for listeners to calibrate politeness judgments. For example, McKinnon and Prieto (2014) show that smiles help Catalan speakers to better distinguish mock impoliteness from genuine impoliteness, especially when the prosodic cues are similar. Interactional research further suggests that smiles or laughter are often produced “at places of possible embarrassment”, namely, delicate occasions of the interaction where problems/rejections/disagreements occur (Haakana 2001: 196). By laughing at delicate interactional placements, Haakana (2001) argues that laughter serves as a remedy to the problem by indicating speakers’ awareness of the delicate nature of the occasion. In terms of rapport management, smiles or laughter at these interactional placements arguably have politeness functions because they are attempts to address possible embarrassment, which show the speaker’s sensitivity to the hearer’s face. Finally, I examined gaze aversion. Previous studies in gesture research and psychology show that hearers tend to avert their gaze when they feel embarrassed or uncomfortable at the speakers’ speech (Kendon 1967; Modigliani 1971). Recently, Brown and Winter (2019) also found a preference of hearers for gaze aversion when being scolded. Gaze aversion, as a multimodal solution to embarrassment and discomfort in interactions, deals with the possible face threats resulting from the speakers’ utterances. Hearers use it to maintain face because by looking away hearers temporarily cut off the embarrassing visual information intake from speakers and indicate their embarrassment and dissatisfaction to their interlocutors (Kendon 1967). Additionally, recent studies in CA suggest a link between gaze aversion and dispreference organization (Kendrick and Holler 2017)

Multimodality in refusals in English as a lingua franca

and divergent stance taking (i.e., disagreements) (Haddington 2006). Specifically, most dispreferred responses are produced with gaze aversion and gaze aversion often accompanies actions that embody divergent stances. Overall, Section 2 shows that refusals are seldom examined in the ELF context and multimodal refusal research is still scarcer. These gaps provide an opportunity for the researcher to examine refusals in a new context (i.e., ELF) and a new angle (i.e., multimodality). Two research questions are investigated in this paper: RQ1: What multimodal cues are used by participants for politeness concerns in ELF refusals? How do they deploy them in interactions? RQ2: What characteristics of ELF refusals can we detect and how are they related to multimodal cues?

3. Methodology 3.1 Participants Ten pairs of Chinese and Indonesian male participants took part in this research with an average age of 21. English interactions in this study between Chinese and Indonesians incontestably form an ELF environment. Participants in this study cannot be simply viewed as foreign language learners whose main purpose is to communicate with the English native-speakers. Rather, they are ELF speakers using English to communicate with other non-native speakers. Both Chinese and Indonesian participants were recruited from universities in Zhejiang Province, China and they did not know each other before the study. All participants have an intermediate English proficiency level. Chinese participants are native speakers of Mandarin and have passed the National College English Test Band 6. Indonesian participants are the native-speaker of Bahasa Indonesia and have learnt English for more than ten years. They use English as a communicative medium for study in China. Seven Indonesian participants report that they only have a beginner level in Chinese, as they have just started to learn Chinese for one year. Three Indonesian participants report that they have advanced Chinese level, as they are Indonesian Chinese.

3.2 Instruments Two role-plays are used to elicit data. Role-plays are widely used for data collection in refusal studies (e.g., Taguchi 2013; Bella 2014; Al-Gahtani and Roever 2018). The pros and cons of role-plays are well-established in the literature (see

107

108

Xianming Fang

Taguchi and Roever 2017 for a review). The present research deploys role-plays because it allows the researcher to collect interactive data where contextual factors can be controlled in a time-effective way. To enhance the authenticity of the roleplays, before the design of the role-plays, a survey2 was distributed to fifty students in China and Australia to investigate what kind of refusal scenarios are more likely to occur in their daily life. Two scenarios were designed based on the survey’s findings. Scenario 1 is about borrowing a book from a classmate. Scenario 2 is about helping with an assignment. Both the two role-plays feature low levels of power and social distance (i.e., talking to a friend and fellow student) but a high rank of imposition (i.e. the refusals and the requests are fairly sensitive). Holtgraves and Yang (1992: 252) suggest that “when any of the three interpersonal variables reach a particularly high level”, speakers “will be polite regardless of the closeness of the relationship with the other person”. That is to say, despite the lower power and social distance, the refuser is still likely to reject the requester politely due to the high level of imposition.

Scenario 1 Your friend (the requester) failed to request a book from the library for this week’s assignment. This week’s assignment is based on the book and will be due in the next few hours. The book is not available in your library now and you have got the last one from the library. Your friend comes to you and wants to get your help. However, you (the refuser) are also working on the assignment and still need to use the book. You can’t help your friend.

Scenario 2 Your friend (the requester) has some problems with the assignment which will be due in the next few hours. He always misses the class and doesn’t take study as a serious thing. He comes up to you (the refuser) to seek your help. However, you have already helped him several times. This time you don’t want to help him. In addition to the role-plays, the study also used interviews to collect participants’ metapragmatic comments regarding politeness as complementary data. The basic interview question for elicitation of participants’ politeness concerns during the tasks was: “Did you consider politeness in your performance and how did you realize it?” Participants’ interview data related to multimodal cues are 2. Consulting previous literature on role-plays about requests and refusals, the survey listed eight topics for participants to choose and an open topic to fill. Participants who choose the given options also need to give a brief description about the context of the requests/refusals. Here are some examples of the topics: borrow books/class notes, borrow money, buy food, help with assignment, and so on.

Multimodality in refusals in English as a lingua franca

used in this paper to help the researcher interpret their performance. Since the researcher can speak both Chinese and English, Chinese participants answered the questions in either Chinese or English, while the Indonesians used English to answer the questions. It is worth clarifying that participants’ comments are treated here as “interactional productions shaped by the context in which they are produced rather than mere representations of the participants’ interests, goals, etc.” (Pomerantz 2012: 504). In other words, participants’ comments represent another type of interaction which is shaped by both the interviewee and the interviewer (researcher), and therefore, cannot be simply viewed as definitive proof of participants’ reasons for behaving in a certain way during the tasks. However, participants’ metapragmatic comments can still provide researchers with close and comprehensive observation directions because what participants have commented on may be overlooked by the researcher or withheld in the performance by the participants (Pomerantz 2005). The metapragmatic comments also offer “some confirmatory evidence that the participants probably had oriented to those concerns” (Pomerantz 2005: 102). For these reasons, participants’ metapragmatic comments are used in this study.

3.3 Research procedures The ten pairs were divided into two groups: in one group (five pairs) the Chinese acted as refusers, while in another group (five pairs) the Chinese acted as requesters. The division into two groups was made to facilitate comparison between Chinese and Indonesian ELF speakers, although such comparisons are not discussed in depth in the current paper. Before the role-plays, each pair of participants was given 20–30 minutes to establish an acquaintanceship, talking about their hometown, hobby, knowledge of their partner’s country, and so on. The participant in the requester role was given two minutes to prepare, while no preparation time was given to the refuser. This is because in daily life, high-imposition requests are usually prepared in advance, while refusals are usually unpredictable. After the role-plays, participants had an immediate interview with the researcher, commenting on their own and their partner’s performance. There were no time limitations for the role-plays but the requesters were asked to make further negotiation with the refusers (i.e., making further requests) after being rejected for the first time. The role-plays and interviews were video-recorded in a quiet room. Three cameras were set up at the front, side and back of the desk. Twenty roleplays of more than 80 minutes in total were collected.

109

110

Xianming Fang

3.4 Data analysis method The data were transcribed following Jefferson’s (2004) CA transcription conventions and Mondada’s (2016) multimodal CA transcription conventions. I used ELAN to annotate multimodal cues. Table 1 lists the categories which occurred in participants’ performance. Participants (refusers and requesters) used various multimodal cues in their role-plays, such as gassho (i.e., putting palms together in front of the chest), palm address (i.e., the palm is directed toward something or somebody), vertical palm gestures (i.e., the hollow of the palm is vertical toward the interlocutor), handshaking, etc. In this paper, I particularly focused on three modalities which are closely related to the speech act of refusal: body positions, smiles with vocal sound, such as smiling voices ha-ha, and smiling facial expressions which are without vocal sound, and gaze. I define gaze aversions that are less than 0.6s as temporary and more than 0.6s as long. 0.6s is chosen as a baseline to interpret the length of the gaze aversion because the average latency in language production is 0.6s (Levinson and Torreira 2015). In this paper, I particularly focused on long gaze aversion rather than temporary gaze aversion because temporary gaze aversion is within the latencies of language production and is likely to serve for speech preparation (Kendon 1967) or word searching (Goodwin 1987). Table 1. Annotated categories. * 1 Body position

Standing or sitting

Multimodality in refusals in English as a lingua franca

Table 1. (continued) 2 Gaze

Mutual gaze

Long gaze aversion (longer than 0.6s) Temporary gaze aversion (less than 0.6s)

3 Facial gestures

Smiles (with vocal sound) Smiling facial expressions (without vocal sound)

4 Hand gestures/ movements

Points

111

112

Xianming Fang

Table 1. (continued) Hand shaking

Gassho

Palm address

Vertical palm gestures

Manipulation of objects

* All pictures are from my data.

Multimodality in refusals in English as a lingua franca

The data were analyzed within the framework of talk-in-interaction. I examined the sequential placements where multimodal cues occurred, the semantic formulas which multimodal cues accompanied, and the previous and next turns that the multimodal turns related to. These help the researcher to understand and make inferences about the role of multimodal cues in interactions, as Kádár and Haugh (2013: 119) remind us that “participants, and thus analysts, can only be sure in our understandings of politeness as the prior and subsequent turns allow”. I don’t separate the Chinese and Indonesian results unless there is a noticeable difference because the focus of this study is the multimodality refusals in ELF rather than cross-cultural comparison.

4.

Findings

This section presents my findings of three multimodal resources with possible politeness functions in participants’ refusals: (1) body position, (2) smiling voices/ smiling facial expressions, and (3) long gaze aversion. All names used in this paper are pseudonyms and participants’ faces are partially concealed for ethical consideration.

4.1 Body position The role-play was set up under a condition that the refusers are already seated in the classroom, pretending to be doing homework or self-study. The requesters were outside the classroom and they needed to open the door and walk to the refusers. No specific instructions were provided regarding what positions participants had to adopt or maintain during the roleplay, which means that the requesters were free to sit down or remain standing, and the refusers were free to remain seated or stand up. In this study six requesters chose to sit down to make the requests and four requesters remained standing throughout the interaction. Differently, refusers’ body positions seem to be more uniform. Almost all of the refusers (n = 9/10) remained sitting throughout the interaction regardless of their interlocutors’ body positions. This resulted in seven interactions where both participants were on the same body plane (six where they were both seated, one where they were both standing) and three where they were on different planes (requester standing, refuser seated). Excerpt 1 shows an example of where the two participants had different body positions. As shown in Figure 1, Tom, the Indonesian refuser, remained sitting throughout the interaction, while Yu, the Chinese requester, kept his standing position. The different body positions between Tom and Yu

113

114

Xianming Fang

show Tom’s maintenance of dominance in the interaction, which is signified through the refuser’s command of space (Burgoon and Dunbar 2006). Specifically, refusers are free to adopt and remain in a comfortable posture (i.e., sitting) in the interaction. Excerpt 1. 27 28

29 30 31

TOM Oh. So, you wanna use this book too? YU yeah. I want it#. fig                #fig.1

Figure 1 TOM OH, MAN. I can’t, I can’t really give it to you right now but if I could finish this within like (1.0), within the, the, the one or two hours, maybe I could lend it to you.

Out of the ten pairs of participants, Ye (Chinese, refuser) was the only refuser who adopted a standing posture. In Excerpt 2, Ye initially had a sitting position (line 1–2, see Figure 2). Then, he adopted a standing posture for handshaking (line 3–4, see Figure 3). The interaction enters the request-refusal sequence from line 4, where Wa produces a pre-pre.3 Ye remained standing until the end of the role-play. Excerpt 2. Help with assignment WA (Indonesian requester) & YE (Chinese refuser) 1

WA ye ye

*Ah, you are here. Hi#. *>> sitting     ---->                      #fig.2

2

YE

Figure 2 Hi.* --->*

3. Pre-pre, also known as “preliminary to preliminary”, refers to “these utterances with action projections, serv[ing] to allow some preliminaries germane to the projected sequence to get accomplished or established before the base sequence itself has its FPP articulated” (Schegloff 2007: 44)

Multimodality in refusals in English as a lingua franca

3 WA YE YE fig

4

WA

5

ye YE

^*(1.3)# ^shake hands -----> *shake hands -----> *  stand up ----->>        #fig.3

Figure 3 You are good*^? Can you do me a favor?        ----->^        ---->* Er, what kind of favor?

In the interview, Ye (Chinese refuser) gave his justification for his standing posture, as he claimed that “I think it is impolite (to sit down) to speak to him”. That is to say, Ye perceived his standing action as polite in the refusal context and therefore consciously adopted this body position. Similarly, in another pair of participants, Ma (Chinese requester) also claimed that he perceived the standing posture as appropriate in the context: “他坐着说明他处于一个比较舒适的状态，然后 我这样去（站着）就可以让他感受到我的态度，是比较有诚意的..” ‘That he sat down showed that he was in a comfortable situation. When I did this (i.e., stood up), I could make him feel my attitude, a more sincere attitude.’ From the requester’s perspective, the standing posture is linked with sincerity, which shows his deference to the refuser in the context. Overall, this section shows that refusers and requesters in ELF talk exhibit different body positions. Refusers (n = 9) uniformly chose to remain seated except for one exception, while the requesters either sat down (n = 6) or remained standing (n = 4) throughout the interaction. Remaining seated appears to be the general norm for refusers, even though standing may be a politer behavior as observed by my participants. The preference for remaining seated is likely to be conferred by the high contextual power of refusers because they have control over the outcome that the requesters urgently seek. Thereby, refusers are more relaxed and free to choose their body positions (see the Discussion for details).

4.2 Smiling voices/smiling facial expressions This section demonstrates that smiling voices/smiling facial expressions (without smiling voices) often accompany refusal explanations and are used to express

115

116

Xianming Fang

criticisms for refusal purposes. In total, eight refusers used smiles/smiling facial expressions that were tightly synchronized with their refusals. Excerpts 3 and 4 exhibit that refusal explanations often accompany smiling voices/smiling facial expressions. The two selected excerpts are from one roleplay (borrowing the book) between Su (Indonesian, refuser) and Xu (Chinese, requester). Excerpt 3 is the pre-expansion of Xu’s request and Su’s “blocking responses”. Excerpt 3. Borrow the book Su (Indonesian refuser) & Xu (Chinese requester) 1 2 3 4 5 6 7 8 9 10

XU su SU XU SU XU

^ Hi, bro^ ^raise head^ Hi How do you do? I’m good. Er: (0.5) er:: have you ever, have you^ finished ^# this week’s assignments?^                                      ^palm address^    sit             down ^ SU Not yet, sorry. XU Oh, I didn’t finish it yet. And I just went to the library. ^Just now^#. And, but I ^couldn’t find the book^. (0.7) ^ points ^              ^  hold   two   hands  ^ Yeah. So I was wondering ^ do^ you have^it^?                          ^...^points---^,,,^ SU Yeah, I have it but *this is ￡the only one￡*. And instead, I am using it.*#                     *     slap the book      *  smiling facial expression  * fig                                                                             #fig.4

Figure 4

After an ostensibly polite greeting from line 1 to line 4, Xu made two pre-requests from line 5 to line 9. The first pre-request in line 5 checks the availability of Su, while the second pre-request in line 9 checks the availability of the book. Xu’s utterances from line 5 to 9 are open to evaluation as “polite”, since he is orienting to being relatively prudent in making a request, recognizing his requests should be based on the assumption of Su’s availability (i.e., he has finished the homework and doesn’t need the book). In line 6, Su addressed the first prerequest, suggesting that he was not available. In line 10, Su addressed the second pre-request. He first acknowledged that he had the book and then explained that he was using the book at present. When making this explanation, he used smiling voices together with a smiling facial expression as shown in Figure 4.

Multimodality in refusals in English as a lingua franca

These explanations constitute a pre-emption, or what Schegloff (2007: 30) coined “blocking responses”; “that is, it raises the possibility that the [request], if tendered, will be declined or rejected, and thereby discourages or blocks the [FPP request] from being tendered at all”. Therefore, the explanation with smiles used in the pre-expansion has a pre-refusal function, as a harbinger of the upcoming refusals. Apart from pre-refusals, smiles also organizedly accompany the base SPP (i.e., refusals). In Excerpt 4, Su mainly used explanations as his refusal strategies. Every time he made explanations, he used either a smiling voice or a smiling facial expression. Excerpt 4. 16 17 18 19

XU SU XU SU

Umm, could you please just, umm= Borrow from the library? Yeah. But, I see, I see, you can see like, *I am now doing the#, ￡[doing]￡ the homework.                                      *   smiling facial expression  -----> fig                                                         #fig.5

xu 20 21 22 23

XU SU

Figure 5                                                              [so] So, [ I ]*    ----> * [so when ] will you finish it? (1.8) I don’t know, *I haven’t, ￡I just￡, I am just starting now*               *      smiling facial expressions             *

Line 16 is the FPP of the requester: Xu uttered “could you please”, and when he was searching for words to complete the request, the refuser helped him to complete the sentence by uttering “borrow from the library?” There is no direct refusal addressing the FPP. Although a direct refusal may be produced by Su in line 20, it is interrupted by Xu’s onset overlapping utterance (Jefferson 1984) which is about further information about when Su can complete the assignment. Thus, Su’s explanation in line 19, consisting of verbal speech and a smiling facial expression (see Figure 5), is interpreted as the SPP (i.e., as the refusal). It serves as a refusal because the onset overlapping indicates that “what is being said in the current turn has been made perfectly available before it reaches completion” (Hayashi 2013: 177). In line 23, when addressing Xu’s detailed enquiries, Su made

117

118

Xianming Fang

an unspecified answer and re-explained the reason with smiles. In brief, a recurrent phenomenon shown in Su’s case is that smiles often accompany his indirect refusal strategy: explanations. That is to say, smiles accompany important parts of refusals. Smiling voices/smiling facial expressions can also accompany the refuser’s criticisms with refusal purposes. In Excerpt 5, Al (Indonesian, refuser) frequently used criticisms (i.e., criticizing the conduct of the requester) as his refusal strategy. When he criticized Lin (Chinese, requester), he always used smiling voices/smiling facial expressions. Excerpt 5. Help with assignments AL (Indonesian refuser) & LIN (Chinese requester) 14 15 16 17 18

LIN ^yeah, I, you know, I could not figure out any part of them.^ ^                        palm presentation                  ^ I don’t really understand the questions, so ^please help me^.                                             ^  palm point  ^ AL what are you doing *on this assignment, bro? come on=.*                    *   smiling  facial expressions    * LIN oh, no. [you know] AL    *[you always] you always like, miss the class#. And you even don’t study.*    *                     smiling         facial expressions                 * fig                                                 #fig.6

Figure 6 LIN no, ^I study, but this I just don’t have enough time.     ^      palm presentation ---> 20 I got a part time job and I have to hang on with my friend, and I always stay up late. 21 You know, yeah^, so I can not arrange my time well. ^So please help me this time.^          ---> ^                                     ^      palm addressed        ^ (Transcription continues) 59 LIN all right. so please help me this time. 60 AL but *you know you are wrong. (0.9) you know that.     *       smiling           --->line 62 61 (2.5) 62 LIN no, I don’t think there is any* questions about that. al                           --->* 63 AL *no, I think you are wrong. *       smiling ---> 64 You never come to the class and then you want me to help you, bro. ha-ha*                                                                    ---->* 65 LIN ^no, no, no^. you are wrong, you are wrong. You know there are many ^wave palms^ 19

Multimodality in refusals in English as a lingua franca

Rather than using direct refusals, Al refused the request by “criticiz[ing] the request/requesters, etc. (i.e., statement of negative feeling)” (Beebe et al. 1990: 73). He criticized Lin’s daily performance and his study attitude (line 16 and 18), which constitutes the SPP (refusal). In lines 19–20, Lin tried to deny the criticism, find excuses for himself, and expand the sequence by renewing the FPP, which was evidence that he interpreted the criticism as a refusal. Al’s criticism in line 18 consists of verbal speech (including exaggerating phrases “you always”) and smiling facial expressions (See Figure 6). These exaggerated formulae suggest that the criticisms are jocular mockery, more specifically, jocular criticism: “interrogating or challenging others in a critical way yet playful tone to indicate disagreement, dissatisfaction, or criticism” (Yang and Ren 2021: 35). As Haugh (2014) points out, “mockery can be projected within a non-serious or humorous frame, and thus as a ‘laughable’” (Haugh 2014: 79), and linguistically it can include “exaggerated, incongruous, and formulaic expressions” (Haugh 2014: 80). Similarly, when addressing Lin’s further requests, Al also adopted jocular criticisms with smiling facial expressions in lines 63–65. From the perspective of the refuser Al, his refusal strategy is open to evaluation as at least not impolite because he framed it as a jocular criticism, refusing Lin in a playful and non-serious way. Haugh (2014) summarized that when recipients interpret mockery as jocular, they typically display agreement with or appreciation of the mockery. However, it is hard to infer that Lin interpreted this criticism in a jocular and non-impolite way, especially in lines 59–67. Lin used “no” three times (line 65) with his vertical palms waved towards Al (Figure 6). Kendon (2004) suggests that vertical palm gestures are used in contexts where the speaker indicates “a wish that what is being done by the interlocutor should be halted” (Kendon 2004: 308). Through the combination of palm gestures and spoken “no”, Lin indicated his disaffiliation with Al. Thus, from Lin’s actions, we can see that even though Al’s criticism is softened by smiling, presumably, it is still somewhat unacceptable to Lin. Overall, this section shows that smiling voices/smiling facial expressions are highly organized in request-refusal sequences in ELF interactions. A recurrent phenomenon is that they often accompany explanations and criticisms with refusal purposes.

4.3 Long gaze aversion Long gaze aversion refers to the refusers’ action of averting gaze from requesters for more than 0.6s. In my data, almost all refusers (n = 9) exhibit long gaze aversions that accompany their refusals.

119

120

Xianming Fang

A common pattern for long gaze aversion is that it organizedly accompanies pre-refusals (gaps, turn-initial delay, preparators), serving as a harbinger for the upcoming refusal. In Excerpt 5 where Ja (Chinese) rejected Sg’s (Indonesian) requests to borrow the book, Ja used long gaze aversion together with a long silence before producing the second pair part. In the interview, Ja claimed that he consciously deployed gaze aversion for addressing embarrassment and maintaining a refusal stance. Excerpt 5. Borrow the book JA (Chinese refuser) & SG (Indonesian requester) 17 18

19

20

SG

yes, but ^   in the library  ^, it ^don’t have the book^ any more.          ^index finger points^     ^  points the book  ^ So can I, can I borrow you, ^ please# borrow, borrow some^, borrow for a while?                             ^          gassho          ^ fig                                     #fig.7

Figure 7 • (1.6)# ja • gaze aversion ---> fig        #fig.8

JA

21 22 23 24

SG ja

Figure 8 *umm, (1.6) but (1.0), umm, the embarrassing situation is that* I have, •now•, *    ,,,,,,,,                                                 * turn pages --->line24                                                               --->•gaze back• •I have to also refer• to this • book to • finish my assignment. • • gaze aversion • gaze back • gaze aversion•    gaze back  • •And will be due in the• next few •hours. So I really• need this book right now. • •gaze aversion       • gaze back•  gaze aversion•   gaze back         • (2.6) umm, I know. But maybe, we can (1.2)*, we can ^     use together      ^.                                               ^horizontal palms waving^                                ---->*

Multimodality in refusals in English as a lingua franca

In line 17 and 18, Sg has made the FPP (request) with a gassho gesture (Figure 7). Upon seeing this gesture, Ja averted his gaze from Sg and employed a long silence (1.6s) in line 19 (Figure 8). Gaze aversion occurs for almost the entire line 20 where Ja made a preparator (the embarrassing situation is that). Since there is no direct refusal addressing the FPP, Ja’s explanations are viewed as the second pair part (lines 21–22). Thus, the silence, long gaze aversion and preparator constitute a pre-rejection, “serv[ing] for co-participants as signals of upcoming dispreferred second pair parts” (Schegloff 2007: 69). The long gaze aversion is arguably a part of the pre-rejection because it is clear that there are two different gaze directions before/during the SPP: before the SPP, the long gaze aversion occurs in lines 19–20, while during the SPP, there is more frequent gaze exchange between the two participants. In the interview excerpt, Ja explained that he used long gaze aversion to reduce embarrassment caused by the requester’s gassho gesture and maintain his refusal stance. Interview excerpt F: why (did you avoid eye contact with the interlocutor)? Ja: because, because I want to reject him, reject him at that time. I don’t want to compromise so quickly. But since his gesture, I feel a little embarrassed, so I put my head down.

5.

Discussion

This paper has shown that body position, smiling voices/smiling facial expressions, and long gaze aversion occur in a very organized way. They accompany important parts of refusals, such as pre-emption, inter-turn gaps (silence), explanations, and criticisms. In this section, I discuss the politeness function of these multimodal cues in refusals.

5.1 Multimodal politeness in refusals To answer the first research question about what multimodal cues are used for by participants and how they are used to show politeness, this study shows that body positions, smiling voices/smiling facial expressions, and gaze aversion are possible resources deployed by refusers for rapport management. The sitting position appears to be a norm for refusers. The preference for the sitting position is influenced by refusers’ high contextual power status. In my research design, the two roles are set as having similar power (P): they are both students. Technically, they are not supposed to have an asymmetric relation.

121

122

Xianming Fang

However, power is not always strictly linked to vertical hierarchy. Spencer-Oatey (2008: 34) suggests that “if a person A has control over positive outcomes (such as bonus payments, improved job conditions) that another person desires, A can be said to have a reward power over B.” In this role-play, the refuser undoubtedly is in a position of high reward power because he is the one who can help the requester to do the assignment. In a role with a higher power position, my refusers uniformly choose to sit, while nearly half of the requesters remain standing when making requests. This finding echoes Brown and Winter’s (2019) finding that higher power speakers adopt more relaxed and flexible postures like sitting, while lower power speakers are often more constrained in multimodal cues and may have to remain standing in some contexts. Notably, in the singular case where a refuser adopted a standing posture, the refuser claimed a politeness function for this posture in the interview. The interpretation of politeness makes sense in terms of power redistribution. When the refuser chooses to stand as the requester does, he relinquishes his higher power status. By standing up, the refuser redistributes the power and creates a relatively symmetric power context, which shows his “rapport-enhancement orientation: a desire to strengthen or enhance harmonious relations between the interlocutors” (Spencer-Oatey 2008: 32). Although there is only one standing refuser case in my data, it still shows that the standing position is a possible and potential multimodal resource for refusers to show their politeness concerns. Smiling voices/smiling facial expressions are frequently found to accompany important parts of refusals such as explanations and criticisms. In fact, the roleplay situation resulted in a lot of smiling voices and smiling facial expressions, and a lot of this can be attributed to the embarrassment and also the enjoyment that the participants experienced in performing a role-play in front of the cameras. However, what is interesting for the current study is that there were also smiling voices/smiling facial expressions that were organized in a way that shows clear concern for rapport management. When smiling voices/smiling facial expressions accompany explanations, they have a noticeable politeness function rather than an emotional function (i.e., “laughable”) because explanations for refusals in my scenarios are not something humorous or amusing. Instead, explanations for refusals have a delicate nature because explanations are the dispreferred responses to the requester’s requests and show that the refuser is at odds with the requester. Haakana (2010) points out that smiling voices/smiling facial expressions can be used to frame the talk and action at hand as being delicate. By smiling in these interactional placements, the refuser displays that they are aware of the possible delicacy of the refusal explanations, specifically, the possible face threats and interactional goal threats caused by the explanations.

Multimodality in refusals in English as a lingua franca

When smiling facial expressions or smiling voices accompany criticism, they help the refuser frame the criticism as jocular. Refusers used them as multimodal downgraders because they “reduce or weaken the force” of the speech act (Spencer-Oatey 2008: 23). Generally, criticisms are risky for rapport management, as Spencer-Oatey (2008) points out that losing face can happen when people are criticized. Yet, thanks to the smiling voices/smiling facial expressions, the refuser’s negative feelings can be masked via the playful and non-serious tone which is achieved through smiling voices/smiling facial expressions. Therefore, using criticism for refusals becomes less rapport-threatening. However, my results show that there is no guarantee that requesters will successfully interpret jocular criticism, despite the politeness calibration function of smiling voices/smiling facial expressions (McKinnon and Prieto 2014). Haugh and Bousfield (2012: 1104) point out that mock impoliteness “is interactionally achieved if both, speaker and hearer(s) project and interpret the meanings/ actions as such given the background knowledge that both the speaker and recipients (whether direct or indirect addressees) are assumed to have access to.” Yet, in the ELF context the background knowledge for achieving successful mockery impoliteness may be limited, given the fact that speakers have different L1 backgrounds and English proficiency levels. Participants’ different understandings of the jocular criticism, I would argue, show the complexity of ELF interactions. Processing mock impoliteness is a challenge for lingua franca users “with potentially differing capacity to accurately produce and interpret such humour types within the intended jocular frame” (Walkinshaw and Kirkpatrick 2020: 71). It also shows that the interpretation of politeness may vary and “we need to relinquish the oft implicit assumption that everyone has the same understanding of politeness in interaction” (Kádár and Haugh 2013: 129). Apart from body positions and smiling voices/smiling facial expressions, long gaze aversion also appears to have a rapport management function in refusals. The long gaze aversion is usually before the refusal. Combining with other modalities with disengagement function such as silence and refusal preparator utterances (e.g., the problem is), long gaze aversion constitutes a strong pre-rejection hint to the speaker that the listener is not interested in the request and implies a potential dispreferred response, as Goodwin (1980: 277) points out that gaze is “one means available to recipients for displaying to a speaker whether or not they are acting as hearers to the speaker’s utterance”. That is to say, before the specific verbal refusal, the refuser has already attempted to convey his refusal stance through the multimodal pre-rejection.

123

124

Xianming Fang

5.2 Refusals and English as a lingua franca To answer the second research question about the characteristics of ELF refusals, this study shows that mitigation appears to be an important characteristic of ELF refusals and multimodal cues are important ways for them to realize mitigation. Despite the preference for the sitting body position displaying the refuser’s high power, the actual refusals are generally mitigated with various supportive moves which are realized through verbal and multimodal means. Similar to previous ELF research on complaints (Konakahara 2017) and disagreements (Matsumoto 2018), this study found an important function of multimodal cues in organizing polite rapport-threatening speech acts. When making explanations and criticisms, multimodal cues such as smiling voices and smiling facial expressions are often deployed by refusers to indicate the delicacy of the situation and mitigate the force of action. Also, refusals are usually foreshadowed and delayed by verbal and multimodal cues, such as silence and long gaze aversion. Previous research claims that ELF is characterized by direct speech acts such as “raw negation, rejection and disagreement” (House 2008: 354). However, such claims were based on a limited set of data, which did not look at rapportthreatening contexts in detail and which only considered European participants. In the current study, by looking at role-plays that included sensitive situations and participants from Asian countries, we see that directness is not necessarily a characteristic of ELF. The preference of mitigated refusals is likely to be influenced by participants’ L1, as both Chinese and Indonesian refusals are indirect and mitigated. Additionally, considering the fact that the Indonesian participants are in China and know Chinese language and culture, there is also a possibility that the mitigated refusals are due to their accommodation to the Chinese participants and culture. Overall, the appearance of mitigated refusals with multimodal politeness (verbal speech plus multimodal cues) in this study is, I would argue, the whole point. It indicates that although ELF interactions pursue transparency and practicality (Seidlhofer 2011), politeness still plays an important role in ELF interactions, especially in a rapport-threatening context. Locher (2018) reminds us that language is not just a means to pass factual content from one person to another, but also a tool to shape relationships and thus to negotiate interpersonal meaning. When people perform a rapport-threatening refusal, it is natural for the speakers to consider politeness, even in the ELF context. Thus, mitigation and other supportive moves in ELF refusals are expected, and refusers may deploy all possible resources to make appropriate refusals, such as body position, facial expressions, and gaze.

Multimodality in refusals in English as a lingua franca

6.

Conclusion

This study closely examined how multimodal cues are deployed by ELF interactants in refusal tasks, illustrating how multimodal cues such as standing, smiling facial expression, smiles, and long gaze aversion are used for rapport management. Data analysis shows that (1) body position (standing) reconstructs the asymmetrical power relationship between the two interactants; (2) smiling voices/smiling facial expressions play a role as a multimodal downgrader, expressing the refuser’s awareness of the delicacy of the situation and mitigating the force of the speech act; and (3) long gaze aversion indicates the speaker’s disinvolvement to the current topic and serves as a pre-rejection. Also, this study shows that politeness understandings should be seen as co-constructed and there exists multiplicity of politeness understandings. Moreover, my research contributes to a different ELF refusal picture in rapport-threatening contexts where mitigated refusals are still widely used for politeness concerns. The paper has important implications for politeness research and refusal studies, which are still concentrated on the analysis of the verbal dimension of politeness. Nowadays, a number of refusal studies still focus on examining the frequency of refusal strategies (i.e., semantic formulas), determining the level of politeness based on this frequency. Yet, this paper shows that multimodal cues are important parts of speakers’ politeness performance. Without considering these multimodal cues, politeness research would be somewhat incomplete. Although the findings in this study are specific to a certain speech act (refusals) produced in a certain context (rapport-threatening role-plays), this study reveals the possibility of multimodal cues in performing politeness in ELF refusals. For future research, more multimodal research concerning other speech acts and intercultural contexts is encouraged.

References Abdolrezapour, Parisa, and Hossein Vahid Dastjerdi. 2013. “Examining Mitigation in Refusals: A Cross-cultural Study of Iranian and American Speech Communities.” Sociolinguistic Studies 6 (3): 519–541. https://doi.org/10.1558/sols.v6i3.519 Al-Gahtani, Saad, and Carsten Roever. 2012. “Proficiency and Sequential Organization of L2 Requests.” Applied Linguistics 33 (1): 42–65. https://doi.org/10.1093/applin/amr031 Al-Gahtani, Saad, and Carsten Roever. 2013. “‘Hi doctor, give me handouts’: Low-proficiency Learners and Requests.” ELT Journal 67 (4): 413–424. https://doi.org/10.1093/elt/cct036 Al-Gahtani, Saad, and Carsten Roever. 2015. “The Development of Requests by L2 Learners of Modern Standard Arabic: A Longitudinal and Cross-Sectional Study.” Foreign Language Annals 48 (4): 570–83. https://doi.org/10.1111/flan.12157

125

126

Xianming Fang

Al-Gahtani, Saad, and Carsten Roever. 2018. “Proficiency and Preference Organization in Second Language Refusals.” Journal of Pragmatics 129: 140–153. https://doi.org/10.1016/j.pragma.2018.01.014

Barron, Anne. 2007. “‘A h no honestly we’re okay’: Learning to Upgrade in a Study Abroad Context.” Intercultural Pragmatics 4 (2): 129–166. https://doi.org/10.1515/IP.2007.009 Beebe, Leslie M., Tomoko Takahashi, and Robin Uliss-Weltz. 1990. “Pragmatic Transfer in ESL Refusals.” In Developing Communicative Competence in a Second Language, ed. by Robin C. Scarcella, Elaine S. Andersen, and Stephen D. Krashen, 55–74. New York: Newbury House. Bella, Spyridoula. 2014. “Developing the Ability to Refuse: A Cross-sectional Study of Greek FL Refusals.” Journal of Pragmatics 61: 35–62. https://doi.org/10.1016/j.pragma.2013.11.015 Bhatti, Joanna, and Vladimir Žegarac. 2012. “Compliments and Refusals in Poland and England.” Research in Language 10 (3): 279–297. https://doi.org/10.2478/v10015-011-0025-x Brown, Lucien, and Bodo Winter. 2019. “Multimodal Indexicality in Korean: ‘Doing Deference’ and ‘Performing Intimacy’ through Nonverbal Behavior.” Journal of Politeness Research 15 (1): 25–54. https://doi.org/10.1515/pr-2016-0042 Burgoon, Judee K., and Norah E. Dunbar. 2006. “Nonverbal Expressions of Dominance and Power in Human Relationships.” In The Sage Handbook of Nonverbal Communication, ed. by Valerie Manusov, and Miles Patterson, 279–298. Thousand Oaks, CA: Sage. https://doi.org/10.4135/9781412976152.n15

Chen, Hongyin. 1996. Cross-cultural Comparison of English and Chinese Metapragmatics in Refusal. Unpublished Doctoral Dissertation, Indiana University. Culpeper, Jonathan. 2011. “Politeness and Impoliteness”. In Sociopragmatics, Handbooks of Pragmatics 5, ed. by Gisle Andersen, and Karin Ajimer, 391–436. Berlin: Mouton de Gruyter. https://doi.org/10.1075/bct.31 Culpeper, Jonathan, Alison Mackey, and Naoko Taguchi. 2018. Second Language Pragmatics: From Theory to Research. New York: Routledge. https://doi.org/10.4324/9781315692388 Félix-Brasdefer, J. César. 2004. “Interlanguage Refusals: Linguistic Politeness and Length of Residence in the Target Community.” Language Learning 54 (4): 587–653. https://doi.org/10.1111/j.1467-9922.2004.00281.x

Firth, Alan. 1996. “The Discursive Accomplishment of Normality: On ‘Lingua Franca’ English and Conversation Analysis.” Journal of Pragmatics 26 (2): 237–259. https://doi.org/10.1016/0378-2166(96)00014-8

Goodwin, Charles. 1980. “Restarts, Pauses, and the Achievement of a State of Mutual Gaze at Turn-Beginning.” Sociological Inquiry 50 (3–4): 272–302. https://doi.org/10.1111/j.1475-682X.1980.tb00023.x

Goodwin, Charles. 1987. “Forgetfulness as an Interactive Resource.” Social Psychology Quarterly 50 (2): 115–130. https://doi.org/10.2307/2786746 Haakana, Markku. 2001. “Laughter as a Patient’s Resource: Dealing with Delicate Aspects of Medical Interaction.” Text & Talk 21 (1): 187–219. https://doi.org/10.1515/text.1.21.1-2.187 Haakana, Markku. 2010. “Laughter and Smiling: Notes on Co-occurrences.” Journal of Pragmatics 42 (6): 1499–1512. https://doi.org/10.1016/j.pragma.2010.01.010 Haddington, Pentti. 2006. “The Organization of Gaze and Assessments as Resources for Stance Taking.” Text & Talk 26 (3): 281–328. https://doi.org/10.1515/TEXT.2006.012 Hassall, Tim. 2020. “Preference Structure in Request Sequences: What about Role-play?” Journal of Pragmatics 155: 321–333. https://doi.org/10.1016/j.pragma.2019.02.020

Multimodality in refusals in English as a lingua franca

Haugh, Michael. 2014. “Jocular Mockery as Interactional Practice in Everyday AngloAustralian Conversation.” Australian Journal of Linguistics 34 (1): 76–99. https://doi.org/10.1080/07268602.2014.875456

Haugh, Michael, and Derek Bousfield. 2012. “Mock Impoliteness, Jocular Mockery and Jocular Abuse in Australian and British English.” Journal of Pragmatics 44 (9): 1099–1114. https://doi.org/10.1016/j.pragma.2012.02.003

Hayashi, Makoto. 2013. “Turn Allocation and Turn Sharing. In The Handbook of Conversation Analysis, ed. by Jack Sidnell, and Tanya Stivers, 167–190. New York: John Wiley & Sons, Incorporated. Holtgraves, Thomas, and Joong-Nam Yang. 1992. “Interpersonal Underpinnings of Request Strategies: General Principles and Differences Due to Culture and Gender.” Journal of Personality and Social Psychology 62 (2): 246–256. https://doi.org/10.1037/0022-3514.62.2.246 House, Juliane. 2008. “(Im)politeness in English as Lingua Franca Discourse.” In Standards and Norms in the English Language. Contributions to the Sociology of Language, ed. by Miriam A. Locher and Jürg Strässler, 351–366. Berlin/Boston: Mouton de Gruyter. Jefferson, Gail. 1984. “Notes on Some Orderlinesses of Overlap Onset.” In Discourse Analysis and Natural Rhetoric, ed. by V. D’Urso, and P. Leonardi, 11–38. Padua: Cleup Editors. Jefferson, Gail. 2004. “Glossary of Transcript Symbols with an Introduction”. In Conversation Analysis: Studies from the First Generation, ed. by Gene H. Lerner, 13–31. Amsterdam: John Benjamins. https://doi.org/10.1075/pbns.125.02jef Kádár, Dániel Z., and Michael Haugh. 2013. Understanding Politeness. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9781139382717 Kendon, Adam. 1967. “Some Functions of Gaze-direction in Social Interaction.” Acta Psychologica 26: 22–63. https://doi.org/10.1016/0001-6918(67)90005-4 Kendon, Adam. 2004. Gesture: Visible Action as Utterance. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511807572 Kendrick, Kobin H., and Judith Holler. 2017. “Gaze Direction Signals Response Preference in Conversation.” Research on Language and Social Interaction 50 (1): 12–32. https://doi.org/10.1080/08351813.2017.1262120

Konakahara, Mayu. 2017. “Interactional Management of Face-threatening Acts in Casual ELF Conversation: An Analysis of Third-party Complaint Sequences.” Journal of English as a Lingua Franca 6 (2): 313–343. https://doi.org/10.1515/jelf-2017-0015 Kwon, Jihyun. 2004. “Expressing Refusals in Korean and in American English.” Multilingua 23 (4): 339–364. https://doi.org/10.1515/mult.2004.23.4.339 Levinson, Stephen C., and Francisco Torreira. 2015. “Timing in Turn-taking and Its Implications for Processing Models of Language.” Frontiers in Psychology 6: 731–748. https://doi.org/10.3389/fpsyg.2015.00731

Liao, Chao-chih, and Mary. I. Bresnahan. 1996. “A Contrastive Pragmatic Study on American English and Mandarin Refusal Strategies.” Language Sciences (Oxford) 18 (3–4): 703–27. https://doi.org/10.1016/S0388-0001(96)00043-5

Locher, Miriam. A. 2018. “Politeness.” In The Encyclopedia of Applied Linguistics, ed. by Carol A. Chapelle. Malden MA: Wiley Blackwell. https://doi.org/10.1002/9781405198431.wbeal0916.pub2

Maíz-Arévalo, Carmen. 2014. “Expressing Disagreement in English as a Lingua Franca: Whose Pragmatic Rules?” Intercultural Pragmatics 11 (2): 199–224. https://doi.org/10.1515/ip-2014-0009

127

128

Xianming Fang

Matsumoto, Yumi. 2018. “Challenging Moments as Opportunities to Learn: The Role of Nonverbal Interactional Resources in Dealing with Conflicts in English as a Lingua Franca Classroom Interactions.” Linguistics and Education 48: 35–51. https://doi.org/10.1016/j.linged.2018.08.007

Matsumoto, Yumi, and Canagarajah, Suresh. 2020. “The Use of Gesture, Gesture Hold, and Gaze in Trouble-in-talk among Multilingual Interlocutors in an English as a Lingua Franca Context.” Journal of Pragmatics 169: 245–67. https://doi.org/10.1016/j.pragma.2020.08.015

McKinnon, Sean, and Pilar Prieto. 2014. “The Role of Prosody and Gesture in the Perception of Mock Impoliteness.” Journal of Politeness Research 10 (2): 185–219. https://doi.org/10.1515/pr-2014-0009

Modigliani, Andre. 1971. “Embarrassment, Facework, and Eye Contact: Testing a Theory of Embarrassment.” Journal of Personality and Social Psychology 17 (1): 15–24. https://doi.org/10.1037/h0030460

Mondada, Lorenza. 2007. “Multimodal Resources for Turn-taking: Pointing and the Emergence of Possible Next Speakers.” Discourse Studies 9: 195–226. https://doi.org/10.1177/1461445607075346

Mondada, Lorenza. 2016. “Challenges of Multimodality: Language and the Body in Social Interaction.” Journal of Sociolinguistics 20 (3): 336–66. https://doi.org/10.1111/josl.1_12177 Mondada, Lorenza. 2019. “Conventions for Multimodal Transcription”. Accessed in October 2021 https://www.lorenzamondada.net/multimodal-transcription Morkus, Nader. 2014. “Refusals in Egyptian Arabic and American English.” Journal of Pragmatics 70: 86–107. https://doi.org/10.1016/j.pragma.2014.06.001 Murray, Neil. 2012. “English as Lingua Franca and the Development of Pragmatic Competence.” ELT Journal 66 (3): 318–26. https://doi.org/10.1093/elt/ccs016 Nelson, Gayle L., Joan Carson, Mahmoud Al Batal, and Waguida El Bakary. 2002. “CrossCultural Pragmatics: Strategy Use in Egyptian Arabic and American English Refusals.” Applied Linguistics 23 (2): 163–89. https://doi.org/10.1093/applin/23.2.163 Pomerantz, Anita. 2005. “Using Participants’ Video-stimulated Comments to Complement Analyses of Interactional Practices. In Conversation and Cognition, ed. by Hedwig te Molder, and Jonathan Potter, 93–113. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511489990.005

Pomerantz, Anita. 2012. “Do Participants’ Reports Enhance Conversation Analytic Claims? Explanations of One Sort or Another.” Discourse Studies 14 (4): 499–505. https://doi.org/10.1177/1461445611434229

Rasmussen, Gitte. 2014. “Inclined to Better Understanding: The Coordination of Talk and ‘Leaning Forward’ in Doing Repair.” Journal of Pragmatics 65: 30–45. https://doi.org/10.1016/j.pragma.2013.10.001

Ren, Wei. 2012. “Pragmatic Development in Chinese Speakers’ L2 English Refusals.” Eurosla Yearbook 12 (1): 63–87. https://doi.org/10.1075/eurosla.12.05ren Schegloff, Emanuel A. 2007. Sequence Organization in Interaction: A Primer in Conversation Analysis. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511791208

Searle, John R. 1976. “A Classification of Illocutionary Acts.” Language in Society 5 (1): 1–23. https://doi.org/10.1017/S0047404500006837

Seidlhofer, Barbara. 2011. Understanding English as a Lingua Franca. Oxford: Oxford University Press.

Multimodality in refusals in English as a lingua franca

Spencer-Oatey, Helen. 2008. Culturally Speaking: Culture, Communication and Politeness Theory. 2nd ed. London/New York: Continuum. Statista. 2021. The Most Spoken Languages Worldwide. Retrieved from https://www.statista .com/statistics/266808/the-most-spoken-languages-worldwide/ Taguchi, Naoko. 2013. “Refusals in L2 English: Proficiency Effects on Appropriateness and Fluency.” In Refusal in Instructional Context and Beyond, ed. by Otilia Martí-Arnándiz, and Patricia Salazar-Campillo, 101–119. New York: Brill. Taguchi, Naoko, and Carsten Roever. 2017. Second Language Pragmatics. Oxford: Oxford University Press. Takahashi, Tomoko, and Leslie M. Beebe. 1987. “The Development of Pragmatic Competence by Japanese Learners of English”. JALT Journal 8 (2): 131–155. Walkinshaw, Ian, and Andy Kirkpatrick. 2020. “‘We want fork but no pork’: (Im)politeness in Humour by Asian Users of English as a Lingua Franca and Australian English Speakers.” Contrastive Pragmatics 2 (1): 52–80. https://doi.org/10.1163/26660393-BJA10010 Yang, Na, and Wei Ren. 2020. “Jocular Mockery in the Context of a Localised Playful Frame: Unpacking Humour in a Chinese Reality TV Show.” Journal of Pragmatics 162: 32–44. https://doi.org/10.1016/j.pragma.2020.04.006

Appendix. Transcription conventions (adapted from Jefferson 2004; Mondada 2019) ** ^^ •• *-----> ----->* >> --->> …… -----,,,,,,,, ￡word￡ (word) (1.4) [hello] Fig #

delimits gestures done by the refuser delimits gestures done by the requester delimits gaze done by the refuser the action described continues across subsequent lines until the same symbol is reached the action described begins before the excerpt’s beginning the action described continues after the excerpt ends action’s preparation action’s apex is reached and maintained action’s retraction indicates smiling voices, or suppressed laughter. indicates uncertain words; no plausible candidate if empty. time between end of a word and beginning of next. overlapping talk exact moment at which a screenshot has been taken is indicated with a specific symbol showing its position within the turn at talk.

129

chapter 5

Indexing social distance through bodily visual practices in two languages Lucien Brown,1 Iris Hübscher,2,3 Hyunji Kim4 & Bodo Winter5 1

Monash University | 2 University of Zurich | 3 Zurich University of Applied Sciences | 4 Kennesaw State University | 5 University of Birmingham

This paper explores the multimodal indexing of social distance in two unrelated languages/cultures: Korean and Catalan. Participants performed several tasks: once with a status superior (“socially distant” condition) and once with a friend (“socially close”). Catalan speakers exhibited more body movements but were less consistent in modulating them according to social distance. The frequency of head nodding increased in the socially distant situation for both languages, and haptics never occurred with superiors. Only Korean speakers consistently reduced their rate of head shaking with the superior, and also reduced their use of adaptors. Gaze aversion showed weak results only for Catalan. The results add to a growing body of evidence that social distance is marked multimodally in similar ways across cultures. Keywords: bodily visual practices, gesture, social distance, power, Catalan, Korean, multimodality

1.

Introduction

Across a range of different languages, speakers are known to modulate their gestures and bodily movements in order to mark various im/politeness-related meanings. In American English, indirect body orientation, lowered eyebrows and expansive gestures have been analysed as “aggravating behaviors”, whereas direct body orientation, raised eyebrows and smaller gesture are “mitigating” (Trees and Manusov 1998). In Yoruba (Orie 2009) as well as many East Asian languages (L. Brown and Prieto 2017), speakers avoid pointing directly to a person with the index finger in order to be polite, particularly if that person is socially superior. Tilting the head backwards can work as a cue for mock politeness in Australian https://doi.org/10.1075/pbns.333.05bro © 2023 John Benjamins Publishing Company

132

Lucien Brown, Iris Hübscher, Hyunji Kim & Bodo Winter

English (Haugh and Bousfield 2012). In Japanese Sign Language, a smaller signing space and a lowered and forward-leaning chin and head position are associated with polite speech register (George 2011). Thus, there are a large number of different body movements that are tied to im/politeness-related meanings, sometimes in culturally specific ways. In this paper, we refer to these body movements that work together with spoken or signed language to mark social or communicative meanings—politeness in this case—as “bodily visual practices” (Ford et al. 2012). One type of politeness-related distinction that is known to be marked through changes in bodily visual practices is social distance. By social distance, we refer to both “vertical distance” (i.e., distance in terms of power or authority) and “horizontal distance” (i.e., distance in terms of solidarity, familiarity and intimacy), following Leech (1983: 126). Social distance is an important distinction modulating the usage of politeness-related resources (see Goldsmith 2007 for review). A number of studies in social psychology (Andersen et al. 2006; Burgoon and Dunbar 2006; Dovidio and Ellyson 1982; Jorgenson 1975; Mehrabian 1972) have shown that speakers perform different bodily visual practices when interacting with a socially distant superior compared to interacting with a socially proximal friend. More recently, these approaches have been applied to the field of politeness research by L. Brown and Winter (2019) and Hübscher et al. (forthcoming), demonstrating that speakers actively recruit a wide range of bodily visual practices in order to perform deferential personas towards status superiors and enact intimate modes of interactions with friends. The current paper builds on the findings of L. Brown and Winter (2019) and Hübscher et al. (forthcoming) as well as L. Brown et al. (forthcoming) by investigating how speakers modulate their bodily visual practices according to social distance. Whereas these previous studies used televised dramas (L. Brown and Winter 2019) and an oral discourse completion task (Hübscher et al. forthcoming), or focused only on manual gesture (L. Brown et al. forthcoming), the current study adds to the existing literature by focusing on interactional data, and by examining a wider range of bodily visual practices that are not manual gestures. By investigating two languages via the same methodology, we are able to explore the extent to which languages with contrasting verbal politeness systems vary in terms of multimodal features. We look at two languages: First, Korean, a language that explicitly marks social distance through the overt use of grammaticized honorifics (L. Brown 2015; Sohn 2001). Second, Catalan, a language which contains no such grammatical system of honorifics and where the politeness system is oriented towards belonging, solidarity and common ground (Sabaté i Dalmau and Curell i Gotor 2007). Through investigating these two languages, our research speaks to questions of universality and specificity in multimodal politeness (see Section 2.1 below).

Indexing social distance through bodily visual practices in two languages

The specific research questions that we addressed were two-fold, and can be summarized as follows: First, during interactional tasks, in what ways do speakers modulate bodily visual practices according to social distance? We look at the following five categories of body movements: 1. 2. 3. 4. 5.

head nodding head shaking adaptors (i.e., touching one’s own body) haptics (i.e., touching the interlocutor’s body) gaze aversion

Second, how do these modulations compare across two languages and cultures (Korean and Catalan) with different systems of verbal politeness?

2.

Background

2.1 Universality versus specificity in multimodal politeness One fundamental question in politeness research regards the extent to which politeness is linguistically and culturally universal (see for example Leech 2014: 81–83). Are there linguistic, cultural or biological principles that underpin politeness as a universal human phenomenon, or is politeness culture-specific? Early theories of politeness led by philosophical work on the nature of language (P. Brown and Levinson 1987; Leech 1983) assumed that politeness was governed by universal mechanisms, even though these theorists based their claims mostly on evidence from English (see Ide 1989). P. Brown and Levinson (1987) saw individual rationality and “face” as the universal human attributes that motivated speakers to use polite language, with the latter defined as the speaker’s “wants” to be unimpeded (dubbed “negative face”) and to be approved of (“positive face”). Negative face in particular was identified as the driving force for speakers to use indirect language when performing sensitive speech acts. However, these universalistic claims were met with a backlash from scholars working on a variety of languages, particularly Asian languages such as Korean and Japanese that have obligatory systems of honorifics which need to be employed in all social situations regardless of concerns for face (Ide 1989; Matsumoto 1988). Some scholars (Leech 1983, 2014; Watts 1989, 1992) accepted that politeness simply worked differently in languages which routinely marked relative social position through the use of honorifics systems. While early politeness theories dealt primarily with lexical and grammatical aspects of politeness, P. Brown and Levinson (1987) also made a universal claim

133

134

Lucien Brown, Iris Hübscher, Hyunji Kim & Bodo Winter

regarding the prosodic qualities of polite speech. Specifically, they predicted that a high-pitched voice should universally signal self-humbling stances and deference via the “natural association” with the voice of a child (P. Brown and Levinson 1987: 268). Around the same time, the assumed connection between high pitch and politeness became a central claim of an influential universalist theory regarding the sound-symbolic use of pitch: Ohala’s (1994) frequency code hypothesis. According to this “biological code” (see Gussenhoven 2004: Chapter 5), a highpitched voice becomes a universal signal for politeness, deference and submission via the association of high-pitched sounds with small body size (i.e., animals with smaller body sizes produce higher-pitched noises). By adopting a high-pitched voice, speakers signal that they are acting in a submissive manner and are not in a position to threaten the interlocutor. The assumption that politeness is submissive behaviour is somewhat congruent with P. Brown and Levinson’s (1987) notion of negative politeness, which encapsulates actions designed to reduce imposition on the interlocutor’s freedom of action. It also resonates with Bax’s (2011: 260) account of the evolution of negative politeness from ritual submission displays, the counterpart to superiority displays. Subsequent studies, however, found only partial support for the politenessrelated dimension of Ohala’s frequency code. Whereas some studies showed that speakers did indeed raise their pitch when speaking politely (e.g., Caballero et al. 2018 for English; Loveday 1981 for Japanese; Orozco 2010 for Spanish), others showed no change in pitch (Idemaru et al. 2019 for Japanese, 2020 for Korean). Yet other studies found that speakers in some cultures actually lowered their pitch to index politeness (e.g., Grawunder et al. 2014 for German; Hübscher et al. 2017 for Catalan; Oh and Cui 2020 for Chinese; Sherr-Ziarko 2019 for Japanese; Winter and Grawunder 2012 for Korean). As pointed out by Winter et al. (2021), these findings used a variety of tasks and also investigated different facets of politeness, making it difficult to draw any solid conclusions. The existing evidence suggests, however, that using high pitch for politeness is not in fact a universal, and that some modes of politeness work differently, including the marking of increased social distance (see L. Brown and Prieto 2017). When interacting with a status superior, rather than using high-pitch to sound submissive or child-like, speakers may prefer to use a voice that is low-pitched, quieter, slowed down and less variable in order to sound composed and formal (Winter and Grawunder 2012). Hübscher et al. (2017) followed by Winter et al. (2021) refer to this strategy of using a “damped down” voice as “prosodic mitigation”. Crucial to the current study, Hübscher et al. (forthcoming) claim that this “damped down” mitigation strategy also extends to the domain of gesture and other bodily visual practices, referring to this phenomenon as “gestural mitigation”. Indeed, Hübscher et al. (forthcoming) found that Catalan speakers used

Indexing social distance through bodily visual practices in two languages

lower rates of facial and bodily cues when social distance was increased, and also reduced their rates of adaptors (i.e., touching their own bodies) and haptic behaviours (i.e., touching the interlocutor). Meanwhile, L. Brown and Winter (2019) showed that characters in Korean televised dramas reduced their rate of manual gestures, eyebrow movements, and touches to their own bodies or the bodies of interlocutors in scenes with superiors. They also adopted more erect but constrained body positions when speaking to superiors, and maintained gaze and direct bodily orientation throughout the interaction. The findings from these two studies also vibe well with an extensive literature in social psychology showing that speakers use more rigid, more attentive and less animated bodily movements when interacting with socially distant people (e.g., Andersen et al. 2006; Burgoon and Dunbar 2006; Burgoon and Newton 1991; Dovidio and Ellyson 1982; Mehrabian 1972). These results suggest that the application of gestural mitigation when interacting in a socially distant context may be shared across different languages. L. Brown et al. (forthcoming) conducted a cross-language comparison of how speakers of Catalan and Korean modulated their manual gestures with respect to social distance. The study used data from a cartoon retelling task, comparing how the same university student participants performed the retelling differently depending on whether their interlocutor is an unfamiliar status superior or a close friend. Results showed that Korean speakers produced fewer gestures overall when addressing the superior, but that Catalan speakers did not. However, speakers of both languages made their gestures less animated by using a smaller gesture space, decreasing the encoding of manner of movement in motion events, and reducing the use of character-viewpoint gestures. In the current paper, we use the same Catalan and Korean corpora from L. Brown et al. (forthcoming) but include extra tasks not used in that paper. We complement the finding regarding gesture in that previous study by looking at five more bodily visual practices–head nodding, head shaking, adaptors, haptics, and gaze aversion–in order to paint a more complete picture of cross-linguistic comparison of how speakers vary their body movements according to social distance.

2.2 Bodily visual practices analysed in the current study We consulted previous research to identify a number of bodily visual practices that could be analysed using the current dataset (summarized in Table 1 with predictions for socially distant context). The first and second bodily visual practices that we look at are head nodding and head shaking. Results from previous studies provide an unclear picture of how these two head movements pattern with social distance. L. Brown and Winter

135

136

Lucien Brown, Iris Hübscher, Hyunji Kim & Bodo Winter

Table 1. Items analysed with predictions Item:

Prediction:

1

head nodding

decrease in socially distant context

2

head shaking

decrease in socially distant context

3

adaptors

decrease in socially distant context

4

haptics

absent in socially distant context

5

gaze aversion

decrease in socially distant context

(2019) for Korean found that head nodding occurred with both status superiors and with status equals at similar frequencies, but did not analyse head shaking. Hübscher et al. (forthcoming) found no difference in head nods, but that head shakes occurred less frequently in the socially distant condition. Kita and Ide (2007) demonstrate that nodding possesses a politeness-related function in Japanese, namely that it works alongside backchannels to mark listener attentiveness. Although the authors note that this mode of behaviour is socially affiliative and occurs when the participants are “on friendly terms” (Kita and Ide 2007: 1250), they do not discuss any clear patterning according to social distance. We predicted that both of these behaviours would follow the general pattern of decreasing in socially distant contexts. The third bodily visual practice that we examined were adaptors, focussing on self-adaptors. These are bodily movements where the speaker touches their own body, including scratching, stroking or rubbing, as well as behaviours such as twirling hair. Previous studies on Korean (L. Brown and Winter 2019) and Catalan (Hübscher et al. forthcoming) showed that speakers touched their own bodies less frequently when interacting with superiors, and similar findings were also shown in Burgoon and Dunbar (2006). We therefore predicted that the frequency of adaptors would decrease in socially distant contexts. The fourth practice that we looked at was haptics, specifically cases where the speaker touches the body of the interlocutor. Haptics take on many physical forms including patting, stroking, holding, hugging and kissing. Studies of touch in interaction show that it takes on a range of functions including the expression of prosocial emotions (e.g., love, gratitude, compassion), the management of the bodily conduct of others, typically children (e.g., touching the back of a child to “shepherd” them towards a specific location), and various functions in diagnosis, treatment and care in medical encounters (see Cekaite and Goodwin 2021 for review). Haptic behaviours are associated with intimacy (Guerrero and Floyd 2006) and have been shown to rarely occur when interacting with superiors (L. Brown and Winter 2019). We predicted that they would only appear in the socially close condition and would be absent in the socially distant condition.

Indexing social distance through bodily visual practices in two languages

Finally, we examined gaze aversion. Previous studies in social psychology show that speakers have less freedom to avert gaze when interacting with superiors (Dovidio and Ellyson 1982; Snyder and Sutker 1977; Strongman and Champness 1968). Some studies claim that Korean may show the opposite pattern due to assumed cultural taboos against maintaining eye contact with superiors, which may be considered insolent or rude (Koo and Jeon 2007). However, the results in L. Brown and Winter (2019) showed little evidence for such claims: speakers maintained gaze more consistently with superiors, except in some specific contexts such as when being scolded or rebuked. We therefore predicted that both Korean and Catalan would show the pattern of supressing gaze aversion with the superior. There are a number of additional features that may show variation according to social distance, but which we decided not to analyse in the current paper. Notably, a number of studies show that speakers adopt direct bodily orientation and more erect postures when interacting with superiors (L. Brown and Winter 2019; Burgoon and Saine 1978; Jorgenson 1975; Mehrabian 1968). However, the methodology for the current study did not allow for systematic analysis of body position since the participants were seated on fixed chairs (see below). In addition, the side-on camera angle did not allow for the analysis of the facial cues (smiles, eyebrow movements, etc.) analysed in L. Brown and Winter (2019) and Hübscher et al. (forthcoming).

3. Methodology 3.1 Data collection The Catalan and Korean data were collected at universities in Barcelona and Seoul, respectively (Universitat Pompeu Fabra, Konkuk University). Our participants took part in dyad recordings with a partner in a soundproof recording booth. The participant and the partner sat facing each other on chairs fixed to the ground. The distance between the front edges of both chairs was 84 cm. Beaulieu (2004) showed that in a situation where participants could position their own chair to interact with an unknown interlocutor, the distance between the chair fronts ranged from 50 to 130 cm, meaning that our seating arrangement was well within this common range.

137

138

Lucien Brown, Iris Hübscher, Hyunji Kim & Bodo Winter

3.2 Procedure Each of our participants (14 for Korean, 12 for Catalan) participated in two data collection sessions: one with a friend and one with a status superior. These two sessions took place on different days within a four-day period, with the order counterbalanced. Each session involved a number of interactional tasks. First, the participants held an unstructured conversation, following the prompt of talking about a movie they had watched recently. Second, participants performed a cartoon retelling (the Tweety Bird cartoon Canary Row), a task used extensively in previous studies on narratives, motion events and gestures (e.g., Kita and Özyürek 2003; McNeill and Duncan 2000). Third, participants performed a map task (Anderson et al. 1991): an activity that involves explaining a route between two points on a map to a partner who has a similar (but not identical) map, and who is tasked with tracing the exact route described to them.1 This activity has been used previously in studies of disfluency (Lickley and Bard 1998), gaze (Boyle et al. 1994), turn taking (Forsyth et al. 2008), and other areas. Finally, participants for both languages performed a roleplay originally used in L. Brown (2011) in which they had to apologize for breaking their partner’s camera (friend condition) or losing the partner’s rare book (superior condition). Including a role-playing task also provided consistency with the literature on the acoustic characteristics of politeness, which has used such tasks extensively (Hübscher et al. 2017; Winter and Grawunder 2012). Roleplays have been widely used in pragmatics research more generally (FélixBrasdefer 2018). The interactions were video recorded using a Canon VIXIA HFM 41 camcorder for Korean, and a Panasonic HPX-171 for Catalan. The average lengths of the recordings were about 12 minutes (friend) and 10 minutes (superior) for Korean, and 17 minutes (friend) and 16 minutes (superior) for Catalan.

3.3 Participants 14 Korean native-speaking participants (7 female and 7 male) and 15 Catalan native-speaking participants (7 female and 8 male) took part as main participants. They were all undergraduate students, aged 22 (range 19–27) on average for Korean and 19 for Catalan (18–24). All Korean participants were from the Seoul/ Gyeonggi area. All Catalan participants were Catalan-dominant (71%) bilingual

1. Only the Catalan data for this task is analysed in the current paper. We checked to see whether excluding the data from this task would change the results. We can confirm that excluding the map task data would not have made any substantive differences to the findings reported here.

Indexing social distance through bodily visual practices in two languages

speakers of Catalan and Spanish living in and around Barcelona. Participants are referred to with a participant number preceded by “C” for Catalan and “K” for Korean. As noted above, participants completed the same tasks with a friend and with a status superior. Participants were asked to bring a same-gender friend with them (although two male Korean speakers brought female friends). The status superiors were confederates recruited by the researchers: a male 60-year-old professor of English literature originating from North Gyeongsang Province for Korean, and a 64-year-old Catalan-dominant bilingual pensioner living in the neighbourhood of the university for Catalan. Neither of the confederates were known to the participants or affiliated with the research. All participants were paid a fee pro-rata to the number of recordings in which they took part and consented to images from the recordings being used in academic publications.

3.4 Coding We used ELAN Version 6.0 (ELAN, 2020) to annotate and count all occurrences of head nodding, head shaking, adaptors, haptics and gaze aversion. The annotation focused solely on the main participant; we did not consider data from the partner for the purposes of this paper.

3.5 Statistical analysis Data analysis was conducted with R version 4.1.1 (R Core Team 2019). The tidyverse package version 1.3.1 was used for data processing and visualization (Wickham et al. 2019). The brms package version 2.16.2 was used for computing Bayesian mixed models (Bürkner 2018). The tidybayes package version 3.0.1 was used to help with visualizing posterior distributions (Kay 2021). All data and analysis code can be found in the following publicly accessible Open Science Framework repository: https://osf.io/nr2yc/ For statistical analysis, we computed mixed negative binomial regressions. This type of generalized linear mixed effects model is appropriate for the analysis of discrete count data in the presence of possible overdispersion (= excess variance) (Winter and Bürkner 2021). Counts were regressed onto the fixed effects factors condition (friend versus superior), language (Catalan versus Korean) and the condition:language interaction. We effect-coded the condition and language factor to aid the interpretation of main effects in the presence of interactions: friend = −0.5, superior = +−0.5; Catalan = −0.5, Korean = +0.5. In line with the fact

139

140

Lucien Brown, Iris Hübscher, Hyunji Kim & Bodo Winter

that negative binomial models use the log link function, all coefficients reported in this paper are log coefficients. We included random effects for participants and task, including by-condition random slopes for both random effects to account for the possibility that participants may respond differently to the condition manipulation, or that the condition manipulation may have differential effects for different tasks. We used the default priors from brms except for weakly informative priors that were set onto the beta coefficients (Lemoine 2019; McElreath 2020), specifically Normal(0, 0.5). This makes estimates more conservative and prevents overfitting, which is especially important when dealing with small sample sizes such as the present one. All models were estimated using MCMC with 6,000 iterations (4,000 warm-up samples discarded; all Rhat = 1.0). Posterior predictive checks (ECDF-overlay) suggested that the model could have plausibly generated the data, which also suggests that a negative binomial model was the appropriate choice. Throughout the paper, we report Cohen’s d as effect size estimate. To compute this, we used the effsize package version 0.8.1 (Torchiano 2019). In line with the output of this package, we report frequentist 95% confidence intervals for the effect size estimate. For the generalized linear mixed effects model, we instead report Bayesian 95% credible intervals. These intervals can be interpreted as indicating a 95% probability that the respective parameter is captured by the interval (Morey et al. 2016). For readers not familiar with Bayesian statistics who may be more familiar with null hypothesis significance testing, we report the posterior probability of the effect having the opposite sign. These posterior probabilities can be interpreted in a continuous fashion, with more or less evidence in support of a coefficient being of a specified sign.

4. Results 4.1 Head nodding In contrast to our prediction that head nodding would decrease in the socially distant condition, our analysis shows that both Korean and Catalan speakers actually produced more head nods when speaking with the superior, as opposed to the friend (Table 2). In terms of rates (head nods per second), all Korean speakers produced more head nods when speaking to the superior (100%), as can also be seen in Figure 2, left. For Catalan, 11 out of 15 speakers produced more head nods when speaking to the superior (73%). The effect size computed on rates was estimated to be small for Catalan (d = 0.45, 95% CI: [−0.30, 1.21]) and large for Korean (d = 1.82, 95% CI: [0.89, 2.75]). Notice that only for Korean does the 95% confi-

Indexing social distance through bodily visual practices in two languages

dence interval of the effect size estimate exclude zero, indicating that the positive effect (more head nods when speaking to the superior) is also comparatively more reliable than in Korean. As shown in Figure 1, right, the posterior distribution of the condition coefficient was largely positive, with a mean estimate of +0.39 (SE = 0.20) and a 95% credible interval clearly skewed towards positive values: [−0.10, +0.74]. The posterior probability of this effect being of opposite sign was low, p(β < 0) = 0.04. The language effect was negative (lower average rates for Korean; log coefficient: −0.63, SE = 0.11), with the 95% credible interval firmly excluding zero, [−0.86, −0.41], and a very low probability of being of opposite sign, p(β > 0) = 0.00 (not a single posterior sample of opposite sign). Finally, there was strong evidence for a condition:language interaction, with a bigger and more consistent condition difference for Korean as opposed to Catalan speakers (log coefficient: +0.45, SE = 0.15). The 95% credible interval of this interaction effect does not overlap with zero, [+0.15, +0.75], and has a very low probability of being of opposite sign, p(β > 0) = 0.00. Table 2. Aggregate counts and percentages of head nods per condition per language (not controlled for durations) Language

Condition

Total

Percentage

Catalan Catalan Korean Korean

friend superior friend superior

1409 1786  436  667

34% 43% 22% 33%

Figure 1. Left: Rates of head nods per second for the friend (orange) and superior (blue) condition, with each line representing a separate participant; downwards-going trends are shaded dark, upwards-going trends are shaded light grey; Right: Posterior distributions of the condition, language, and condition:language interaction coefficients (logged) from the corresponding generalized linear mixed effects model

141

142

Lucien Brown, Iris Hübscher, Hyunji Kim & Bodo Winter

Head nods were typically produced when participants expressed alignment or active listenership, and were often synchronized with verbal backchannels. In the examples below (Figures 2 and 3), Catalan participant C9 produces repeated head nods when producing backchannels to the superior in the unstructured conversation (Figure 2b), whereas in a similar situation with the friend backchannels occur with no head nods (Figure 2a). In the Korean example in Figure 3, participant K10 nodded five times in quick succession while producing three backchannels (a, yey, yey) after the superior told him that the most recent movie he had seen was Sully (3b). In contrast, when the friend of the same participant told him that he had recently seen the film Now is Good, K10 produces a single filler and does not nod at all (3a). In this way, participants incorporated frequent and repetitive nodding behaviours synchronized with backchannels when interacting with the superior. Participant C9: Unstructured conversation (a) friend

(b) superior

Friend:

Si la veus no li facis gaire cas ‘If you see it don’t pay too much attention to it’

Superior:

Van venir els Queens ‘The Queens came’

C9:

Ah val Ah OK

C9:

Ah Ah

Figure 2. Nodding with the superior (Catalan)

Indexing social distance through bodily visual practices in two languages

Participant K10: Unstructured conversation (a) friend

Friend:

K10:

(b) superior

“나우 이즈 굿” 봤어 nawu icu kwus pwasse ‘I watched Now is Good’

Superior: 또 사람들이 잘 안 보는 “설리 허드슨 강”의 기

어 e ‘A h’

K10:

적 tto salamtul-i cal an ponun Selli hetusunkang-uy kicek ‘And Sully which not many people watch’ 아, 예예 a, yey, yey ‘Ah, I see’

Figure 3. Nodding with the superior (Korean)

4.2 Head shaking Table 3 shows the counts and percentages of head shakes, revealing no condition difference for Catalan, but a small difference for Korean. While the overall rates of head shakes per second are extremely low, for Korean they were very consistent: 12 out of 14 speakers (86%) produced lower rates of head shakes in the socially distant condition (Figure 4, left). This picture was markedly different for Catalan, where only 8 out of 15 speakers (53%) produced lower rates of head shakes in the socially distant condition. For Catalan, the effect size was negligible (d = −0.09, 95% confidence interval: [+0.84, −0.66]). For Korean, the effect size was large (d = −0.93, 95% confidence interval: [−0.12, −1.75]). The generalized linear mixed effects model revealed, again, a very consistent language difference, with Catalan speakers producing many more head shakes (log coefficient of Catalan versus Korean: −2.13, SE = 0.19). The 95% credible interval of the language coefficient does not overlap with zero, [−2.50, −1.72]. The posterior probability of this language difference being of opposite sign was very low, p(β > 0) = 0.00. In terms of condition difference, there was an overall tendency for the socially distant condition to have lower rates of head shakes (−0.28, SE = 0.20), but the 95% credible interval of this coefficient overlapped with zero [−0.65, +0.14]. The posterior probability of the effect being of opposite sign was relatively low, p(β > 0) = 0.07, but clearly not enough to confidently rule out an effect in the opposite direction. Most importantly, there was strong evidence for a condition:language interaction (log coefficient: −0.66, SE = 0.28), the 95% credible interval of

143

144

Lucien Brown, Iris Hübscher, Hyunji Kim & Bodo Winter

which excluded zero, [−1.20, −0.12], with a very low posterior probability of being of opposite sign, p(β > 0) = 0.01. This interaction is in line with the picture seen in Figure 4, with the condition difference being much more pronounced and consistent for Korean, despite overall lower rates of head shakes. Table 3. Aggregate counts and percentages of head shakes per condition per language (not controlled for durations) Language

Condition

Total

Percentage

Catalan

friend

718

17%

Catalan

superior

731

17%

Korean

friend

 56

 3%

Korean

superior

 16

 1%

Figure 4. Left: Rates of head shakes per second for the friend (orange) and superior (blue) condition, with each line representing a separate participant; downwards-going trends are shaded dark, upwards-going trends are shaded light grey; Right: Posterior distributions of the condition, language, and condition:language interaction coefficients (logged) from the corresponding generalized linear mixed effects model

In the following Example (Figure 5), Korean participant K5 uses head shakes when disagreeing with the friend’s playful assertion in the roleplay that repairing the camera is going to cost as much as 1,500,000 KRW (around 1,025 USD, as of December 2022). These kinds of disagreements about the details of resolving the role play situation never occurred in the socially distant condition, particularly since in Korean culture it is relatively taboo to directly contradict a social superior (L. Brown 2011: 80). Head shakes did not only co-occur with explicit disaffiliation, however. In the next Example (Figure 6), participant K6 produces a succession of head shakes when stating that the movie Fantastic Beasts and Where to Find Them has no

Indexing social distance through bodily visual practices in two languages

Participant K10: Roleplay friend

Friend:

내가 찾아봤는데 nay-ka chacapwassnuntey A/S가 150만 원 정도 한다더라 A/S-ka 150-man wen cengto hantatela ‘I looked into it and they said that the repairs are going to cost 1,500,000 won’

P5:

아닌 것 같은데 anin kes kathuntey ‘I don’t think so’

Figure 5. Head shaking with the friend (Korean) Participant K10: Unstructured conversation (a) friend

(b) superior

내용은 없는데 볼거리는 많아. nayyong-un epsnuntey polkeli-nun manha. ‘(The movie) doesn’t have any story, but it has lots of visual elements to enjoy’

영화 자체만으로만 평가하자면 좀 약간 딱 Yenghwa cachey-manuloman phyengka hacamyen com yakkan ttak

내용이 그닥 많지는 않아가지고 nayyong-i kutak manhcinun anhakaciko ‘If assessing the movie itself, it’s a little bit- because it doesn’t really have that much of a story’

Figure 6. Head shaking with the friend (Korean)

story, with the head shakes appearing to add emphasis to her claim. In contrast, when making the same statement with the superior, she cants her head to the side, which is known to be a marker of uncertainty (McClave 2000). With the superior, the verbal content of her claim about the movie’s lack of a story line is also

145

146

Lucien Brown, Iris Hübscher, Hyunji Kim & Bodo Winter

mitigated by her use of the adverb kutak ‘(not) really’ and the predicate manhcinun anhakaciko ‘doesn’t have much’ rather than the stronger epsnuntey ‘doesn’t have any’ with the friend. In sum, the headshaking is used here to strengthen the speaker’s epistemic commitment to negative propositional content with the friend, whereas the same speaker uses a head cant to reduce epistemic commitment with the superior.

4.3 Adaptors Table 4 shows the counts for adaptors, revealing slightly more adaptors for the friend as opposed to the socially distant condition, especially in the case of Korean. Catalan speakers did not consistently modulate the rate of adaptors per second (see Figure 7, left), with 8 out of 15 speakers producing lower rates in the socially distant condition (53%). For Korean, nearly everybody produced lower rates of adaptors in the socially distant condition: 11 out of 14 speakers (79%). The effect size was negligible for Catalan (d = −0.09, 95% confidence interval: [−0.84, +0.66]), and medium for Korean (d = −0.78, 95% confidence interval: [−1.58, +0.03]). For Catalan, the 95% confidence interval of the effect size was firmly centred on zero, for Korean it barely included zero. Table 4. Aggregate counts and percentages of adaptors per condition per language (not controlled for durations) Language

Condition

Catalan Catalan Korean Korean

friend superior friend superior

Total

Percentage

839 779 362 219

20% 19% 18% 11%

Consistent with the picture suggested by Figure 7, left, the generalized linear mixed effects model revealed no across-the-board condition difference (log coefficient: −0.16, SE = 0.21). As can be seen in Figure 7, the 95% credible interval of this condition difference firmly overlapped with zero: [−0.58, +0.31]; posterior probability of being of opposite sign, p(β > 0) = 0.18. There also was no strong evidence for a condition:language interaction (log coefficient: −0.21, SE = 0.21), the 95% credible interval also overlapped with zero, [−0.63, +0.22]; posterior probability of opposite sign: p(β > 0) = 0.16. There was a very weak indication for a language difference, with Korean speakers producing slightly lower rates of adaptors than Catalan speakers (−0.28, SE = 0.20), but here, too, the 95% credible interval overlapped with zero, [−0.66, +0.11]. The posterior probability of this effect being

Indexing social distance through bodily visual practices in two languages

of opposite sign is relatively low, p(β > 0) = 0.08, but clearly not enough to rule out an effect of opposite direction.

Figure 7. Left: Rates of adaptors per second for the friend (orange) and superior (blue) condition, with each line representing a separate participant; downwards-going trends are shaded dark, upwards-going trends are shaded light grey; Right: Posterior distributions of the condition, language, and condition:language interaction coefficients (logged) from the corresponding generalized linear mixed effects model

In the following example from the Tweety cartoon retelling (Figure 8), participant K9 encounters trouble with recalling and/or communicating the detail of how Sylvester was watching Tweety through binoculars. With the friend, he produces a bilabial click and touches his nose with his right index finger when attempting to recall this detail. Self-touching is known to be associated with cognitive processes such as word searches (Harrigan 1985). In contrast, when uttering the identical content with the superior and pausing to repair his utterance, no adaptors are produced.

4.4 Haptics Only five instances of the main participants initiating haptic behaviours occurred in the data – three for Korean produced by two participants, and two for Catalan produced by two participants. All occurrences were in the socially close condition, and all appeared in the roleplay task. The number of occurrences was too small to warrant statistical analysis. However, the pattern whereby these appeared exclusively in intimate interactions is consistent with previous studies (Andersen et al. 2006; L. Brown and Winter 2019). The types of haptic behaviours differed between the two languages. All three tokens from Korean were from male participants touching the friend’s knees, which is a fairly common form of touching in intimate male interactions in

147

148

Lucien Brown, Iris Hübscher, Hyunji Kim & Bodo Winter

Participant K9: Tweety task (a) friend

고양이가, 쩝, 망원경으로 koyangi-ka, ccep, mangwenkyengulo ‘The cat, (bilabial click) through a telescope’

(b) superior

이제 고양이가 창문을 통해서, 망원경을 통해서, icey koyangi-ka changmwun-ul thonghayse, mangwenkyeng-ul thonghayse ‘The cat [watches the bird] through the window, through binoculars’

Figure 8. Adaptors with the friend (Korean)

Korean. As shown in Figure 9, these knee touches were synchronized either with the apology head act mian(hata) ‘sorry’ (a and c), or when introducing the offense (b). It therefore appears that these knee touches can take on mitigating functions in intimate interactions. Notably, the knee touch in example (b) occurs just after a hissing noise ssup, which is known to be a hesitation device that takes on politeness functions (Winter and Grawunder 2012; L. Brown et al. 2022). (a) K9

(b) K9

(c) K10

미안하다 mianhata ‘sorry’

씁, 그리고 내가 솔직하게 하나

미안 mian ‘sorry’

더 말할 게 있어. ssup, kuliko nay-ka solcikhakey hana te malhal key isse ‘(hiss), and to be honest I have one more thing that I need to tell you’

Figure 9. Knee touches with friend (Korean)

Both of the examples of the main participant performing haptics in the Catalan data were handshakes occurring at the end of the roleplay when the two participants agreed the next course of action. The interaction involving participant

Indexing social distance through bodily visual practices in two languages

C5 featured one of these occurrences (Figure 10d) where C5 and the friend agree to share the cost of the broken camera. This same interaction also featured three instances where the friend touched the main participant (Figure 10 a–c) while the latter explained how she dropped the camera while being attacked by a cat. The friend touches the main participant’s hands (10a), face (10b) and arm (10c) as she gathers information about the cat attack and enacts the possible ways that it unfolded. All examples here are accompanied by smiles, suggesting that they are performing a relational function. Needless to say, these kinds of physical touches and animated co-constructed enactments would be unlikely to occur in an interaction with a superior. Participant C5: Roleplay (a)

C5:

En plan “aquí” ‘Like “here”’

Friend:

Aquí i el gat va fer així i va tirar la cámera al terra ‘Here and the cat went like this and threw the camera on the floor’

(b)

Friend: Et va fer així a la cara? ‘Did he do that to your face?’

(c)

(d)

C5:

A la cara tampoc, al braç ‘Not at the face (either), on the arm’

C5:

Friend:

Al braç ‘On the arm’

Friend: Vale ‘OK’

Figure 10. Haptic behaviours with friend (Catalan)

Vale, em sembla correcte ‘OK, I think that’s correct’

149

150

Lucien Brown, Iris Hübscher, Hyunji Kim & Bodo Winter

4.5 Gaze aversion There was a slight tendency for there to be on average more gaze aversions in the socially close than in the socially distant condition (Table 5). This picture, however, was not consistent across speakers, as clearly shown by Figure 11, left. In terms of rates (gaze aversions per second), 9 out of 15 Catalan speakers (60%) produced more gaze aversions in the socially close condition, whereas only 4 out of 14 Korean speakers (29%) followed this pattern. Korean speakers thus actually averted their gaze slightly more frequently in the socially distant condition. Effect sizes, however, revealed that the difference was not consistent for either language. There was a small effect for Catalan (d = −0.47, 95% CI: [−1.23, +0.29]) and a negligible effect for Korean (d = +0.13, 95% CI: [−0.64, +0.92]). For both effect size estimates, the 95% confidence interval firmly included zero. A similar picture is revealed by the corresponding generalized linear mixed effects model (see Figure 11, right), which reveals no indication of a condition difference, with a really small coefficient (log estimate: −0.04, SE = 0.22) whose 95% interval was centred at zero, [−0.48, + 0.42]; posterior probability of being of opposite sign, p(β > 0) = 0.40. There equally was no indication whatsoever for a condition:language interaction (log estimate: +0.03, SE = 0.13). The posterior distribution of this coefficient was also centred at zero; 95% credible interval: [−0.22, +0.29], posterior probability of being of opposite sign, p(β < 0) = 0.39. There was, however, stronger evidence for a language effect (log estimate: −0.33, SE = 0.09). As shown in Figure 11, right, the 95% credible interval of the language coefficient does not include zero, [−0.50, −0.16]. The posterior probability of an effect of opposite sign was very low, p(β > 0) = 0.00. This shows that Catalan speakers produced overall a much higher rate of gaze aversions than Korean speakers. However, neither language modulated this rate consistently as a function of social context. Table 5. Aggregate counts and percentages of gaze aversions per condition per language (not controlled for durations) Language

Condition

Total

Percentage

Catalan

friend

2904

69%

Catalan

superior

2403

57%

Korean

friend

938

47%

Korean

superior

849

42%

Indexing social distance through bodily visual practices in two languages

Figure 11. Left: Rates of gaze aversions per second for the friend (orange) and superior (blue) condition, with each line representing a separate participant; downwards-going trends are shaded dark, upwards-going trends are shaded light grey; Right: Posterior distributions of the condition, language, and condition:language interaction coefficients (logged) from the corresponding generalized linear mixed effects model

Interestingly, it was the roleplay task where participants from both languages increased consistency in modulating gaze according to the condition. We interpret this as showing that maintaining gaze when interacting with a status superior is particularly important in rapport-threatening situations such as, in this case, when making an apology. The fact that Korean participants followed their Catalan counterparts in reducing gaze aversion when apologizing to the status superior goes against the frequently cited folk belief that speakers of Korean and other East Asian languages need to avoid direct eye contact when interacting with status superiors (Koo and Jeon 2007). However, closer analysis of the videos showed that Korean speakers did tend to organize their gaze aversions so that they occurred at particularly sensitive points of the role play. In Figure 12 below, participant K1 bowed her head and therefore broke eye contact when uttering the illocutionary force indicating device (IFID) coysong-hapnita ‘sorry’ towards the superior. With the friend, she uses a different IFID, mian-hata, which tends to be limited to interactions with intimates and status equals/subordinates (Hatfield and Hahn 2011), and maintains her gaze on the friend while rubbing her hands together. In contrast, in the Catalan Example (Figure 13), participant C16 maintains eye contact while uttering the formulaic ho sento ‘I’m sorry’ to the superior, whereas her production of the somewhat more emotionally charged apology IFID em sap greu ‘I’m sorry’ to the friend features gaze aversion at the start.

151

152

Lucien Brown, Iris Hübscher, Hyunji Kim & Bodo Winter

K1: Tweety task (a) friend

(b) superior

교수님이 말씀만 하고 빨리 안 끝내 주셔서… 진, 진짜

죄송합니다 coysonghapnita

미안해 kyoswunim-i malssum-man hako ppalli an kkuthnay cwusyese … cin, cincca mianhay ‘My professor did not let me go and kept talking…’

진작 연락을 좀 드렸어야 하는 건데 … cincak yenlak-ul com tulyesseya hanun kentey… ‘I am sorry. I should have reached out to you in advance…’

Figure 12. Gaze aversion with the superior C16: Tweety task (a) friend

(b) superior

Em sap greu ‘I’m sorry’

Ho sento ‘I’m sorry’

Figure 13. Gaze aversion with friend (Catalan)

5.

Discussion

The results showed that speakers modulated their use of body movements according to social distance in most of the categories that we analysed (see Table 6). The most consistent difference was observed for head nods, of which both Catalan and Korean speakers produced more in the socially distant situation, although the Korean speakers were more consistent. Haptics appeared at low frequencies, but only ever in the socially close situation for both languages. Only Korean speakers consistently reduced their rate of head shaking with the superior, and also showed

Indexing social distance through bodily visual practices in two languages

a tendency to reduce their use of adaptors. Gaze aversion showed a small effect only for Catalan. Table 6. Summary of results in relation to social distance (the long dash ‘–’ indicates lack of evidence for a preference) Korean

Catalan

Head nodding

more

more

Head shaking

fewer

–

Adaptors

fewer (tendency)

–

Haptics

fewer (low frequency)

fewer (low frequency)

Gaze aversion

–

fewer (tendency)

The general pattern is for speakers to reduce the frequency of body movements practices in the socially distant condition, except for head nods, which show an increase (see below for discussion of head nods). These findings complement our previous observations that speakers of Korean and Catalan constrain their use of manual gesture when interacting with a status superior (L. Brown et al. forthcoming). Moreover, it is consistent with the strategy of “gestural mitigation” proposed by Hübscher et al. (forthcoming). When interacting with a status superior, speakers adopt an overall strategy of moving less, and therefore they appear less animated and less expressive. These patterns of “gestural mitigation” are also congruent with changes in other modalities that accompany shifts in social distance. When interacting with status superiors, speakers also use a more monotonous and less expressive voice (Hübscher et al. 2017; Idemaru et al. 2019; Winter and Grawunder 2012), and curtail expressiveness on the verbal dimension by using fewer ideophones (Kim et al. 2021). Taken together, these results show that indexing social distance towards a status superior is not so much about making yourself appear small and submissive (Ohala 1994) or making yourself sound like a child (P. Brown and Levinson 1987), but rather adopting a formal and solemn persona where the expression of emotional stances is subdued. Although we only collected data from two languages, there are reasons to think that these patterns of “gestural mitigation” (Hübscher et al. forthcoming) may be shared across a number of languages and cultures. The results reported here are from two unrelated languages from distant cultures, and are largely consistent with the extant literature in social psychology mainly from English (e.g., Burgoon and Dunbar 2006; Burgoon and Saine 1978; Dovidio and Ellyson 1982; Mehrabian 1968, 1972; Snyder and Sutker 1977) and a smaller number of stud-

153

154

Lucien Brown, Iris Hübscher, Hyunji Kim & Bodo Winter

ies in politeness research both for spoken languages (L. Brown and Winter 2019; Hübscher et al. forthcoming) and signed languages (George 2011; Mapson 2014). Our study therefore contributes to a growing body of evidence that speakers adopt less frequent, less animated and less expressive bodily conduct for politenessrelated functions, specifically for social distance with status superiors. Our results also speak to the specific reasons for why particular behaviours are suppressed when interacting with a socially distant interlocutor. The qualitative analysis of examples from the data shows that head shaking is used either when disaffiliating with the interlocutor, or to strengthen epistemic commitment to negative propositional content. These are stances that are likely to be less appropriate in interactions with an unknown superior. As for the suppression of adaptors, this also can be interpreted as coming from the need to appear more formal and tensed in front of a status superior. In addition, since adaptors are associated with the expression of affect (e.g., Harrigan 1985), their lower frequency in interactions with superiors also reflects the more emotionally-controlled persona that is required when interacting with status superiors, whereas affect can be freely expressed with a friend. Finally, although the relative formality of the data collection context (i.e., structured tasks recorded in the lab) resulted in very low levels of haptic behaviour overall, their total absence in the superior situation confirms that bodily touching is associated with a degree of intimacy and affective expression that is largely incompatible with interactions with status superiors (see Andersen et al. 2006; L. Brown and Winter 2019; Guerrero and Floyd 2006). The increase of head nods in the socially distant condition is one pattern that goes contrary to this general pattern. Our analysis showed that speakers used repeated head nods paired with backchannels to display attentiveness, active listenership and compliance when interacting with superiors, thus showing the same patterns that Kita and Ide (2007) previously identified for Japanese. Head nods are known to be marker of prosocial behaviour (Feygina and Henry 2015), which show alignment to the interlocutor in a way that is also relatively unobtrusive in that it allows them to continue their turn. The increases in head nods in the socially distant condition contrasts directly with the tendency to decrease the frequency of head shakes, which are known to be markers of rejection, uncertainty, doubt and various forms of disalignment (Rice and Hinnell 2015). The fact that increased head nods displayed the clearest difference both for Korean and Catalan shows that the politeness-related functions of head nods are not limited to Japanese (Kita and Ide 2007), and extend even into European languages such as Catalan where different politeness ideologies are at play. Although our results show the general patterns discussed above, they also point to important cultural differences. Notably, the Korean speakers were more consistent than the Catalan speakers in how they modulated body movements

Indexing social distance through bodily visual practices in two languages

according to relative social distance, even though it was the Catalan speakers who produced more of these behaviours overall. The finding that Korean speakers are more sensitive to social distance in their bodily conduct is consistent with findings on verbal politeness reported for Korean (Byon 2004, 2005) and other languages with developed systems of honorifics such as Japanese (Fukushima 1996; Takahashi and Beebe 1993). In particular, the fact that only the Korean speakers constrained their use of head shakes appears to be linked to the fact that expressing disagreement and causing discomfort to social superiors is relatively taboo in Korean culture (L. Brown 2011), whereas this is not necessarily the case for Catalan. However, the results show that the Catalan participants were also certainly sensitive to social distance, albeit less so than the Korean participants. This suggests that social indexing is still an important politeness-related action even in languages that do not have developed honorific systems such as Catalan. The results therefore bring into question any attempts to partition off honorific languages from any over-arching theory of politeness. The finding that Korean speakers did not avert their gaze more often in the socially distant condition also has important implications for cross-cultural understanding of politeness. It is often reported, typically anecdotally, that looking a status superior in the eye is impolite in Korean culture (Koo and Jeon 2007) and elsewhere in East Asia (Argyle and Cook 1976; Uono and Hietanen 2015). However, the current study found no strong support for an East-West distinction in the overall modulation of gaze according to social distance. This follows on L. Brown and Winter (2019), who actually found that Korean speakers maintained gaze more faithfully with a superior, thus following a cross-cultural pattern whereby social superiors attract more direct gaze (Dovidio and Ellyson 1982; Snyder and Sutker 1977; Strongman and Champness 1968). We believe that gaze aversion in interactions with superiors in Korean and other East Asian cultures may be reserved for particularly rapport-sensitive stages of interaction such as when producing an apology IFID (see Figure 12) or when being scolded or rebuked (see L. Brown and Winter 2019), rather than being the default mode of conduct.

6.

Conclusion

This paper has demonstrated that speakers change their use of a variety of body movements depending on the relative social distance of the interlocutor. We found that speakers of Korean–a language in which social distance is rigorously indexed at the verbal level–modulate their bodily behaviour in line with social distance in a more consistent way, but that speakers of Catalan are still

155

156

Lucien Brown, Iris Hübscher, Hyunji Kim & Bodo Winter

somewhat sensitive to the distinction. Together with previous studies that have looked at acoustic (L. Brown et al. 2014; Hübscher et al. 2017; Winter et al. 2021; Winter and Grawunder 2012), gestural (L. Brown et al. forthcoming; L. Brown and Winter 2019; Hübscher et al. forthcoming) and morpho-lexical (Kim et al. 2021) dimensions of these same languages, we see an overall strategy whereby speakers display mitigation across various modalities in order to create a persona that is less animated and expressive but more attentive and compliant when interacting with a socially distant other. This study therefore contributes towards a multimodal picture of politeness and the marking of social distance, which shows a mixture between universality and specificity across different languages and cultures.

Funding This work was supported by the Core University Program for Korean Studies through the Ministry of Education of the Republic of Korea and Korean Studies Promotion Service of the Academy of Korean Studies (AKS-2017-OLU-2250002). The Catalan data collection costs and work were covered by the postdoctoral fellowship awarded to the second author by the URPP Language and Space. Bodo Winter was supported by the UKRI Future Leaders Fellowship MR/ T040505/1.

Acknowledgments We are thankful to Grace Oh at Konkuk University for her immense assistance with the Korean data collection, and to Soyeon Kim, who worked as an RA on the Korean data. We are grateful to Kaori Idemaru and Eric Pederson for their input at different stages of this project.

References Andersen, Peter, Laura Guerrero, and Susanne Jones. 2006. “Nonverbal Behavior in Intimate Interactions and Intimate Relationships.” In The SAGE Handbook of Nonverbal Communication, ed. by Valerie Manusov, and Miles Patterson, 259–278. Thousand Oaks, CA: Sage. https://doi.org/10.4135/9781412976152.n14 Anderson, Anne, Miles Bader, Ellen Bard, Elizabeth Boyle, Gwyneth Doherty, Simon Garrod, Stephen Isard, Jacqueline Kowtko, Jan McAllister, and Jim Miller. 1991. “The HCRC map task corpus.” Language and Speech 34 (4): 351–366. https://doi.org/10.1177/002383099103400404

Argyle, Michael, and Mark Cook. 1976. Gaze and Mutual Gaze. Cambridge: Cambridge University Press.

Indexing social distance through bodily visual practices in two languages

Bax, Marcel. 2011. “An Evolutionary Take on (Im)politeness: Three Broad Developments in the Marking out of Socio-proxemic Space.” Journal of Historical Pragmatics 12 (1–2): 255–282. https://doi.org/10.1075/jhp.12.1-2.11bax Beaulieu, Catherine. 2004. “Intercultural Study of Personal Space: A Case Study.” Journal of Applied Social Psychology 34 (4): 794–805. https://doi.org/10.1111/j.1559-1816.2004.tb02571.x Boyle, Elizabeth, Anne Anderson, and Alison Newlands. 1994. “The Effects of Visibility on Dialogue and Performance in a Cooperative Problem Solving Task.” Language and Speech 37 (1): 1–20. https://doi.org/10.1177/002383099403700101 Brown, Lucien. 2011. Korean Honorifics and Politeness in Second Language Learning. Amsterdam: John Benjamins. https://doi.org/10.1075/pbns.206 Brown, Lucien. 2015. “Honorifics and Politeness.” In The Handbook of Korean Linguistics, ed. by Lucien Brown, and Jaehoon Yeon, 303–319. Malden, NJ: Wiley Blackwell. https://doi.org/10.1002/9781118371008.ch17

Brown, Lucien, Hyunji Kim, Iris Hübscher and Bodo Winter. Forthcoming. “Gestures are Modulated by Social Context: A Study of Multimodal Politeness in two Cultures.” Gesture. Brown, Lucien, Hyunji Kim, and Bodo Winter. 2022. “Is it polite to hiss?: Nonverbal Sound Objects as Markers of (Im)politeness in Korean.” Frontiers in Communication 7: 854066. https://doi.org/10.3389/fcomm.2022.854066

Brown, Lucien, and Pilar Prieto. 2017. “(Im)politeness: Prosody and Gesture.” In The Palgrave Handbook of Linguistic (Im)politeness, ed. by Jonathan Culpeper, Michael Haugh, and Dániel Z. Kádár, 357–379. New York: Springer. https://doi.org/10.1057/978-1-137-37508-7_14 Brown, Lucien, and Bodo Winter. 2019. “Multimodal Indexicality in Korean: “Doing Deference” and “Performing Intimacy” through Nonverbal Behavior.” Journal of Politeness Research 15 (1): 25–54. https://doi.org/10.1515/pr-2016-0042 Brown, Lucien, Bodo Winter, Kaori Idemaru, and Sven Grawunder. 2014. “Phonetics and Politeness: Perceiving Korean Honorific and Non-honorific Speech Through Phonetic Cues.” Journal of Pragmatics 66: 45–60. https://doi.org/10.1016/j.pragma.2014.02.011 Brown, Penelope, and Stephen C. Levinson. 1987. Politeness: Some Universals in Language Usage. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511813085 Burgoon, Judee, and Norah Dunbar. 2006. “Nonverbal Expressions of Dominance and Power in Human Relationships.” In The SAGE Handbook of Nonverbal Communication, ed. by Valerie Manusov, and Miles Patterson, 279–297. Thousand Oaks, CA: Sage. https://doi.org/10.4135/9781412976152.n15

Burgoon, Judee, and Deborah Newton. 1991. “Applying a Social Meaning Model to Relational Message Interpretations of Conversational Involvement: Comparing Observer and Participant Perspectives.” Southern Journal of Communication 56 (2): 96–113. https://doi.org/10.1080/10417949109372822

Burgoon, Judee, and Thomas Saine. 1978. The Unspoken Dialogue: An Introduction to Nonverbal Communication. Boston, MA: Houghton Mifflin School. Bürkner, Paul-Christian. 2018. “Advanced Bayesian Multilevel Modeling with the R Package brms.” The R Journal 10 (1): 395–411. https://doi.org/10.32614/RJ-2018-017 Byon, Andrew. 2004. “Sociopragmatic Analysis of Korean Requests: Pedagogical Settings.” Journal of Pragmatics 36 (9): 1673–1704. https://doi.org/10.1016/j.pragma.2004.05.003 Byon, Andrew. 2005. “Apologizing in Korean: Cross-cultural Analysis in Classroom Settings.” Korean Studies 29: 137–166. https://doi.org/10.1353/ks.2006.0003

157

158

Lucien Brown, Iris Hübscher, Hyunji Kim & Bodo Winter

Caballero, Jonathan, Nikos Vergis, Xiaoming Jiang, and Marc Pell. 2018. “The Sound of Im/politeness.” Speech Communication 102: 39–53. https://doi.org/10.1016/j.specom.2018.06.004

Cekaite, Asta, and Marjorie Goodwin. 2021. “Touch and Social Interaction.” Annual Review of Anthropology 50 (1): 203–218. https://doi.org/10.1146/annurev-anthro-101819-110402 Dovidio, John, and Steve Ellyson. 1982. “Decoding Visual Dominance: Attributions of Power Based on Relative Percentages of Looking while Speaking and Looking while Listening.” Social Psychology Quarterly 45 (2): 106–113. https://doi.org/10.2307/3033933 Félix-Brasdefer, J. César. 2018. “Role Plays.” In Methods in Pragmatics. Handbooks of Pragmatics 10, ed. by Andreas H. Jucker, Klaus P. Schneider, and Wolfram Bublitz, 305–331. Berlin: de Gruyter. https://doi.org/10.1515/9783110424928-012 Feygina, Irina, and P. J. Henry. 2015. “Culture and Prosocial Behavior.” In The Oxford Handbook of Prosocial Behavior, ed. by David Schroeder, and William Graziano, 188–208. Oxford: Oxford University Press. Ford, Cecilia, Sandra Thompson, and Veronika Drake. 2012. “Bodily Visual Practices and Turn Continuation.” Discourse Processes 49 (3–4): 192–212. https://doi.org/10.1080/0163853X.2012.654761

Forsyth, Richard, David Clarke, and Phoenix Lam. 2008. “Timelines, Talk and Transcription: A Chronometric Approach to Simultaneous Speech.” International Journal of Corpus Linguistics 13 (2): 225–250. https://doi.org/10.1075/ijcl.13.2.05for Fukushima, Saeko. 1996. “Request Strategies in British English and Japanese.” Language Sciences 18 (3–4): 671–688. https://doi.org/10.1016/S0388-0001(96)00041-1 George, Johnny. 2011. “Politeness in Japanese Sign Language (JSL): Polite JSL Expression as Evidence for Intermodal Language Contact Influence.” PhD dissertation, University of California. Goldsmith, Daena J. 2007. “Brown and Levinson’s Politeness Theory.” In Explaining Communication: Contemporary Theories and Exemplars, ed. by Bryan B. Whaley and Wendy Samter, 219–236. Mahwah, NJ: Lawrence Erlbaum. Grawunder, Sven, Marianne Oertel, and Cordula Schwarze. 2014. “Politeness, Culture, and Speaking Task: Paralinguistic Prosodic Behavior of Speakers from Austria and Germany.” In Proceedings of the 7th International Conference on Speech Prosody: Social and Linguistic Speech Prosody, 159–163. https://doi.org/10.21437/SpeechProsody.2014-20 Guerrero, Laura, and Kory Floyd. 2006. Nonverbal Communication in Close Relationships. Abingdon: Routledge. https://doi.org/10.4324/9781410617064 Gussenhoven, Carlos. 2004. The Phonology of Tone and Intonation. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511616983 Harrigan, Jinni. 1985. “Self-touching as an Indicator of Underlying Affect and Language Processes.” Social Science and Medicine 20 (11): 1161–1168. https://doi.org/10.1016/0277-9536(85)90193-5

Hatfield, Hunter, and Jee-Won Hahn. 2011. “What Korean Apologies Require of Politeness Theory.” Journal of Pragmatics 43 (5): 1303–1317. https://doi.org/10.1016/j.pragma.2010.10.028 Haugh, Michael, and Derek Bousfield. 2012. “Mock Impoliteness, Jocular Mockery and Jocular Abuse in Australian and British English.” Journal of Pragmatics 44 (9): 1099–1114. https://doi.org/10.1016/j.pragma.2012.02.003

Hübscher, Iris, Joan Borràs-Comes, and Pilar Prieto. 2017. “Prosodic Mitigation Characterizes Catalan Formal Speech: The Frequency Code Reassessed.” Journal of Phonetics 65: 145–159. https://doi.org/10.1016/j.wocn.2017.07.001

Indexing social distance through bodily visual practices in two languages

Hübscher, Iris, Cristina Sánchez-Conde, Joan Borràs-Comes, Laura Vincze, and Pilar Prieto. forthcoming. “Multimodal Mitigation: How Facial and Body Cues Index Social Meaning in Catalan Requests.” Ide, Sachiko. 1989. “Formal Forms and Discernment: Two Neglected Aspects of Universals of Linguistic Politeness.” Multilingua 8 (2–3): 223–248. https://doi.org/10.1515/mult.1989.8.2-3.223

Idemaru, Kaori, Bodo Winter, and Lucien Brown. 2019. “Cross-cultural Multimodal Politeness: The Phonetics of Japanese Deferential Speech in Comparison to Korean.” Intercultural Pragmatics 16 (5): 517–555. https://doi.org/10.1515/ip-2019-0027 Idemaru, Kaori, Bodo Winter, Lucien Brown, and Grace Oh. 2020. “Loudness Trumps Pitch in Politeness Judgments: Evidence from Korean Deferential Speech.” Language and Speech 63 (1): 123–148. https://doi.org/10.1177/0023830918824344 Jorgenson, Dale. 1975. “Field Study of the Relationship between Status Discrepancy and Proxemic Behavior.” The Journal of Social Psychology 97 (2): 173–179. https://doi.org/10.1080/00224545.1975.9923337

Kay, Matthew. 2021. “Tidybayes: Tidy Data and Geoms for Bayesian Models.” R Package Version 3.0.1. https://doi.org/10.5281/zenodo.1308151 Kim, Hyunji, Bodo Winter, and Lucien Brown. 2021. “Beyond Politeness Markers: Multiple Morphological and Lexical Differences Index Deferential Meanings in Korean.” Journal of Pragmatics 182: 203–220. https://doi.org/10.1016/j.pragma.2021.06.006 Kita, Sotaro, and Sachiko Ide. 2007. “Nodding, Aizuchi, and Final Particles in Japanese Conversation: How Conversation Reflects the Ideology of Communication and Social Relationships.” Journal of Pragmatics 39 (7): 1242–1254. https://doi.org/10.1016/j.pragma.2007.02.009

Kita, Sotaro, and Aslı Özyürek. 2003. “What does Cross-linguistic Variation in Semantic Coordination of Speech and Gesture Reveal?: Evidence for an Interface Representation of Spatial Thinking and Speaking.” Journal of Memory and Language 48 (1): 16–32. https://doi.org/10.1016/S0749-596X(02)00505-3

Koo, Hyunjung, and Jungmi Jeon. 2007. Hwapep-uy ilon-kwa silcey [The theory and reality of communication]. Seoul: Pagijong. Leech, Geoffrey. 1983. Principles of Pragmatics. Abingdon: Routledge. Leech, Geoffrey. 2014. The Pragmatics of Politeness. Oxford: Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195341386.001.0001

Lemoine, Nathan. 2019. “Moving beyond Noninformative Priors: Why and How to Choose Weakly Informative Priors in Bayesian Analyses.” Oikos 128 (7), 912–928. https://doi.org/10.1111/oik.05985

Lickley, Robin, and Ellen Bard. 1998. “When can Listeners Detect Disfluency in Spontaneous Speech?” Language and Speech 41 (2): 203–226. https://doi.org/10.1177/002383099804100204 Loveday, Leo. 1981. “Pitch, Politeness and Sexual Role: An Exploratory Investigation into the Pitch Correlates of English and Japanese Politeness Pormulae.” Language and Speech 24 (1): 71–89. https://doi.org/10.1177/002383098102400105 Mapson, Rachel. 2014. “Polite Appearances: How Non-manual Features Convey Politeness in British Sign Language.” Journal of Politeness Research 10 (2): 157–184. https://doi.org/10.1515/pr-2014-0008

Matsumoto, Yoshiko. 1988. “Reexamination of the Universality of Face: Politeness Phenomena in Japanese.” Journal of Pragmatics 12 (4): 403–426. https://doi.org/10.1016/0378-2166(88)90003-3

159

160

Lucien Brown, Iris Hübscher, Hyunji Kim & Bodo Winter

McClave, Evelyn. 2000. “Linguistic Functions of Head Movements in the Context of Speech.” Journal of Pragmatics 32 (7): 855–878. https://doi.org/10.1016/S0378-2166(99)00079-X McElreath, Richard. 2020. Statistical Rethinking: A Bayesian Course with Examples in R and Stan (2nd ed.). Boca Raton, FL: CRC press. https://doi.org/10.1201/9780429029608 McNeill, David, and Susan Duncan. 2000. “Growth Points in Thinking-for-speaking.” In Language and Gesture, ed. by David McNeill, 141–161. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511620850.010 Mehrabian, Albert. 1968. “Inference of Attitudes from the Posture, Orientation, and Distance of a Communicator.” Journal of Consulting and Clinical Psychology 32 (3): 296–308. https://doi.org/10.1037/h0025906

Mehrabian, Albert. 1972. Nonverbal Communication. Chicago, IL: Aldine Atherton. Morey, Richard, Rink Hoekstra, Jeffrey Rouder, Michael Lee, Eric-Jan Wagenmakers. 2016. “The Fallacy of Placing Confidence in Confidence Intervals.” Psychonomic Bulletin and Review 23 (1): 103–123. https://doi.org/10.3758/s13423-015-0947-8 Oh, Grace, and Mao Cui. 2020. “The Acquisition of Acoustic Correlates of Politeness by Native Chinese Speakers.” Linguistic Research 37: 113–134. Ohala, John. 1994. “The Frequency Code Underlies the Sound-symbolic Use of Voice Pitch.” In Sound Symbolism, ed. by Johanna Nichols, John Ohala, and Leanne Hinton, 325–347. Cambridge: Cambridge University Press. Orie, Ọlanikẹ Ọla. 2009. “Pointing the Yoruba Way.” Gesture 9 (2): 237–261. https://doi.org/10.1075/gest.9.2.04ori

Orozco, Leonor. 2010. Estudio sociolingüístico de la cortesía en tratamientos y peticiones. Datos de Guadalajara. PhD dissertation, El Colegio de México. R Core Team. 2019. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. Rice, Sally, and Jennifer Hinnell. 2015. “Head, Shoulders, Knees, and Toes: The Partitioning of the Body in the Embodied Marking of Stance.” Paper presented at 13th International Cognitive Linguistic Conference (ICLC13). Sabaté i Dalmau, Maria, and Hortènsia Curell i Gotor. 2007. “From ‘Sorry very much’ to ‘I’m ever so sorry’: Acquisitional Patterns in L2 Apologies by Catalan Learners of English.” Intercultural Pragmatics 4 (2): 287–315. Sherr-Ziarko, Ethan. 2019. “Prosodic Properties of Formality in Conversational Japanese.” Journal of the International Phonetic Association 49 (3): 331–352. https://doi.org/10.1017/S0025100318000117

Snyder, Robert, and Louis Sutker. 1977. “The Measurement of the Construct of Dominance and its Relationship to Nonverbal Behavior.” The Journal of Psychology 97 (2): 227–230. https://doi.org/10.1080/00223980.1977.9923967

Sohn, Ho-Min. 2001. The Korean Language. Cambridge: Cambridge University Press. Strongman, Kenneth, and Brian Champness. 1968. “Dominance Hierarchies and Conflict in Eye Contact.” Acta Psychologica 28: 376–386. https://doi.org/10.1016/0001-6918(68)90026-7 Takahashi, Tomoko, and Leslie Beebe. 1993. “Cross-linguistic Influence in the Speech Act of Correction.” In Interlanguage Pragmatics, ed. by Gabriele Kasper and Shoshana Blum-Kulka, 158–169. Oxford: Oxford University Press. Torchiano, Marco. 2019. Effsize: Efficient Effect Size Computation. https://doi.org/10.5281/zenodo.1480624

Indexing social distance through bodily visual practices in two languages

Trees, April, and Valerie Manusov. 1998. “Managing Face Concerns in Criticism: Integrating Nonverbal Behaviors as a Dimension of Politeness in Female Friendship Dyads.” Human Communication Research 24 (4): 564–583. https://doi.org/10.1111/j.1468-2958.1998.tb00431.x Uono, Shota, and Jari Hietanen. 2015. “Eye Contact Perception in the West and East: A Crosscultural Study.” Plos One 10 (2): e0118094. https://doi.org/10.1371/journal.pone.0118094 Watts, Richard J. 1989. “Relevance and Relational Work: Linguistic Politeness as Politic Behavior.” Multilingua 8 (2/3): 131–166. https://doi.org/10.1515/mult.1989.8.2-3.131 Watts, Richard J. 1992. “Linguistic Politeness and Politic Verbal Behaviour: Reconsidering Claims for Universality.” In Politeness in Language: Studies in its History, Theory and Practice, ed. by Richard J. Watts, Sachiko Ide, and Konrad Ehlich, 43–69. Berlin: De Gruyter Mouton. https://doi.org/10.1515/9783110886542-005 Wickham, Hadley, Mara Averick, Jennifer Bryan, Winston Chang, Lucy McGowan, Romain François, Garret Grolemund, Alex Hayes, Lionel Henry, and Jim Hester. 2019. “Welcome to the Tidyverse.” Journal of Open Source Software 4 (43): 1686. https://doi.org/10.21105/joss.01686

Winter, Bodo, and Paul-Christian Bürkner. 2021. “Poisson Regression for Linguists: A Tutorial Introduction to Modeling Count Data with Brms.” Language and Linguistics Compass e12439. https://doi.org/10.1111/lnc3.12439 Winter, Bodo, and Sven Grawunder. 2012. “The Phonetic Profile of Korean Formal and Informal Speech Registers.” Journal of Phonetics 40 (6): 808–815. https://doi.org/10.1016/j.wocn.2012.08.006

Winter, Bodo, Grace Oh, Iris Hübscher, Kaori Idemaru, Lucien Brown, Pilar Prieto, and Swen Grawunder. 2021. “Rethinking the Frequency Code: A Meta-analytic Review of the Role of Acoustic Body Size in Communicative Phenomena.” Philosophical Transactions B, 376 (1840), 20200400. https://doi.org/10.1098/rstb.2020.0400

161

chapter 6

Multimodal manifestation of ta’ârof in Persian Farbod Farahandouz1,2 & Shima Moallemi3 1

Sorbonne Nouvelle University | 2 National Institute for Oriental Languages and Civilizations | 3 Utrecht University

The present research investigates the multimodal aspect of ta’ârof, a politeness ritual essential in many interpersonal interactions in Persian. We studied the opening sequences of interactions in an Iranian talk show to illustrate how gestures are used to express self-lowering and other-raising strategies between the host of the show and his guests. To analyze our corpus, we used NEUROGES® coding system to annotate gestures. Three categories of gestures (on body, on person, and egocentric deictic) were investigated and the results showed that they all have a pragmatic performative function and contribute to the meaning construction of ta’ârof in greeting sequences. In addition, age and gender have been identified as influential factors in the use of gesture to express politeness in Persian. Keywords: Ta’ârof, Persian, politeness, gesture, multimodality

1.

Introduction

Kinesics and its relationship to language and culture, introduced by the works of Birdwhistell (1968) and Hall (1971, 1984), is now well developed as a branch of linguistics and is widely considered in the analysis of interactions. However, the branch of pragmatics that deals with the study of im/politeness has mainly focused on the verbal modality of exchanges (L. Brown and Prieto 2017), even though P. Brown and Levinson (1987) had mentioned kinesic factors in their work on the universals of politeness a long time ago. The studies carried out on Persian politeness (Beeman 1986; Koutlaki 2002; Sharifian 2011; Miller et al. 2014) are no exception and almost all the research focuses on the verbal aspect of the exchange, with some minor allusions to multimodal dimensions (Sahragard 2003; Izadi 2015; Yaqubi 2020; Beeman 2020). The aim of this article is to explore the multimodal aspects of the politeness ritual of ta’ârof with a focus on gesture analysis. https://doi.org/10.1075/pbns.333.06far © 2023 John Benjamins Publishing Company

164

Farbod Farahandouz & Shima Moallemi

Regarding multimodal factors in the study of im/politeness, L. Brown and Prieto (2017) underline the importance of the speaker’s voice as well as bodily and facial expressions in the comprehension of im/politeness in communication. They suggest that “by adopting certain prosodic patterns, speakers can change what may on the surface appear to be a polite utterance into an impolite one, and vice versa” (2017: 362). McKinnon and Prieto (2014) showed that prosodic cues and gestures are two factors that differentiate genuine impoliteness sentences from mock impoliteness sentences in Catalan. For Catalan speakers, genuine impoliteness was associated with prosodic and gestural features of anger, while mock politeness was linked to an expression of joy and was associated with speakers smiling, shaking their heads, or slightly raising their arms to the midtorso. Considering an intercultural perspective, Yaqubi (2020) showed that facial expressions were one of the factors that helped non-native Persian audiences watching Iranian subtitled films to distinguish ostensible invitations and refusals, which are frequent in the practice of ta’ârof in Persian, from genuine forms of these speech acts. Even though this study was not directly about multimodal factors implicated in Persian politeness, the results showed the role played by facial expressions in the practice of ta’ârof by Iranians. In Persian, the politeness ritual of ta’ârof is essential in many interpersonal interactions and is considered as one of the most important cultural values in Iranian society. Etymologically, ta’ârof is derived from the Arabic root arafa which means ‘to meet and get to know each other’, while in Persian it refers to a complex and ritualized practice of politeness. One of the main purposes of ta’ârof is, as underlined by Sharifian (2011: 144), its interactional function as it is used to negotiate and lubricate social relations. It also provides an opportunity for interlocutors to demonstrate an aspect of their identity and create a self-image, such as being someone who is welcoming, hospitable, humble, or respectful toward others. Ta’ârof can be manifested in interactions as ostensible offers or invitations that serve to express attention and sympathy to the interlocutor but that are not to be taken at face value. Thus, the expected answer and the polite one in these situations is the rejection of the offer or the invitation. The speaker, knowing that by refusing the interlocutor is practicing ta’ârof, will not be offended by this refusal and will insist on his words by renewing his invitation or offer. This offer-rejection sequence can continue for several speech turns, and interlocutors may even use metapragmatic phrases such as “Please, don’t do ta’ârof” or “I assure you, I’m not doing ta’ârof”. Ta’ârof is not only about invitations or offers, but it can also be realized as hesitation in making requests, giving frequent compliments, offering the piece of clothing or other items on which one has been complimented, or expressing reluctance to readily accept money in business transactions and in situations

Multimodal manifestation of ta’ârof in Persian

where a debt is being returned (Sharifian 2011; Koutlaki 2002). Of course, in these two last situations, the interlocutor should not accept the item offered or insist on paying what they owe. It’s important to add that whereas ta’ârof has a positive connotation and is widely practiced and expected in most interactional situations, it can reflect a negative meaning if it is “used in ways that might be interpreted as impolite, manipulative, or self-interested” (Beeman 2020: 203). As some behaviors are associated with ritualized linguistic expressions used to perform ta’ârof, we will discuss both behavioral and linguistic aspects below. Gestures can also be observed during the behavioral and linguistic ta’ârof and that is precisely the object of our research. Our corpus consists of a readily available video of an Iranian talk-show on YouTube as data. By studying the greeting sequences between the host of the show and his guests, we intend to analyze the implication of gestures as a non-verbal taxème (Kerbrat Orecchioni 1994) in the construction of bonding and differentiation (Izadi 2015) by interlocutors in ta’ârof situations. This study can contribute to a better understanding of multimodal factors implicated in the practice of politeness in Persian.

2.

Linguistic and behavioral ta’ârof

According to Kerbrat-Orecchioni (1994), the way society conceives interpersonal relationships can influence its communicative behaviors. Depending on the frequency or the rareness of the use of taxèmes in communication, one can distinguish societies with hierarchical and egalitarian communicative tendencies. Kerbrat-Orecchioni (1994: 74) defines taxème as “any behavior, verbal or nonverbal, likely to mark a hierarchical relationship between the interactants” that specifies the social status of the interlocutors. For example, in the context of an Iranian family, the youngsters are supposed to use the V-form to address someone older than them but the older member of the family (for example an older cousin or an uncle) can naturally address them by applying the T-form. The use of these asymmetrical address terms, as a linguistic taxème, indicate the existence of a hierarchical relationship between the interlocutors. Concerning multimodal taxèmes, we would refer to the body posture that young people are supposed to adopt in front of older people (especially in the family context) in Iran. Indeed, children and young people should adopt an upright posture and avoid lying down or extending their legs in the presence of their uncles, aunts, or grandparents, otherwise, it would be considered as a lack of respect (Moallemi 2019: 146). Moreover, in a university context, whenever students see their professors in the corridors, they stand up and greet them, which is not common in the French university context, for example. Compared to Persian, fewer taxèmes are observed in French,

165

166

Farbod Farahandouz & Shima Moallemi

as even in a professional context, the use of the reciprocal T-form between interlocutors is quite common. However, we can observe that it’s always the superior that initiates the use of the T-form (“on peut se tutoyer?” meaning ‘could we use tu to address each other?’) and imposes it somehow, because the interlocutor with an inferior status cannot generally refuse it or it would be considered rude. Thus, even in a more egalitarian culture, we can observe the use of taxèmes, even though they would be less frequent than in a hierarchical society. In Persian, the large number of taxèmes, which can be seen through the frequent use of honorifics (pronouns, verbs, fixed expressions, or phrases), can be symbolic of a more hierarchical society (Moallemi 2020). These taxèmes are indeed used in the framework of ta’ârof. In the words of Beeman, “the essence of the language of ta’ârof is the symbolic social elevation (other-raising) of the addressee and the symbolic lowering of oneself (self-lowering)” (2020: 205). In fact, Persian proposes three linguistic variables for pronouns and verbs, going from neutral to self-lowering and other-raising forms (Izadi 2015; Beeman 2020). For instance, for the neutral verb âmadan (‘to come’), other-raising variables such as tashrif âvardan (‘bring presence’) and eftekhâr-e hozour dâdan (‘to give me the honor of your presence’), and the selflowering form khedmat residan (‘to be at one’s service’) are used (Izadi 2015: 83). Ritualized phrases can also be employed to express deference, and in this case, they generally have a fixed response. The phrase mâ mokhlese shomâ hastim (‘we are your sincere and obedient [friend]’) is a self-lowering form used mostly by male speakers to exalt the status of the addressee and is mostly accompanied by a hand placed on one’s chest. In return, the addressee will reply with mâ kouchike shomâ hastim (‘we are your inferior’), which is another self-lowering form and the typical answer to the first phrase. The pronoun we used instead of the firstperson singular is also a deferential form and shows modesty and humbleness.1 The linguistic aspect of ta’ârof is mostly accompanied or supported by specific body movements, gestures and vocal intonation, which are factors that can indicate whether an invitation or an offer is genuine or ostensible. There is a Persian expression that can illustrate well the importance of multimodal factors in the practice of ta’ârof. When a person uses a stereotypical ritualized phrase for example to greet someone or makes invitations or offers, but the intonation or the body postures or gestures they use do not show their commitment to what they are say1. The first person plural pronoun ‘we’ refers here to what is called in French “nous de modestie”, which is generally used by an author or a conference speaker (Riegel et al. 1994) and is different from the ‘we of majesty’ that refers to a religious or civil authority. However, in Persian, the ‘we of modesty’ is frequently used, especially in the school context by the pupils to express deference, and, in some ritualized phrases, it is the pronoun ‘we’ which is most popularly used.

Multimodal manifestation of ta’ârof in Persian

ing, or they do not insist enough in offering or inviting after the interlocutor has rejected the offer or invitation, one can criticize by saying that they just did a dry and empty ta’ârof (‘ta’ârof-e khoshko khâli’) meaning that the speaker had an artificial way of expressing politeness. This metapragmatic comment shows that the ritualized utterances involving ta’ârof alone are not enough to express politeness and other factors such as the repetition of ta’ârof turns (offer or invitation) and also multimodal factors are expected and can be indicative of the degree of politeness of a speaker. L. Brown (2013) also showed that in Korean honorific markers are not intrinsically deferential and as some of them can be used sarcastically, they cannot be strictly associated with politeness-related meanings. This research underlines the importance of a concordance between verbal politeness and the mode of delivery via nonverbal elements and shows that it can influence the way politeness is evaluated in interactions. Opening and closing sequences, being, in general, quite ritualized (KerbratOrecchioni 1994), give rise to the practice of behavioral ta’ârof, such as standing up when someone enters the room or leaves it, and inviting someone to sit down first. In all these social behaviors, the whole body and especially the hands are involved in expressing deference and respect, particularly in formal situations. The hands are both folded in front of oneself, or one hand is placed on the chest while the other indicates the location where the other party should sit, for instance. The posture is bent a little forward, with the eyes cast down. Even though these examples of behavioral ta’ârof are not culture-specific and can be easily understood in an intercultural context, as we will see in the analyses below, some details in the gestures that accompany them could be specific considering gender and age factors. Sahragard (2003) and Beeman (2020) refer to an instance of behavioral ta’ârof that could be identified as culture-specific. It concerns inviting the guest to sit in the place of honor, which according to the authors, is the farthest place from the door.2 We should also add that this attribution of place of honor is sometimes influenced by other criteria such as the size of the room, the number of people present in the room, the furniture configuration as well as the location of the fireplace (in cold seasons) or the direction of air conditioner or fan, etc. In Persian culture, hosts will consider these factors before proposing a place to guests. As a sign of self-lowering, the guest can at first refuse to sit in this place, suggesting that other guests, older than them, deserve this place or that they feel

2. This concept could be traced back to ancient times when Iranian people used to sit on carpets and it was necessary to keep the guests as far away as possible from the entrance where they took off their shoes and which was considered less clean, but this hypothesis needs further exploration.

167

168

Farbod Farahandouz & Shima Moallemi

comfortable enough in a normal place. The host then insists, and the guest ends up consenting. Ta’ârof can be interpreted as an attempt to express respect and give a positive image of the one who practices it or can be a means of bonding or differentiation. Inspired by the works of Goffman (1967) and P. Brown and Levinson (1987) on the notions of face, face work, and face-threatening act (FTA), Koutlaki (2002) defines Persian face through the two concepts of shaxsiat (‘personality’) and ehterâm (‘respect’). The shaxsiat of an individual can be observed through one’s social behavior and hence can influence the way others evaluate or judge them. Ehterâm is observed in the practice of ta’ârof, especially by using nonverbal and verbal taxèmes (like honorifics). The use of a proper honorific with the proper person (considering their status) can enhance the face of the two interlocutors. For example, when a speaker uses an other-raising address term, they enhance the face of the interlocutor and express their respect. However, the self-lowering strategy used in the practice of ta’ârof is not face-threatening for the speaker; rather, it enhances their own shaxsiat because they can show mastery of the social norms and be considered as someone humble and respectful in return. In the same way, according to Koutlaki (2002), refusing an offer as part of the practice of ta’ârof is not an FTA, as it would be interpreted according to P. Brown and Levinson’s conception of positive face (1987: 62). It is rather a face-enhancing act, as the offer is an expression of politeness, and being an ostensible speech act (Isaacs and Clark 1990), the desired and the expected answer is actually the rejection of the offer. Criticizing P. Brown and Levinson’s definition of face as mainly individualistic and cognitivist, ignoring its social aspect, Izadi (2015) goes further and proposes a different definition of face in Persian as being socially co-constructed. Inspired by Arundale’s (2010) work which considers face as a “relational phenomenon which is dynamically co-constructed in interactions”, Izadi (2017: 211) proposes a definition of face in Persian through the emic concepts of âberu (lit. ‘water of the face’)3 and shaxsiat. Âberu “embodies the image of a person, a family, or a group, particularly as viewed by others in the society” (Sharifian 2007: 36). According to Sharifian (2011: 141), this social image “is tied to a large number of social norms in relation to financial status, behavior, both linguistic and nonlinguistic, and social relations and network”. Similar to the definition given by Koutlaki (2002), the shaxsiat of a person stands for one’s personality, or character, and is dynamically co-constructed in interactions. Someone’s shaxsiat is evaluated according to the way one behaves, speaks and even dresses (Izadi 2017: 211). These two concepts are linked as “one’s âberu is what one thinks others think of one’s shaxsiat, 3. Âb (water) refers metaphorically to one’s honor, reputation or prestige and ru(y) refers to someone’s face.

Multimodal manifestation of ta’ârof in Persian

whereas others’ âberu is what one thinks of their shaxsiat” (Izadi 2017: 211). The social image or âberu is very important in Iranian culture and people try not to lose their individual or group âberu. The practice of ta’ârof as one of the main social norms in Iranian culture can contribute to maintaining someone’s âberu and bring a positive image of her/his shaxsiat in the eyes of others. According to Izadi (2015: 84), ta’ârof is also an attempt to achieve both bonding and differentiation in social interactions. When people use ta’ârof to express their respect to each other, they create a degree of differentiation as they recognize the social status of each other and the degree of intimacy or formality they share. At the same time, by practicing behavioral ta’ârof (e.g., inviting a neighbor to come over for dinner before saying goodbye or proposing to someone to sit in the place of honor or to enter first, etc.) people try to bond with their interlocutors. Even though the invitation or the offer is not genuine, it shows attention to others or the will to put the other’s comfort or desire first. The degree of bonding can also vary. If the interlocutor accepts the invitation or the offer, they both achieve stronger bonding. On the contrary, if they limit the interaction to an ostensible invitation and refusal, the degree can be inferior, but nevertheless existing. The importance of bonding in the practice of ta’ârof can be illustrated with an intercultural example. Moallemi (2019) showed that even though young Iranians find the practice of ta’ârof time-consuming and annoying when they are in the Iranian context, the absence of this practice in an intercultural context, in France for instance, leads to a negative evaluation of their foreign interlocutor as someone who is not cordial or warm compared to Iranians. Concerning the relationship between bonding/differentiation and the concept of âberu, Izadi (2017) mentions that people try to avoid negative evaluation of their âberu by people with whom they try to bond. Appropriate use of ta’ârof in social interactions can contribute to the maintenance of the positive image of a person. Based on the above definitions of ta’ârof, we will study the bonding/differentiation that is created using verbal and nonverbal taxèmes in opening sequences. In particular, we will study on-body, on-person, and egocentric deictic gestures (Lausberg 2013, 2019) which we explain in the next section.

3. Data and method This research has a multimodal conversation analytic design for which we collected our data from ready-to-use videos on YouTube. Video data or any kind of ethnographic visual data (still image, sculpture, petroglyphs, etc.) is the prerequisite for doing research on interaction and gesture. For this research, we use YouTube data that we think is relevant to our research question as the videos we

169

170

Farbod Farahandouz & Shima Moallemi

chose contain salutation sequences which, as mentioned above, contain ta’ârof exchanges between interlocutors. Norris explains that “as long as there is some kind of (inter)action shown in the video” (2019: 87), researchers could use and apply their framework on the data. For our study, we chose the first season of an Iranian talk show, produced in 2017. Dorehami is a talk show on Iranian public television that is broadcast once a week. We collected ten episodes of this talk show, in which the host, who is a middle-aged man, invites celebrities, actors, actresses, athletes, etc., and asks them general questions about their careers and also personal questions. Each episode is about sixty minutes long and recorded in a studio in the presence of an audience of fifty to sixty persons. In the first phase of the data collection, we selected five male guests aged between 30 and 80 years old and five women guests between 30 to 60 years old. In the second phase of our data analysis, we extracted only greeting sequences of these episodes that contain obvious practices of ta’ârof. Also, one midconversation sequence was added to the corpus to illustrate self-lowering and other-raising rituals. Since the beginning, this talk show has had different seasons. We have chosen the first season especially because of the configuration and the decoration of the scene which allow us to see body posture, the entrance, and the path from the entrance to the place where the conversation and greeting sequences take place (Figure 1).

Figure 1. Scene configuration and decoration. ‘A’ shows the entrance, and ‘B’ shows the place where the conversations take place

We should mention that the interior decor of the TV studio represents a typical old Iranian house with different doors to the rooms, and several windows toward the garden or yard. As shown in Figure 1, the entrance is located on the left side of the studio and the place where the conversations take place is in the mid-

Multimodal manifestation of ta’ârof in Persian

dle of the stage. As mentioned above, different criteria can influence the attribution of the place of honor. Here, it seems that because of the studio configuration in which the guests should face the audience and constraints of camera angle, the sofa in the middle is considered to be the place of honor, which is not physically the furthest place from the entrance. Different classifications and transcription conventions have emerged since the first attempts for classifying gestures. Among different attempts to classify gestures in the last decades, we can mention Efron (1972) and Ekman and Friesen (1969), who worked merely on gesture representations. More recently, McNeill (1992) came up with a classification of manual gestures which has been followed widely for analyzing co-speech gestures, which are hand and arm movements produced simultaneously with speech. Other conventions were also developed by Bressem (2013), as well as Bressem, Ladewig and Müller (2013), who introduced a framework for linguistic analysis and gesture analysis for studying syntax, sentence structure, phonology, and their relationship with gesture. For this study, we chose the NEUROGES® coding system for nonverbal behavior and gesture, introduced by Lausberg and Sloetjes (2009) and Lausberg (2013, 2019), which is used for studying hand movements and their relation to cognitive, emotional, and interactive processes. As mentioned by the developer of the NEUROGES® coding system, it is “designed as a tool for basic empirical research on movement behavior and its link to cognitive, emotional, and interactive processes” (Lausberg 2013: 85). For this research, we focused on haptic gestures in Persian greeting interactions, and it seems to us that the NEUROGES® coding system offers wider options for the analysis of this kind of gesture than McNeill’s model (1992) which is not really adapted for the study of haptic gestures. Also, NEUROGES® is different from McNeill’s gesture classification in several respects. For instance, it helps to analyze not only in-space movements but also symmetrical and asymmetrical movements of both hands, to which we could add the notion of hand dominance. Another notable advantage of this coding system is its very detailed subcategories for haptics or touch. More precisely, touching on (one’s own) body, on objects attached to the body (e.g., clothes, eyeglasses, etc.), on another person’s body, or on objects are all clearly distinguished in this coding system (cf. focus values of NEUROGES®). We analyze only hand movements for our research but NEUROGES® gives the possibility to code the upper limbs as well as leg movements. The NEUROGES® coding system is composed of three modules and 58 values.4 Module I is developed for “basic segmentation and classification of move4. The diagram is accessible via this link https://neuroges-bast.info/storage/neuroges-pages /April2022/qP1Z4WaQVbmYJ2fCwkA8.jpeg

171

172

Farbod Farahandouz & Shima Moallemi

ment behavior” (Lausberg 2013: 96) and it deals with the structure of hand movements. It is divided into activation, structure, and focus categories. For the activation category, we distinguish movements and rest position. In this category, the idea of movement is based on Kendon’s definition of the gesture unit (1972, 2004): the hand moves from a rest or initial position to a region in space and then returns to the original, rest or initial position.5 When a movement is detected and annotated, we can then analyze the next category, the structure category in which we analyze the type of movement based on its trajectory and its dynamics in space. Based on these two distinct phases (activation and structure) we have five different patterns for movements: phasic, irregular, repetitive, aborted, and shift. The last category in the first module is the focus category in which hand movements are analyzed regarding the location where they act (on). This category has six different values: within body, on body, on attached object, on separate object, on person, and in space (focus category). In particular, these values provide a framework for analyzing any form of physical contact on the surface of one’s own body, an object, or another person’s body. On body refers to hand movements on the body surface. It should be mentioned that “dynamic contact with objects that are attached to the other person’s body is coded as well with the value on person” (Lausberg 2013: 109) such as touching the eyeglasses, watch, etc. We should also mention that other authors in the field of multimodality like Burgoon et al. (2010); Richmond et al. (2012); Knapp et al. (2014) and Moore (2020) use other terms such as haptics, tactics (tactile behavior), zero proxemics (without space between interactants) or, more commonly, touch to refer to the physical contact between parts of the body. Module II of this coding system deals with bilateral limb movement and the symmetricity of (hand) movements based on research on split-brain patients to show the different levels of stability of interhemispheric cooperation (Lausberg 2013: 98). For our research, we do not focus on this category as we did not see, at least for the current research, any significant relationship between hand dominance and its pragmatic functions. Module III of NEUROGES® consists of two categories: function and type, and offers “an analysis of conceptual body movements, i.e., gestures and actions, regarding their emotional, cognitive, physical, or practical functions” (Lausberg 2019: 175). In our study, we analyzed egocentric deictic gestures. This value is used to classify gestures used to designate a target or a location in space. In the gesture studies literature, we could also find the terms deictic (McNeill 1992; Kendon

5. Gesture unit refers to what Kendon (2004) calls excursion which is constituted of the three phases of relaxation, stroke and return.

Multimodal manifestation of ta’ârof in Persian

2004) or pointing gestures (Lausberg 2019: 24) to refer to this value (Cf. the analysis algorithm of NEUROGES® system). For the purpose of studying the relation between gesture and speech in the expression of politeness in greeting sequences in Persian, we, first, extracted the greeting sequences of the ten videos selected for our data. We then used ELAN (ver. 5.9, 2020) to transcribe speech and annotate hand gestures based on the NEUROGES® coding system, to annotate the focus categories, based on the explanations above. Transcription of speech helped us to see how hand movements are performed in relation to the verbal components to form an utterance. Afterward, all the greeting sequences of the videos were compared to see the occurrence of on-body and of on-person hand movements (Module I of NEUROGES®) as well as egocentric deictic gestures (Module III of NUEROGES®) and their association with ritualized verbal politeness phrases. This comparison was made based on the age and the gender of each guest on the talk show. We should also mention that we considered the fact that the NEUROGES® coding system is not based on speech and word-to-hand movement associations. But we enriched our analysis with speech transcription as an added value intending to study not only the nonverbal form of the ta’ârof but also to examine the relationship between the verbal and the nonverbal aspect of this practice.

4. Results At the very beginning of each footage, the studio audience applauds, and simultaneously the host stands up and goes near the stage entrance to welcome the guest. It is worth mentioning that this posture change can be considered as behavioral ta’ârof and, as an other-raising strategy, since it represents a nonverbal taxème used to express respect to the guest. The visual inspection of the opening sequences reveals that in all excerpts, similar on-body movements were performed by guests when saying doroud or salâm (both are equivalent of hello in English) as a kind of self-lowering strategy (Figure 2). Besides, in four out of ten sequences, we see this gesture precede the speech in the gesture tier in ELAN. This underlines the importance of nonverbal taxèmes in the practice of politeness in this situation. Concerning the host’s gestures in these opening sequences, we noticed a difference in the type of on-body gesture used with male and female guests. In greeting sequences with male guests, the host puts his hand on his chest, but when greeting female guests, he puts his left hand over the right hand and keeps both hands in front of him (Figure 3). Being the host of a TV show watched by thousands of Iranians, he tries to behave according to the specific social norms of the

173

174

Farbod Farahandouz & Shima Moallemi

Figure 2. On-body gestures performed by three different guests in greeting sequences

interaction and tries to respond to the expectations of the audience in order to maintain his social image (âberu). The behavior expected in this situation (a formal interaction with a woman that is broadcast on national TV ) is to act with regard to hayâ. According to Dehkhoda Persian-Persian Dictionary (Dehkhoda et al. 1998), behaving in the framework of hayâ refers to any kind of behavior that forbids oneself from doing something to avoid committing an ugly act and to avoid condemnation and blame. In other words, hayâ is “an inhibiting as well as an impelling force aroused when one faces situations in which a social or religious taboo is violated or is prone to transgression” (Bakhtiar 2018: 142). In everyday life, this concept can have different behavioral manifestations. For instance, in public places, it is important to adopt an appropriate body posture that would not be too slouched such as manspreading, or avoiding staring at someone unknown, or using words or phrases with sexual connotations in the presence of the opposite sex. For men, in some social contexts, it could be inappropriate to gaze at women and some strictly religious men even avoid looking women directly in the eyes. Nonetheless, the same behavior may be considered rude by women who interpret this as a lack of mutual attention and consider it as a deviation from hayâ as they expect eye contact in social interactions. Thus, hayâ has different degrees of perception and understanding based on the interactional context and one’s educational, social and cultural background, as well as religious beliefs. According to Bakhtiar (2018: 149), “showing hayâ by means of avoiding violating social norms and values is to demonstrate that one possesses âberu”.

Figure 3. The host performs a different gesture when greeting female guests

Multimodal manifestation of ta’ârof in Persian

As the interaction between an unrelated man and a woman is subject to religious constraints, the host adopts gestures and postures that show limited movements. He tries to respect social distance (proxemics) with female guests and to have an upright posture (Figure 3) in order to avoid transgressing a religious taboo. We can even see in the second picture of Figure 3 that the host slightly leaned back to increase the distance between himself and the female guest. The body posture adopted by the host and his smaller gestures,6 compared to on-body gestures performed with male guests, are indicative of a self-lowering strategy to give more space to the guests. In fact, by adjusting his gesture to female guests, the host expresses his respect to them and at the same time, he wants to make a good impression and to communicate a positive image of himself (to the female guest but also to the TV audience) as a person who acts regarding hayâ. By adopting this behavior, he tries to bond not only with the women guests, but also with the audience that believes in religious taboos. The second result of our analysis is the occurrence of egocentric deictic gestures. As a matter of fact, the host points to the sofa which is the place of honor and invites the guest to sit there. The entrance being on the left side of the stage (Figure 1), the guests are always invited to sit on the sofa, which is the farthest place from the entrance, and best placed as it directly faces the audience. This other-raising strategy confirms the above-mentioned idea that the host honors the guests by offering them the best sitting place (Figure 4).

Figure 4. The egocentric deictic gesture performed by the host

An interesting example of egocentric deictic gesture was observed in an excerpt where a guest was already present on the stage when a second younger guest was invited to join. As we see in Figure 5, the first guest is seated on the left side of the sofa but when the second guest arrives, he performs an egocentric deictic gesture and offers him his place. We should add that the older guest is a retired football player and a coach whereas the younger guest is a young football player.

6. This refers to gesture zones (center-center, center, periphery, and extreme periphery) based on McNeill (1992).

175

176

Farbod Farahandouz & Shima Moallemi

Although the first guest is older and has a hierarchical position, as he was on the stage first, when the new guest arrives, he temporarily considers himself the host and tries to show his hospitality and respect toward the new younger guest by practicing an other-raising ta’ârof strategy. Thus, he tries to reduce the status differentiation due to the age factor and attempts to bond with the new guest. Furthermore, even though the two factors of age and experience put the first guest in a higher position, his attempt to reduce hierarchy shows his desire to communicate a positive image of himself as a humble and modest person. In return, the second guest invites the older one to sit in the place of honor and recognizes his higher social position, as is expected in the practice of ta’ârof. By not accepting the offer of the older guest, the second guest expresses his respect for him and acknowledges his attempt to bond with him.

Figure 5. The older guest offers his place on the sofa to the younger one. Before the arrival of the second guest (left) and after his arrival (right)

Another sequence of ta’ârof follows just before sitting when the younger guest performs an egocentric deictic gesture to invite his interlocutor to sit first (Figure 6). His gesture is verbally accompanied by befarmâ’id (‘please’) which is a typical other-raising utterance in this situation. The first guest replies also with the ritualized expression khâhesh mikonam (‘you’re welcome’) and the same egocentric deictic gesture. The younger guest sits first and then the older one sits down. By practicing ta’ârof in this situation, the younger guest is in a sense asking permission from his status superior interlocutor to sit down and, by doing so, expressing his respect to him. Actually, the fact that two sequences of ta’ârof follow in the opening sequence between the two guests, one initiated by the older guest and the second by the younger guest, can be indicative of their intention of bonding. The older guest tries to reduce the status differentiation and, reciprocally, the younger guest, by asking permission to sit, recognizes the status of his interlocutor. In our analysis, we also observed that the host performs on-person gestures with male guests as part of the gestural greeting. Specifically, he places his open

Multimodal manifestation of ta’ârof in Persian

Figure 6. The younger guest invites the older one to sit first

hand on the back of the male guests as soon as they arrive on the stage, with a long hold phase that lasts until sitting at the indicated place (Figure 7). It is also possible to consider this as a deictic posture, pointing to the place of honor, because of the long duration of the hold phase.

Figure 7. On-person gesture performed by the host

If touching was absent when greeting male guests, it could be interpreted as impoliteness or as not being warm enough. Actually, in Iranian culture, compared to western culture (French for example), physical contact between men is more common. Moallemi (2019) showed that in an intercultural context, Iranian young men in France complain that they don’t know how to express their friendship to a male friend or express their joy at finding a close friend after a long time. Multimodal factors such as vocal intonation and physical contact are mentioned as elements that play an important role in the “Iranian friendship” between men. This concerns egalitarian relationships, but it seems that the status differentiation frame does not indicate any prohibition of physical contact either (besides touching between unrelated men and women that is officially forbidden in public places), as the on-person gesture performed by the host on the older guest was not received as an offense but as showing his intention to take care of an older guest. This practice is different from what is accepted in Korean, for example, where it is mostly the superior that initiates other-touches, but inferiors rarely touch superiors (L. Brown and Winter 2019). A remarkable point about this gesture is the duration of it, performed by the host vis-à-vis the older guests. In order to show the duration of each performed

177

178

Farbod Farahandouz & Shima Moallemi

gesture, first, we used the ELAN annotation selection tool, then we entered the exact duration on an Excel table. The table contains two factors: one factor indicates the candidates’ gender and their age (under/over 60 years old) and the other is dedicated to the duration of the gesture. We finally used the automatic chart generator to export the following graph (Figure 8)

Figure 8. Duration of the physical hand contact for on-person gestures by second. M and F stand for gender (male and female) and +/− 60 shows the age (below or above 60 years old)

As is shown in Figure 8, the duration of the physical body contact of this gesture is longer (11.29 and 15.4 seconds) with older guests compared to younger interlocutors (2.01, 3.65, 4.29 seconds). This observation confirms the importance of age as a status differentiation factor in Persian. The duration of the on-person gesture in this situation also shows the necessity of using an honorific gesture to express and emphasize respect to an older interlocutor. In this situation, it’s the nonverbal taxème that helps the host to express more politeness. What we observe here is the performative function of a pragmatic gesture. In addition, giving attention to elders is very important in Iranian culture and can explain the duration of the gesture performed. Figure 8 also shows the absence of the on-person gesture with female guests. This is due to the religious law that limits physical contact between unrelated men and women. Besides the occurrence of on-person gestures in the greeting sequences, we analyzed a mid-conversation sequence between an older guest and the host of the show. This sequence does not represent a typical example of behavioral ta’ârof but

Multimodal manifestation of ta’ârof in Persian

it is interesting in terms of im/politeness. Since the guest is being rather talkative, the host has difficulties controlling the conversational turns but as the guest is in a higher social position due to his age, the host cannot interrupt him directly or he would be considered rude. The host, therefore, engages in an indirect strategy to communicate his desire to take back the speech turn: he moves to sit directly beside the guest and puts his hands on his knee (Figure 9). The guest understands the indirect request made by his interlocutor and introduces the lateral sequence, “yes, I will end it now”, before continuing with the rest of the story. It seems that this promise was not convincing for the host, who continues keeping his hands on the knee of the interlocutor. Observing that, the guest reacts by putting his hand on the wrist of his interlocutor in order to, indirectly, ask him to wait and let him finish his story.

Figure 9. On-person gestures performed by the host (left) and the guest (right)

In this excerpt, which is an illustration of the interactive function of gesture (Kendon 2004: 159), we observed a conflictual interaction engaged by the on-body gestures of the two interlocutors. Even though the host uses gestures as an indirect way to take back the speech turn, the face threat that it creates is considerable for the guest, as can be seen in his reaction. We can conclude that the degree of separation produced by this nonverbal exchange is high and the interlocutors do not reach any bonding.

5.

Conclusion

In this article, we studied an instance of the gestural manifestation of the politeness ritual of ta’ârof. The analyses of the greeting sequences in an Iranian talk show demonstrated the repetition of three gestures. In this regard, on-body, onperson, and egocentric deictic gestures were observed as parts of the practice of ta’ârof in greeting sequences. The current study found that, as nonverbal taxèmes, each of these gestures, along with the verbal ta’ârof, represents a deferential strategy. On-body gestures can be interpreted as a self-lowering strategy whereas

179

180

Farbod Farahandouz & Shima Moallemi

egocentric deictic and on-person gestures are representative of an other-raising strategy. On-person and egocentric deictic gestures contribute to the creation of bonding between the host and the guest of the talk show. However, on-person gesture (Figure 9), used as an indirect request in order to avoid interrupting an old guest, did not create any bonding while it contributed to separation between the interlocutors. Considering that, unlike the other sequences, this last example did not correspond to a typical ta’ârof interaction, we can question the relevance of this gesture as associated with politeness in situations that do not involve ta’ârof. In other words, we can suggest that on-person gestures can be considered polite only in situations that involve ta’ârof and in which these gestures are conventionalized to be an other-raising strategy. A possible explanation for this observation may be the place on which the host enacts the gesture. In Figure 7, the contact point (the host’s hand) is at the back or near the shoulder of the guests. Whereas, in the last sequence, shown in Figure 9, the contact point is the knee which is not conventionalized in the practice of ta’ârof. Egocentric deictic gestures are a part of the behavioral ta’ârof that consists of inviting a high-status guest to sit in the place of honor. In this situation, the gesture could be used individually or with the verbal ta’ârof and could be accompanied by the other-raising expression befarmâ’id (‘please’). In so far as on-body gestures contribute to the expression of ta’ârof and the meaning construction of the utterance, they either take part in the practice of politeness as a nonverbal taxème (self-lowering) or emphasize verbal politeness. In addition, concerning the host’s on-body gestures, we can note that there is a kind of gesture alignment with regard to the guest’s gender. The gender of the guest was observed to be a factor of difference in the data as the host did not perform any on-person gestures toward female guests and only rarely used on-body (on chest) gestures in welcoming them. Also, for on-person gestures, the difference in hold phase duration was an interesting factor that could vary for the host according to the age of the interlocutor. We can thus conclude that the factors of age and gender influence the gestural strategies used to express politeness in Persian, as the verbal phrases used by the host were the same for all guests independent of these two factors. This finding underlines the importance of the consideration of both linguistic and multimodal strategies in the study of the Persian politeness system of ta’ârof. A further quantitative investigation would be necessary in order to study the factors of gender and age in the practice of ta’ârof and gesture in relation to im/ politeness. Moreover, we believe that other multimodal aspects, such as vocal intonation and pitch, deserve to be studied in relation to genuine and ostensible offers and invitations in Persian.

Multimodal manifestation of ta’ârof in Persian

Bibliography Arundale, Robert B. 2010. “Constituting Face in Conversation: Face, Facework, and Interactional Achievement.” Journal of Pragmatics, Face in Interaction, 42 (8): 2078–2105. https://doi.org/10.1016/j.pragma.2009.12.021

Bakhtiar, Mohsen. 2018. “Emotion Concepts in Context: Figurative Conceptualizations of Hayâ ‘Self-Restraint’ in Persian.” In Linguistic Taboo Revisited, ed. by Andrea Pizarro Pedraza, 141–60. De Gruyter Mouton. https://doi.org/10.1515/9783110582758-008

Beeman, William O. 1986. Language, Status, and Power in Iran. Bloomington: Indiana University Press. Beeman, William O. 2020. “Ta’ārof: The Key to Iranian Social Behavior.” In Handbook of Pragmatics, ed. by Jan-Ola Östman and Jef Verschueren, 203–224, Amsterdam: John Benjamins. https://doi.org/10.1075/hop.22.taa1 Birdwhistell, Ray L. 1968. “L’analyse kinésique.” Translated by Michèle Lacoste. Langages 3 (10): 101–6. https://doi.org/10.3406/lgge.1968.2553 Bressem, Jana. 2013. “70. A Linguistic Perspective on the Notation of Form Features in Gestures.” In Body – Language – Communication. An International Handbook on Multimodality in Human Interaction, ed. by Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, and Sedinha Teßendorf, 1079–98. Berlin: De Gruyter Mouton. https://doi.org/10.1515/9783110261318.1079 Bressem, Jana, Silva H. Ladewig, and Cornelia Müller. 2013. “71. Linguistic Annotation System for Gestures”. In Body – Language – Communication. An International Handbook on Multimodality in Human Interaction, edited by Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, and Sedinha Teßendorf, 1098–1124. Berlin: De Gruyter Mouton. https://doi.org/10.1515/9783110261318.1098 Brown, Lucien. 2013. “‘Mind your own esteemed business’: Sarcastic Honorifics Use and Impoliteness in Korean TV Dramas.” Journal of Politeness Research, 9 (2): 159–186. https://doi.org/10.1515/pr-2013-0008

Brown, Lucien, and Pilar Prieto. 2017. “(Im)politeness: Prosody and Gesture.” In The Palgrave Handbook of Linguistic (Im)Politeness, ed. by Jonathan Culpeper, Michael Haugh, and Dániel Z. Kádár, 357–79. London: Palgrave Macmillan UK. https://doi.org/10.1057/978-1-137-37508-7_14

Brown, Lucien, and Bodo Winter. 2019. “Multimodal Indexicality in Korean: ‘Doing Deference’ and ‘Performing Intimacy’ through Nonverbal Behavior.” Journal of Politeness Research 15 (1): 25–54. https://doi.org/10.1515/pr-2016-0042 Brown, Penelope, and Stephen C. Levinson. 1987. Politeness: Some Universals in Language Usage. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511813085 Burgoon, Judee K., Laura K. Guerrero, and Kory Floyd. 2010. Nonverbal Communication. Boston: Taylor & Francis, Routledge. Dehkhoda, Ahmad, Jafar Shahidi, Hassan Ahmadi Givi, Mohammad Moein, and Hamid Hassani. 1998. Dehkhoda Dictionary (Vol. 15). Tehran: Rozaneh. Efron, David. 1972. Gesture, Race and Culture. The Hague: Mouton. Ekman, Paul, and Wallace V. Friesen. 1969. “The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding.” Semiotica 1 (1): 49–98. https://doi.org/10.1515/semi.1969.1.1.49

181

182

Farbod Farahandouz & Shima Moallemi

ELAN (Version 5.9) [Computer software]. 2020. Nijmegen: Max Planck Institute for Psycholinguistics, The Language Archive. Retrieved from https://archive.mpi.nl/tla/elan” Goffman, Erving. 1967. Interaction Ritual: Essays on Face-to-Face Behavior. New York: Doubleday. Hall, Edward Twitchell. 1971. La dimension cachée. Translated by Amélie Petita and Françoise Auteur de la postface Choay. Paris: Éditions du Seuil. Hall, Edward Twitchell. 1984. Le langage silencieux. Translated by Jean Mesrie and Barbara Niceall. Paris: Éditions du Seuil. Isaacs, Ellen A., and Herbert H. Clark. 1990. “Ostensible Invitations.” Language in Society 19 (4): 493–509. https://doi.org/10.1017/S0047404500014780 Izadi, Ahmad. 2015. “Persian Honorifics and Im/Politeness as Social Practice.” Journal of Pragmatics 85: 81–91. https://doi.org/10.1016/j.pragma.2015.06.002 Izadi, Ahmad. 2017. “Culture-Generality and Culture-Specificity of Face.” Pragmatics and Society 8 (2): 208–230. https://doi.org/10.1075/ps.8.2.03iza Kendon, Adam. 1972. “Some Relationships Between Body Motion and Speech: An Analysis of an Example.” In Studies in Dyadic Communication. Pergamon General Psychology Series, ed. by Aaron Siegman and Benjamin Pope, 177–210. New York: Pergamon Press. https://doi.org/10.1016/B978-0-08-015867-9.50013-7

Kendon, Adam. 2004. Gesture: Visible Action as Utterance. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511807572 Kerbrat-Orecchioni, Catherine. 1994. Les interactions verbales. [Verbal interactions], Tome 3. Paris: A. Colin. Knapp, Mark L., Judith A. Hall, and Terrence G. Horgan. 2014. Nonverbal Communication in Human Interaction. 8th ed. Boston, MA: Wadsworth Cengage Learning. Koutlaki, Sofia A. 2002. “Offers and Expressions of Thanks as Face Enhancing Acts: Tæ’arof in Persian.” Journal of Pragmatics 34 (12): 1733–1756. https://doi.org/10.1016/S0378-2166(01)00055-8

Lausberg, Hedda. 2013. Understanding Body Movement: A Guide to Empirical Research on Nonverbal Behaviour, with an Introduction to the NEUROGES® Coding System. Frankfurt am Main: Peter Lang. Lausberg, Hedda. 2019. The Neuroges Analysis System for Nonverbal Behavior and Gesture: The Complete Research Coding Manual Including an Interactive Video Learning Tool and Coding Template. Berlin: Peter Lang. https://doi.org/10.3726/b15103 Lausberg, Hedda, and Han Sloetjes. 2009. “Coding Gestural Behavior with the NEUROGESELAN System.” Behavior Research Methods. https://doi.org/10.3758/BRM.41.3.841 McKinnon, Sean, and Pilar Prieto. 2014. “The Role of Prosody and Gesture in the Perception of Mock Impoliteness.” Journal of Politeness Research 10 (2): 185–219. https://doi.org/10.1515/pr-2014-0009

McNeill, David. 1992. Hand and Mind: What Gestures Reveal about Thought. Chicago: University of Chicago Press. Miller, Corey, Rachel Strong, Mark Vinson, and Claudia M. Brugman. 2014. Ritualized Indirectness in Persian: Taarof and Related Strategies of Interpersonal Management. University of Maryland Center for Advanced Study of Language. https://www.casl.umd .edu/wp-content/uploads/2016/02/RITUALIZED-INDIRECTNESS-IN-PERSIAN-2014 .pdf

Multimodal manifestation of ta’ârof in Persian

Moallemi, Shima. 2019. “Expatriation En France et Développement de Compétences Pragmatiques (et) Interculturelles. D’une Enquête Auprès d’Iraniens Expatriés à l’expérimentation d’une Démarche Didactique Dans Des Instituts de FLE à Téhéran.” PhD dissertation, Sorbonne-Nouvelle. https://www.theses.fr/2019PA030029 Moallemi, Shima. 2020. “Les enjeux Identitaires du développement de la compétence pragmatique interculturelle chez les apprenants iraniens de FLE.” Le Langage et l’homme 1: 21–38. Moore, Nina-Jo. 2020. Nonverbal Communication: Studies and Applications. San Diego: Cognella Academic Publishing. Norris, Sigrid. 2019. Systematically Working with Multimodal Data: Research Methods in Multimodal Discourse Analysis. Hoboken, NJ: John Wiley & Sons. https://doi.org/10.1002/9781119168355

Richmond, Virginia P., James C. McCroskey, and Mark Hickson. 2012. Nonverbal Behavior in Interpersonal Relations. 7th ed. Boston: Pearson/Allyn & Bacon. Riegel, Martin, Jean-Christophe Pellat, and René Rioul. 1994. Grammaire méthodique du français. Paris: Presses universitaires de France. Sahragard, Rahman. 2003. “A Cultural Script Analysis of a Politeness Feature in Persian.” Conference talk presented in Proceeding of the 8th Conference of Pan-Pacific Association of Applied Linguistics, Kibi International University, Okayama. Sharifian, Farzad. 2007. “3. L1 Cultural Conceptualisations in L2 Learning.” In Applied Cultural Linguistics Implications for Second Language Learning and Intercultural Communication, ed. by Farzad Sharifian, and Gary B. Palmer, 33–51. John Benjamins Publishing Company. https://benjamins.com/catalog/celcr.7.04sha. https://doi.org/10.1075/celcr.7.04sha Sharifian, Farzad. 2011. Cultural Conceptualisations and Language: Theoretical Framework and Applications. Amsterdam, Netherlands. https://doi.org/10.1075/clscc.1 Yaqubi, Mojde. 2020. “Subtitling of Ostensible Speech Acts (OSAs): Towards Proposing a Guideline.” Revista Española de Lingüística Aplicada/Spanish Journal of Applied Linguistics 33 (2): 641–66. https://doi.org/10.1075/resla.18033.yaq

183

chapter 7

Within-speaker accommodation behavior in apology-centered interactions The role of socio-pragmatic factors Omnia Ibrahim1 & Iris Hübscher2, 3 1

Saarland University | 2 University of Zurich | 3 Zurich University of Applied Sciences

The present study explores the extent to which vocal accommodation in conflicting situations can be explained by socio-pragmatic factors such as interpersonal relationships (i.e., politeness) and pragmatic events occurring in apology-centered interactions. A total of 28 dyadic conversations where 14 target speakers (8 males and 6 females) interacted in apology-centered role-plays with a status superior and a friend interlocutor were annotated phonetically, orthographically, pragmatically and analyzed separately. Accommodation was measured with the difference-in-distance paradigm comparing the first and last 30% of the conversations. The main results indicate that the conversations with a status superior are characterized by more divergent behavior than the conversations with a friend. These findings suggest that interpersonal power dynamics between interlocutors could serve as a predictor for accommodation behavior. Keywords: vocal accommodation, F0 variability, interpersonal relationships, apology-centered interactions

1.

Introduction

Speakers exhibit a great deal of acoustic-phonetic variability due to both physiological (e.g. age, gender and vocal track size) and psychological bases (e.g. emotions, stress). Furthermore, in everyday life, speech dynamically varies according to different factors such as the situation (e.g. formal vs. informal), speech function (e.g. request or apology), topic, and interlocutors, amongst others (Leongómez et al. 2021). All of these factors lead to within-speaker variability and are known to serve a communicative purpose (see Cooke et al. 2014 for a review).

https://doi.org/10.1075/pbns.333.07ibr © 2023 John Benjamins Publishing Company

186

Omnia Ibrahim & Iris Hübscher

Moreover, when communicating, speakers constantly position themselves in relation to the ongoing social interaction. Depending on whether a person interacts with a senior or a peer, the socio-pragmatic qualities of their speech is adapted in one way or another (see e.g. Winter et al. 2021). This adaptation may lead to an increase or a decrease in the interlocutors’ speech similarity, which is known as accommodation/alignment. Ibrahim et al. (2019) suggested that accommodation between speakers is not necessarily a function of the duration of a conversation. Rather, situational factors, like winning a game, can have a greater effect on speakers’ convergence behavior. Indeed, accommodation has been proposed as a strategy to signal and manage the social distance between speakers (Giles et al. 1991). Furthermore, in negotiation interactions, Sagi and Diermeier (2017) observed that interlocutors who reach an agreement show a gradual increase in language similarity over the course of the negotiation. Importantly, speakers also modulate their voice to signal im/politeness related meanings. Studies have shown variable pitch modifications in polite speech. Ohala (1984: 327) proposed the frequency code hypothesis stating that social messages such as “deference, politeness, submission, lack of confidence, are signaled by high and/or rising F0 whereas assertiveness, authority, aggression, confidence, and threat are conveyed by low or falling F0”. According to this hypothesis, a highpitched voice becomes a universal signal for politeness, deference and submission through the association between high pitch and small body size (e.g., the voice of a child). Following studies have, however, produced mixed results and have only found partial support for the relationship between politeness and high pitch. While a number of studies have demonstrated support for this hypothesis (e.g., Caballero et al. 2018, for Canadian English; Loveday 1981 for Japanese; Orozco 2008 for Spanish), several recent studies have shown that the overall pitch height was actually lowered in order to signal politeness (Grawunder et al. 2014 for German; Hübscher et al. 2017 for Catalan; Oh and Cui 2020 for Chinese, Sherr-Ziarko 2019 for Japanese; Winter and Grawunder 2012 for Korean; see also Winter at al. 2021 for a meta-analytic review). Most of the above-mentioned studies have investigated the relation between pitch height and politeness in the speech act of request; however, so far to our knowledge little attention has been paid to another speech act, namely apologies. Similar to requests, apologies have been studied intensively in association with im/politeness (see e.g. Zhang 2001; Harris et al. 2006; Kasanga and Lwanga-Lumu 2007; Jucker 2018; Aijmer 2019). Holmes (1998: 217) stated that “the apology is quintessentially a politeness strategy”. Interpersonal apologies involve the speaker acknowledging that something has gone wrong and needs to be put right (Deutschmann 2003). Blum-Kulka and Olshtain (1984) going back to Olshtain and Cohen (1983) identified the main structural components of an apology as (A) an expression of responsibility/blame

Within-speaker accommodation behavior in apology-centered interactions

(B) an explanation or account (C) an offer of repair and (D) a promise of forbearance. Many subsequent categorizations of apologies are based on these early works but also have been adapted and developed (see e.g., Deutschmann 2003). At this moment, it is still unclear to what extent politeness-related meanings such as social distance have an effect on the accommodation behavior of interlocutors in a situation which requires an apology. Hence in the view of apology as a politeness strategy, the present study analyzes within-speaker variations of accommodation behavior in Catalan dyadic apology-centered conversations, and addresses the following questions: a. To what extent do speakers change their F0 values in an apology-centered interaction depending on whether they interact with a status superior versus a friend (+ / – social distance)? b. Do speakers show different F0 accommodation behavior (convergence or divergence) depending on the social distance between the interlocutors? c. Will the observed patterns differ as a function of the speaker’s gender (male vs. female)?

2.

Background

2.1 Vocal accommodation behavior Humans respond more positively to people who are similar to their personality as claimed by similarity-attraction theory (Reis 2007). For that reason, when people engage in speech interaction, they tend to align their vocal characteristics with those of their interlocutors either by becoming more similar (convergence) or less similar (divergence) to each other. The tendency for interlocutors to become more similar in their speaking behavior has been referred to in various disciplines as convergence (e.g. Pardo 2006), entrainment (e.g. Levitan and Hirschberg 2011), alignment (e.g. Pickering and Garrod 2004, 2013, 2021), accommodation (e.g. Giles et al. 1991; Shepard et al. 2001), or adaptation (e.g. Bell et al. 2003). In the present study, we use the term “convergence”, defined as a strategy used to adapt to another person’s behavior while “divergence” is used to accentuate the verbal and multimodal differences between communicators. According to communication accommodation theory, convergence between conversational partners is somehow expected over the course of the conversation (Giles et al. 1991). Numerous studies view convergence as a default and sometimes uncontrolled behavior during the conversation (Bourhis 1977; Delvaux and Soquet 2007; Babel 2012). Lewandowski (2012) found that native English speakers

187

188

Omnia Ibrahim & Iris Hübscher

still converge toward their native German-speaking interlocutors’ accents even though they have been explicitly instructed not to change their pronunciation to accommodate to their interlocutors’ non-native accents. Moreover, Brennan and Clark (1996) discovered that speakers design their speech specifically for their conversational partners, and they adapt to their interlocutor’s new conceptualization of objects over the course of a conversation (Brennan 1991).

2.2 Function of accommodation behavior Studies have shown that similarity in language use is indicative of the conceptual alignment of interlocutors (Pickering and Garrod 2004, 2021). Furthermore, convergence has been associated with rapport, trust, and communicative efficiency; therefore, it appears to facilitate conversational success while phenomena like divergence are typically received negatively (Giles et al. 1991). Miles et al. (2009) argue that interactional accommodation reduces working memory load and facilitates information flow because freed-up cognitive resources can be directed toward the perception of the other person. Our perception of the accommodation behavior of our interlocutors will determine how we interpret and evaluate the conversation. It is thought to contribute to setting up a conversational common ground between the speakers and to facilitate mutual understanding. Convergence has been observed especially when cooperation between participants is desirable (Manson et al. 2013), such as goal-oriented interactions when describing a map or playing a game (Pardo 2006; Reichel et al. 2018), or in the speech of married couples engaging in marital therapy sessions (Lee et al. 2010). Indeed, interactive spontaneous conversation settings, along with social motivation and focus on the task, appear to provide most naturalistic opportunities for accommodation behavior (Coles-Harris 2017).

2.3 Factors affecting accommodation behavior There is mixed evidence whether accommodation between interlocutors increases in a linear fashion through the interaction’s duration, or whether it has more dynamic characteristics, but studies have claimed that accommodation increases with the duration of an interaction (Heath 2017). Episodic models of speech production (Pierrehumbert 2001) were proposed to explain the speakers’ ability to accommodate over the course of the conversation (Heath 2017); according to that model when we perceive an instance of a particular speech sound, it becomes part of the definition of that sound in our mind; and the following productions of instances of that sound are influenced by the sound’s new definition (Pierrehumbert 2001). Other studies have questioned this view of accommodation

Within-speaker accommodation behavior in apology-centered interactions

phenomena. Bane et al. (2012) showed in their study of voice onset time accommodation that the convergence in the speakers’ vocal characteristics is not in a single direction throughout the conversation’s duration. This suggests that accommodation might not be linear over time and is conditioned by the social roles of the speakers. One potential reason for this variation in the degree of accommodation is due to social factors, such as social characteristics and the relationship between the participant and interlocutor or model talker (Pardo et al. 2012). Relevant factors include gender (Pardo et al. 2012), dialect (Giles et al. 1991; Drager et al. 2010), interlocutor status (Bane et al. 2012), and attitude towards the model talker (Babel 2012). Situational factors (such as effects of the conversational topic or task) also contribute to the degree of accommodation between speakers in human-human and human-machine interactions (Cohen Priva and Sanker 2018; Levitan et al. 2012). In the current paper, the influence of social distance between the interlocutors will be examined in more detail. We expect more divergent behavior when our participants interact with a status superior interlocutor, and convergence or maintenance behavior with a same-status interlocutor (friend). Based on previous studies, females have shown more convergence behavior than male speakers (Reichel et al. 2018; Weise et al. 2019). Therefore, we expect to observe more convergence from our female speakers compared to our male speakers.

2.4 Accommodation effects Accommodation phenomena are manifested in various levels both verbal (e.g., spoken and written conversations) and multimodal (i.e., mirroring the interlocutor’s facial expressions or gestures (Rasenberg et al. 2020). It is exhibited in the speaker’s choice of referring expressions (Brennan and Clark 1996), syntax (Branigan et al. 2000; Reitter et al. 2006), linguistic style (Niederhoffer et al. 2002; Danescu-Niculescu-Mizil et al. 2011), and pronunciation (Pardo 2006) in humancomputer as well as human-human conversation (Bell et al. 2000; Bell et al., 2003; Coulston et al. 2002; Brennan 1991; Brennan and Clark 1996; Thomason et al. 2013). In the acoustic domain, accommodation has been investigated through multiple variables, such as speaking rate (Pardo 2006), intensity (Natale 1975), fundamental frequency (F0) (Kurtić and Gorisch 2018; Zellers and Schweitzer 2017), or the interaction between those features (Levitan and Hirschberg 2011). In the present work, we focused on fundamental frequency (F0) for its function for both linguistic information and communicative cues associated with the speaker’s age, gender, and/or emotional state.

189

190

Omnia Ibrahim & Iris Hübscher

It was reported that accommodation in F0 is more frequent in conversations than accommodation of other prosodic parameters (De Looze et al. 2014). Furthermore, listeners typically adapt more rapidly to their conversational partners’ F0 (Zellers and Schweitzer 2017). For example, listeners are able to identify the location of F0 values relative to an individual speaker’s range (Honorof and Whalen 2005). This may be a result of listeners’ expectations regarding average F0 for different sex’s voices (Bishop and Keating 2012). It has also been found that F0 accommodation has a conversational function in turn-taking in overlapping speech (Kurtić and Gorisch 2018; Zellers and Schweitzer 2017; Levitan et al. 2015a). F0 is a crucial interactive conversation structuring parameter. In their recent work, Kurtić and Gorisch (2018) investigated the relation between F0 accommodation and turn-taking using the same two approaches previously proposed by Zellers and Schweitzer (2017), F0 initializing (local context with adjacent turns) and normalizing (model of other speaker’s F0 norms). They found that F0 accommodation is only relevant as a turn competitive resource in overlaps that start clearly before a speaker transition. Their results suggested that both F0 initialization and normalization take place when speakers compete for the turn taking. Accommodation has been measured along two primary timescales referred to as global and local; global accommodation is measuring accommodation at the level of the conversation by comparing the beginning to the end, while local accommodation is measuring the accommodation behavior in each turnswitching utterance (Levitan and Hirschberg 2011; Lee et al. 2014). In the current paper, we will focus on the global scale of accommodation at the conversation level. The traditional method of measuring accommodation is difference of distance (DID), which is defined as the change in the absolute distances between the interlocutors (i.e., the speaker and their interlocutor or model talker). Studies usually use two different recording tasks for researching accommodation: shadow tasks and conversational interactions. In shadow tasks, speakers are exposed to recordings of a model talker, which they repeat after, and a comparison is made of their speech before and after exposure (Pardo et al. 2013; Babel and Bulatov 2012). On the other hand, for more natural settings, researchers extract the accommodation from the conversational interactions (beginning and end, or different turns). This presents additional complications in defining reference points for the speakers because both of the interlocutors are potentially changing (Pardo et al. 2018). In respect to the studied languages, cross-linguistic evidence of accommodation behavior has been observed, where speakers become more similar to their interlocutors in the local (Levitan et al. 2015b) and global (Beňuš et al. 2014) domain of their conversations. Despite accommodation studies having been

Within-speaker accommodation behavior in apology-centered interactions

heavily focused on the English language (Lehnert-LeHouillier et al. 2020; Babel et al. 2014; Levitan et al. 2012; Gregory and Webster 1996), empirical evidence of accommodation has also been documented in numerous languages: French (Bailly and Martin 2014), German (Gessinger et al. 2021; Michalsky and Schoormann 2017; Schweitzer et al. 2017; Schweitzer and Lewandowski 2014), Italian (Savino et al. 2016), Japanese (De Looze et al. 2014), Polish (Karpiński et al. 2014), Slovak (Beňuš et al. 2014), and Swedish (Ibrahim et al. 2019; Edlund et al. 2009). Together, these studies indicate that accommodation behavior is a universal and language independent phenomenon. However, for better understanding of the phenomenon, it is crucial to investigate accommodation in more languages and in different social situations. To the best of our knowledge, the present study is the first to investigate vocal accommodation in Catalan.

3. Methods 3.1 Participants Fourteen Catalan-speakers (eight males and six females with an age range from 18 to 24) participated in this experimental task as the main speakers, along with fourteen participants as the same status (friend) interlocutors (same gender as the main speaker) and one male speaker acted as the status superior interlocutor (the same person in all conversations). They were all students at the Universitat Pompeu Fabra in Barcelona and native speakers of Catalan. All the speakers considered Catalan to be their dominant language relative to Spanish (mean percentage of Catalan in total daily language use = 75.83%; SD = 13.62). Participants received a token payment for their participation. No participant reported any speech or language disorders.

3.2 Stimuli For this study, we used 28 role-play conversations extracted from the corpus used by Brown et al. (this volume) and Brown et al. (forthcoming). The corpus contains audiovisual recordings in a soundproof room (video camera: Panasonic HPX-171 and clip-on microphones: Sennheiser EW100). Each main speaker attended two data collection sessions with a different interlocutor (the order was counterbalanced): A) A session with a friend who the participants knew well (same gender, similar age), B) A session with a status superior (professor) who the participants have not met previously. In the complete corpus, each session included four different interactional tasks. First, the participants held an unstruc-

191

192

Omnia Ibrahim & Iris Hübscher

tured conversation about movies that they had recently watched. Second, the main participant performed a cartoon retelling (the Tweety Bird cartoon Canary Row). Third, they performed a map task, where the direction giver had to explain a route between two points on a map and the direction taker had to track the exact route described to them. Finally, participants performed a role-play originally used in Brown (2011), which is the task that will be analyzed in the present paper. In this role-play the main participants had to apologize for breaking their partner’s camera (friend condition) or for losing a partner’s rare book (superior condition).

3.3 Annotation To be able to conduct a comprehensive analysis of vocal accommodation, all 28 conversations underwent three types of annotations: phonetic, orthographic and pragmatic. While the visual part of the role-play was not used in the current study, we still decided to show the depth of the visual annotation undertaken in the corpus, as it is particularly interesting for future work on multimodal accommodation studies (Table 1 and Figure 1). Table 1. Description of tiers annotated for both interlocutors separately Tier number

Annotation type

Description

Tier 1 Tier 2 Tier 3 Tier 4 Tier 5 Tier 6

Orthographic Head gesture Eye gaze Gesture type Vertical Transversal

Tier 7

Saggital

Tier 8

Hand shape

Tier 9 Tier 10 Tier 11

Hand(s) Adaptor Pragmatic annotation Phonetic annotation

Orthographic transcription Head nods, head shakes and tilts Averted eye gaze vs. gaze directed at the interlocutor Deictic, iconic, metaphoric and beat Large (=peak above chest) vs. small (peak below chest) Large (=peak beyond shoulders) vs. small (=within shoulder length) Large (=peak to the right of mid-point) vs. small (=peak to the left of mid-point) Closed (=at least three fingers being pressed together) vs. open (=everything else) Right hand, left hand, both hands Description of body part touched see Table 2

Tier 12

see Figure 2

Within-speaker accommodation behavior in apology-centered interactions

Figure 1. An example of the multiple level of annotation of the corpus in ELAN software of the main speaker (parallel to this, the interlocutor’s speech and gesture is annotated)

Figure 1 displays a screenshot of the very end of a conversation when the main speaker apologizes one more time for losing the book in the metro, by saying Ho sento molt, eh. De veritat (‘I’m very sorry. Really.’ ). While he says this in the beginning, he touches both hands, then produces a small beat gesture and also concurrently a small head shake. The pragmatic event is an expression of apology (see Table 2). For the phonetic annotation, the corpus has been automatically annotated for words, syllables and phonemes using WebMAUS (Kisler et al. 2017) for the Catalan language. Then, we performed manual revisions and corrections of boundaries and labels to guarantee maximum precision using Praat (Boersma and Weenink 2020) (see Figure 2). For the pragmatic annotation, we distinguished between ten different labels which helped us to roughly distinguish between different pragmatic acts. We expanded the traditional classification of apology strategies by Blum-Kulka and Olshtain (1984) with novel labels (such as assent vs. dissent) for coding the reactions of the apology receiver and the negotiation strategies used to solve the conflict. There are two types of potential apologies: being late for the meeting labeled as A, and the main apology for losing the book or breaking the camera labeled as B. In our conversations, each utterance was labeled by one of these pragmatic events described in Table 1. The events are not sequential as they highly depend on the dynamics of the conversation. In Table 1, all labels in the column on the left

193

194

Omnia Ibrahim & Iris Hübscher

were used by the main speaker only, as they are the ones who express the apology, while the labels in the column on the right were used by both speakers and their interlocutors. Table 2. Labels used for the pragmatic annotation with corresponding examples Events expressed by main speakers only

Events expressed by main speakers and their interlocutors

A1/B1 Expression of apology Ho sento (‘I’m sorry’)

A6/B6 Request for information Què ha passat? (‘What happened?’)

A2/B2 Acknowledgment of responsibility És culpa meva totalment (‘It is entirely my fault’)

A7/B7 Request for action Però clar, ara hem de decidir algo, què fem? (‘But of course, now we have to decide something, what do we do?’)

A3/B3 Explanation or account estava parlant amb un profe (‘I was talking to a professor’)

A8/B8 Expressing emotions (surprise/disbelief/ anger) Pff, és que era una caméra que la tenia fa molt de temps, tio, quina rabia. (‘Buff, it was a camera which I had since a long time, dude, what a rage’)

A4/B4 Offer of repair Jo t’ho dic que t’ho pago jo (‘I’m telling you that I pay for it’)

A9/B9 Assent Accepto el teu odi cap a mi (‘I accept your hatred against me’)

A5/B5 Promise of non-occurrence Intentaré que no torni a passar (‘I try to not let it happen again’)

A10/B10 Dissent (positive/ negative) Moltes gràcies, però que sigui l’últim cop que arribes tard (‘Thank you very much but this is the last time that you arrive late’)

Figure 2. Example of phonetic transcription output in Praat software

Within-speaker accommodation behavior in apology-centered interactions

3.4 Measuring accommodation For the data analysis, the fundamental frequency median values of the interlocutors were automatically extracted (formula: “To Pitch: 0, 75, 500”) using the Praat software (version 6.1.14). The F0 values were measured separately for each voice interval. Before measuring accommodation, raw F0 values in the contours were converted to the octave-median (OMe) scale, proposed by (De Looze and Hirst 2014), defined by the formula: OMe = log2 (Hz/median); where median corresponds to the median value of F0 for the recording. In the current paper, we measured accommodation using the difference-ofdistance (DID) method. We used k-sample versions of the Anderson-Darling (AD) test, which is a nonparametric test (distribution free test) that does not assume that data are coming from normal distributions. It helps in measuring the similarity between the interlocutors’ values and captures accommodation broadly. Other researchers have used t-tests, which determine if there is a significant difference between the means of two interlocutors’ distributions (Levitan and Hirschberg 2011). The described measure assumes that the extracted acoustic values are normally distributed. Unfortunately, the F0 values are often not well described by a statistically normal distribution (Simpson 2009; Traunmüller and Eriksson 1995). The mean value over a large period of time has been shown to not be a suitable measure for acoustic features and to be somehow misleading. If F0 is scaled linearly (in Hz), there is some positive skewness (Mikeev 1971). In addition, it has been observed that some speakers show a bimodal F0-distribution, in particular when speaking with increased vocal effort (Rappaport 1958). Furthermore, in their recent work, Cohen Priva and Sanker (2019) demonstrate a limitation of using the t-test for DID as it is based on comparing the distance without reference to the raw values for each speaker. DID measures require one single value to represent the acoustic feature, which is usually the mean value of the speaker’s voice over the span of time. In order to measure accommodation at the conversation level (Figure 3), we attempted to identify cases in which speakers’ distributions were more similar to each other later in the conversation. For our analysis, we used two parts of the conversation: the first 30% (where we expected there to be less accommodation) and the final 30% (where we expected more accommodation). For each part, we compared the F0 distributions of the two interlocutors using the AndersonDarling (AD) measure, which helps to capture convergence broadly. This test is implemented in the kSamples R package (Scholz and Zhu 2017). The larger the AD test value, the more different the two distributions are considered to be and vice versa. Accommodation was measured with the difference-in-distance paradigm comparing the first and last 30% of the conversations. The difference in

195

196

Omnia Ibrahim & Iris Hübscher

distance (DID) between the conversation’s beginning score and the conversation’s final score was computed as a measure of accommodation (accommodation = final score – beginning score). We inferred convergence when the results are negative.

Figure 3. Measuring vocal accommodation using difference in distance method (DID), we calculated the similarity between the two interlocutor’s F0 values in the first 30% and last 30% of the conversation

3.5 Statistical analysis Extracted acoustic features were statistically analyzed by fitting linear mixed effects models (LME) using the lmer package (Bates et al. 2015) in R (R Core Team 2018) and evaluating model fits using the lmerTest package (Kuznetsova et al. 2017). A linear mixed-effects model was constructed for each acoustic feature as a dependent variable. Interlocutor (talking to friend or to status superior) and speaker’s gender (male or female) were the fixed effects and coded as simple contrasts, while speaker and item were the random factors. In case of convergence, errors or singularity (i.e. overfit), the maximal random structure was reduced stepwise. The significance of fixed effects was evaluated by performing maximum likelihood t-tests using Satterthwaite approximations to degrees of freedom. After comparison of the models, the final

Within-speaker accommodation behavior in apology-centered interactions

model for F0 was an interaction model: lmer(F0_mean ~ Interlocutor * Gender + (1 | Speaker) + (1 | Item)), while the final model for F0 variability (standard deviation) was a simple model: lmer(F0_stdev ~ Interlocutor + Gender + (1 | Speaker) + (1 | Item)).

4. Results In this section, we first statistically analyzed the effect of social distance (status superior vs. friend) on the F0 variations of our target Catalan speakers. Second, we explored the target speakers’ F0 accommodation towards their interlocutors. Third, we investigated the impact of the speakers’ gender on their accommodation behavior. Finally, we looked at the accommodation of individual speakers.

4.1 F0 variation depending on social distance between the interlocutors To answer the first research question, i.e., to what extent F0 changes as a function of the interlocutor’s status, we analyzed the F0 and F0 variability of our target speakers when they speak to a friend (F) and to a status superior (P) (see Figure 4 and Figure 5). As expected, most target speakers (grey lines) modulate their F0 pattern depending on the social distance: the target speakers tend to speak with lower F0 in asymmetric conversations with the status superior. More interestingly, we found an interaction between interlocutor (F, P) and gender: the magnitude of change performed by our female speakers was steeper than the male speakers (see Figure 4). The results of the LMM for F0 and F0 variability are presented in Table 3. The fixed effects results revealed a main effect for interlocutor’s status and gender (< 0.0001 ***) on F0 and F0 variability. Only F0 showed an interaction effect between the interlocutor and gender.

4.2 F0 accommodation behavior as a function of social distance In the previous section we found that speakers modulate their F0 in response to the social distance between themselves and their interlocutor. In the current section, we explored the target speakers’ F0 accommodation towards their interlocutors. Figure 6 shows the percentage of observed vocal accommodation in our 28 analyzed conversations: Convergence (red), where speakers become more similar, maintenance (green), where each speaker maintains their own F0 values and divergence (blue), where speakers become more different by the end of conversation. In both F0 and F0 variability, we found a divergence behavior in socially dis-

197

198

Omnia Ibrahim & Iris Hübscher

Figure 4. F0 values of our target speakers when speaking to a friend or status superior, the gray lines represent mean values of each individual speaker

Figure 5. F0 variability of our target speakers when speaking to a friend or to a status superior, the gray lines represent mean values of each individual speaker

Within-speaker accommodation behavior in apology-centered interactions

Table 3. Results of linear mixed effects models for F0 and F0 variability Feature F0 (interaction model)

F0 variability (simple model)

Coeff.

St.Error

T-value

p-value

Interlocutor (Status superior)

−56.416

1.176

−47.971

  < 0.0001 ***

Gender (male)

−69.369

8.584

 −8.081

  < 0.0001 ***

Status superior*male

 44.967

1.629

 27.602

  < 0.0001 ***

Interlocutor (Status superior)

  −3.3408

 0.2622

−12.743

  < 0.0001 ***

Gender (male)

  −3.0950

 1.0309

 −3.002

< 0.001 **

tant interactions. More than 70% of conversations with the status superior showed divergence, while we found more variable accommodation behavior (in F0) with a preference for convergence (in F0 variability) in the peer (friend) conversations.

Figure 6. Percentage of vocal accommodation in our analyzed conversations: F0 (left) and F0 variability (right)

4.3 F0 accommodation behavior as a function of speaker gender Figures 7 and 8 show the vocal accommodation patterns (convergence, maintenance or divergence) in our conversations but divided by gender: males (N = 16) and females (N = 12). Males’ conversations showed similar patterns as the general observations in both F0 and F0 variability displayed in the previous section. In females’ conversations with friends, we did not find any convergence at all; mainly they maintained their F0 style across the conversations. For conversations with

199

200

Omnia Ibrahim & Iris Hübscher

the status superior, they showed divergence. Interestingly, females showed the exact same accommodation patterns (33.3% convergence, 16.7% maintenance and 50% divergence) in their F0 variability when interacting with their friends or the status superior.

Figure 7. Percentage of F0 accommodation in conversations between males (left) and conversation between females (right)

Figure 8. Percentage of F0 variability accommodation in conversations between males (left) and conversation between females (right)

Within-speaker accommodation behavior in apology-centered interactions

4.4 Within-speaker analysis of accommodation To get an impression of the individual accommodation behavior with the two different interlocutors, we looked at the variation of accommodation states (convergence, maintenance, or divergence) of each target speaker when they talk with a friend or a status superior. Figure 9 shows the 14 target speakers and their individual accommodation behavior. 65% of our speakers show variability in their accommodation behaviors when they speak to a friend and to a status superior, while 35% show the same pattern between the two interlocutors.

Figure 9. F0 accommodation of individual target speakers. In the middle are 14 target speakers (TS) and linked to right conversations with status superior and left conversations with friends. The links represent the accommodation status in the conversations. Blue means the TS has the same accommodation status in both conversations, while red means different status

Let us take a closer look at target speaker number 1 as an example of the dynamics of accommodation during the conversation when he speaks with the status superior (professor) and when he speaks with the same status interlocutor (his friend). Figure 10 displays his conversation with the status superior. The global accommodation state in this conversation is divergence, as the target speaker and the status superior have become more different at the end than at the beginning. When adding the pragmatic events during the conversation from both interlocutors, we can observe that our interlocutors get similar really quickly

201

202

Omnia Ibrahim & Iris Hübscher

when the target speaker expresses his apology before they diverge again towards the end.

Figure 10. F0 values of our first target speaker when he was talking to the status superior (Divergence)

In the following conversation with his friend (Figure 11), our target speaker produces a typical sequence of apology, which has been found often in our data: explain – apology – offer of repair – negotiate. However, surprisingly, despite the fact that his friend expresses his disbelief and anger, the two interlocutors maintain their vocal similarity and show convergence.

5.

Discussion and conclusion

The present study explored within-speaker vocal accommodation in twenty-eight dyadic apology-centered conversations. Each speaker had two conversations, one with a status superior and one with a same-status interlocutor. We first statistically analyzed the effect of the interlocutors’ status (status superior vs. friend) on the F0 variations of our target Catalan speakers. We found that our target speakers tend to speak with lower F0 in asymmetric conversations with superiors. This result is in line with previous research on the effect of social distance on F0 in Catalan (Hübscher et al. 2017) and a similar trend has also been shown in other languages (e.g., Grawunder et al. 2014 for German; Oh and Cui 2020 for Chinese; SherrZiarko 2019 for Japanese; Winter and Grawunder 2012 for Korean). As pointed out by Winter at al. (2021), while no solid conclusions can be drawn as a variety of

Within-speaker accommodation behavior in apology-centered interactions

Figure 11. F0 values of our first target speaker when he was talking to a friend (Convergence)

different tasks have been used, the results do indeed show that using high pitch for politeness is not a universal as proposed by Ohala (1984) through his frequency code hypothesis and which often still has been assumed as a default. Importantly, in the present study, we were able to show that in Catalan the lowering of F0 when talking to a status superior does not only apply in data elicited through an oral discourse elicitation task (Hübscher et al. 2017) but also in an interactive setting with an actual status superior interlocutor present. Furthermore, interestingly, we found an interaction between the interlocutor (status superior, friend) and gender: the magnitude of change of our female speakers was steeper than the male speakers. One potential explanation is that the difference is not due to the gender of the speaker, but due to whether the interactions are same-gender or different-gender. Female speakers had a female friend and a male status superior interlocutor, while for male speakers they had conversations with a male friend and a male status superior. Studies have shown that team composition influences the direction and magnitude of the accommodation behavior observed (Reichel et al. 2018; Weise et al. 2019). In relation to the second and third questions, we further focused on conversation-level accommodation, where we compared the speakers’ similarity during the first and last 30% parts of the conversations. We would expect divergence behavior of our target speakers with a status superior who is of higher status, older and unfamiliar to the main speaker, while with a same status interlocutor, friend, who is of same age/gender and familiar to the main speaker, we would expect more convergence behavior especially after they reach agreement. It

203

204

Omnia Ibrahim & Iris Hübscher

should be noted that vocal accommodation effects reported in previous literature vary widely in magnitude. In the study by Schweitzer and Lewandowski (2014), participants showed more convergence towards interlocutors perceived as more competent as compared to those rated as more friendly, while in our study, the conversations with the status superior were characterized with more divergence than conversations among friends. A potential reason is that our target speakers are familiar with their friends, and this result is an effect of long-term convergence. In female conversations, we found a tendency to diverge more or at least maintain F0 levels, compared to male speakers’ conversations. Phonetic alignment or convergence is generally considered to be a method for building rapport and has been shown to occur more often during positive interactions (Lee et al. 2010). Relatedly, divergence has been described as a technique for accentuating individual and cultural differences as well as expressing disdain towards the interlocutor (Bourhis 1977). It is plausible that the type of task under study (apology) may have caused our target speakers to react negatively towards the interlocutor. Apologies are seen as a threat to the speaker’s own positive face since the apologizer admits to having done something untoward (Grainger and Harris 2007). In individual speakers’ cases, 64% of our target speakers show variability in their accommodation states when they speak to a friend and to a status superior. This finding aligns with the observation of individual differences in the pattern of phonetic convergence (e.g. Pardo et al. 2016). Although the current study is based on a small sample of speakers, we conclude that interpersonal power dynamics between interlocutors could serve as a predictor for accommodation behavior. Possible limitations of the present study are that (1) our results are limited to one language/culture and therefore cannot be generalized. Also, (2) this study focused on global conversation level accommodation behavior, and we observed a noticeable variability over the course of a conversation which is not interpretable only in terms of social distance. Further research is thus needed to explore the dynamics of accommodation locally within the conversation in relation to different pragmatic events and across different languages/cultures. Finally, our study contributes to the broader literature on accommodation by investigating a different language (Catalan) and by having implications for how we understand the apology speech act within politeness research. Furthermore, the study of within-speaker variability, in response to the interlocutor or speech act, is of particular interest to the field of forensic phonetics since samples in forensic speaker comparisons tend to be mismatched for speaking situation and style.

Within-speaker accommodation behavior in apology-centered interactions

Funding This research was supported by the innovation pool 2020 of the University Research Priority Program (URPP) “Language and Space”, and by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – Project-ID 232722074 – SFB 1102.

Acknowledgments We would like to thank Elisa Pellegrino for her valuable input during the development of this project and the two co-editors of this volume, Andreas H. Jucker and Lucien Brown for their very helpful suggestions and improvements of a previous version of this contribution.

References Aijmer, Karin. 2019. “‘Ooh whoops I’m sorry!’ Teenagers’ Use of English Apology Expressions.” Journal of Pragmatics 142: 258–269. https://doi.org/10.1016/j.pragma.2018.10.017 Babel, Molly. 2012. “Evidence for Phonetic and Social Selectivity in Spontaneous Phonetic Imitation.” Journal of Phonetics 40 (1): 177–189. https://doi.org/10.1016/j.wocn.2011.09.001 Babel, Molly, and Dasha Bulatov. 2012. “The Role of Fundamental Frequency in Phonetic Accommodation.” Language and Speech 55 (2): 231–248. https://doi.org/10.1177/0023830911417695 PMID: 22783633 Babel, Molly, Grant McGuire, Sophia Walters, and Alice Nicholls. 2014. “Novelty and Social Preference in Phonetic Accommodation.” Laboratory Phonology 5 (1): 123–150. https://doi.org/10.1515/lp-2014-0006

Bailly, Gerard, and Amelie Martin. 2014. “Assessing Objective Characterizations of Phonetic Convergence.” In Proceedings of Interspeech, Singapore, 2011–2015. https://doi.org/10.21437/Interspeech.2014-458

Bane, Max, Peter Graff, and Morgan Sonderegger. 2012. “Longitudinal Phonetic Variation in a Closed System.” The 46th Chicago Linguistics Society. Bates, Douglas, Martin Mächler, Ben Bolker, and Steve Walker. 2015. “Fitting Linear MixedEffects Models Using lme4.” Journal of Statistical Software 67 (1): 1–48. https://doi.org/10.18637/jss.v067.i01

Bell, Linda, Johan Boye, Joakim Gustafson, and Mats Wirén. 2000. “Modality Convergence in a Multimodal Dialogue System.” In Proceedings of Gtalog 2000, Fourth Workshop on the Semantics and Pragmatics of Dialogue, 29–34. Bell, Linda, Joakim Gustafson, and Mattias Heldner. 2003. “Prosodic Adaptation in HumanComputer Interaction.” In Proceedings of ICPHS’03, 833–836. Beňuš, Štefan, Agustín Gravano, Rivka Levitan, Sarah Ita Levitan, Laura Willson, and Julia Hirschberg. 2014. “Entrainment, Dominance and Alliance in Supreme Court Hearings.” Knowledge Based System 7: 3–14. https://doi.org/10.1016/j.knosys.2014.05.020 Bishop, Jason, and Patricia Keating. 2012. “Perception of Pitch Location within a Speaker’s Range: Fundamental Frequency, Voice Quality and Speaker Sex.” The Journal of the Acoustical Society of America 132 (2): 1100–1112. https://doi.org/10.1121/1.4714351

205

206

Omnia Ibrahim & Iris Hübscher

Blum-Kulka, Shoshana, and Elite Olshtain. 1984. “Requests and Apologies: A Cross-cultural Study of Speech Act Realization Patterns (CCSARP).” Applied Linguistics 5 (3): 196–213. https://doi.org/10.1093/applin/5.3.196

Boersma, Paul, and David Weenink. 2020. “Praat: Doing Phonetics by Computer [Computer Program].” Version 6.1.08, from http://www.praat.org/ Bourhis, Richard Y. 1977. “The Language of Intergroup Distinctiveness.” In Language, Ethnicity, and Intergroup Relations, ed. by Howard Giles, 119–135. London: Academic Press. Branigan, Holly P., Martin J. Pickering, and Alexandra A. Cleland. 2000. “Syntactic Coordination in Dialogue.” Cognition 75 (2): B13–B25. https://doi.org/10.1016/S0010-0277(99)00081-5

Brennan, Susan E. 1991. “Conversation with and through Computers.” User Modeling and Useradapted Interaction 1 (1): 67–86. https://doi.org/10.1007/BF00158952 Brennan, Susan E., and Herbert H. Clark. 1996. “Conceptual Pacts and Lexical Choice in Conversation.” Journal of Experimental Psychology: Learning, Memory and Cognition 22 (6): 1482–1493. Brown, Lucien. 2011. Korean Honorifics and Politeness in Second Language Learning. Amsterdam/Philadelphia: John Benjamins. https://doi.org/10.1075/pbns.206 Brown, Lucien, Hyunji Kim, Iris Hübscher, and Bodo Winter. (Forthcoming). “Gestures are Modulated by Social Context: A Study of Multimodal Politeness across two Cultures.” Gesture. Caballero, Jonathan, Nikos Vergis, Xiaoming Jiang, and Marc Pell. 2018. “The Sound of Im/politeness.” Speech Communication 10: 39–53. https://doi.org/10.1016/j.specom.2018.06.004

Cohen Priva, Uriel, and Chelsea Sanker. 2018. “Distinct Behaviors in Convergence across Measures.” In Proceedings of the 40th Annual Meeting of the Cognitive Science Society, 1515–1520. Cohen Priva, Uriel, and Chelsea Sanker. 2019. “Limitations of Difference-in-difference for Measuring Convergence.” Laboratory Phonology: Journal of the Association for Laboratory Phonology 10 (1), 15: 1–29. https://doi.org/10.5334/labphon.200 Coles-Harris, Evan Hugh. 2017. “Perspectives on the Motivations for Phonetic Convergence.” Language and Linguistics Compass 11 (12): e12268. https://doi.org/10.1111/lnc3.12268 Cooke, Martin, Simon King, Maëva Garnier, and Vincent Aubanel. 2014. “The Listening Talker: A Review of Human and Algorithmic Context-induced Modifications of Speech.” Computer Speech and Language 28 (2): 543–571. https://doi.org/10.1016/j.csl.2013.08.003 Coulston, Rachel, Sharon Oviatt, and Courtney Darves. 2002. “Amplitude Convergence in Children’s Conversational Speech with Animated Personas.” In Proceedings of Interspeech, 2689–2692. Danescu-Niculescu-Mizil, Cristian, Michael Gamon, and Susan Dumais. 2011. “‘Mark my words!’ Linguistic Style Accommodation in Social Media.” In Proceedings of WWW, 745–754. https://doi.org/10.1145/1963405.1963509 De Looze, Celine, and Daniel Hirst. 2014. “Theome (Octave-median) Scale: A Natural Scale for Speech Melody.” In Proceedings of the 7th International Conference on Speech Prosody, Dublin, 910–914. De Looze, Céline, Stefan Scherer, Brian Vaughan, and Nick Campbell. 2014. “Investigating Automatic Measurements of Prosodic Accommodation and its Dynamics in Social Interaction.” Speech Communication 58: 11–34. https://doi.org/10.1016/j.specom.2013.10.002

Within-speaker accommodation behavior in apology-centered interactions

Delvaux, Véronique, and Alain Soquet. 2007. “The Influence of Ambient Speech on Adult Speech Productions through Unintentional Imitation.” Phonetica 64 (2–3): 145–173. https://doi.org/10.1159/000107914

Deutschmann, Mats. 2003. Apologising in British English. PhD diss., Umeå Universitet. Sweden. Drager, Katie, Jen Hay, and Abby Walker. 2010. “Pronounced Rivalries: Attitudes and Speech Production.” Te Reo 53: 27–53. Edlund, Jens, Mattias Heldner, and Julia Hirschberg. 2009. “Pause and Gap Length in Face-toface Interaction.” In Proceedings of Interspeech, Brighton, 2779–2782. https://doi.org/10.21437/Interspeech.2009-710

Gessinger, Iona, Eran Raveh, Ingmar Steiner, and Bernd Möbius. 2021. “Phonetic Accommodation to Natural and Synthetic Voices: Behavior of Groups and Individuals in Speech Shadowing.” Speech Communication 127: 43–63. https://doi.org/10.1016/j.specom.2020.12.004

Giles, Howard, Nikolas Coupland, and Justine Coupland. 1991. “Accommodation Theory: Communication, Context, and Consequence.” In Contexts of Accommodation: Developments in Applied Sociolinguistics, ed. by Howard Giles, Justine Coupland, and Nikolas Coupland, 1–68. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511663673.001

Grainger, Karen, and Sandra Harris. 2007. “Special Issue: Apologies: Introduction.” Journal of Politeness Research 3 (1): 1–9. https://doi.org/10.1515/PR.2007.001 Grawunder, Sven, Marianne Oertel, and Cordula Schwarze. 2014. “Politeness, Culture, and Speaking Task: Paralinguistic Prosodic Behavior of Speakers from Austria and Germany.” In Proceedings of Speech Prosody, Dublin, Ireland, 159–163. https://doi.org/10.21437/SpeechProsody.2014-20

Gregory, Stanford W., and Stephen Webster. 1996. “A Nonverbal Signal in Voices of Interview Partners Effectively Predicts Communication Accommodation and Social Status Perceptions.” Journal of Personality and Social Psychology 70 (6): 1231–1240. https://doi.org/10.1037/0022-3514.70.6.1231

Harris, Sandra, Karen Grainger, and Louise Mullany. 2006. “The Pragmatics of Political Apologies.” Discourse & Society 17 (6): 715–737. https://doi.org/10.1177/0957926506068429 Heath, Jevon. 2017. “How Automatic is Phonetic Convergence? Evidence from Working Memory.” In Proceedings of the Linguistic Society of America, 35. Holmes, Janet. 1998. “Apologies in New Zealand English.” In The Sociolinguistics Reader: Gender and Discourse, ed. by Jenny Cheshire, and Peter Trudgill. 201–239. London: Arnold. Honorof, Douglas N., and D. H. Whalen. 2005. “Perception of Pitch Location within a Speaker’s F0 Range.” Journal of the Acoustical Society of America 117: 2193–2200. https://doi.org/10.1121/1.1841751

Hübscher, Iris, Joan Borràs-Comes, and Pilar Prieto. 2017. “Prosodic Mitigation Characterizes Catalan Formal Speech: The Frequency Code Reassessed.” Journal of Phonetics 65: 145–159. https://doi.org/10.1016/j.wocn.2017.07.001 Ibrahim, Omnia, Gabriel Skantze, Sabine Stoll, and Volker Dellwo. 2019. “Fundamental Frequency Accommodation in Multi-party Human-robot Game Interactions: The Effect of Winning or Losing.” In Proceedings of Interspeech, 3980–3984. https://doi.org/10.21437/Interspeech.2019-2496

207

208

Omnia Ibrahim & Iris Hübscher

Jucker, Andreas H. 2018. “Apologies in the History of English: Evidence from the Corpus of Historical American English (COHA).” Corpus Pragmatics 2 (4): 375–398. https://doi.org/10.1007/s41701-018-0038-y

Karpiński, Maciej, Katarzyna Klessa, and Agnieszka Czoska. 2014. “Local and Global Convergence in the Temporal Domain in Polish Task-oriented Dialogue.” In Proceedings of Speech Prosody, 743–747. https://doi.org/10.21437/SpeechProsody.2014-137 Kasanga, Luanga A., and Joy-Christine Lwanga-Lumu. 2007. “Cross-cultural Linguistic Realization of Politeness: A Study of Apologies in English and Setswana.” Journal of Politeness Research 3 (1): 65–92. https://doi.org/10.1515/PR.2007.004 Kisler, Thomas, Uwe Reichel, and Florian Schiel. 2017. “Multilingual Processing of Speech via Web Services.” Computer Speech & Language 45: 326–347. https://doi.org/10.1016/j.csl.2017.01.005

Kurtić, Emina, and Jan Gorisch. 2018. “F0 Accommodation and Turn Competition in Overlapping Talk.” Journal of Phonetics 71: 376–394. ISSN 0095-4470. https://doi.org/10.1016/j.wocn.2018.09.006

Kuznetsova, Alexandra, Per B. Brockhoff, and Rune H. B. Christensen. 2017. “lmertest Package: Tests in Linear Mixed Effects Models.” Journal of Statistical Software 82 (13): 1–26. https:// www.jstatsoft.org/v082/i13, https://doi.org/10.18637/jss.v082.i13 Lee, Chi-Chun, Matthew Black, Athanasios Katsamanis, Adam Lammert, Brian Baucom, Andrew Christensen, Panayiotis G. Georgiou, and Shrikanth Narayanan. 2010. “Quantification of Prosodic Entrainment in Affective Spontaneous Spoken Interactions of Married Couples.” In Proceedings of Interspeech, 793–796. https://doi.org/10.21437/Interspeech.2010-287

Lee, Chi-Chun, Athanasios Katsamanis, Matthew P. Black, Brian R. Baucom, Andrew Christensen, Panayiotis G. Georgiou, and Shrikanth S. Narayanan. 2014. “Computing Vocal Entrainment: A Signal-derived PCA-based Quantification Scheme with Application to Affect Analysis in Married Couple Interactions.” Computer, Speech and Language 28 (2): 518–539. https://doi.org/10.1016/j.csl.2012.06.006 Lehnert-LeHouillier, Heike, Susana Terrazas, Steven Sandoval, and Rachel Boren. 2020. “The Relationship between Prosodic Ability and Conversational Prosodic Entrainment.” In Proceedings of Speech Prosody, 769–773. https://doi.org/10.21437/SpeechProsody.2020-157 Leongómez, Juan David, Katarzyna Pisanski, David Reby, Disa Sauter, Nadine Lavan, Marcus Perlman, and Jaroslava Varella Valentova. 2021. “Voice Modulation: From Origin and Mechanism to Social Impact.” Philosophical Transactions of the Royal Society B: Biological Sciences 376 (1840). https://doi.org/10.1098/rstb.2020.0386 Levitan, Rivka, Štefan Beňuš, Agustín Gravano, and Julia Hirschberg. 2015a. “Entrainment and Turn-taking in Human-human Dialogue.” In AAAI 2015 Spring Symposium on Turntaking and Coordination in Human-Machine Interaction, 44–51. Levitan, Rivka, Štefan Beňuš, Agustín Gravano, and Julia Hirschberg. 2015b. “Acousticprosodic Entrainment in Slovak, Spanish, English and Chinese: A Cross- linguistic Comparison.” In Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGdial), 325–334. https://doi.org/10.18653/v1/W15-4644 Levitan, Rivka, Agustín Gravano, Laura Willson, Štefan Beňuš, Julia Hirschberg, and Ani Nenkova. 2012. “Acoustic-prosodic Entrainment and Social Behavior.” In Proceedings of the North American Chapter of the Association for Computational Linguistics Conference: Human Language Technologies, Stroudsburg, PA, USA, 11–19.

Within-speaker accommodation behavior in apology-centered interactions

Levitan, Rivka, and Julia Hirschberg. 2011. “Measuring Acoustic-prosodic Entrainment with Respect to Multiple Levels and Dimensions.” In Proceedings of Interspeech, 3081–3084. https://doi.org/10.21437/Interspeech.2011-771

Lewandowski, Natalie. 2012. “Automaticity and Consciousness in Phonetic Convergence.” The Listening Talker conference, Edinburgh, 70–71. Loveday, Leo. 1981. “Pitch, Politeness and Sexual Role: An Exploratory Investigation into the Pitch Correlates of English and Japanese Politeness Formulae.” Language and Speech 24: 71–89. ( https://doi.org/10.1177/002383098102400105) Manson, Joseph H., Gregory A. Bryant, Matthew M. Gervais, and Michelle A. Kline. 2013. “Convergence of Speech Rate in Conversation Predicts Cooperation.” Evolution of Human Behavior 34 (6): 419–426. https://doi.org/10.1016/j.evolhumbehav.2013.08.001 Michalsky, Jan, and Heike Schoormann. 2017. “Pitch Convergence as an Effect of Perceived Attractiveness and Likability.” In Proceedings of Interspeech, Stockholm, 2253–2256. https://doi.org/10.21437/Interspeech.2017-1520

Mikeev, Y. V. 1971. “Statistical Distribution of the Periods of the Fundamental Tone of Russian Speech.” Soviet Physics-Acoustics 16: 474–477. Miles, Lynden K., Louise K. Nind, and C. Neil Macrae. 2009. “The Rhythm of Rapport: Interpersonal Synchrony and Social Perception.” Journal of Experimental Social Psychology 45: 585. https://doi.org/10.1016/j.jesp.2009.02.002 Natale, Michael. 1975. “Convergence of Mean Vocal Intensity in Dyadic Communication as a Function of Social Desirability.” Journal of Personality and Social Psychology 32 (5): 790–804. https://doi.org/10.1037/0022-3514.32.5.790 Niederhoffer, Kate G., and James W. Pennebaker. 2002. “Linguistic Style Matching in Social Interaction.” Journal of Language and Social Psychology 21 (4): 337–360. https://doi.org/10.1177/026192702237953

Oh, Eunhae, and Mao Cui. 2020. “The Acquisition of Acoustic Correlates of Politeness by Native Chinese Speakers.” Linguistic Research 37: 113–134. Ohala, John J. 1984. “An Ethological Perspective on Common Cross-language Utilization of F0 of Voice.” Phonetica 41: 1–16. https://doi.org/10.1159/000261706 Olshtain, Elite, and Andrew Cohen. 1983. “Apology: A Speech Act Set.” In Sociolinguistics and Language Acquisition, ed. by Nessa Wolfson, and Elliott Judd, 18–36. Rowley, MA: Newbury House. Orozco, Leonor. 2008. “Peticiones corteses y factores prosódicos.” In Fonología Instrumental, ed. by Esther Herrera Zendejas, and Pedro Martin Butragueno, 335–355. Mexico City, México: El Colegio de México. Pardo, Jennifer S. 2006. “On Phonetic Convergence During Conversational Interaction.” Journal of American Acoustic Society 119: 2382–2393. https://doi.org/10.1121/1.2178720 Pardo, Jennifer S., Rachel Gibbons, Alexandra Suppes, and Robert M. Krauss. 2012. “Phonetic Convergence in College Roommates.” Journal of Phonetics 40: 190–197. https://doi.org/10.1016/j.wocn.2011.10.001

Pardo, Jennifer S., Kelly Jordan, Mallar Rollienei, Caitlin Scanlon, and Eva Lewandowski. 2013. “Phonetic Convergence in Shadowed Speech: The Relation Between Acoustic and Perceptual Measures.” Journal of Memory and Language 69 (3): 183–195. https://doi.org/10.1016/j.jml.2013.06.002

Pardo, Jennifer S., Adelya Urmanche, Sherilyn Wilman, and Jaclyn Wiener. 2016. Phonetic Convergence across Multiple Measures and Model Talkers. Attention, Perception and Psychophysics 79: 637–659. https://doi.org/10.3758/s13414-016-1226-0

209

210

Omnia Ibrahim & Iris Hübscher

Pardo, Jennifer S., Adelya Urmanche, Sherilyn Wilman, Jaclyn Wiener, Nicholas Mason, Keagan Francis, and Melanie Ward. 2018. “A Comparison of Phonetic Convergence in Conversational Interaction and Speech Shadowing.” Journal of Phonetics 69: 1–11. https://doi.org/10.1016/j.wocn.2018.04.001

Pickering, Martin J., and Simon Garrod. 2004. “Toward a Mechanistic Psychology of Dialogue.” Behavioral and Brain Sciences 27 (2): 169–190. https://doi.org/10.1017/S0140525X04000056

Pickering, Martin J., and Simon Garrod. 2013. “An Integrated Theory of Language Production and Comprehension.” Behavioral and Brain Sciences 36 (4): 329–347. https://doi.org/10.1017/S0140525X12001495

Pickering, Martin J., and Simon Garrod. 2021. Understanding Dialogue: Language Use and Social Interaction. Cambridge: Cambridge University Press. https://doi.org/10.1017/9781108610728

Pierrehumbert, Janet B. 2001. “Exemplar Dynamics: Word Frequency, Lenition, and Contrast.” In Frequency and the Emergence of Linguistic Structure, ed. by Joan Bybee, and Paul Hopper, 137–157. Amsterdam: John Benjamins. https://doi.org/10.1075/tsl.45.08pie R Core Team. 2018. “R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing.” Vienna, Austria, https://www.R-project.org/ Rappaport, Werner. 1958. “Über Messungen der Tonhöhen Verteilung in der deutschen Sprache.” Acustica 8: 220–225. Rasenberg, Marlou, Asli Özyürek, and Mark Dingemanse. 2020. “Alignment in Multimodal Interaction: An Integrative Framework.” Cognitive Science 44 (11): e12911. https://doi.org/10.1111/cogs.12911

Reichel, Uwe D., Štefan Beňuš, and Katalin Mády. 2018. “Entrainment Profiles: Comparison by Gender, Role, and Feature Set.” Speech Communication 100: 46–57. https://doi.org/10.1016/j.specom.2018.04.009

Reis, Harry. 2007. “Similarity-attraction Effect.” In Encyclopedia of Social Psychology, ed. by Roy F. Baumeister, and Kathleen D. Vohs, 876–876. Thousand Oaks, CA: SAGE Publications. https://doi.org/10.4135/9781412956253.n517 Reitter, David, Johanna D. Moore, and Frank Keller. 2006. “Priming of Syntactic Rules in Task-oriented Dialogue and Spontaneous Conversation.” In Proceedings of the 28th Annual Conference of the Cognitive Science Society (CogSci), 685–690. Sagi, Eyal, and Daniel Diermeier. 2017. “Language Use and Coalition Formation in Multiparty Negotiations.” In Proceedings of the Annual Conference of the Cognitive Science Society (CogSci), 259–271. https://doi.org/10.1111/cogs.12325 Savino, Michelina, Loredana Lapertosa, Alessandro Caffò, and Mario Refice. 2016. “Measuring Prosodic Entrainment in Italian Collaborative Game-based Dialogues.” In SPECOM, vol. 9811 of Lecture Notes in Computer Science, 476–483. Springer. https://doi.org/10.1007/978-3-319-43958-7_57

Scholz, Fritz, and Angie Zhu. 2017. “kSamples: K-Sample Rank Tests and their Combinations.” r package version 1.2–7. [Online]. Available: https://CRAN.R-project.org/package =kSamples Schweitzer, Antje, and Natalie Lewandowski. 2014. “Social Factors in Convergence of F1 and F2 in Spontaneous Speech.” In The International Seminar on Speech Production. Cologne.

Within-speaker accommodation behavior in apology-centered interactions

Schweitzer, Katrin, Michael Walsh, and Antje Schweitzer. 2017. “To See or Not to See: Interlocutor Visibility and Likeability Influence Convergence in Intonation.” In Proceedings of Interspeech, Stockholm, 919–923. https://doi.org/10.21437/Interspeech.2017-1248

Shepard, C. A., Howard Giles, and B. A. Le Poire. 2001. “Communication Accommodation Theory.” In The New Handbook of Language and Social Psychology, ed. by Peter W. Robinson, and Howard Giles, 33–56. New York: Wiley. Sherr-Ziarko, Ethan. 2019. “Prosodic Properties of Formality in Conversational Japanese.” Journal of the International Phonetic Association 49 (3): 331–352. https://doi.org/10.1017/S0025100318000117

Simpson, Adrian P. 2009. “Phonetic Differences Between Male and Female Speech.” Language and Linguistics Compass 3 (2): 621–640. https://doi.org/10.1111/j.1749-818X.2009.00125.x Thomason, Jesse, Hui V. Nguyen, and Diane Litman. 2013. “Prosodic Entrainment and Tutoring Dialogue Success.” In Proceedings of the 16th International Conference on Artificial Intelligence in Education, 750–753. https://doi.org/10.1007/978-3-642-39112-5_104 Traunmüller, Hartmut, and Anders Eriksson. 1995. “The Frequency Range of the Voice Fundamental in the Speech of Male and Female Adults.” Manuscript, Department of Linguistics, University of Stockholm. Weise, Andreas, Sarah Ita Levitan, Julia Hirschberg, and Rivka Levitan. 2019. “Individual Differences in Acoustic-prosodic Entrainment in Spoken Dialogue.” Speech Communication 115: 78–87. https://doi.org/10.1016/j.specom.2019.10.007 Winter, Bodo, and Sven Grawunder. 2012. “The Phonetic Profile of Korean Formal and Informal Speech Registers.” Journal of Phonetics 40 (6): 808–815. https://doi.org/10.1016/j.wocn.2012.08.006

Winter, Bodo, Grace Eunhae Oh, Iris Hübscher, Kaori Idemaru, Lucien Brown, Pilar Prieto, and Sven Grawunder. 2021. “Rethinking the Frequency Code: A Meta Analytic Review of the Role of Acoustic Body Size in Communicative Phenomena.” Philosophical Transactions B. B 376: 20200400. https://doi.org/10.1098/rstb.2020.0400 Zellers, Margaret, and Antje Schweitzer. 2017. “An Investigation of Pitch Matching Across Adjacent Turns in a Corpus of Spontaneous German.” In Proceedings of Interspeech, 2336–2340. https://doi.org/10.21437/Interspeech.2017-811 Zhang, Hang. 2001. “Culture and Apology: The Hainan Island Incident.” World Englishes 20 (3): 383–391. https://doi.org/10.1111/1467-971X.00222

211

chapter 8

Perceptual changes between adults and children for multimodal im/politeness in Japanese Takaaki Shochi,1,2 Albert Rilliard3,4 & Donna Erickson5 1

CLLE UMR5263 CNRS, Pessac, France | 2 LaBRI UMR5800 CNRS, Talence, France | 3 Université Paris Saclay, CNRS, LISN | 4 Universidade Federal do Rio de Janeiro | 5 Haskins Laboratories

Japanese im/politeness attitudes are conveyed by audiovisual prosody in tandem with lexical markers. This chapter reports on two experiments about the acquisition by Japanese elementary school children of prosodic codes and social categories. The first experiment studied the perceived degree of politeness and its social use, and the second, the perceived similarity between the expressivities conveyed in pairs of expressions. An analysis of the audiovisual performances showed the types of changes in pitch and range in line with symbolic frequency and effort codes. The perceptual results showed that children learn to use and recognize im/polite expressions in a socially adequate fashion between 6 to 10 years old, thus showing an underlying growing cultural coherence gained with age. Keywords: perceptual changes, multimodal expressivity, im/politeness, Japanese

1.

Introduction

During human communication, multimodal cues are used for conveying and receiving social affective information and play an important role for establishing good interpersonal relationships (e.g., Swerts and Krahmer 2005; Swerts 2011; Rilliard et al. 2009, 2014; Erickson et al. 2020). Different from basic emotions (Izard 2007) which are probably shared among human beings as a phylogenetically developed competence (see Fernández-Dols and Russell 2017 for discussions), social affects, such as politeness, irritation, irony, contempt etc., are closely linked to language and culture. It follows, therefore, that these expressions can vary across languages and cultures, and as such their prototypical characteristics https://doi.org/10.1075/pbns.333.08sho © 2023 John Benjamins Publishing Company

214

Takaaki Shochi, Albert Rilliard & Donna Erickson

and socially-conformed use must be learned (Zinck and Newen 2008; Scherer 2013). Symbolic codes have been proposed in the literature that link the recurrent correlation between a physiologically derived constraint (the size of the vocal folds or the link between fundamental frequency and effort) and changes in the voice, and propose these relations are used to encode the reverse relationship: because larger animals tend to have lower voices than conspecifics, lowering one’s voice sends a message of “larger size”, and thus more power etc., under Ohala’s Frequency code (1983, 1994). From this observation, he predicts the use of higher/ lower pitch for a series of situations that include relations of power – and typically useful here, in the case of “politeness”, a higher pitch would be predicted. Gussenhoven’s Effort code (2004) links a greater effort of the speaker in a given speech act as a marker of its importance for the speaker’s goals; greater effort leads to larger pitch span – and marking interest in an interaction is a potentially polite behavior, thus the prediction of larger pitch span following the Effort code. Social affective expressions are closely linked to im/politeness strategies, and are thus determined by three things for the management of face: distance, relative power and absolute ranking (Brown and Levinson 1987; see also Culpeper, Bousfield, and Wichmann 2003; Culpeper 2010, 2016; or works in Culpeper, Haugh, and Kádár 2017, for impoliteness specificities). Distance is a social variable which describes a symmetric social dimension of proximity between the speaker and the hearer, in terms of similarity and frequency of interaction. Relative power indicates, still in Brown and Levinson’s (1987) view, the amount of imposition the speaker can put on the hearer, as well as the self-evaluation of the social status difference between speaker and hearer, that may legitimate this potential to impose on / control the other (on distance and power, see also Spencer-Oatey 1996). Power is based on social hierarchy, which is what is seen in Japanese society. Absolute ranking refers to “a culturally and situationally defined ranking of impositions by the degree to which they are considered to interfere with an agent’s wants of self-determination or of approval” (Brown and Levinson 1987: 77). In Japanese society, the cultural concepts of politeness are highly codified by social status (rank), age and situation, and seem to fit well with Brown and Levinson’s account of im/politeness strategies. The sociological variables relating to im/politeness strategies are strongly influenced by age and social hierarchy, culturally-determined factors which play a decisive role in establishing good relationships among conversation partners, that may account for the “discernment” aspect of social interaction in a high context culture like the Japanese one (Hill et al. 1986; Ide 2002). See Kiyama, Tamaoka, and Takiura (2012) for a detailed discussion of the applicability of Brown and Levinson’s face model to Japanese culture, as a rebuttal to earlier studies, e.g. Ide (1989, 2006) and Matsumoto (1988), suggesting that Brown and Levinson might be problematic for Asian cultures.

Perceptual changes between adults and children for multimodal im/politeness in Japanese

Previous perceptual studies have examined perception of the vocal and visual characteristics of im/politeness strategies. Nadeu and Prieto (2011) have shown the importance of facial information to interpret the acoustic information linked to the expression of politeness in Catalan. Building on the “audio-visual prosody” paradigm (Swerts and Krahmer 2005), studies have shown that the visual cues have a role, together with audio and lexical ones, in the expression and identification process of many linguistic functions, notably in relation to the acquisition of social awareness in children (Swerts 2011), and capacities related to the management of linguistic im/politeness strategies. These studies include works studying the reception of im/politeness cues in listeners of various language backgrounds, for example French, English, Brazilian Portuguese, and Japanese in Rilliard et al. (2014). Also see Idemaru, Winter, and Brown (2019) for a comparison of Japanese and Korean polite speech, and Idemaru et al. (2020) for Korean deferential speech. Other studies showed variation in the reception of audiovisual prosodic cues to social affect cross-culturally (Mixdorff et al. 2017; Rilliard et al. 2017; Shochi, Aubergé, and Rilliard 2007). Concerning studies of audio-visual prosody specifically with regard to first language acquisition, Swerts (2011) examined the relation between visual prosody and social awareness in children, reporting that as children become older, they become more socially aware, and this is expressed in their prosody. For Swerts, audio-visual prosody refers to both auditory cues as well as visual cues, such as body language, posture and facial expressions. With age, the child becomes “more knowledgeable about the conventions that exist within a specific community on how to associate specific expressions with specific pragmatic meanings” (2011: 382). Armstrong et al. (2018) examined how 3–5-year-old Catalan children understand belief states using auditory and facial prosodic information, and found that children were better at understanding prosodic meanings as they grew older; also, they did better with audio visual prosodic information than just audio alone or visual alone. Work by Hübscher et al. (2017) and Hübscher, Wagner, and Prieto (2020) reported that children as young as 3 years old use both auditory and visual prosody to understand sociopragmatic meanings. They recognize politeness from prosodic or visual cues even before they have fully acquired various lexical and morphosyntactic politeness markers. Shochi, Erickson, et al. (2009) looking at Japanese 9- and 10-year-old children, reported that facial cues helped in understanding politeness and impoliteness expressions, and that the visual cues were perhaps more important for children than for adults. Long (2010) looking only at the auditory component of prosody, reported that children as young as the first grade begin to be aware of the difference between expressions of apology and gratitude, and by the 7th or 9th grades, their knowl-

215

216

Takaaki Shochi, Albert Rilliard & Donna Erickson

edge is similar to that of adults. These findings suggest that language socialization is a process which takes place in the first seven to nine years of school. For an interesting description of how young children in Japan are taught the general concepts of politeness, see Burdelski (2010): he shows how e.g., enacting role-plays and training social routines (request, acceptation, offer, invitations) help children develop positive, other-oriented behaviors that correspond to Japanese social rules. We are not aware of any studies about when children learn about different categories of politeness; however, generally, around the 5th grade, they start to learn some of the lexical aspects of politeness (Harada 2011). Our current study, in contrast to our previous studies, examines the multimodal aspect of im/politeness expressions in Japanese society, as it relates to the acquisition of these expressions by children from 6 to 9 years old. Specifically, the current paper proposes a new acoustic and visual analysis of the stimuli, based on two new perceptual experiments we report here. The im/politeness expressions we examine are the Japanese concepts of teinei, seii, kyoshuku, heijo, and zonzai (broadly translated as ‘polite’, ‘sincere’, ‘walkingon-eggs’, ‘declaration’, and ‘impolite’). These particular expressions were chosen because they represent various types of im/politeness expressions in Japanese. These translations are rough, and cannot be understood without a brief introduction to the hierarchical social structure of Japanese culture, which largely differs from western categories. (The reader is referred to e.g., Long 2010; Ofuka et al. 2000 and references therein for descriptions of the relationship between Japanese culture and sociopragmatic expressions.) In contrast to many western societies where the conversational “other” is often defined in terms of “social intimacy”, in Japanese society, the conversational “other”, the one to whom the conversation is directed, is culturally defined in terms of their hierarchical relation (e.g., age, social status, power) to the speaker. Social intimacy can play a role, but usually in the use of impolite expressions, discussed below. In general, in Japan the addressee is either at the same level as or below the speaker in terms of age and/ or social status and power, or above the speaker’s level. The social situation that is measuring the distance (e.g., school, family, business, etc.) also plays an important role: older sibling/younger sibling, older classmate/younger or same age classmate, more experienced or older colleague in the workplace/ less experienced or younger colleague, etc. Given this importance of social hierarchy, expressions of im/politeness are highly formulaic (Ide 1989). Within the framework of Japanese society, the expressions are all formulaic codes pertaining to how to maintain “wa” (和, ‘harmony, peace’), in various social situations; that is, each type of “politeness” is a formulaic expression with a specific name and a specific prosodic expressivity, in addition to a grammatical structure.

Perceptual changes between adults and children for multimodal im/politeness in Japanese

The term teinei (丁寧) politeness, is used for polite behavior with people like bus drivers, shop clerks, etc. – people who are not well-known, to show respect for a person who is a relative stranger. It can also be used casually with friends. The literal meaning of teinei is ‘to pay attention’ to how to speak; it is the generic term for polite speech which we may translate as ‘courtesy politeness’. Seii (誠意) politeness, deals with how to express politeness in difficult situations with someone either higher than you, or the same level; the focus is on how to enthusiastically respect the sincere feelings of the addressee. We may translate this type of politeness as ‘sincere politeness’. The literal meaning of seii (誠意) is ‘honest, sincere feeling’. An example of using seii might be expressing respect to another person, even though you disagree with that person. Ruth Benedict, in her classic work “The Chrysanthemum and the Sword”, describes a situation in which seii was violated (see Benedict 1947: 159–160). A young man tells an older couple that he wants to go to the United States. They reply with a certain amount of incredulity, which was interpreted by the young man as indicating disrespect to his sincere feelings and the result was the speaker cut off relations with them because their rudeness was unforgiveable. Benedict, in her explanation of the speaker’s reactions, points out that sneering, or sarcasm in Japanese culture, is a “wounding of the heart”, a type of impoliteness that is difficult, if not impossible, to forgive. This anecdote about seii illustrates the extent to which respect of the other’s feelings is extremely important; violation of another’s feelings verges on immorality. It also helps the third author of this paper understand better why her “playful” sarcasm was not tolerated by her Japanese colleagues and friends, often leading to severe breakdowns in the relationship, which took a lot of work to mend. Kyoshuku (恐縮) has the literal meaning of ‘shrink fear’: the speaker “shrinks himself ” to show proper “fear/respect” toward his/her superior. Kyoshuku is a formulaic politeness strategy concerning how to handle difficult situations with a superior, such as a boss, or a school principal, situations in which showing respect to the superior and his/her point of view is of utmost priority. Kyoshuku politeness conveys the speaker’s ashamedness / embarrassment that they are not able to act in accordance with the expectations of the person who is hierarchically superior to them (Sadanobu 2004). For example, if a professor has to tell his superior, the school officer, that s/he cannot attend the weekend-scheduled entrance exams due to a personal prior commitment, or, if an employee has to tell their boss that the boss has made a mistake in decision-making, or if a shop clerk has to apologize to the customer for something, regardless of whether it was the shop clerk’s fault or not, etc., kyoshuku politeness is used. In each case, the priority of the speaker is to respect the superior person’s feelings. By using kyoshuku politeness, this goal is achieved; it avoids a potential conflicting situation, which may at times

217

218

Takaaki Shochi, Albert Rilliard & Donna Erickson

result in getting a favor from the hierarchically superior person. Although young children are never in adult-type boss-employee situations, still they encounter situations where they need to politely reject a senpai’s (older classmate) opinion in a way to express respect for the senpai’s feelings (see Ide 2002, for examples of hierarchical honorifics used by school-aged children). At what age children begin to use kyoshuku is one of the topics of investigation of this paper. It is during the 5th school grade that the lexical items related to honorific form (尊敬語 sonkeigo), humble form (謙譲語 kenjogo), and courtesy politeness (丁寧 teinei) are formally taught in the national education program. The prosodic characteristics of kyoshuku politeness, as well as the other types of politeness, are discussed toward the end of this introduction. Note that for kyoshuku, as well as for seii, the expressions have formulaic grammatical and prosodic characteristics, independent of how the speaker truly feels about a particular person or fact, thus further underlying the culturallyprescribed formulae for Japanese expressions of im/politeness. Also note that both kyoshuku and seii perhaps require more cultural awareness of respect for position and age than does teinei, which is more similar to western concepts of politeness. As far as we know, there is little research on seii; for kyoshuku, see Sadanobu (2004) and some of our previous studies, e.g., Shochi (2008); Shochi et al. (2008). Kyoshuku, which has no lexical equivalent in English, has roughly been translated as ‘walking on eggs’ (e.g., Rilliard et al. 2017), because walking on eggs is the nearest available in the western inventory of im/politeness, in that the speaker tries to avoid hurting the other’s feelings, often of a person who is somewhat difficult to get along with. However, kyoshuku, as explained above is clearly very different from walking on eggs. Heijo (平叙) politeness is used when the focus is on the facts, not the relationship of the speaker to the addressee. The literal meaning of heijo is ‘plain sentence’; we may translate it as ‘neutral politeness’, or ‘declaration’. This type of politeness presents information or facts to the hearer without any affective meaning. Zonzai (ぞんざい) can be roughly translated as ‘rude’, but has complexities. It is normatively spoken to someone with the same or lower social level. Literally, zonzai means ‘rude, careless, sloppy’, and refers to speech in which the speaker is disrespectful to the addressee. Zonzai may be expressed in two ways: first when the speaker voluntarily expresses his “rude” and “arrogant” attitude to the hearer; that is the case of zonzai in the stimuli we examine. It can also be expressed by the absence (lack) of polite attitude, and the overt display that the hearer is not part of the speaker’s group and is thus a strategy of positive impoliteness (damaging the hearer’s positive face; Culpeper 1996). Since age is an important factor to determine hierarchy in East Asian cultures, a senpai (older) person has a right (greater possibility) to behave with zonzai to a kohai (younger) person. Thus, zonzai can

Perceptual changes between adults and children for multimodal im/politeness in Japanese

be spoken to someone who is an equal or a younger person to indicate that “I am senpai (older); you are kohai (younger)”. For instance, in elementary school, a senpai fifth grader might speak to a kohai fourth grader with zonzai. Another example is that sometimes at an after-business drinking party, a kohai might drink too much, thus risking behaving with zonzai with a higher person; however, in this situation, the zonzai behavior often tends to be tolerated. Interestingly, zonzai can be used in close relationships, probably mostly between members of the same sex and social status, as a way to indicate intimacy and bonding, frequently seen in current popular anime. Listing five expressions of politeness in the above order might suggest a linearity of politeness in Japanese culture, from most polite to least polite; however, this is misleading. Expressions of politeness in Japan are not linear, as will be elaborated upon in the course of this paper. With regard to prosodic features, studies show that social affects are signaled by various acoustic cues, such as fundamental frequency (F0) characteristics (intonation contour, F0 height, and amount of change in F0), loudness, duration and voice quality (Scherer 2001; Rilliard et al. 2009) and listeners’ perceptual patterns are influenced by the gender and the maturity (age) of the listener (Roseberry-McKibbin and Brice 1999). With regard to acoustic cues of politeness in Japanese, numerous studies (e.g., Kawano 1995; Hong 1993; Ofuka et al. 2000; Ito 2002; Shochi 2008) have reported that politeness is signaled by increased F0 as well as increased duration, especially on the final vowel (e.g., Ofuka et al. 2000). However, Sherr-Ziarko (2019), in his examination of formality, politeness and F0, found that F0 tends to be higher in informal Japanese speech, thus suggesting F0 might be viewed as “multi-faceted” issue (2019: 348). As for impoliteness, the final syllable is also lower and longer (Ofuka et al. 2000), and often, stressed alveolar trill [r] can be heard (Calvetti 2020). The prosodic cues of kyoshuku are a tense, harsh voice, accompanied by a marked head lowering (Sadanobu 2004), along with a short duration and flat F0 (Shochi 2008). To date, no research that we know of has been done with seii. Work by Shochi et al. (2008); Rilliard et al. (2009) showed that Japanese adult listeners are able to perceive social affective expressions based on auditory or visual prosody, as well as a combination of both; specifically, L1 listeners were able to recognize a set of twelve Japanese attitudes, while American English and French listeners, without knowledge of the Japanese language, showed important confusions between some expressions. More specifically, the kyoshuku expression was not categorized within the polite expressions by non-native listeners, but linked to irritation or zonzai, while the other politeness expressions received high identification scores (Shochi 2008). One motivation of our current work is thus to inves-

219

220

Takaaki Shochi, Albert Rilliard & Donna Erickson

tigate the interrelations between expressions of politeness in Japanese, in order to measure if and when Japanese children start to identify social im/polite expressions and link them to adequate social contexts as adults do. The current paper proposes a re-analysis of a perceptual judgement experiment with children (Shochi, Erickson, et al. 2009), in the light of new results from a study on pair comparisons with young (6–9 yo) children. It proposes a new analysis of these two sets of perceptual results in line with a new acoustic and visual analysis of the stimuli. Section 2 presents the stimuli used and its analysis, and the listeners involved in the test as well as the experimental paradigms. Section 3 describes the results, and Section 4 gives an analysis of the main findings and their implications for the strategies of politeness in Japanese.

2. Methods 2.1 Listeners The listeners all spoke Kumamoto dialect. Given that the standard Tokyo dialect is prevalent in national television programs, most listeners had familiarity with the standard Tokyo dialect. Concerning the first experiment, three groups of listeners participated: The first group consisted of 29 adults (18 females, 11 males; mean age = 23); the second, 28 children in 4th grade (G4) in the Japanese school system (13 females, 15 males; mean age = 9.9 years); the third, 50 children in 5th grade (G5) (31 females, 19 males; mean age = 10.9 years). The total number of participants for this experiment was 107. In the second experiment, four age groups participated, with all listeners different from the first experiment. The first group was composed of 18 children in 1st Grade (11 females, 7 males; mean age: 6.1), the second, 19 children in 2nd Grade (13 females, 6 males; mean age: 7.4), the third, 19 children in 4th Grade (9 females, 10 males; mean age: 9.5), and the fourth, 40 adults (28 females, 12 males; mean age: 21.6). The total number of participants was 96. The explanation for the different age groups in experiment 2, compared to experiment 1, is as follows: The aim of experiment 2 was two-fold: to examine children of a larger set of ages and to compare the results with experiment 1. Experiment 1, done prior to experiment 2, only examined 4th and 5th graders, since the questionnaire task in experiment 1 was too difficult for younger students. Adults took the experiment in a quiet room individually using a computer and headset. Children took the experiment in a school classroom with a computer and headset for each child. The task was described and after explanations and questions, they put their headsets on and took the experiment. No listener reported

Perceptual changes between adults and children for multimodal im/politeness in Japanese

any hearing problems. The difference in size of each group is due to the fact that the children took the test as a class and the class size varied. In accordance with the ethical rules of each of the elementary schools, the parents signed a consent form before the tests allowing their children to participate in the experiments.

2.2 Stimuli Five Japanese expressions related to politeness were examined: teinei (TEI), seii (SEI), kyoshuku (KYO), heijo (HEI) and impolite zonzai (ZON). Each audiovisual stimulus was recorded using the same affectively neutral sentence as shown in (1) to express the prosodic attitudes (Shochi, Rilliard, et al. 2009) by a trained Japanese language teacher, the first author of this paper, who has considerable experience teaching social affective expressions to learners of Japanese. (1) 名古屋 で飲みます Nagoya denomimas Nagoya have-a-drink ‘Let’s have a drink in Nagoya’

Given that polite expressions in Japan are highly formulaic, the recordings represent exemplar productions of the five types of politeness following the definition and the context inspired by previous research (Sadanobu 2004; Kawano 1995; Hong 1993; Ohara 2001; Ofuka et al. 2000; etc.). The five polite expressions were validated previously by adult Tokyo dialect listeners (Rilliard et al. 2009; Shochi et al. 2008). The topic of gender variation in the production of these types of politeness is a topic for future exploration. The recordings were done in a soundproof room at LIMSI, France using a digital DV camera (Canon XM1 3CCD) and an omnidirectional AKG C414B microphone placed 40 cm from the speaker’s mouth. The distance to the microphone and the recording level were kept constant, thus allowing for reliable intensity measurements (i.e., comparable between sentences) if the absolute level (SPL) is unknown. The microphone was connected to a USBPresound device connected to a computer outside the room, recording the speech signal at 44.1 kHz, 16 bits. In order to replace the camera sound with the high-quality sound recorded by the microphone, synchronization was achieved with hand claps performed between each sentence. Video clips were encoded with a cinepack codec with a 784 × 576 pixels resolution. Each stimulus was presented to the elementary school listeners either with audio-only, video-only or audio-video modalities. The acoustic and facial movements typical of each of the five expressions are described in the next section.

221

222

Takaaki Shochi, Albert Rilliard & Donna Erickson

2.3 Acoustic and facial analysis 2.3.1 Acoustic and facial measurements The following acoustic parameters were extracted from the five utterances of Nagoya de nomimasu, spoken with teinei, seii, kyoshuku, hyojo, and zonzai, using Praat (Boersma and Weenink 2020): (i) the voice’s fundamental frequency (F0) measured in semitones (with reference to 1 hertz), (ii) the signal intensity (in dB), and (iii) the moraic durations (in seconds). Then, the following visual parameters were estimated from the video thanks to the OpenFace software (Baltrušaitis et al. 2018; Baltrušaitis, Mahmoud, and Robinson 2015): (iv) the amplitude (on a 1 to 4 scale) of a set of Action Units (AU) (Ekman, Friesen, and Hager 2002) from the upper face (AU01, 02, 04, 05, 06, 07, 09 & 45), and (v) a set of AU from the lower face (AU10, 12, 14, 15, 17, 20, 23; AU on lip opening and jaw drop were not considered because they were biased by articulation). An overview of the AUs can be found in Table 1. Then, (vi) the rotations of the head were measured over three axes (pitch, yaw, and roll); and (vii) gaze direction was estimated along x- and yaxes (i.e., x: left/right movements, with negative values indicating right gaze; y: up/down movements, with negative value indicating up gaze). Left and right are given from the position of the speaker. Table 1. Description of the action units used in this study, as found in (Ekman, Friesen, and Hager 2002) Action Unit

Description

AU01 AU02 AU04 AU05 AU06 AU07 AU09 AU10 AU12 AU14 AU15 AU17 AU20 AU23 AU45

Inner brow raiser Outer brow raiser Brow lowerer Upper lid raiser Cheek raiser Lid tightener Nose wrinkler Upper lip raiser Lip corner puller Dimpler Lip corner depressor Chin raiser Lip stretcher Lip tightener Blink

The detail of the facial expressions is displayed through a series of still images extracted from each video, and presented in Figure 1.

Perceptual changes between adults and children for multimodal im/politeness in Japanese

Figure 1. Each row features a series of six still images extracted regularly from the video of each social affect and cropped so to focus on the head; the expressions are presented in the following order, from the top: teinei, seii, kyoshuku, heijo, zonzai

These audio-visual measurements were taken each 5 ms for F0 and intensity, for each mora for moraic duration, and at each video frame (one each 40 ms) for visual parameters; they were aligned and linearly interpolated in order to get vectors with a 5 ms time frame (except for duration). These values were then regrouped (taking their mean value) for each mora of the sentence to create the figures displayed in the next section that present the mean and confidence intervals for the values observed on one mora. 2.3.2 Acoustic parameters variation F0 and intensity levels (Figure 2) tend to mark the difference between attitudes, with louder performance, but not higher F0, for zonzai. Kyoshuku shows the flattest curve, while on the contrary the teinei, seii and heijo expressions show large pitch spans (especially teinei); seii has a relatively high pitch for a low intensity performance. As for the interaction between F0 and intensity, we see that for the two expressions of politeness, seii and teinei, F0 increases while intensity decreases. The other three expressions, heijo, kyoshuku and zonzai, suggest that F0 and intensity work together. The pattern of high F0 and low intensity found for the two polite expressions can be seen as characteristics of low dominance utterances, and opposed to the strategy of power display used for zonzai with higher

223

224

Takaaki Shochi, Albert Rilliard & Donna Erickson

intensity and relatively low pitch, along the lines of the Frequency code (Ohala 1983, 1994) – with teinei also using a large pitch span typical of the Effort Code’s predictions (Gussenhoven 2004). On the contrary, kyoshuku did not fit any of the predictions of these two symbolic codes, as will be discussed later.

Figure 2. Plot of the mora’s means and confidence intervals for F0 (top plot), intensity (middle plot), moraic duration (bottom), for each attitude (left to right graphs). Duration values are raw values for each mora

As for duration, the final mora /su/ shows the longest duration in the kyoshuku expression. In general, however, the moras with the open vowel /a/ (i.e., /ma/, /na/) have the longest durations, with the /ma/ in zonzai being the longest. Lengthening of the penultimate mora for zonzai is a prosodic characteristic of impoliteness expressions (see e.g., Ofuka et al. 2000; Calvetti 2020). 2.3.3 Action Units Only AU movements higher than an intensity of one are considered here: under that rule, most action units are silent for these expressions for this speaker, except

Perceptual changes between adults and children for multimodal im/politeness in Japanese

for the kyoshuku expression (see Figure 3). In the case of kyoshuku, the AUs 04 (brow lowerer), 07 (lid tightener), 10 (upper lip raiser), and 17 (chin raiser) are activated during the sentence, forming a frowned face; slight markings of AUs 12 (lip corner puller), 20 (lip stretcher), and 23 (lip tightener) are also used (not shown) and participate in the global contraction of the face.

Figure 3. Plot of the mora’s means and confidence intervals for AUs 04, 07, 10 and 17 for each attitude (left to right graphs)

225

226

Takaaki Shochi, Albert Rilliard & Donna Erickson

2.3.4 Head movements The rotations of the head are shown in Figure 4 for each expression. The main movements are observed for (i) kyoshuku, with the head going down throughout the sentence, and turned toward the right (with a slight roll on the left that may result from the other movements); (ii) zonzai, with head up at the beginning, turned toward the left during the sentence (movement opposite to that of kyoshuku); (iii) teinei and seii, with slight head down at the end of the sentence (larger movement for teinei). Notice that teinei shows a wider F0 span than seii, suggesting a possible connection between F0 and head position. The head is mostly still for heijo.

Figure 4. Plot of the mora means for head pitch (top, positive value for head down), yaw (middle, positive for right movement), and roll (bottom, positive for left), for each attitude (left to right graphs)

Perceptual changes between adults and children for multimodal im/politeness in Japanese

2.3.5 Gaze direction Gaze directions are shown in Figure 5. During the zonzai expression, the speaker is staring at the addressee while his head is up, which makes it a typical “looking down” type of pose (head up, with gaze directed at the interlocutor). For kyoshuku, the gaze is down throughout the sentence, while for teinei and seii, the gaze goes down to a lesser extent, especially at the end of the sentence. Gaze also goes rapidly down in the middle of the sentence for heijo. For heijo, there is a rapid fall of F0 on the third mora of the utterance. Notice that head and gaze movements tend to synchronize.

Figure 5. Plot of the mora means for the two gaze directions, red for horizontal (positive for left) and blue for vertical positive for down), for each attitude (left to right graphs)

Table 2 presents a summary of the main facial and vocal changes found in each expression, basing the description on the heijo expression as the most neutral one, and describing the others in relation to it. Table 2. Summary table of the main characteristics of each expression. Note that for AUs, the movements that do not go above an intensity of 1 were not considered here Measure

Teinei

Seii

Kyoshuku

Heijo

Zonzai

F0

highest start; lowest end: larger range

initial equal to heijo, fall down to the mean (not lower)

flat; close to mean

high initial, falling below the mean for a low flat end

Relatively low (given the high vocal effort), decreasing F0

Intensity

peak on /no/

peak on /mi/

peak on /mi/; fall after

above the mean for /nagoya/; fall on the 2nd word (see F0)

strongest initial, with important decrease on the second word

227

228

Takaaki Shochi, Albert Rilliard & Donna Erickson

Table 2. (continued) Measure

Teinei

Seii

Kyoshuku

Heijo

Zonzai

Lengthening

long /masu/

long /ma/

long /masu/

long /ma/ strict isochrony on the other

long /ma/ important lengthening (twice heijo duration)

Action units

0

0

AU04+07: brow frown AU10+17: frowned face

0

0

Head pitch

++ at sentence end

+ at sentence end

+++ along the sentence (bowing)

0

- (start)

Head yaw

0

0

Turn right

0

Turn left

Head roll

0

0

Tilt left

0

Tilt left

Gaze horizontal

0 (addressee direction)

0

Left (indirect)

0

0

Gaze vertical

down at end

down at end

down

rapid down in the middle

straight

2.4 Experimental paradigm Two experimental methods for analyzing the perception of five different stimuli being labeled as “teinei, seii, kyoshuku, heijo, and zonzai” were carried out: (i) the perceived degree of politeness in terms of the category of people who may be addressed in such a way by the speaker, and (ii) the similarity between pairs of expressions. For the first experiment, the test consisted of 15 different stimuli (5 attitudes * 3 modalities), each presented three times to listeners in a random order. For the second experiment, the same five stimuli were used in a pair-comparison paradigm, where participants had to rate the similarity between two expressions, in terms of the comparability of speaker behavior; ratings were given on a 1 to 9 scale. All pairs of the five expressions were presented in audio-only, visual-only, and audiovisual modalities. 2.4.1 Experiment 1 Stimuli were presented to the subjects in three experimental conditions according to the modality of presentation: audio-only (A), video-only (V ), audio-video

Perceptual changes between adults and children for multimodal im/politeness in Japanese

(AV ). In order to balance the effect of the presentation order, half the subjects began with the video-only stimuli and the other half with the audio-only ones. Each stimulus could be listened to only once. After each presentation, subjects answered the question: “Who may this person talk to in this way?” The possible answers are: “Teacher”, “Higher grade student/person”, “Lower grade student/person”, “Classmate” and “Unknown”. This question was intended to check the kind of interpersonal relationship associated with the different attitudes by the subject within a hierarchical social framework that children could easily understand. They were then asked to judge the degree of politeness (teinei) of the stimulus on a scale ranging from impolite to polite (encoded on a 0–10 scale); the middle of the scale corresponding to a neutral expression. The listeners were asked to use a slide bar displayed on the computer screen to move a marker to indicate their judgments; the position of the marker was set to the middle of the scale for each presentation, and subjects had to move it according to their perception of the degree of politeness. Each subject participated in the three experimental conditions (corresponding to the three modalities of presentation) during the same sitting. All tests were run on a computer interface, and listeners wore quality headsets. 2.4.2 Experiment 2 Participants were asked to judge the difference between pairs of stimuli, on a 1 to 9 scale (1 for almost identical stimuli, 9 for completely different ones). Stimuli were based on performances of the same sentence presented in Example (1), and produced so as to convey five different prosodic attitudes, teinei, seii, kyoshuku, heijo, and zonzai. Each possible pair of the five performances (i.e., ten pairs), in each modality (A, V, AV ), were created in AB or BA order, separated by a short pause, in order to be presented to participants. Participants were instructed to give their answers as fast as possible on the basis of their first impression. The AB/BA presentation order, as well as the order of presentation for modalities (either A-V-AV or V-A-AV), was balanced across participants.

2.5 Statistical processing 2.5.1 Experiment 1 Results of the two questions were analyzed separately. The first question (the category of people who may be addressed in such a way by the speaker) was analyzed using a correspondence analysis (CA; Husson, Lê, and Pagès 2017). On the basis of contingency tables counting the number of answers in each category of

229

230

Takaaki Shochi, Albert Rilliard & Donna Erickson

answer – “Teacher” (Prof.), “Higher grade student/person” (HiGr), “Lower grade student/person” (LoGr), “Classmate” (Clm) and “unknown” (Unkn) – for each presented attitude, the proximity of attitudes and of social status was described. Different analyses were carried out for each of the three groups of judges (adults, G4 and G5), mixing answers of the three modalities. The answers to the second question – the degree of politeness – were analyzed using the GLM repeated measure procedure of SPSS. There were three within-subject factors: the presented attitudes (A – 5 levels, fixed factor), the modality of stimuli (M – 3 levels, fixed factor) and the three repetitions of each stimulus (R – 3 levels, fixed factor). Two between-subject factors were used: age of subjects/level in school (G4, G5, or adult) (L – 3 levels, fixed) and the order of presentation of modalities during the test (O – 2 levels, fixed). Following Max and Onghena (1999), the Huynh-Feldt correction was applied to all results of the repeated measures ANOVA. 2.5.2 Experiment 2 The analysis of the second experiment follows the methodology presented in the work by Romney and colleagues (Romney et al. 1996; Romney, Moore, and Rusch 1997; Romney et al. 2000): it consists of applying a correspondence analysis to the similarity matrix obtained from the pair comparison results, in order to obtain a spread of the presented stimuli on a multidimensional space. This spread of stimuli is seen as representing the structure of perceptual proximity between the stimuli. Similarities and differences in this spread of stimuli across groups is then quantified to compare their coherence in terms of “shared knowledge” (Moore et al. 1999). The next step consists in studying the spread of stimuli across the perceptual space. First, for each pair of stimuli composed of two attitudes A and B, their similarity score was obtained from subjects’ answers: distance estimations obtained from subjects were converted into similarity scores, with a 10 for pairs of identical attitudes (AA pairs, not presented to participants), and a 10 minus the measured distances for the heterogeneous pairs (the distance being on a 1 to 9 scale); this was done because correspondence analysis is designed to work on similarity data (cf. Romney et al. 2000). Then, these similarity scores were stacked in a 5×5 similarity matrix (experiments are based on five attitudes: each line of the matrix contains the similarity of a given attitude vis-à-vis the set of the five possible attitudes). One such 5×5 matrix was created for each modality (A, V, AV ) and each subject. All of the 5×5 matrices were stacked in a 1440 (5 attitudes × 3 modalities × 96 subjects) × 5 matrix, hereafter referred to as X. A Correspondence Analysis (CA) was performed on X (Romney et al. 2000). The row scores obtained from the CA were standardized (Kumbasar, Romney, and Batchelder 1994), so that for

Perceptual changes between adults and children for multimodal im/politeness in Japanese

each subject, the values are set to a mean of 0 and a variance equal to the singular value of each dimension of the CA. This standardization allows scaling of the data for possible differences in the use of the answer scale between subjects. The effect of such a normalization may at worst reduce the strength of the observed variations, according to Romney et al. (1996). The CA on matrix X gives a representation of the perceived distances between the 5 attitudes, for each subject and in each modality. Raw results give a cloud of points where the main tendencies can already be seen. By grouping scores from subjects of the same grade and from the same modality, it is possible to obtain a figure of the effects of age and modality on the representation of these 5 attitudes. The output of the CA has 4 dimensions, but the first two factors for the row account respectively for 38.7% and 34% of the total variance: i.e., a 2D plot shows 72.8% of the variance in the distance between attitudes. In the corresponding plots (see the result section), all individuals will not be represented; rather, it shows the mean coordinates on the first two factors for each category observed (with the ellipses around the means representing the 97.5% confidence limit from the means). The categories used to group answers were: (i) the three presentation modalities (A, V, AV ) and (ii) the four age groups (1st grade, 2nd grade, 4th grade and Adults). We then look at the shapes of the perceptual spaces. From the spread of stimuli in the CA space, the shapes of the “perceptual spaces” obtained, thanks to the spread of points on the CA space, are compared across groups, using the method presented in Romney et al. (2000), inspired by Rao and Suryawanshi (1996). The comparison is based on the measurement of the Euclidean distances between all pairs of points representing attitudes in the CA space (the four dimensions are used). The obtained distances give a description of the shape of the five points in the CA space, which is supposed to reflect the “perceptual space” defined for these five attitudes and for each participant in each modality. To compare the perception of the different participants, the correlations between the vectors describing these perceptual spaces are calculated for each pair of participants in each modality. The result is a 288×288 correlation matrix (3 modalities × 96 subjects), on which a Principal Component Analysis (PCA) is run. A plot of the mean position of the point by age group or by modality groups allows the observation of changes in shapes linked to these two factors. Finally, the variance in the results is used to estimate the proportion of “shared knowledge” across participants. If one takes the square root of these correlation vectors (i.e., the correlation between the shapes of the five-point spread obtained for each participant), it allows an estimation of what was coined by Romney et al. (2000) as the knowledge shared by the different groups. The principle is that the more correlated the two “perceptual spaces” are, the more the two participants

231

232

Takaaki Shochi, Albert Rilliard & Donna Erickson

share similar cultural knowledge for the data under investigation. Values reported can be viewed as the ratio of shared knowledge between different groups of participants. A high within- or between-group ratio demonstrates their coherence in the rating task; conversely, a low score demonstrates large variation in responding to the task.

3. Results 3.1 Experiment 1 3.1.1 Question 1 Results for question 1, about who may be addressed in such a way by the speaker, are summarized in Figure 6. The figure shows the spread of attitudes and potential addressees along the two first dimensions (that represent about 90% of the variances) of the CA run for each group. The main similarity across age groups is the systematic association by all subjects of the answer “Lower grade student/person” to the zonzai expression on one side of the first dimension, while the other four expressions are on the other side of this axis. Amongst these polite expressions, adults associate kyoshuku to “Higher grade student/person”, teinei and seii to “Teacher”, while heijo is used with “Unknown” addressee. No systematic associations are made with “Classmate”. Both G4 and G5 children tend to associate teinei and seii with “Teacher”; G4 associate kyoshuku with “Classmate”, while G5 with “Higher Grade student”, in a fashion similar to adults. These results suggest that G5 students are approaching an adult-like understanding of kyoshuku, but G4 students still do not have a complete grasp of the social use of kyoshuku. 3.1.2 Question 2 Figure 7 presents the mean degree of “politeness” given by the three test groups to the attitudes, in each presentation modality. Diagrams are used in Figure 7 as a way to compare the similarities and differences among the age groups and the modalities. In terms of overall differences in levels of politeness, we see that teinei is clearly more polite than heijo, with zonzai clearly least polite. So, on one hand we see a gradual negative progression of politeness levels. On the other hand, we see that seii and kyoshuku are perceived in all modalities and by all age groups as less polite than teinei, indicating that these two types of politeness fall outside the polite-impolite dichotomy, thus, underlining the non-linear, multidimensional character of politeness expressions in Japanese culture.

Perceptual changes between adults and children for multimodal im/politeness in Japanese

Figure 6. Position of the stimuli (blue points) and of the addressees (red points) presented to each age group (Adult, G5, G4) along the first two dimensions of the CA (see text)

233

234

Takaaki Shochi, Albert Rilliard & Donna Erickson

Figure 7. Mean and confidence interval of politeness levels attributed to the five expressions (x-axis) by each group of judges (individual lines), in each modality (individual plots)

For the audio modality, we see for all three groups a progressive increase in ratings of politeness: zonzai