HCI International 2019 – Late Breaking Papers: 21st HCI International Conference, HCII 2019, Orlando, FL, USA, July 26–31, 2019, Proceedings [1st ed. 2019] 978-3-030-30032-6, 978-3-030-30033-3

Table of contents :
Front Matter ....Pages i-xxi
Front Matter ....Pages 1-1
Design Approach for Investigating Multimodal Communication in Dismounted Soldier-Robot Interaction (Daniel Barber, Rhyse Bendell)....Pages 3-14
Accessibility Studies: Abuses, Misuses and the Method of Poietic Design (Gian Maria Greco)....Pages 15-27
Design Techniques of Ambient Media Advertisements and Message Comprehension (Yen Hsu, Chia-Jung Lee, Pei-Ying Yang)....Pages 28-39
How to Define the Passenger’s Hazard Perception Level by Combining Subjective and Objective Measures? (Qianjing Hu, Gang Guo, Huaji Wang, Mingqing Tang, Chen Lv, Dongpu Cao)....Pages 40-53
Applying Co-creation Principles to Requirement Elicitation in Manufacturing (Manuel Oliveira, Andrea Bettoni, Eva Coscia, Hans Torvatn)....Pages 54-61
Exploring Color-Universal Design Considering Kansei Differences: Color-Vision Types and Impressions of Color Images (Takashi Sakamoto, Yasuyo G. Ichihara)....Pages 62-72
Systematic Mapping Study: Use of Design Science in Creation and Evaluation of UX Artifacts (Zlatko Stapić, Dijana Plantak Vukovac)....Pages 73-84
Development of a Generator System of Parameterized Questionnaires to Evaluate the Usability of Web Pages (William J. Yamada, Marcelo Morandini, Thiago A. Coleti, Cinthyan Renata S. C. Barbosa)....Pages 85-100
Vehicle Human-Machine Interaction Interface Evaluation Method Based on Eye Movement and Finger Tracking Technology (Mengjin Zeng, Gang Guo, Qiuyang Tang)....Pages 101-115
Front Matter ....Pages 117-117
Optimizing Exploratory Workflows for Embedded Platform Trace Analysis and Its Application to Mobile Devices (Jonathan Ah Sue, Peter Brand, Joachim Falk, Ralph Hasholzner, Jürgen Teich)....Pages 119-139
Evaluating System Sufficiency in a Multimodal, Multiuser Sensemaking Environment Designed for Intelligence Analysis (Shannon Briggs)....Pages 140-152
CiSA: An Inclusive Chatbot Service for International Students and Academics (Jeongyun Heo, Jiyoon Lee)....Pages 153-167
Some Mathematical and Practical Aspects of Decision-Making Based on Similarity (Peeter Lorents, Maryna Averkyna)....Pages 168-179
Explorative Visualization of Food Data to Raise Awareness of Nutritional Value (Anna Lunterova, Ondrej Spetko, George Palamas)....Pages 180-191
A New Paradigm of Addressing the Complexity of Entrepreneurial Community Design Leveraging Augmented Reality (Ke Ma, Yixiang Zhang, Jing Cao)....Pages 192-201
Sentiment Analysis Through Machine Learning for the Support on Decision-Making in Job Interviews (Julio Martínez Zárate, Sandra Mateus Santiago)....Pages 202-213
Front Matter ....Pages 215-215
Stereohaptics Toolkit for Dynamic Tactile Experiences (Ali Israr, Siyan Zhao, Zachary Schwemler, Adam Fritz)....Pages 217-232
A Review of Augmented Reality-Based Human-Computer Interaction Applications of Gesture-Based Interaction (Chutisant Kerdvibulvech)....Pages 233-242
The OTC (Object to Camera) Approach to Visualize Behind Stories of Museum Exhibits (Si Jung SJ Kim, Alexis Sanchez, John Farhad Hanifzai, Francis Palispis, Keitaro Nishimura)....Pages 243-252
Through the Realities of Augmented Reality (Thitirat Siriborvornratanakul)....Pages 253-264
Front Matter ....Pages 265-265
Development of a Puzzle Game to Learn Coding for Elementary Students (Jaisoon Baek, Gyuhwan Oh)....Pages 267-279
Which Virtual Piano Keyboard for Children with Autism? A Pilot Study (Maria Claudia Buzzi, Marina Buzzi, Marco Maugeri, Gabriella Paolini, Maria Teresa Paratore, Alessandra Sbragia et al.)....Pages 280-291
ShadowHunter: Facilitating Children’s Outdoor Exploration with Shadows (Yang Chen, Yuyu Lin, Lijuan Liu, Ziyu Liu, Cheng Yao, Fangtian Ying)....Pages 292-305
An Innovative Employment of Virtual Humans to Explore the Chess Personalities of Garry Kasparov and Other Class-A Players (Khaldoon Dhou)....Pages 306-319
The Relationship Between Game Elements and Player Emotions by Comparing Game Frameworks (Junyao Hu, Tao Xi)....Pages 320-329
Research on Multimedia Teaching in Universities Under Human-Computer Interaction Environment (Xiaoyan Niu)....Pages 330-340
Employing a Voice-Based Emotion-Recognition Function in a Social Chatbot to Foster Social and Emotional Learning Among Preschoolers (Tsai-Hsuan Tsai, Hsien-Tsung Chang, Shin-Da Liao, Hui-Fang Chiu, Ko-Chun Hung, Chun-Yi Kuo et al.)....Pages 341-356
Recommender Systems for an Enhanced Mobile e-Learning (Oswaldo Vélez-Langs, Isaac Caicedo-Castro)....Pages 357-365
Front Matter ....Pages 367-367
Using CFD Technology to Simulate a Model of Human Thermoregulation in the Stable Temperature Environment (Sina Dang, Hongjun Xue, Xiaoyan Zhang, Jue Qu, Chengwen Zhong, Siyu Chen)....Pages 369-381
Interface Design for Boccia Robot Considering Operation Characteristic (Minzhi Deng, Ruya Chen, Shun Song, Junjie He, Rintaro Onishi, Rytaro Suzuki et al.)....Pages 382-394
Development and Evaluation of a Model of Human Comfort and Cognitive Ability for Moderate Differences in Thermal Environment (Shane T. Mueller, Yin-Yin (Sarah) Tan, Isaac Flint)....Pages 395-411
Measuring Nursing Workload in an Intensive Care Unit Using NGOMSL Model (Sivamanoj Sreeramakavacham, Jung Hyup Kim, Laurel Despins)....Pages 412-421
Modeling Drone Crossing Movement with Fitts’ Law (Kaito Yamada, Hiroki Usuba, Homei Miyashita)....Pages 422-432
Research on the Differences of Risk Perception Ability Between Novice and Experienced Drivers (Jie Zhang, Gang Guo, Peizhi Wang, Qiuyang Tang, Zijian Zhang)....Pages 433-447
Front Matter ....Pages 449-449
Feasibility Assessment of Cloud SaaS Enabled Collaboration and Information Confidentiality for the Public Accounting Industry (Lior Baron, Tzipora Halevi)....Pages 451-467
Frictionless Web Payments with Cryptographic Cardholder Authentication (Francisco Corella, Karen Pomian Lewison)....Pages 468-483
A Robust and Real-Time Face Anti-spoofing Method Based on Texture Feature Analysis (Aasim Khurshid, Sergio Cleger Tamayo, Everlandio Fernandes, Mikhail R. Gadelha, Mauro Teofilo)....Pages 484-496
Impacts of Emotional Ambient Sounds on Face Detection Sensitivity (Dong Liu, Pei-Luen Patrick Rau)....Pages 497-506
Human-Centered Manufacturing Challenges Affecting European Industry 4.0 Enabling Technologies (Manuel Oliveira, Emrah Arica, Marta Pinzone, Paola Fantini, Marco Taisch)....Pages 507-517
When to Take a Break? Exploring the Role of Peers on Sedentary Office Workers’ Resting Behaviors (Ling Qin, Xu Sun, Bingjian Liu, Luis Moreno Leyva)....Pages 518-527
NTSB Investigation of Weather-Related Aviation Incidents and Accidents (Sathya Silva, Paul Suffern)....Pages 528-537
Front Matter ....Pages 539-539
Evaluating Citizen Scientists’ User Experience and Engagement Using a Mobile Watershed Data Management App (Ann Fruhling, Shannon L. Bartelt-Hunt, Alan Kolok)....Pages 541-554
UX Analysis of the North Korean Information Technology (Xianglian Han, Sung Woo Kim, Chunghak Oh)....Pages 555-568
Integration and Development of Science, Technology and Innovation in Developed Countries: Perspective from Ocean Policy (Yuta Komori, Yasuyuki Matsuura, Riho Gojo)....Pages 569-579
Dhaka University Telemedicine Programme, Targeting Healthcare-Deprived Rural Population of Bangladesh and Other Low Resource Countries (K. Siddique-e Rabbani, Abdullah Al Amin, Zihad Tarafdar, Md. Abu Yousuf, A. K. M. Bodiuzzaman, Ahmad Imtiaz Khan et al.)....Pages 580-598
Differences in Relationships and Risk Factors Associated with Hypertension, Diabetes, and Proteinuria Among Urban and Rural Adults in Bangladesh (Fumihiko Yokota, Ashir Ahmed)....Pages 599-610
Back Matter ....Pages 611-612

Citation preview

LNCS 11786

Constantine Stephanidis (Ed.)

HCI International 2019 – Late Breaking Papers 21st HCI International Conference, HCII 2019 Orlando, FL, USA, July 26–31, 2019 Proceedings

Lecture Notes in Computer Science Founding Editors Gerhard Goos Karlsruhe Institute of Technology, Karlsruhe, Germany Juris Hartmanis Cornell University, Ithaca, NY, USA

Editorial Board Members Elisa Bertino Purdue University, West Lafayette, IN, USA Wen Gao Peking University, Beijing, China Bernhard Steffen TU Dortmund University, Dortmund, Germany Gerhard Woeginger RWTH Aachen, Aachen, Germany Moti Yung Columbia University, New York, NY, USA

11786

More information about this series at http://www.springer.com/series/7409

Constantine Stephanidis (Ed.)

HCI International 2019 – Late Breaking Papers 21st HCI International Conference, HCII 2019 Orlando, FL, USA, July 26–31, 2019 Proceedings

123

Editor Constantine Stephanidis University of Crete and Foundation for Research and Technology Hellas (Forth) Heraklion, Greece

ISSN 0302-9743 ISSN 1611-3349 (electronic) Lecture Notes in Computer Science ISBN 978-3-030-30032-6 ISBN 978-3-030-30033-3 (eBook) https://doi.org/10.1007/978-3-030-30033-3 LNCS Sublibrary: SL3 – Information Systems and Applications, incl. Internet/Web, and HCI © Springer Nature Switzerland AG 2019 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Foreword

The 21st International Conference on Human-Computer Interaction, HCI International 2019, was held in Orlando, FL, USA, during July 26–31, 2019. The event incorporated the 18 thematic areas and affiliated conferences listed on the following page. This year the HCI International (HCII) conference introduced the additional option of “late-breaking work.” This applies both for papers and posters with the corresponding volumes of the proceedings published after the conference. A total of 5,029 individuals from academia, research institutes, industry, and governmental agencies from 73 countries submitted contributions, and 1,274 papers and 209 posters were included in the pre-conference proceedings. In addition, 46 papers and 56 posters were included in the post-conference proceedings. These contributions address the latest research and development efforts and highlight the human aspects of design and use of computing systems. The contributions thoroughly cover the entire field of human-computer interaction, addressing major advances in knowledge and effective use of computers in a variety of application areas. The volumes constituting the full set of the pre-conference and post-conference proceedings are listed in the following pages. I would like to thank the program board chairs and the members of the program boards of all thematic areas and affiliated conferences for their contribution to the highest scientific quality and the overall success of the HCI International 2019 conference. This conference would not have been possible without the continuous and unwavering support and advice of the founder, Conference General Chair Emeritus and Conference Scientific Advisor Prof. Gavriel Salvendy. For his outstanding efforts, I would like to express my appreciation to the communications chair and editor of HCI International News, Dr. Abbas Moallem. July 2019

Constantine Stephanidis

HCI International 2019 Thematic Areas and Affiliated Conferences

Thematic areas: • HCI 2019: Human-Computer Interaction • HIMI 2019: Human Interface and the Management of Information Affiliated conferences: • EPCE 2019: 16th International Conference on Engineering Psychology and Cognitive Ergonomics • UAHCI 2019: 13th International Conference on Universal Access in Human-Computer Interaction • VAMR 2019: 11th International Conference on Virtual, Augmented and Mixed Reality • CCD 2019: 11th International Conference on Cross-Cultural Design • SCSM 2019: 11th International Conference on Social Computing and Social Media • AC 2019: 13th International Conference on Augmented Cognition • DHM 2019: 10th International Conference on Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management • DUXU 2019: 8th International Conference on Design, User Experience, and Usability • DAPI 2019: 7th International Conference on Distributed, Ambient and Pervasive Interactions • HCIBGO 2019: 6th International Conference on HCI in Business, Government and Organizations • LCT 2019: 6th International Conference on Learning and Collaboration Technologies • ITAP 2019: 5th International Conference on Human Aspects of IT for the Aged Population • HCI-CPT 2019: First International Conference on HCI for Cybersecurity, Privacy and Trust • HCI-Games 2019: First International Conference on HCI in Games • MobiTAS 2019: First International Conference on HCI in Mobility, Transport, and Automotive Systems • AIS 2019: First International Conference on Adaptive Instructional Systems

Conference Proceedings Volumes Full List Pre-conference Proceedings Volumes 1. LNCS 11566, Human-Computer Interaction: Perspectives on Design (Part I), edited by Masaaki Kurosu 2. LNCS 11567, Human-Computer Interaction: Recognition and Interaction Technologies (Part II), edited by Masaaki Kurosu 3. LNCS 11568, Human-Computer Interaction: Design Practice in Contemporary Societies (Part III), edited by Masaaki Kurosu 4. LNCS 11569, Human Interface and the Management of Information: Visual Information and Knowledge Management (Part I), edited by Sakae Yamamoto and Hirohiko Mori 5. LNCS 11570, Human Interface and the Management of Information: Information in Intelligent Systems (Part II), edited by Sakae Yamamoto and Hirohiko Mori 6. LNAI 11571, Engineering Psychology and Cognitive Ergonomics, edited by Don Harris 7. LNCS 11572, Universal Access in Human-Computer Interaction: Theory, Methods and Tools (Part I), edited by Margherita Antona and Constantine Stephanidis 8. LNCS 11573, Universal Access in Human-Computer Interaction: Multimodality and Assistive Environments (Part II), edited by Margherita Antona and Constantine Stephanidis 9. LNCS 11574, Virtual, Augmented and Mixed Reality: Multimodal Interaction (Part I), edited by Jessie Y. C. Chen and Gino Fragomeni 10. LNCS 11575, Virtual, Augmented and Mixed Reality: Applications and Case Studies (Part II), edited by Jessie Y. C. Chen and Gino Fragomeni 11. LNCS 11576, Cross-Cultural Design: Methods, Tools and User Experience (Part I), edited by P. L. Patrick Rau 12. LNCS 11577, Cross-Cultural Design: Culture and Society (Part II), edited by P. L. Patrick Rau 13. LNCS 11578, Social Computing and Social Media: Design, Human Behavior and Analytics (Part I), edited by Gabriele Meiselwitz 14. LNCS 11579, Social Computing and Social Media: Communication and Social Communities (Part II), edited by Gabriele Meiselwitz 15. LNAI 11580, Augmented Cognition, edited by Dylan D. Schmorrow and Cali M. Fidopiastis 16. LNCS 11581, Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management: Human Body and Motion (Part I), edited by Vincent G. Duffy

x

Conference Proceedings Volumes Full List

17. LNCS 11582, Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management: Healthcare Applications (Part II), edited by Vincent G. Duffy 18. LNCS 11583, Design, User Experience, and Usability: Design Philosophy and Theory (Part I), edited by Aaron Marcus and Wentao Wang 19. LNCS 11584, Design, User Experience, and Usability: User Experience in Advanced Technological Environments (Part II), edited by Aaron Marcus and Wentao Wang 20. LNCS 11585, Design, User Experience, and Usability: Application Domains (Part III), edited by Aaron Marcus and Wentao Wang 21. LNCS 11586, Design, User Experience, and Usability: Practice and Case Studies (Part IV), edited by Aaron Marcus and Wentao Wang 22. LNCS 11587, Distributed, Ambient and Pervasive Interactions, edited by Norbert Streitz and Shin’ichi Konomi 23. LNCS 11588, HCI in Business, Government and Organizations: eCommerce and Consumer Behavior (Part I), edited by Fiona Fui-Hoon Nah and Keng Siau 24. LNCS 11589, HCI in Business, Government and Organizations: Information Systems and Analytics (Part II), edited by Fiona Fui-Hoon Nah and Keng Siau 25. LNCS 11590, Learning and Collaboration Technologies: Designing Learning Experiences (Part I), edited by Panayiotis Zaphiris and Andri Ioannou 26. LNCS 11591, Learning and Collaboration Technologies: Ubiquitous and Virtual Environments for Learning and Collaboration (Part II), edited by Panayiotis Zaphiris and Andri Ioannou 27. LNCS 11592, Human Aspects of IT for the Aged Population: Design for the Elderly and Technology Acceptance (Part I), edited by Jia Zhou and Gavriel Salvendy 28. LNCS 11593, Human Aspects of IT for the Aged Population: Social Media, Games and Assistive Environments (Part II), edited by Jia Zhou and Gavriel Salvendy 29. LNCS 11594, HCI for Cybersecurity, Privacy and Trust, edited by Abbas Moallem 30. LNCS 11595, HCI in Games, edited by Xiaowen Fang 31. LNCS 11596, HCI in Mobility, Transport, and Automotive Systems, edited by Heidi Krömker 32. LNCS 11597, Adaptive Instructional Systems, edited by Robert Sottilare and Jessica Schwarz 33. CCIS 1032, HCI International 2019 - Posters (Part I), edited by Constantine Stephanidis

Conference Proceedings Volumes Full List

xi

34. CCIS 1033, HCI International 2019 - Posters (Part II), edited by Constantine Stephanidis 35. CCIS 1034, HCI International 2019 - Posters (Part III), edited by Constantine Stephanidis

Post-conference Proceedings 36. LNCS 11786, HCI International 2019 – Late Breaking Papers, edited by Constantine Stephanidis 37. CCIS 1088, HCI International 2019 – Late Breaking Posters, edited by Constantine Stephanidis and Margherita Antona

http://2019.hci.international/proceedings

HCI International 2019 (HCII 2019) The full list with the Program Board Chairs and the members of the Program Boards of all thematic areas and affiliated conferences is available online at:

http://www.hci.international/board-members-2019.php

HCI International 2020 The 22nd International Conference on Human-Computer Interaction, HCI International 2020, will be held jointly with the affiliated conferences in Copenhagen, Denmark, at the Bella Center Copenhagen, July 19–24, 2020. It will cover a broad spectrum of themes related to HCI, including theoretical issues, methods, tools, processes, and case studies in HCI design, as well as novel interaction techniques, interfaces, and applications. The proceedings will be published by Springer. More information will be available on the conference website: http://2020.hci.international/. General Chair Prof. Constantine Stephanidis University of Crete and ICS-FORTH Heraklion, Crete, Greece E-mail: [email protected]

http://2020.hci.international/

Contents

User Experience Design and Evaluation Design Approach for Investigating Multimodal Communication in Dismounted Soldier-Robot Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . Daniel Barber and Rhyse Bendell Accessibility Studies: Abuses, Misuses and the Method of Poietic Design . . . Gian Maria Greco Design Techniques of Ambient Media Advertisements and Message Comprehension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yen Hsu, Chia-Jung Lee, and Pei-Ying Yang How to Define the Passenger’s Hazard Perception Level by Combining Subjective and Objective Measures? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qianjing Hu, Gang Guo, Huaji Wang, Mingqing Tang, Chen Lv, and Dongpu Cao

3 15

28

40

Applying Co-creation Principles to Requirement Elicitation in Manufacturing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Manuel Oliveira, Andrea Bettoni, Eva Coscia, and Hans Torvatn

54

Exploring Color-Universal Design Considering Kansei Differences: Color-Vision Types and Impressions of Color Images . . . . . . . . . . . . . . . . . Takashi Sakamoto and Yasuyo G. Ichihara

62

Systematic Mapping Study: Use of Design Science in Creation and Evaluation of UX Artifacts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zlatko Stapić and Dijana Plantak Vukovac

73

Development of a Generator System of Parameterized Questionnaires to Evaluate the Usability of Web Pages . . . . . . . . . . . . . . . . . . . . . . . . . . . William J. Yamada, Marcelo Morandini, Thiago A. Coleti, and Cinthyan Renata S. C. Barbosa Vehicle Human-Machine Interaction Interface Evaluation Method Based on Eye Movement and Finger Tracking Technology . . . . . . . . . . . . . . . . . . Mengjin Zeng, Gang Guo, and Qiuyang Tang

85

101

xviii

Contents

Information, Visualization and Decision Making Optimizing Exploratory Workflows for Embedded Platform Trace Analysis and Its Application to Mobile Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jonathan Ah Sue, Peter Brand, Joachim Falk, Ralph Hasholzner, and Jürgen Teich

119

Evaluating System Sufficiency in a Multimodal, Multiuser Sensemaking Environment Designed for Intelligence Analysis . . . . . . . . . . . . . . . . . . . . . Shannon Briggs

140

CiSA: An Inclusive Chatbot Service for International Students and Academics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jeongyun Heo and Jiyoon Lee

153

Some Mathematical and Practical Aspects of Decision-Making Based on Similarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Peeter Lorents and Maryna Averkyna

168

Explorative Visualization of Food Data to Raise Awareness of Nutritional Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Anna Lunterova, Ondrej Spetko, and George Palamas

180

A New Paradigm of Addressing the Complexity of Entrepreneurial Community Design Leveraging Augmented Reality. . . . . . . . . . . . . . . . . . . Ke Ma, Yixiang Zhang, and Jing Cao

192

Sentiment Analysis Through Machine Learning for the Support on Decision-Making in Job Interviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . Julio Martínez Zárate and Sandra Mateus Santiago

202

Virtual and Augmented Reality Stereohaptics Toolkit for Dynamic Tactile Experiences . . . . . . . . . . . . . . . . Ali Israr, Siyan Zhao, Zachary Schwemler, and Adam Fritz A Review of Augmented Reality-Based Human-Computer Interaction Applications of Gesture-Based Interaction . . . . . . . . . . . . . . . . . . . . . . . . . Chutisant Kerdvibulvech The OTC (Object to Camera) Approach to Visualize Behind Stories of Museum Exhibits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Si Jung SJ Kim, Alexis Sanchez, John Farhad Hanifzai, Francis Palispis, and Keitaro Nishimura Through the Realities of Augmented Reality. . . . . . . . . . . . . . . . . . . . . . . . Thitirat Siriborvornratanakul

217

233

243

253

Contents

xix

Learning and Games Development of a Puzzle Game to Learn Coding for Elementary Students . . . Jaisoon Baek and Gyuhwan Oh

267

Which Virtual Piano Keyboard for Children with Autism? A Pilot Study . . . . Maria Claudia Buzzi, Marina Buzzi, Marco Maugeri, Gabriella Paolini, Maria Teresa Paratore, Alessandra Sbragia, Caterina Senette, and Amaury Trujillo

280

ShadowHunter: Facilitating Children’s Outdoor Exploration with Shadows. . . Yang Chen, Yuyu Lin, Lijuan Liu, Ziyu Liu, Cheng Yao, and Fangtian Ying

292

An Innovative Employment of Virtual Humans to Explore the Chess Personalities of Garry Kasparov and Other Class-A Players . . . . . . . . . . . . . Khaldoon Dhou

306

The Relationship Between Game Elements and Player Emotions by Comparing Game Frameworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Junyao Hu and Tao Xi

320

Research on Multimedia Teaching in Universities Under Human-Computer Interaction Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiaoyan Niu

330

Employing a Voice-Based Emotion-Recognition Function in a Social Chatbot to Foster Social and Emotional Learning Among Preschoolers . . . . . Tsai-Hsuan Tsai, Hsien-Tsung Chang, Shin-Da Liao, Hui-Fang Chiu, Ko-Chun Hung, Chun-Yi Kuo, and Chih-Wei Yang Recommender Systems for an Enhanced Mobile e-Learning . . . . . . . . . . . . . Oswaldo Vélez-Langs and Isaac Caicedo-Castro

341

357

Human and Task Models in HCI Using CFD Technology to Simulate a Model of Human Thermoregulation in the Stable Temperature Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . Sina Dang, Hongjun Xue, Xiaoyan Zhang, Jue Qu, Chengwen Zhong, and Siyu Chen Interface Design for Boccia Robot Considering Operation Characteristic . . . . Minzhi Deng, Ruya Chen, Shun Song, Junjie He, Rintaro Onishi, Rytaro Suzuki, Ryuta Motegi, Naoyuki Takesue, Shin Tsuchiya, Yoshiki Shimomura, Nami Shida, Osamu Nitta, and Keiko Kasamatsu

369

382

xx

Contents

Development and Evaluation of a Model of Human Comfort and Cognitive Ability for Moderate Differences in Thermal Environment . . . . . . . . . . . . . . Shane T. Mueller, Yin-Yin (Sarah) Tan, and Isaac Flint

395

Measuring Nursing Workload in an Intensive Care Unit Using NGOMSL Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sivamanoj Sreeramakavacham, Jung Hyup Kim, and Laurel Despins

412

Modeling Drone Crossing Movement with Fitts’ Law . . . . . . . . . . . . . . . . . Kaito Yamada, Hiroki Usuba, and Homei Miyashita Research on the Differences of Risk Perception Ability Between Novice and Experienced Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jie Zhang, Gang Guo, Peizhi Wang, Qiuyang Tang, and Zijian Zhang

422

433

Design and User Experience Case Studies Feasibility Assessment of Cloud SaaS Enabled Collaboration and Information Confidentiality for the Public Accounting Industry. . . . . . . . Lior Baron and Tzipora Halevi Frictionless Web Payments with Cryptographic Cardholder Authentication . . . Francisco Corella and Karen Pomian Lewison A Robust and Real-Time Face Anti-spoofing Method Based on Texture Feature Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Aasim Khurshid, Sergio Cleger Tamayo, Everlandio Fernandes, Mikhail R. Gadelha, and Mauro Teofilo Impacts of Emotional Ambient Sounds on Face Detection Sensitivity . . . . . . Dong Liu and Pei-Luen Patrick Rau Human-Centered Manufacturing Challenges Affecting European Industry 4.0 Enabling Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Manuel Oliveira, Emrah Arica, Marta Pinzone, Paola Fantini, and Marco Taisch When to Take a Break? Exploring the Role of Peers on Sedentary Office Workers’ Resting Behaviors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ling Qin, Xu Sun, Bingjian Liu, and Luis Moreno Leyva NTSB Investigation of Weather-Related Aviation Incidents and Accidents . . . Sathya Silva and Paul Suffern

451 468

484

497

507

518 528

Contents

xxi

HCI, Innovation and Society Evaluating Citizen Scientists’ User Experience and Engagement Using a Mobile Watershed Data Management App . . . . . . . . . . . . . . . . . . . Ann Fruhling, Shannon L. Bartelt-Hunt, and Alan Kolok UX Analysis of the North Korean Information Technology . . . . . . . . . . . . . Xianglian Han, Sung Woo Kim, and Chunghak Oh Integration and Development of Science, Technology and Innovation in Developed Countries: Perspective from Ocean Policy . . . . . . . . . . . . . . . Yuta Komori, Yasuyuki Matsuura, and Riho Gojo Dhaka University Telemedicine Programme, Targeting Healthcare-Deprived Rural Population of Bangladesh and Other Low Resource Countries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K. Siddique-e Rabbani, Abdullah Al Amin, Zihad Tarafdar, Md. Abu Yousuf, A. K. M. Bodiuzzaman, Ahmad Imtiaz Khan, Papia Chowdhury, Kamrul Hussain, Shahed Md. Abu Sufian, Maruf Ahmad, Md. Moniruzzaman, and Ashir Ahmed Differences in Relationships and Risk Factors Associated with Hypertension, Diabetes, and Proteinuria Among Urban and Rural Adults in Bangladesh: -Findings from Portable Health Clinic Research Project 2013–2018- . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fumihiko Yokota and Ashir Ahmed Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

541 555

569

580

599

611

User Experience Design and Evaluation

Design Approach for Investigating Multimodal Communication in Dismounted Soldier-Robot Interaction Daniel Barber(&) and Rhyse Bendell University of Central Florida, Institute for Simulation and Training, Orlando, FL 32826, USA {dbarber,rbendell}@ist.ucf.edu Abstract. For several decades there has been continuous growth in the field of robotics, with recent trends driving towards a vision of humans collaborating in a cohesive unit with automated counterparts. Enabling true mixed-initiative teaming between a human and robot will require communication capabilities and cognition comparable to human teammates. Multimodal communication is a framework in which interfaces can be created supporting the flexible selection of different modalities (e.g. speech, gestures) for these transactions. A major challenge for human factors researchers investigating human robot collaboration with multimodal interfaces is the current limitations of robots. Therefore, simulations and wizard-of-oz type experiments are heavily employed to measure performance, workload, and other factors in future mixed-initiative scenarios. Although these techniques facilitate experimentation, it can be difficult to transition findings to working prototypes of today’s robots. For example, a researcher finds an effective way to convey a robots decision making rationale in a simulation-based study, but has no working robot that can drive the content in reality. Furthermore, the literature regarding multimodal communication with robots applied to the military domain is limited. For example, evaluation of different modalities as part of an interrupting task has been explored in driving scenarios, but not between robot(s) and soldiers. In many cases there is conflict in findings across domains. To address this challenge, this paper describes how a multimodal interface for a real robotic platform developed under the U.S. Army Research Labs Robotics Collaborative Technology Alliance (RCTA) was extended to support standalone simulation of interactions and integration with simulated virtual environments. This functionality enables researchers to assess new interaction techniques using the same software that will interact with a real platform to facilitate transition of their research. Furthermore, experiment design approaches including theory-based tasks in a military relevant mission (Cordon and Search) are discussed. Keywords: Multimodal interfaces
Human-robot interaction Tactile display
Communication modalities

© Springer Nature Switzerland AG 2019 C. Stephanidis (Ed.): HCII 2019, LNCS 11786, pp. 3–14, 2019. https://doi.org/10.1007/978-3-030-30033-3_1




Simulation




4

D. Barber and R. Bendell

1 Introduction Robotic technologies have recently made significant advances towards the goal of autonomy. Though some fields may not require or even benefit from autonomous robotic systems (e.g., perhaps we will never be comfortable with fully autonomous surgeons), there are significant improvements that can be made in areas such as search and rescue, emergency response, military operations, etc. which involve high workload, hazardous scenarios. Putting aside fears of potentially malicious AI (a la Isaac Asimov, Stanley Kubrick, or William Gibson), it may be possible to employ such systems to replace humans in life-risking operations to not only protect but to save lives [1–3]. As improvements are made to robotic technologies, it is important for researchers to stay ahead of the curve and attend to the potential directions in which autonomous robots may develop. Anticipating near-future advances improves our ability to curb dangerous approaches as well as optimize the way that technologies are leveraged. A promising direction that is currently the focus of a great deal of research is the transition of near-autonomous robots from tools that may be used to accomplish an operator’s goals into teammates that may assist with a human’s goal while also attending to separate objectives [4, 5]. The dream of single operator, multi-robot teams may still be a ways off [6, 7]; however, multi-human teams which incorporate intelligent robotic teammates are very nearly a reality. That said, there are a number of challenges beyond the technological (e.g., computer vision, object recognition, artificial intelligence networks, etc.) that will need to be tackled before humans can effectively partner with robots [8]. Our understanding of human-human teaming has been systematically improved for centuries, yet gaps still exist in our knowledge of how best to optimize team performance; the introduction of non-human intelligences into the mix will necessarily complicate matters further [9–11]. Fortunately, we have some understanding of how humans partner with non-humans as a result of human-animal teams which have proven to be extremely effective in a host of scenarios from law enforcement to search and rescue [12]. Human-dog teams, for instance, have functioned effectively both with and without the assistance of technology (see Bozkurt et al. [13] for an interesting use of technology in this field). A fundamental difference between current human-nonhuman teams and the vision for the future is that autonomous robots may be designed to communicate with a human far more naturally than animals (e.g., speech, graphical depictions, intuitive data arrays) [14]. Though current methods of human-robot communication are primarily limited to video feeds and environmental data transmission [15–17] the future of human-robot teams (HRT) will depend on the expansion of communication modalities to support more natural interactions [10, 18–21] (see Fig. 1).

Design Approach for Investigating Multimodal Communication

5

Fig. 1. A diagrammatic representation of the type of multi-modal communication that will be needed to facilitate future HRT communications: bi-directional auditory communication, visual and touch/button interaction facilitated by an interface, and tactile communication which would likely only need to be one way and is represented here as a tactile belt.

2 Multimodal Communication and Teaming Currently, the needs of human members of HRTs are not clearly understood [22], but it is known that in the context of high-workload situations human performance benefits from reducing cognitive load as much as possible [23–25]. Multi-modal communication (MMC) presents a solution to the danger of overloading human operators by leveraging the fact that the brain is more capable of attending to more information if it is split across modalities [26, 27]. By flexibly utilizing both explicit (e.g., speech, visuals, tactile displays) and implicit communication modes, MMC techniques offload information processing demands to facilitate interactions and improve performance [21]. To emphasis this point, further, one should consider the operational environment in which human-robot teaming will take place. For a soldier, the mission space has potential for noise, low visibility, and many dynamic events. Speech may be the

6

D. Barber and R. Bendell

primary method of human-human communication, but visual signals (gestures), or touch (hand on shoulder) provide signal redundancy to ensure the message is received and understood. Consider the case of a HRT consisting of a dismounted soldier on patrol while interacting with a robotic teammate that is conducting a search of an area: initial communications may primarily be speech based commands and acknowledgements, but once the robot begins to encounter objects of interest it must either send images, video feed, or descriptions to their human counterpart. The human, in this case, would already be experiencing visual load as a result of their patrol, so although images may be most effective for communicating findings it may be better in that situation to generate speech descriptions to which the soldier may attend without reducing the quality of the patrol. The additional of tactile messages may also emphasis the content in speech further, [28]. It is presently impossible to wholly determine the optimal ways to support teaming in such a scenario because the teammates and MMC tools that will be needed to run assessments in the field do not exist.

3 Researching Future HRI: Physical and Virtual Simulation A challenge for current investigations of HRI is the lack of functional autonomous robots with which to test interactions in relevant scenarios. A possible approach to tackling that issue is to create representative tools/teammates which are capable of simulating a given scenario; however, implementing a real-world simulation of even giving the appearance of needed capability requires significant resources. A far less demanding approach is to virtually simulate capabilities in a laboratory setting, though this approach yields findings that are less applicable to real world interactions. Fundamentally, the incongruence between the experience of participating in an experiment and actually interacting with a robot in an ongoing mission reduces the degree to which such investigations can accurately predict interaction outcomes and thereby deduce methods for supporting effective teaming. Though it is important to note those drawbacks, simulating near-future HRI is currently the most viable approach to preparing for the arrival of usable technologies, and therefore it is more relevant to consider the type and design of simulations that can most accurately approximate reality. Physical simulations are a common tool for evaluating human performance, and can be used for nearly any scenario that does not incorporate undue risk (e.g., firefighters may run drills in burning buildings, but running search and rescue drills in the presence of real radioactive fallout is likely not advisable or necessary) [29]. Jentsch et al. 2010 describe the pros and cons of a scaled Model Operations in Urban Terrain (MOUT) physical simulation that makes heavy use of the “wizard-of-oz” technique to simulate interactions between human operators and remote robotic teammates in a military context. Here, the wizard-of-oz technique essentially describes the use of predetermined events, confederates, and faked information/communications to create the illusion of conducting a real mission in a relevant “environment” without requiring functional autonomous systems or a real world environment in which to test them. Real world technologies and spaces are not required by this approach, however, a great deal of effort, preparation, and maintenance is still necessary to employ a physical simulation such as the MOUT as the faked technologies and environments must still be implemented (see Fig. 2) [30].

Design Approach for Investigating Multimodal Communication

7

Pulley system for operating UAVs

Graduate student for scale

MOUT Urban Environment

Fig. 2. The scaled MOUT environment including a model urban environment, remote control “autonomous” ground vehicles, and a pulley system for controlling “autonomous” aerial vehicles.

Another approach to simulating near-future HRI is to employ virtual reality simulations. Several such simulations have been developed in the last decade, particularly for the investigation of military focused HRI. The Mixed-Initiative Experimental (MIX) Testbed is one such simulation which provides simulated ground and air robotic systems in a 3D environment as well as an operator control unit (OCU) which allows users to interact with the virtual systems [32]. A benefit of the MIX simulator over physical simulations such as the MOUT is that it does not require a large room to run nor confederates to operate the simulated autonomous teammates. On the other hand, the use of confederates does allow for a larger degree of flexibility than preprogrammed interactions or capabilities which constrain interaction possibilities From an experimental perspective, however, the repeatability of predetermined actions is often better than allowing too much freedom (e.g., two participants who engaged in different interactions may not be directly compared). The MIX testbed has supported several investigations focused on HRT, but did not address the need for MMC which supports verbal communication and virtual multi-tasking (though participants could, of course, be tasked with real-world tasks while interacting with the MIX).

8

D. Barber and R. Bendell

The Virtual Test Bed (VTB) was developed as an extension of the MIX to more effectively study multi-modal communication in HRTs. It not only includes a virtual monitoring task which simulates ongoing activities of a human team member (executed by the user), but also incorporates a prototype Multi-Modal Interface (MMI) developed with the support of the Robotics Collaborative Technology Alliance (RCTA). The RCTA MMI is a real-world device designed explicitly to support multi-modal communication for HRTs in military operations; even though the tasks and events simulated in the VTB are faked, the interactions themselves are identical to what would be experienced in near-future teams (note that the MMI software is presented within the VTB, the physical device itself is not used). The one drawback of the original VTB was that users had to engage in their tasks and interactions through a desktop computer and monitor and did not have a viewpoint which immersed them in the virtual environment. Advances in commercial virtual reality displays (primarily head mounted displays and motion tracked controllers) allowed the issue of immersion to be properly addressed and led to another overhaul of the VTB: extension into immersive virtual reality.

Fig. 3. Participant view of a perimeter monitoring task taking place in a simulated urban environment in the VRMIX.

Design Approach for Investigating Multimodal Communication

9

The immersive version of the MIX/VTB style HRI testing environment, the VRMIX, relies on an HTC Vive and paired handheld controller to put participants right into a virtually simulated mission environment (see Fig. 3). Much like the VTB, the VRMIX has the capacity to simulate autonomus teammates in addition to a variety of tasks and operations for human-in-the-loop experimentation [31]; however, it also has the vital capability of allowing users to interact with a simulated MMI that they can “hold” (insofar as they hold and interact with a controller which is represented in VR) and use to communicate with a remote autonomous teammate. As indicated in Fig. 1, several wearable devices may be required to support MMC with the RCTA MMI, potentially including wireless headset, a tablet of visual display, and a tactile display such as a tactor belt or vest depending on need. The VRMIX simulation testbed has the capacity to simulate interaction through each of those required modalities by generating simulated events which trasmit real data to most any device in order to elicit natural communication experiences. As such, the VRMIX provides a unique method for evaluating HRT effectiveness in relevant scenarios while supporting environments, tasks, and interactions. Moreover, VRMIX support dismounted HRT while environments like the scaled MOUT and MIX testbed do not.

4 HRI Research Example: Dismounted Solider-Robot Teams Although it is known that cognitive overload may be ameliorated by distributing loading between processing modes, the optimal method for balancing loading and especially for implementing a system to execute that balancing is far from being well understood. Accordingly, it is important at this stage in MMC research to address the issues of when, why, and how to make use of each available modality. Here, we describe a possible research approach for investigating the specific performance benefits introduced by the tactile modality as a supplementary avenue of communication for human-robot teams. Tactile communication is a potential untapped resource for use in augmenting robot-to-human communication. The use of haptic cues and feedback is a well known method for notifications in commercial products like cell phone and gaming consoles. Moreover, efforts are ongoing to deliver hands-free navigation using tactile belts for the U.S. Army, enabling a Soldier to maintain light discipline and keep their “heads-up” while traveling in a pre-defined point, [31]. Research to extend a tactile displays ability to convey content similar or complementary to speech are still early, but show promise. Barber, et al., conducted a series of studies to determine the feasibility of delivering two-word phrases within a HRT task, [28]. Results of this effort showed the ability of participants to receive simple reports such as “danger to the north” with high reliability and low response times. Although promising, it is still unclear if one may benefit from having this additional delivery modality paired with speech or other forms of communication from a robot.

10

D. Barber and R. Bendell

The virtual simulation approach that we propose here for investigating tactile communications within MMC makes use of both the VRMIX environment, and the Cordon and Search (C&S) operational context briefly described in the introduction. Though the focus of such an experiment would be centered on the potential performance augmenting effects of tactile communication, it is important to keep in mind that auditory and visual communication may also be provided and evaluated as comparative standards. The basic approach that we suggest is a systematic combination/evaluation of communication modalities in a mission context that allows for empirical performance evaluation (e.g., assessing performance of the Cordon and Search task with the provision of communication via visual, visual + auditory, visual + tactile, auditory + tactile, etc.). The experimental approach would ideally directly compare each combination of communication modalities with respect to two primary outcomes: performance of the monitoring tasks that are vital to the Cordon aspect of C&S, as well as performance of the Search component of the task. The paradigm that we propose separates the two elements of the task between a human and robotic teammate such that the human (the participant in the context of human subjects research) completes the perimeter monitoring component of the C&S task while also monitoring reports from an autonomous robotic teammate as they ostensibly complete the search component. Using a wizardof-oz approach, the robot teammate searches the inner cordon area and reports findings using the proposed combinations of modalities. Accordingly, the tasks administered to the human teammate would include the perimeter monitoring task as well as a dedicated robot monitoring or communication task. Evaluation of communication modality effectiveness would therefore be accomplished via investigation of improved or reduced performance of the communication task. An important a priori decision for such an experiment is the quantification of performance metrics. Performance of a perimeter monitoring task may be quantified well enough through signal detection metrics (see 30, 33), however, the success and effectiveness of communication exchanges as they relate to team performance is not so easily determined by a generalizable paradigm. We suggest that one effective approach to measuring the effectiveness of communications is to measure the development and maintenance of team situational awareness in the context of a given mission context. Situational awareness (SA) generally refers to ones understanding of the current state as well as potential future states of a given situation as it relates to a known (or developing) set of objectives or goals. Given the context of a C&S operation, we propose that evaluation of the quality (and therefore measured effectiveness) of SA should follow from the nature of the search at hand. Consider, for example, a C&S operation that is concerned with the identification of potential bomb-making materials in an urban environment. While the human teammate is busy conducting a monitoring task to ensure the safety of the operation, a robotic teammate may sweep the cordoned area and relay information regarding the presence of potentially dangerous materials. Awareness of the robotic teammate’s location, current/past/future actions, findings, and status are all important aspects of communicable information which lends itself to evaluation in that context. Accordingly, if the robotic teammates actions, findings, and communications are predetermined (as they necessarily would be then the following performance data may be assessed during the course of the mission and in the context of communication modalities:

Design Approach for Investigating Multimodal Communication

11

overall team performance in identifying hazardous materials, the human’s ability to recall and recognize information regarding reports that occur during the mission, self-reported and objective outcomes with respect to the human teammate’s ability to interpret the communications sent by their robotic counterpart, the development and quality of situational awareness as it develops over the course of a mission and in the context of various loading conditions, and the overall performance of the human’s primary perimeter monitoring task. Note also that it may be relevant to consider individual differences in the ways that human teammates interact with their virtual counterpart as these variations have been shown to have a significant effect on both individual and team behaviors. Particularly, considering the virtual reality administration method and military focused scenario, some relevant experience should be accounted for to avoid confounding effects. Table 1 below details the possible independent and dependent variables included in the proposed investigative approach. Table 1. Design approach for investigating MMC Independent Individual Differences • Biological Sex • Video game experience • Virtual reality experience • Military experience (e.g. rank, deployment, time in service, etc.) • Education level • Robotics Experience Multi-modal communication set: • Auditory • Visual • Auditory + Visual • Auditory + Tactile • Visual + Tactile • Visual + Auditory + tactile Threat detection task demand levels: • Low • High

Dependent Correlated with • Mental workload (NASA-TLX score) • Working memory (recall probe score) • Situation awareness (SA probe scores) • Multiple Resource Questionaire (MRQ)

Effects on: –Mental workload (TLX score) –Situation awareness (SA probe scores) –Physiological response – Microsoft Band 2 (HRV, IBI, HR) –Response time (IRT, SDT) –Identification percent accuracy (SDT) –Identification error rate (SDT) –Effects on task performance: • Percent accuracy • Error rate • Response time

Additional measures that may be relevant to the evaluation of the effectiveness of MMC pairings/sets include measures of perceived workload which may include selfreport measures (such as the NASA-Task Load Index, see [33]) or physiological response measures such as Heart-Rate Variability (HRV), Inter-Beat Interval, or Galvanic Skin Response (GSR) which have been shown to correlate with workload, [18, 34]. Physiological responses may prove especially useful for future implementations of MMC in real world systems as they have the potential to provide real time information that could be used to tailor communication modalities on-the-fly in order to optimize performance.

12

D. Barber and R. Bendell

5 Conclusions The purpose for this paper is to capture current gaps in the development and assessment of multimodal communication in squad-level human robot teams. Although advances are rapid in the area of machine intelligence, the ability to perform as a cohesive team in environments relevant to the military is not yet achieved. Therefore simulation techniques are necessary to explore how well research findings from human-human communication translate to human-robot teams, as well as how to take advantage of the “super-human” capabilities robots can provide. The VRMIX testbed described provides a platform to emulate future military relevant scenarios with soldier-robot teams to evaluate multimodal communication strategies. Finally, an example experiment focusing on Cordon & Search with a robot team member to investigate tactile communications is proposed with goal of advancing our understanding of how tactile messages may improve communication and situation awareness. Acknowledgement. This research was sponsored by the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF-10-2-0016. The views and conclusions contained in this document are those of the author’s and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.

References 1. Burke, J.L., Murphy, R.R.: Human-robot interaction in USAR technical search: two heads are better than one. In: 13th IEEE International Workshop on Robot and Human Interactive Communication, 2004. ROMAN 2004, pp. 307–312. IEEE, September 2004 2. Casper, J., Murphy, R.R.: Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center. IEEE Trans. Syst. Man Cybern. Part B Cybern. 33(3), 367–385 (2003). https://doi.org/10.1109/TSMCB.2003.811794 3. Micire, M.J.: Evolution and field performance of a rescue robot. J. Field Robot. 25, 17–30 (2008) 4. Phillips, E.K.: Supporting situation awareness through robot-to-human information exchanges under conditions of visuospatial perspective taking (Doctoral dissertation). University of Central Florida, Orlando (2016). http://stars.library.ucf.edu/etd/4936 5. Phillips, E., Ososky, S., Grove, J., Jentsch, F.: From tools to teammates: toward the development of appropriate mental models for intelligent robots. Proc. Hum. Factors Ergon. Soc. Ann. Meeting 55(1), 1491–1495 (2011) 6. Chen, J.Y., Barnes, M.J., Qu, Z.: RoboLeader: an agent for supervisory control of multiple robots. In: Proceedings of the 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI 2010), pp. 81–82 (2010) 7. Jentsch, F., Fincannon, T.: Multiple robot-multiple operator control and teamwork: lessons learned and design guidelines. In: Proceedings of Infotech@ Aerospace 2012 (2012) 8. Mouloua, M., Parasuraman, R.: Human Performance in Automated Systems: Current Research and Trends. Lawrence Erlbaum Associates, Publishers, Hillsdale (1994)

Design Approach for Investigating Multimodal Communication

13

9. Jentsch, F., Evans, A., Feldman, M., Hoeft, R., Rehfeld, S., Curtis, M.: A scale MOUT facility for studying human-robot interaction and control. In: Proceedings for the Army Science Conference 2005, (24) (2004) 10. Sawyer, B.: Effects of signal probability on multi-tasking-based distraction in driving, cyberattack, & battlefield simulation. University of Central Florida Electronic Theses and Dissertations, 5169 (2015) 11. Fincannon, T., Jentsch, F., Sellers, B., Talone, A.: Best practices in human operation of robotic/unmanned vehicles: a technical review of recommendations regarding the human-torobot ratio. Proc. Hum. Factors Ergon. Soc. Ann. Meeting 57(1), 1268–1272 (2013) 12. Phillips, E., Schaefer, K.E., Billings, D.R., Jentsch, F., Hancock, P.A.: Human-animal teams as an analog for future human-robot teams: Influencing design and fostering trust. J. Hum. Robot Interact. 5(1), 100–125 (2016). https://doi.org/10.5898/JHRI.5.1.Phillips 13. Bozkurt, A., et al.: Toward cyber-enhanced working dogs for search and rescue. IEEE Intell. Syst. Cyborg Intell. 29, 32–39 (2014) 14. Phillips, E., Rivera, J., Jentsch, F.: Developing a tactical language for future robotic teammates. Proc. Hum. Factors Ergon. Soc. Ann. Meeting 57(1), 1283–1287 (2013) 15. Yanco, H.A., Drury, J.L.: Rescuing interfaces: a multi-year study of humanrobot interaction at the AAAI Robot Rescue Competition. Auton. Robots 22(4), 333–352 (2007) 16. Chen, J.Y.C., Haas, E.C., Barnes, M.J.: Human performance issues and user interface design for teleoperated robots. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 37(6), 1231–1245 (2007) 17. Brown, J., Gray, J.P., Blanco, C., Juneja, A., Alberts, J., Reinerman, L.: Handsfree, heads-up control system for unmanned ground vehicles (No. TARDEC- 22037) (2011). http://www. dtic.mil/dtic/tr/fulltext/u2/a547423.pdf 18. Barber, D., Carter, A., Harris, J., Reinerman-Jones, L.: Feasibility of wearable fitness trackers for adapting multimodal communication. In: Yamamoto, S. (ed.) HIMI 2017. LNCS, vol. 10273, pp. 504–516. Springer, Cham (2017). https://doi.org/10.1007/978-3-31958521-5_39 19. Barber, D., Lackey, S., Reinerman-Jones, L., Hudson, I.: Visual and Tactile Interfaces for Bi-Directional Human Robot Communication. SPIE Defense, Security, and Sensing Unmanned Systems Technology. Baltimore, Maryland USA (2013) 20. Bischoff, R., Graefe, V.: Dependable multimodal communication and interaction with robotic assistants. In: 11th IEEE International Workshop on Robot and Human Interactive Communication, pp. 300–305. IEEE (2002) 21. Lackey, S.J., Barber, D.J., Reinerman-Jones, L., Badler, N., Hudson, I.: Defining nextgeneration multi-modal communication in human-robot interaction. In: Human Factors and Ergonomics Society Conference. HFES, Las Vegas (2011) 22. Sterling, B., Perala, C.: Workload, stress, and situation awareness of soldiers who are controlling unmanned vehicles in future urban operations. Army Research Lab Technical report, ARL-TR-4071 (2007) 23. Lu, S., Wickens, C., Prinet, J., Hutchins, S., Sarter, N., Sebok, A.: Supporting interruption management and multimodal interface design: three meta-analyses of task performance as a function of interrupting task modality. Hum. Factors 55(4), 697–724 (2013) 24. Stader, S.: Impacts of complexity and timing of communication interruptions on visual detection tasks. UCF Stars Library: Electronic Theses and Dissertations (4571) (2014) 25. Wickens, C.D.: Multiple resources and performance prediction. Theor. Issues Ergon. Sci. 3, 159–177 (2002). https://doi.org/10.1080/14639220210123806 26. Duncan, J., Martens, S., Ward, R.: Restricted attentional capacity within but not between sensory modalities. Nature 397, 808–810 (1997) 27. Kahneman, D.: Attention and Effort. Prentice-Hall, Englewood Cliffs (1973)

14

D. Barber and R. Bendell

28. Barber, D., Reinerman-Jones, L., Matthews, G.: Toward a tactile language for human-robot interaction: two studies of tacton learning and performance. Hum. Factors J. Hum. Factors Ergon. Soc. (2014). https://doi.org/10.1177/0018720814548063 29. Jentsch, F., Evans, W., Ososky, S.: Model world: military HRI research conducted using a scale MOUT facility. In: Published in Human-Robot Interactions in Future Military Operations, pp. 419–431 (2010) 30. Bendell, R., Vasquez, G., Jentsch F.: Human performance with autonomous robotic teammates: research methodologies and simulations. In: Human Performance in Automated Systems: Current Research and Trends (2019, in press) 31. Elliott, L.R., Schmeisser, E.T., Redden, E.S.: Development of tactile and haptic systems for U.S. infantry navigation and communication. In: Smith, M.J., Salvendy, G. (eds.) Human Interface 2011. LNCS, vol. 6771, pp. 399–407. Springer, Heidelberg (2011). https://doi.org/ 10.1007/978-3-642-21793-7_45 32. Barber, D., Nicholson, D., Davis, L., Chen, J.: The mixed-initiative experimental testbed for collaborative human robot interactions. In: International Symposium on Collaborative Technologies and Systems (2008) 33. Vasquez, G., Bendell, R., Talone, A.B., Jentsch, F.: Development of a signal detection-based task for research on distributed human-robot teaming within immersive virtual reality. Proc. Hum. Factors Ergon. Soc. 62(1), 1479–1483 (2018) 34. Matthews, G., Reinerman-Jones, L., Barber, D., Abich IV, J.: The psychometrics of mental workload: multiple measures are sensitive but divergent. J. Hum. Factors Ergon. Soc. (2014). https://doi.org/10.1177/0018720814539505

Accessibility Studies: Abuses, Misuses and the Method of Poietic Design Gian Maria Greco1,2(&) 1

Autonomous University of Barcelona, Barcelona, Spain [email protected] 2 University of Vigo, Vigo, Spain

Abstract. Over the past several decades, accessibility has been increasingly pervading a vast range of fields, producing a large number of new ideas, theories, and innovations that have already proven to be quite fruitful. A closer look at how accessibility has entered and developed in various research fields shows that said fields have experienced fundamental changes: a shift from particularist accounts to a universalist account of access, a shift from maker-centred to usercentred approaches, and a shift from reactive to proactive approaches. Through these processes, accessibility has birthed new areas within those fields, that have been gradually converging to constitute the wider field of accessibility studies. The nature and position of accessibility studies has now become a central topic. This ongoing progression of conceptual clarification may bear some misunderstanding and misinterpretations along the way. In the paper, I first briefly review the principal traits of the process of formation of accessibility studies; then address some possible misconceptions; and finally, introduce a first, very general sketch of poietic design, a method proper to accessibility studies. Keywords: Accessibility revolution
Accessibility studies
Audiovisual translation
Design
Human rights
Maker-user gap Maker-expert-user gap
Media accessibility
Poietic design
Universal access in human-computer interaction




1 Introduction Over the past decades, accessibility has been increasingly pervading a vast range of fields, producing a large number of new topics, theories, and innovations that have already proven to be quite fruitful. Upon closer examination of how accessibility has broken into and developed in these fields shows that the fields themselves have been experiencing a series of shifts. Through these processes, accessibility has given rise to new areas within those fields, that have been gradually converging to constitute the wider field of accessibility studies (AS) [1–4]. As in any process of emancipation, where an individual claims her autonomy and individuality once having reached maturity, this ongoing progression of conceptual clarification may bear some misunderstanding and misinterpretations along the way. They are physiological, due precisely to the emancipatory dialectic between an individual (AS) and her peers (other well-established fields). My goal in this paper is threefold: (a) to briefly review the © Springer Nature Switzerland AG 2019 C. Stephanidis (Ed.): HCII 2019, LNCS 11786, pp. 15–27, 2019. https://doi.org/10.1007/978-3-030-30033-3_2

16

G. M. Greco

principal traits of the process of formation of AS in order to (b) add a few more tiles to the mosaic of AS while (c) addressing some possible misconceptions. Which means that this paper should be read as an additional contribution on the path towards the academic maturity of AS. For this reason, the paper is divided into two parts. In part one, I briefly recall the main characteristics of the process of formation of AS and mention some of its defining features. This part, which corresponds to the next three sections, summarises aspects that I have addressed more extensively in [1]. Therefore, I refer the reader to that paper for a more detailed discussion. In the second part, I begin by addressing some misconceptions that may lead to misuses and abuses of AS and then conclude by introducing a first, very general sketch of a method proper to AS, namely poietic design.

2 The Accessibility Revolution Access is a central concept in human life. As discussed by Lakoff in his analysis of freedom as a metaphor: “you are not free to go somewhere, get something, or do something if access is blocked, or if there is no path (or road or bridge) to it. Freedom requires not just the absence of impediments to motion but also the presence of access. Inhibiting freedom is, metaphorically, not just throwing up roadblocks, holding one back, taking away power, imposing burdens or threats or harm, but also failing to provide access. […] The metaphor of freedom as freedom of motion thus has two important parts: freedom from and freedom to. Freedom from concerns those things that can keep you from moving. Freedom to concerns making sure there is access” [5]. The freedom examined by Lakoff is not freedom as a human right but the very general, archetypical concept of freedom. The importance of access for human life and thought has become even more evident through the debate on human rights. Within this context, accessibility began to make its way to the forefront towards the end of WWI and then subsequently, through the widespread movement that led to the adoption of the Universal Declaration of Human Rights. Human rights rest upon two intertwined grounds: human dignity and access. The concept of human dignity usually sets a minimum standard of quality of life an individual is entitled to for the sole reason of being a human being. Such a standard is often defined by a series of material and immaterial goods thought to be essential for every individual. Yet, setting up a public education system does not suffice for a state to claim that it is respecting the right to education for all, if said system does not also provide the means to access for every student. Guaranteeing the conditions for the existence of those fundamental goods is a necessary but not sufficient condition for human dignity. The possibility to have actual access to them also needs to be in place [6]. Thus the reason why human rights, e.g. the human right to education, are expressed in terms of the “human right to access to”, e.g. “the human right to access to education” [4]. Having access means, for example, being able to use, interact with, and enjoy those fundamental goods. That is, accessibility entails both quantity and quality of experience. Though human rights consist of a (series of) theory(ies) and there is plenty of scholarship that rejects them, they have indeed shed light on the crucial role of access in many human activities, bringing it to the forefront of theoretical, social and political

Accessibility Studies: Abuses, Misuses and the Method of Poietic Design

17

debate. Thanks to the cultural revolution they have produced, it has become clear that access is a necessary requirement in the most varied aspects of our lives. From this point of view, accessibility then acts as a proactive principle, which calls for a proactive attitude to comply with the access requirement [4]. In order for the accessibility revolution to fully blossom however, a second condition was needed; this time related to the nature of our world and the ways in which we access it. It needed the information revolution. A 2004 preparatory document for the UNESCO World Summit on the Information Society warns about the “reconfiguration of access” enabled by information and communication technologies (ICTs), which is challenging “fundamental social and political notions of freedom, control, personal responsibility, and shared community values” [7]. By reconfiguring in an unprecedented way how we access the world, ourselves and others, ICTs are creating new social inequalities through the formation of multiple divides, beyond the traditional framing of a digital divide between those who have physical access to ICTs and those who do not. In the information society, accessibility becomes the grounds on which power negotiations and social struggles take place. Rooted in the human rights revolution and boosted by the information revolution, accessibility has been leading a revolution of its own. The question of access tackles the very foundations of our society. It has become so all-encompassing that some say we are living in “the age of access” [8]. This is highly evident in research. The revolutionary effects of accessibility have been producing a paradigm shift in various fields, from transportation studies to human-computer interaction, from geography to engineering, from design to sustainability studies, from translation studies to cultural heritage, from education to tourism studies, just to name a few [9–14]. Briefly mentioning two cases may help highlight this point. Acknowledging that providing access to digital information goes beyond mere technical issues, researchers have started to develop an entirely new approach based on accessibility in order to investigate issues of digital sustainability. It is an approach that has subsequently produced foundational ramifications for the whole field of sustainability studies. Since “access concerns can be considered a prerequisite for sustainability”, then accessibility becomes “a necessary step towards conceptualizing the sustainability of human societies and their development” [12]. The transformative effects of accessibility are even more pronounced in the case of transportation studies, where accessibility has played an essential role since at least the 1950s. Over the years, it has grown so as to become one of its main concepts, to the point that many scholars have been rethinking the whole field, because “[accessibility] changes how we think about and measure transport problems and the scope of solutions that are considered for addressing them. As with the Copernican revolution, this shift changes what we consider the system’s centre: traffic-based planning places motor vehicles at the centre, while accessibility-based planning places people at the centre of the transport system” [15].

18

G. M. Greco

3 The Shifts Produced by Accessibility A closer look at how accessibility has made its way into and then evolved in various research fields shows that these fields have been experiencing some fundamental changes: a shift from particularist accounts to a universalist account of access, a shift from maker-centred to user-centred approaches, and a shift from reactive to proactive approaches. Accessibility usually entered these fields through an initial focus on a specific group, often persons with disabilities. Over time, the focus was progressively widened to include other groups, until finally reaching a universal scope. That is, these fields have moved from various particularist accounts, that frame access as exclusively or mainly concerning specific groups of people, towards a more universalist (or integrated/holistic) account, where access concerns all human beings. The field of human-computer interaction is a clear example. Over the past few decades, the focus on accessibility within this field has given rise to the subdomain called “universal access in human-computer interaction” (UAHCI) [16]. UAHCI was initially grounded on “approaches to accessibility mainly targeted toward providing access to computer-based applications by users with disabilities” [17]. Over time, it gradually expanded its focus to other groups until embracing a universal vision by acknowledging that “accessibility can no longer be considered as a specific problem of people with disabilities [but of] society at large” [14]. The shift is even more evident in the field of audiovisual translation (AVT), the “branch of translation studies concerned with the transfer of multimodal and multimedial texts into another language and/or culture” [18]. When accessibility first came into this field, scholars started to use the term “media accessibility” (MA) to refer to a very specific subdomain of AVT, that concerned with “subtitling for the deaf and the hard of hearing (SDH) and audio description (AD) for the blind and the visually impaired” [19]. Even though it was at times expanded to include other modalities, like audio subtitling and sign language interpreting, this first particularist account framed MA as both specifically related to persons with sensory disabilities and exclusively limited to a precise set of AVT services and modalities. Over time, scholars shifted towards a second particularist account of MA, according to which MA concerned not only sensory but also linguistic barriers [20, 21]. Recently, scholars have started to advocate for the shift to a universalist account, which defines MA as concerning access to media and non-media objects, services and environments through media solutions, for any person who cannot or would not be able to, either partially or completely, access them in their original form [1, 4, 22]. The universalist definition does not limit MA to any specific group but rather, focuses on the functional processes involved in the interaction between users’ specificities, the particular contexts within which they act or are placed, and the means to address those specificities in such contexts. According to this account, MA comprises three categories: solutions that allow access to media objects, services, and environments; solutions that allow access to media objects, services, and environments through media tools; and solutions that allow access to non-media objects, services, and environments through media instruments.

Accessibility Studies: Abuses, Misuses and the Method of Poietic Design

19

While the first particularist account frames MA as a sub-area of AVT and the second particularist account makes it overlap with AVT itself, both frame MA as a subarea of translation studies. Inversely, by allowing for the inclusion of other groups and access services that would have been otherwise excluded from particularist-based MA, the universalist account favours a convergence of the different conceptions of MA and of MA services developed in other fields. In turn, this positions MA as a broader interdisciplinary area that criss-crosses many fields, including AVT, but that cannot be entirely nor exclusively reduced to any of them because it is a proper subdomain of a new field, which I would refer to as AS. The different positions of MA are exemplified in Fig. 1, which should obviously be considered a mere schematisation of their multilayered enmeshment.

Fig. 1. A simplified schematisation of the three accounts of the area of media accessibility.

The changing tide towards a universalist account of accessibility has been interlaced with a second movement, namely, the increasing attention towards users as bearers of valuable knowledge for the investigation of accessibility processes and phenomena. For years, the dominant attitude was based on the assumption that maker’s knowledge is the only one that matters. Whether it be the design of some technology or a theatre performance, artefacts were devised according to the maker’s point of view or, in the best case scenario, according to the makers’ interpretation of users’ needs and capabilities [23]. A major consequence of maker-centred approaches has been a complex series of gaps between the different stakeholders involved. Two of the most prominent are what I have referred to as the maker-user gap, a multifaceted gap that can exist between those who make and those who use an artefact, and the makerexpert-user gap, which places makers, experts and users at opposite ends of a triangular spectrum of the design process [1, 2, 24, 25]. The need to bridge these gaps has spurred a shift towards inclusive design practices based on user-centred approaches: the knowledge of users, experts and other stakeholders needs to be fully taken into account in the design process because it is as important as maker’s knowledge. Evidence of this second shift can be found, for example, in the privileged status that reception studies has achieved in the various fields affected by accessibility.

20

G. M. Greco

The recognition that the knowledge of users and experts is as important as the maker’s has been posing questions as to how this knowledge should be integrated within the design process, how it should be acquired, and how it should be used. Pursuant to [25–27], the process of artefact design can be broken down into a (series of) ex-ante, in itinere, and ex-post stage(s). For years, access concerns were mainly addressed by adopting reactive approaches. Once produced, artefacts were often modified through ex-post solutions, that is, add-ons in order to render them accessible [28]. In other rare cases, accessibility was addresses at in itinere stages. Ex-post and in itinere solutions bear many limitations, for they may produce a “loss in functionality [or] provide limited and low-quality access” [17]. In some cases, addressing access at the ex-post or in itinere stages is the only possible way. In other cases however, adopting a reactive approach means renouncing accessibility completely, because “it is impossible to ‘glue’ accessibility onto some of the systems as an afterthought or postmanufacture process” [29]. This has led to a shift towards proactive approaches, which entail “a purposeful effort to build access features into a product as early as possible (e.g., from its conception to design and release)” [17]. Moreover, reactive approaches have often led to the late involvement of accessibility experts, drastically decreasing the chances of making an artefact accessible. These issues are well-known problems in many areas, like web accessibility, where “a main factor for the lack of Accessibility at the Web is the major knowledge gap that normally exists between developers and Accessibility specialists [as well as the] common practice to consider Accessibility at the very last stages of the development process, or when applications are already coded” [30]. In order to tackle this problem, researchers and industry have long been devising specific methodologies that place accessibility concerns – as well as involve users and experts – from the early stages of web application development.

4 The Formation of Accessibility Studies In order for a new field to be born, one strong, yet not necessarily well-defined, “unique, or at least central, concern” must exist first [31]. This central idea then sets in motion a dialectic between endogenous and exogenous forces that may lead to the formation of the new field [32], if exogenous forces prove to be stronger than the endogenous ones, as will be described below. Some of these forces that scholars widely agree upon are: (a) interdisciplinarity, (b) the formation of a research community, and (c) forms of opposition by well-established fields. When a new problem emerges on the knowledge horizon, if it cannot be tackled using exclusively the tools of a specific field, a common practice among researchers is to join forces and set up a multidisciplinary programme, each drawing on ideas and methods from her own discipline, to then “split apart unchanged when the work is done” [33]. Yet, some issues are so unique or challenging that the mere juxtaposition of different fields and methods is not sufficient. They demand the “integration and synthesis of ideas and methods”, which often leads to the creation of “new hybrid research fields” [33]. Access issues have long proven to demand such an interdisciplinary approach, urging researchers from the most diverse fields to not only share their knowledge and tools but to integrate them and devise new ones in order to successfully overcome such challenges.

Accessibility Studies: Abuses, Misuses and the Method of Poietic Design

21

The uniqueness of accessibility issues has attracted an increasingly broader range of researchers, who have been hybridising their knowledge and methods in order to address said issues. During this journey they have acquired new profiles that neither fully conform to their original fields nor fit within classical boundaries between fields. Regardless of where they started from, they all end up having more in common amongst themselves than with colleagues from their original fields. The area of MA is once more a clear case. Though it was bred within the field of translation studies, MA problems have attracted scholars from the most vastly divergent fields. Researchers from engineering to tourism studies, from filmmaking to computer science, from psychology to the performing arts, have been joining forces, sharing their own methods and creating new ones in order to tackle MA problems. These partnerships have caused them to gravitate beyond the borders of their original fields, detaching themselves from their original colleagues, and forming a new community of peers. While the formation of a new community around accessibility and its interdisciplinary nature have been acting as exogenous forces, they have been opposed by endogenous forces trying to bar that very process. Seeing as “no discipline willingly abdicates its mandated sovereignty” [34], well-established fields tend to resist the formation of a new field, perceived as either a competitor or a threat to their fiefdoms. They tend to shield themselves behind the walls of their orthodoxy, so that their “response is often only to create new subfields – a seemingly endless proliferation that incorporates members of the emerging community of scholars within the larger enterprise without any debate about the significance of their challenge” [35]. In [1] I have discussed several instantiations of these centripetal forces in relation to MA.

5 Some Misuses and Abuses of Accessibility Studies In the multifaceted process summarised in the previous sections, accessibility has been acting as a magnet. It slowly began applying its attractive force on many fields, leading to the creation of specific subdomains. It simultaneously pulled both these and their researchers out of the spheres of influence of their original fields and has ultimately brought them together on a new, common ground, namely accessibility studies. The reader may refer to Fig. 2 which illustrates this, albeit in a very abstract way. Obviously the relationship between AS and other fields is much more complex, seeing as many fields intersect both among themselves and with AS. AS is the research field concerned with (a) the critical investigation of accessibility processes and phenomena, and (b) the design, implementation and evaluation of accessibility-based and accessibility-oriented methodologies. For a discussion of the definition of AS and its implications, I refer the reader to [1]. As for the goal of this paper, concerning part (a) of the definition, it is sufficient to recall Bradley’s words on accessibility in digital sustainability: “[access] is not only about the ability to find and retrieve an item, but also the ability to use, view, listen to, interact with, display, or run the digital item in such a way that users can be assured that what they are viewing satisfies their needs” [36]. Accessibility lies at the heart of a vast gamut of issues, such as acceptability, adaptability, availability, flexibility, personalisation, and usability.

22

G. M. Greco

Fig. 2. A simplified schematisation of the formation process of the field of accessibility studies.

Together with many other issues, they are all relevant topics of investigation in AS, as long as some caution is taken, as I will discuss later. AS has been, de facto, a field for some time, and it is now a central topic in scholarly debate. The journey of a new field towards academic emancipation and recognition is seldom linear and smooth, and may carry some physiological confusion. Therefore, a bit of clarification is called for. In the remaining part of this section, I will address a few of the possible misinterpretations that may lead to misuses and abuses of AS. The list is neither exhaustive nor conclusive. The ultimate goal of the following paragraphs is to contribute to the metatheoretical analysis of the nature of AS while to preventively clear from the path some obstacles that AS may face along the way. Obstacles that, for example, may take the form of argumentations that accept AS but undermine some of its traits, ultimately aiming to keep it under the control of some other field. This even includes fields that may be willing to undergo intense structural renovations to their “castles” so as to make room for AS and exploit its potentials, while keeping it chained to prevent it from walking away from their kingdom. Once more, I will refer mainly to MA to exemplify my point. A first source of confusion may be related to the distinctiveness of AS. Some may embrace AS, even warmly, while minimising the importance of clarifying the reciprocal positioning and relationship between AS and other fields. Let us consider a possible case in relation to MA, where some may dismiss the value of the distinction between AVT and MA as merely a question of one’s point of view. Instead of being a weakness, this is actually a decisive reason for the need to address such a distinction. Having clarity around the perspective from which one conducts her analysis is critical to avoid being stuck in a conceptual, epistemological or methodological muddle. Consider an ancient artefact. A chemist analyses the composition of its materials, a philosopher its aesthetical features. The object being analysed is the same, but the perspectives from which they observe it and the level at which they conduct their analyses differ substantially. In their investigation, the chemist and the philosopher are

Accessibility Studies: Abuses, Misuses and the Method of Poietic Design

23

each guided by a different set of questions, they use different methods, and they elaborate different interpretative models. The chemist may even use the philosopher’s results to flesh out some new idea in her own field through metaphorical or analogical thinking. Yet, this does not make the philosopher’s statements on the aesthetics of the artefact chemistry statements. Tracing a distinction between MA and AVT does not mean repudiating the role (still being) played by the latter in the development of the former, nor does it mean denying that the two intersect and have much in common. However, they look at the world through different lenses. They are guided by different questions, each of which influence the ways they investigate a problem, the explanations they formulate, and, ultimately, the solutions they devise. As a subfield of translation studies, AVT is concerned with translation, and when it observes the world it frames it in terms of translation problems. As a subdomain of AS, MA is concerned with accessibility, and when it observes the world it frames it in terms of access problems. Obviously, some accessibility problems may be related to translation. Yet, while MA and AVT clearly do intersect: (a) not all translation problems are accessibility problems; and (b) not all accessibility problems are translation problems; therefore (c) not all MA problems are AVT problems and vice versa (see Fig. 1). The fact that MA and AVT intersect and may borrow from each other does not make the need for a distinction a mere fanciful exercise. Mechanical engineering models are used in medicine to gain insight into some mechanisms of the human body and biology. This does not weaken the distinction between medicine and mechanical engineering nor does it make the former a subfield of the latter. As a subdomain of AS, access concerns are ultimately central in MA. Clearly distinguishing between MA (and AS too) and other fields is eventually critical for how one addresses and responds to those concerns. Otherwise, one may run the risk of curing a cold with a hammer. Precisely for these reasons a second possible claim one may advance, i.e. that AVT has become a subdomain of MA, should be discarded as well. Once more, while all problems of MA are accessibility problems, not all problems of AVT are accessibility problems. Which means that, as shown in the third image of Fig. 1, MA is indeed a proper subset of AS (that is, all elements of the MA set are elements of the AS set), while AVT simply intersects with MA (as well as with AS). The distinctiveness of AS may also give rise to a second form of confusion, that is, the interpretation of AS as either a mere extension or evolution of some other field. This claim usually plays a hypernym game with the concepts involved. For example, one can use the concept of translation to talk about design, saying that designing a house is the process of translating the requests of a client into a blueprint and then into a physical building. This formulation plays on the polysemic trait of the words “translation” and “design”. While it could be a catchy metaphor or analogy, perhaps useful in inspiring some insights into how the process of design works, accepting it literally would mean classifying design as a hyponym of translation, and thus a hypernym trap that leads to conclude that the field of design is a subfield of translation studies. Similarly, claiming that “accessibility is a form of translation and translation is a form of accessibility” [37, see also 38] has a deep heuristic value that, for example, can help us grasp the connection between translation and accessibility at some levels as well as the role they play in addressing and solving social issues. However, it should not be interpreted literally; otherwise it would fall into the hypernym trap. Thus,

24

G. M. Greco

considering AS as a mere extension or subdomain of some other field, say translation studies, begets the very same controversial conclusion. Ultimately, the hypernym game would engender a field so generic and overpopulated to the point it would become useless. A third source of confusion may be due to the misinterpretation of AS as a theory instead as a field. AS is not a specific theory of accessibility developed within some fields or subfields, but a field of its own. As such, AS does not aim to provide a unified theory of accessibility but rather, to stimulate the formulation of and, in addition, host many theories of accessibility. Theories that investigate, assess and explain problems, processes and phenomena through the lens of accessibility, while addressing the associated theoretical and the social issues. For example, a theory that rejects the value of users’ knowledge in the production of access services has full citizenship in AS. It would probably come into conflict with other theories hosted within AS, but this is part of the healthy internal dialectic of any field.

6 Conclusion: Accessibility and Poietic Design As mentioned above, AS can be defined as the research field concerned with (a) the critical investigation of accessibility processes and phenomena, and (b) the design, implementation and evaluation of accessibility-based and accessibility-oriented methodologies. In previous works as well as in the pages above, I have focused on (a). In these concluding paragraphs I will briefly examine (b). AS is mature enough to both host a family of theories regarding accessibility-related issues as well as to be organised into subfields, such as access ethics, the subfield of AS that investigates the ethical issues raised by accessibility in relation to human life and society. In the course of its emergence, through the enmeshment of the areas that have come together to form it, AS has borrowed specific methodological approaches, and then mixed and restructured them to create its own methods, each concerned with different aspects involved in accessibility. One of these is what I shall refer to as poietic design (PD), where “design” should be intended in the most general sense, from policies to software. PD stems from and complies with the inner features of accessibility and applies them to the process of design. The long path that has led to the emergence of AS and the revolution experienced by the different fields have clearly demonstrated the poietic trait of accessibility. The focus on users put forward by accessibility, for example, does not imply a secondary role of the makers. On the contrary, it shows that the design process is a co-construction where makers and users, as well as other agents, must work together. Each plays a role that cannot be ignored. Accessibility calls for the responsibility we have both collectively and individually as co-creators and agents. As a method, PD consists of a series of principles: 1. The Principle of Universality: accessibility concerns all, not exclusively specific groups or individuals. 2. The Principle of Personalisation: one size does not fit all. The design should be able to respond to the specificities of individual users.

Accessibility Studies: Abuses, Misuses and the Method of Poietic Design

25

3. The Principle of User-centrality1: design should focus on users and their specificities. 4. The Principle of Epistemic Inclusivity: users and other stakeholders, including experts, are bearers of valuable knowledge for the design of artefacts. 5. The Principle of Participation: design should be carried out through the active participation of the stakeholders involved. 6. The Principle of Proactivism: accessibility should be addressed ex-ante, not expost. The list above should not be considered exhaustive. It merely sketches some of the overarching principles of PD. Others may be added which refer to additional fundamental aspects of accessible design, such as usability and expertise. While I leave a more detailed account of PD, its principles and ethical implications to future work, a few words about the Principle of Personalisation may help to clarify the general point. This principle tells us that universalism should not lead to the annihilation of users’ differences and limitation of their freedom. “Design for All” (or Universal Design) identifies the horizon within which the design process should take place: “all” (universalism) means that design has to (potentially) address all users, but not treat them as a homogeneous group of indistinguishable persons. Personalisation means “design for one”, that is, we need to design artefacts that can respond to the specificities of each individual. It is design that guarantees individual’s choice and makes the adaptation and customisation to her needs, to the environments within which she acts, as well as other factors, possible. PD is both prescriptive and a descriptive, backward- and forward-looking. It defines the conditions necessary to design an accessible artefact or to make an artefact accessible, in addition to offering an analytic tool for the diagnosis of problems and the devising of strategies of intervention. The poietic trait of accessibility does not simply provide a conceptual tool for the design process, as seized by PD; it also shows the vast potential impact inherent in AS, for example through reframing and successfully tackling old problems. A clear case is the social model of disability. While this model has represented a stepping stone away from the medical model, for years now disability studies scholars have been highlighting its limitations and suggesting the need to move forward, beyond the social model of disability. Yet, this has proven to be very difficult from a disability studies perspective. On the other hand, as I argued in [3, 24, 39], once the problem is framed from its novel perspective, AS allows us to move from a social model of disability to an atimic model of accessibility (or social model of accessibility), where disability is but an instantiation of a general process of deterioration or negation of the equal status of human beings as imposed by society to anyone who faces barriers to access. Acknowledgements. I am grateful to Constantine Stephanidis and Margherita Antona for their support as well as to the two anonymous referees and the participants to the 13th conference on Universal Access in Human-Computer Interaction held in Orlando (Florida, US) on 26-31 July 2019. I wish to thank Josh Branson, Kate Dangerfield, Louise Fryer, and Pablo Romero-Fresco

1

I owe the name to Louise Fryer.

26

G. M. Greco

for their comments on previous drafts. Teressa Canosa skilfully copyedited the final version of this paper. All remaining mistakes are mine. The research presented in this paper has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 752659 (project UMAQ). The author is a member of TransMedia Catalonia (2017SGR113), a research group funded by the Catalan government under the SGR funding scheme, and the GALMA research group, funded by the Galician government under the scheme Proxecto de Excelencia 2017.

References 1. Greco, G.M.: The nature of accessibility studies. J. Audiovis. Transl. 1, 205–232 (2018) 2. Greco, G.M.: The need for accessibility studies. In: Paper Presented at the Conference the Future of Media Accessibility: Issues and Visions, Lecce, Italy (2017) 3. Greco, G.M.: L’accessibilità culturale come strumento per i diritti umani di tutti. In: Cetorelli, G., Guido, M.R. (eds.) Il patrimonio culturale per tutti. Fruibilità, riconoscibilità, accessibilità, pp. 94–102. Direzione Generale Musei, Ministero dei Beni e delle Attività Culturali e del Turismo, Roma (2017) 4. Greco, G.M.: On accessibility as a human right, with an application to media accessibility. In: Matamala, A., Orero, P. (eds.) Researching Audio Description. New Approaches, pp. 11– 33. Palgrave Macmillan, London (2016) 5. Lakoff, G.: Whose Freedom? The Battle Over America’s Most Important Idea. Farrar, Straus and Giroux, New York (2006) 6. Francioni, F.: The rights of access to justice under customary international law. In: Francioni, F. (ed.) Access to Justice as a Human Right, pp. 1–55. Oxford University Press, Oxford (2007) 7. Dutton, W.H.: Social Transformation in an Information Society: Rethinking Access to You and the World. UNESCO, Paris (2004) 8. Rifkin, J.: The Age of Access: How the Shift from Ownership to Access is Transforming Modern Life. Penguin, London (2001) 9. Buhalis, D., Darcy, S.: Introduction: from disabled tourists to accessible tourism. In: Buhalis, D., Darcy, S. (eds.) Accessible Tourism: Concepts and Issues, pp. 1–20. Channel View Publications, Bristol (2011) 10. Janelle, D.G., Hodge, D.C. (eds.): Information, Place and Cyberspace: Issues in Accessibility. Springer, New York (2000). https://doi.org/10.1007/978-3-662-04027-0 11. Levine, J., Grengs, J., Merlin, L.: From Mobility to Accessibility: Transforming Urban Transportation and Land-Use Planning. Cornell University Press, Ithaca (2019) 12. Prodan, A.C.: The sustainability of digital documentary heritage. In: Albert, M.-T., Bandarin, F., Pereira Roders, A. (eds.) Going Beyond. HS, pp. 59–69. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-57165-2_5 13. Pullin, G.: Design Meets Disability. The MIT Press, Cambridge (2009) 14. Stephanidis, C., Emiliani, P.L.: Connecting to the information society: a European perspective. Technol. Disabil. 10, 21–44 (1999) 15. Litman, T.: Evaluating Accessibility for Transportation Planning: Measuring People’s Ability to Reach Desired Goods and Activities. Victoria Transport Policy Institute, Victoria (2007) 16. Stephanidis, C. (ed.): The Universal Access Handbook. CRC Press, Boca Raton (2009)

Accessibility Studies: Abuses, Misuses and the Method of Poietic Design

27

17. Emiliani, P.L.: Perspectives on accessibility: from assistive technologies to universal access and design for all. In: Stephanidis, C. (ed.) The Universal Access Handbook, pp. 2.1–2.18. CRC Press, Boca Raton (2009) 18. Pérez-González, L.: Audiovisual translation. In: Baker, M., Saldanha, G. (eds.) Routledge Encyclopedia of Translation Studies, pp. 13–20. Routledge, London (2008) 19. Orero, P.: Audiovisual translation: a new dynamic umbrella. In: Orero, P. (ed.) Topics in Audiovisual Translation, pp. VII–XIII. Benjamins, Amsterdam (2004) 20. Díaz Cintas, J.: Audiovisual Translation Today. A Question of Accessibility for All. Translating Today, pp. 3–5 (2005) 21. Orero, P., Matamala, A.: Accessible Opera: overcoming linguistic and sensorial barriers. Perspect. Stud. Translatology 15, 262–277 (2007) 22. Greco, G.M.: Towards a Pedagogy of Accessibility. The Need for Critical Learning Spaces in Media Accessibility Education and Training. Linguistica Antverpiensia 19 (forthcoming) 23. Norman, D.A.: The Design of Everyday Things. Basic Books, New York (2013) 24. Greco, G.M.: The normative dimension of cultural accessibility. In: Paper presented at the First Italian Workshop on Cultural Accessibility, Lecce, Italy (2013) 25. Greco, G.M.: Come e perché organizzare un evento culturale accessibile: Dalla teoria alla pratica. In: Paper presented at the Conference ArtLab13, Lecce, Italy (2013) 26. Greco, G.M., Pedone, L.: Accessibilità e sicurezza dei luoghi di spettacolo. Note su criteri impositivi, criteri prescrittivi e buone prassi. AGM, Lecce (2015) 27. Greco, G.M., Pedone, L., Monsellato, E., Rizzo, B., Spinelli, E.: Guida per eventi accessibili ed ecosostenibili. Social Sound-Puglia Sounds, Regione Puglia, Lecce (2012) 28. Adams, R.: User modeling: a universal access perspective. In: Stephanidis, C. (ed.) The Universal Access Handbook, pp. 24.21–24.19. CRC Press, Boca Raton (2009) 29. Vanderheiden, G.C.: Universal design and assistive technology in communication and information technologies: alternatives or complements? Assistive Technol. Official J. RESNA 10, 29–36 (1998) 30. Martín, A., Cechich, A., Rossi, G.: Accessibility at early stages: insights from the designer perspective. In: Paper Presented at the W4A2011–Submission Type Technical (2011) 31. Pietig, J.: Is education a discipline? Educ. Forum 48, 365–372 (1984) 32. Farjoun, M.: The dialectics of institutional development in emerging and turbulent fields. Acad. Manage. J. 45, 848–874 (2002) 33. Committee on Facilitating Interdisciplinary Research, National Academy of Sciences, National Academy of Engineering, Institute of Medicine: Facilitating interdisciplinary research. The National Academies Press, Washington, D.C (2005) 34. Klein, J.T.: The rhetoric of interdisciplinarity: boundary work in the construction of new knowledge. In: Lunsford, A.A., Wilson, K.H., Eberly, R.A. (eds.) The SAGE Handbook of Rhetorical Studies, pp. 265–283. SAGE, Thousand Oaks (2009) 35. Salter, L., Hearn, A.M.V.: Outside the Lines: Issues in Interdisciplinary Research. McGillQueen’s University Press, Montreal (1996) 36. Bradley, K.: Defining digital sustainability. Libr. Trends 56, 148–163 (2007) 37. Díaz Cintas, J., Orero, P., Remael, A.: Media for all: a global challenge. In: Díaz Cintas, J., Orero, P., Aline (eds.) Media for All, pp. 11–20. Rodopi, Amsterdam (2007) 38. Remael, A.: Media accessibility. In: Gambier, Y., van Doorslaer, L. (eds.) Handbook of Translation Studies, vol. 3, pp. 95–101. John Benjamins, Amsterdam/Philadelphia (2012) 39. Greco, G.M.: Accessibility, human rights, and the ghetto effect. In: Paper Presented at the Conference Wounded Places. On the Integrity of the Body, Beirut, Lebanon (2016)

Design Techniques of Ambient Media Advertisements and Message Comprehension Yen Hsu1

, Chia-Jung Lee1(&)

, and Pei-Ying Yang2

1

2

The Graduate Institute of Design Science, Tatung University, Taipei, Taiwan R.O.C. [email protected] Ming-Chi University of Technology, Taipei, Taiwan R.O.C.

Abstract. Ambient media advertising is an innovative outdoor advertising format. This type of advertising typically involves placing unexpected advertisements in unusual locations to draw the attention of consumers and leave a memorable impression on consumers. Ambient media advertisements are often integrated into daily surroundings, and these novel advertisements make consumers’ daily lives more fun and creative. Unexpected ambient media advertisements draw consumers’ attention, provoke their positive attitudes toward the advertisement, and increase purchase intentions and revenues. Ambient media advertisements that attract people’s attention often employ unique media to convey messages, and different presentation designs are applied to incorporate novel and unpredictable factors in the visual display to stimulate the audiences’ interests in advertisement messages. For this study, 238 advertisement samples were acquired from the website Bored Panda Ambient Media. After experts eliminated similar samples, a total of 190 ambient media advertisement samples were adopted for this research. By analyzing these samples, five design techniques (i.e., metaphor, similes, allegory, metonymy, and analogy) and three media (i.e., site media, reproducing media, and mechanical media) were identified, and a total of 15 ambient media advertisement sample combinations were obtained. The differences in the message comprehension level were then determined among these samples. The results are as follows:

(1) Among various media with all design techniques, site media resulted in the highest message comprehension. (2) Among all design techniques under various media, the technique of metaphor resulted in the highest message comprehension. (3) Regarding the effect of media and design technique interactions, the combination of metaphor and site media resulted in the highest message comprehension. Keywords: Ambient media
Design technique
Media Message comprehension
Message communication

© Springer Nature Switzerland AG 2019 C. Stephanidis (Ed.): HCII 2019, LNCS 11786, pp. 28–39, 2019. https://doi.org/10.1007/978-3-030-30033-3_3




Design Techniques of Ambient Media Advertisements

29

1 Introduction Ambient media advertising first appeared in British media around 1999 before becoming a standard advertising format in the advertising industry [1]. At the time of its inception, this type of advertising attracted great attention among British marketing and advertising media enterprises [2]. Ambient media advertising is an innovative outdoor advertisement format, in which unexpected advertisements are placed in unusual locations to draw consumers’ attention [3] and provide unique experiences to the public [4]. Ambient media advertisements have more novelty and are more surprising for recipients than any other advertising media. By placing unexpected advertisements in unusual locations, this type of advertisements successfully encourages their positive attitudes toward the advertisement [5]. Ambient media advertisements provoke new social phenomenon, which increases purchase intentions and sales revenue [6]. In ambient media advertising, a common advertising approach is to integrate advertisements into daily life and use creativity to add some fun to people’s lives. By finding a common point between product features and the carrier, advertisers use the happiness and value generated by such novelties to increase ambient media recognition by the public [7]. Ambient media advertisements are usually placed at unexpected locations for targeted audiences. These advertisements are usually well-planned to surprise and excite the audience. Ambient media advertisements often employ unique media to convey messages, and different presentation designs are applied to incorporate novel and unpredictable factors in the visual display to stimulate the audiences’ interests in advertisement messages [8]. An increasing number of ambient media advertisements have employed the approach of on-set interactive experiences to achieve promotion purposes [9]. Successful ambient media advertisements effectively communicate advertisement messages through the optimal medium by integrating advertisement contents with designs and media [1]. In response to the increasing popularity of ambient media, the present study analyzed and categorized design techniques and media of ambient media advertisements and then analyzed the differences in the level of message comprehension among different types of advertisements. The objectives of this study are as follows: (1) Categorize design techniques and medium types of ambient media. (2) Analyze the effect of design techniques of ambient media advertisements on message comprehension under various media. (3) Determine the effect of various media of ambient media advertisements on message comprehension under various design techniques. (4) Determine the effect of media and design technique interactions on message comprehension.

30

Y. Hsu et al.

2 Literature Review The present study focused on determining the process and employment of the rhetoric approach as design techniques for ambient media advertisements. Based on relevant literature review, this study analyzed ambient media advertisements and theories for design techniques to determine the different media characteristics of advertisements and message comprehension. This section is divided into four main topics: ambient media, design techniques, media, and message comprehension. 2.1

Ambient Media

Media are the form of presentation of advertisements, and advertisement messages are information conveyed by companies to the public. Advertisements must provoke the public’s interest in the product and promote the brand image. The increasing demand of products and services has led to the generation of various advertisement formats and types. People have learned to apply selection and elimination to process the high amount of advertisement messages. Such behaviors affect the views and purchase intention of consumers [10]. Extraordinary ideas can draw consumers’ attention and persuade them to purchase the product. Without extraordinary ideas, advertisements may be overlooked by consumers and hence do not achieve their purpose. Ideas imply creativity [11]. The creativity of ambient media is often based on advertisement messages, and creative ambient media are often realized in locations/spaces that are visually appealing, creating mutual feelings and achieving extensive communication. Advertisement creativity increases the beauty, artistic, and truthful level of the work. Therefore, creativity is the key to the success and quality of an advertisement [12]. 2.2

Design Techniques

Languages were created by humans to properly, correctly, and lively convey messages and thoughts. During the process of language use, rhetoric techniques were invented. Rhetoric techniques optimize the effectiveness of language use and emotional expressions [13]. Design techniques refer to the message communication method of an advertisement. Advertisements provide product information that companies want to convey to targeted consumers; advertisements are presented creatively through design techniques and texts to market the product [14]. According to the research results obtained by Trevino (1986), Wimberly et al. (2000), Jeong (2008), Booth (2010), Radden and Kövecses (1999), and Gentner (1983), the present study categorized the design techniques employed by ambient media into five categories. (1) Metaphor: The carrier can represent the tenor; both objects are considered equal, and their similitudes or differences can only be understood through comprehensive thinking [15]. The public interprets another symbolic element when facing the product. The metaphoric design technique cleverly integrates a cultural element into the product design to create a new creative product. The technique of metaphor

Design Techniques of Ambient Media Advertisements

(2)

(3)

(4)

(5)

2.3

31

involves pairing two objects and producing new combinations and comprehensions of the relationships and contexts projected by these two objects [16]; for example, bus handles shaped as hands implies contact with unknown individuals. Similes: Similitudes between the tenor and the carrier are obvious and can be understood without much thinking [17]. For example, containers on a truck represent USB devices with enormous storage capacity. The capacity of storage is the similitude of the two objects. Allegory: This technique involves counterintuitive thinking by the audience; the product and service portrayed are contradictory to the audience’s expectations [18]. For example, the image of a prisoner sitting on the floor of an elevator trying to open the door symbolizes imprisonment in tiny spaces and other possible torture, which contradicts the emphasis of human rights in modern society. Metonymy: This technique employs partial elements, symbols, or product features to refer to the entire object or concept, and the referral fits the audience’s expectation [19]. For example, regarding the image of well-organized clothes stored in each step of a staircase, the visual of well-organized storage represents the products and services of IKEA, and such an interpretation fits the expectation of the public. Analogy: The tenor and the carrier are absolutely not related, and contexts or written texts are often required to facilitate comprehension [20]. For example, comparing colorful iPods with paint recycling fields implies the effective reuse and recycle of the paint. Media

In Understanding Media: The Extensions of Man, Mcluhan proposed that media are messages [21]. People comprehend various messages simultaneously using more than one sense and their comprehension is affected by the conveyance method. Messages and communication media are symbiotic, and such relationships are presented by title font, color, and images of various specifications when people browse through the platform [22]. Ambient media transform messages into symbols through different channels or methods. The nature of these media determines the extent to which the message is conveyed. The media can be divided into three categories and can be applied individually or in combination [23]: (1) Site media: A communicator is a media. The media communicate through language, facial expression, and hand gestures. Therefore, a communicator must be present. This type of media is confined to the scene and can only occur on site. The communicator conveys the message content through behaviors, actions, or facial expressions such as sitting down or grabbing other people. (2) Reproducing media: The media were created from the perspective of cultural aesthetics such as images, paintings, and photographs. The media comprise communication works that can exist independently and serve as the basis for site media because they can be reproduced and are creative. Communication is achieved through images, photographs, or nature; for example, images and visual aids such as clouds or the color of the sky.

32

Y. Hsu et al.

(3) Mechanical media: This category refers to the use of telephones, televisions, and elevators, which are the transporters of site media and reproducing media. The main differences between reproducing media and mechanical media are that mechanical media operate through mechanics; therefore, technicality must be considered to determine the mechanical output. For mechanical media, the technical aspect is used to communicate messages; for example, the movement of an escalator or the cold air from an air conditioner [23]. 2.4

Message Comprehension

Consumers’ understanding and comprehension regarding advertisements in surroundings influence their attitude and intention as well as the subsequent purchase behaviors. Therefore, advertisement message comprehension plays a critical role in consumers’ purchase decision-making [24]. Through message comprehension, consumers interpret stimulations from the environment and infer knowledge, meanings, and faith. Such inferences then interact with consumers’ existing knowledge to produce personal inferences and messages [25]. The following factors influence message comprehension: (1) products, product formats, brands, and marketing methods: Relevant message attributes and benefits obtained by consumers from the product affect consumers’ perception of the product [26] and (2) new product benefits, complexity, and perceived risks affect consumers’ perception regarding new products [27]. This study aims to determine the differences in the message comprehension level among advertisement samples. On the basis of the research results of Feiereisen, Wong, and Broderick (2008), the present study assessed the level of message comprehension from the perspectives of the advertisement slogan, image descriptions, and main features of a product or service.

3 Experiment Ambient media communicate a wide variety of concerns including charity, environmental protection, ecosystem, sports, and brand marketing. To understand the design techniques commonly adopted for ambient media advertisements, relevant cases and samples were obtained from the website of Bored Panda Ambient Media. According to the categorization of sites by the website, a total of 238 advertisement samples were obtained in the preliminary stage. Subsequently, further examination was conducted to combine or eliminate similar samples to reduce the possibility of repeated data. Finally, a total of 190 ambient media advertisement samples were obtained. These samples were categorized according to five design techniques (i.e., metaphor, similes, allegory, metonymy, and analogy) and three media (i.e., site media, reproducing media, and mechanical media). The 5  3 experimental design was adopted to generate a total of 15 ambient media advertisement sample combinations. Moreover, 60 participants were recruited for each media. The between-group design was adopted for the experiment. A total of 180 participants were recruited. Before the test, the participants were allowed 5 min to familiarize themselves with the experiment tool and testing samples and to

Design Techniques of Ambient Media Advertisements

33

read the operating instructions. The participants were also given time to practice the operation to quickly learn the procedure and content of the experiment and reduce test errors. The operation procedure is as follows: the participants were instructed to look at the center of the screen and click the left button of the mouse for the testing sample to appear on the screen. Each sample was programmed to be displayed for 10 s. Once the sample disappeared from the screen, the participants began to respond to the items. The message comprehension level of these samples was tested. 3.1

Materials

Table 1 presents the 15 ambient media advertisement samples. The samples with images on topics with lesser relevance or with a messy background were eliminated. The image in the sample advertisement and its design concept must be consistent with the advertisement theme; the advertisement text must be readable and be consistent with the original meaning of the situation.

Table 1. Sample numbering according to the media and design techniques Media

Design technique Metaphor Similes

Allegory

Metonymy

Site media

Sample 1 Metaphor + Site Sample 2 Metaphor + Reproducing Sample 3 Metaphor + Mechanical

Sample 7 Allegory + Site Sample 8 Allegory + Reproducing Sample 9 Allegory + Mechanical

Sample 10 Sample 13 Metonymy + Site Analogy + Site Sample 11 Sample 14 Metonymy + Analogy + Reproducing Reproducing Sample 12 Sample 15 Metonymy + Analogy + Mechanical Mechanical

Reproducing media Mechanical media

Sample 4 Similes + Site Sample 5 Similes + Reproducing Sample 6 Similes + Mechanical

Analogy

4 Results 4.1

Effect of Design Techniques on Message Comprehension Under Different Media

The experiment with all the ambient media advertisements revealed significant differences in the message comprehension level among advertisements using different media (F value = 8.973, p = .000 < .05). Table 2. Advertisement comprehension level according to different media Media Design techniques Site media Reproducing media Mechanical media *p < .05 **p < .01 ***p < .001

N 300 300 300

M 5.011 4.598 4.543

Std. F value p value 1.4747 8.973 .000*** 1.4754 1.4906

Ranking 1 2 3

34

Y. Hsu et al.

The mean value of the comprehension level of an advertisement using site media was higher than that of advertisements using the other two media, and the mean values were in the following order: M = 5.011 > M = 4.598 > M = 4.543. This finding implies that the participants had the highest comprehension level for advertisements using site media (Table 2). Slight differences were observed in the mean values of the comprehension level of reproducing and mechanical media. The simple main effects of these three media variables revealed that for the design techniques of similes, allegory, metonymy, and analogy, the comprehension level of various media differed significantly, whereas for the technique of metaphor, nonsignificant differences were observed (F value = 0.943, p = .391 > .05). The technique of similes showed significant differences in structure (F value = 4.126, p = .018 < .05), and a post hoc test revealed that message comprehension was higher for site media than for mechanical media. The technique of allegory showed significant differences in structure (F value = 33.853, p = .000 < .05), and the post hoc test revealed higher message comprehension for site media than for mechanical media as well as higher message comprehension for reproducing media than for mechanical media. The technique of metonymy showed significant differences in structure (F value = 9.672, p = .000 < .05), and the post hoc test revealed higher message comprehension for site media than for reproducing media. The technique of analogy showed significant differences in structure (F value = 7.578, p = .001 < 0.05), and the post hoc test revealed higher message comprehension for mechanical media than for site media as well as higher message comprehension for mechanical media than for reproducing media (Table 3). Table 3. Analysis of variance (ANOVA) of main effects of advertisement message comprehension (media) Source Media (A) SS Df MS F value Metaphor (b1) 2.594 2 1.297 0.943 Similes (b2) 12.337 2 6.169 4.126 Allegory (b3) 121.019 2 60.509 33.853 Metonymy (b4) 40.137 2 20.068 9.672 Analogy (b5) 32.182 2 16.091 7.578 *p < .05 **p < .01 ***p < .001

4.2

p value .391 .018* .000*** .000*** .001***

Scheffé – a1 > a3 a1 > a3, a2 > a3 a1 > a2 a3 > a1, a3 > a2

Effect of Media on Message Comprehension Under Different Design Techniques

According to Table 4, different design techniques exerted significant effects on advertisement message comprehension (F value = 28.851, p = .000 < 0.05). Particularly, the mean of the message comprehension level for advertisements employing metaphors was higher than that for advertisements employing similes, allegory, metonymy, and analogy, and the mean values showed the following order: M = 5.488 > M = 4.911 > M = 4.864 > M = 4.187 > M = 4.136. The results revealed that the participants had a uniform perspective of message comprehension for advertisements employing the technique of metaphor (Table 4).

Design Techniques of Ambient Media Advertisements

35

Table 4. Advertisement comprehension level according to different design techniques Design Techniques N Media Metaphor 180 Similes 180 Allegory 180 Metonymy 180 Analogy 180 *p < .05 **p < .01 ***p < .001

M 5.488 4.911 4.864 4.187 4.136

Std. F value p value 1.1724 28.851 .000*** 1.2439 1.5632 1.5086 1.5098

Ranking 1 2 3 4 5

The main effects test results revealed that under site media, reproducing media, and mechanical media, the design techniques significantly affected the message comprehension level of the participants. Significant differences were observed in site media (F value = 18.144, p = .000 < .05), and the post hoc test revealed higher message comprehension for the technique of metaphor than for the techniques of metonymy and analogy as well as higher message comprehension for the techniques of similes, allegory, and metonymy than for the technique of analogy. Reproducing media showed significant differences (F value = 26.233, p = .000 < .05), and the post hoc test revealed higher message comprehension for the techniques of metaphor, similes, and allegory than for the techniques of metonymy and analogy. Mechanical media showed significant differences (F value = 18.144, p = .000 < .05), and the post hoc test revealed higher message comprehension for the technique of metaphor than for the techniques of allegory and metonymy as well as higher message comprehension for the techniques of similes and metonymy than for the technique of allegory (Table 5). Table 5. The main effects of advertisement message comprehension (design techniques) according to the ANOVA Source Design Techniques (B) Site media (a1)

Reproducing media (a2)

Mechanical media (a3)

SS

Df

MS

F value

p value

Scheffé

128.393

4

32.098

18.144

.000***

170.762

4

42.690

26.233

.000***

b1 b2 b4 b1 b2 b2 b3 b1 b2 b5

98.771

*p < .05 **p < .01 ***p < .001

4

24.693

18.144

.000***

> > > > > > > > > >

b4, b5, b5 b4, b4 b5, b5 b3, b3, b3

b1 > b5 b3 > b5, b1 > b5, b3 > b4, b1 > b4,

36

4.3

Y. Hsu et al.

Effect of Media and Design Technique Interactions on Message Comprehension

Two-way ANOVA revealed significant effects of interactions between the three media and the five design techniques (F value = 11.923, p = .000 < .05). The results indicated that the interaction between media and design technique had moderating effects on message comprehension (Table 6). Table 6. Significance of media and design technique interaction on message comprehension Independent variables SS Df Media (A) 39.321 2 Design techniques (B) 228.977 4 Media (A)* 168.948 8 Design techniques (B) Error 1567.530 885 *p < .05 **p < .01 ***p < .001

MS 19.660 57.244 21.119

F value 11.100 32.319 11.923

p value .000*** .000*** .000***

1.771

Table 7. Effect of design technique and media interactions on advertisement message comprehension No. Design techniques Media Site Sample 1 Metaphor Reproducing Sample 2 Mechanical Sample 3 Site Sample 4 Similes Reproducing Sample 5 Mechanical Sample 6 Site Sample 7 Allegory Reproducing Sample 8 Mechanical Sample 9 Site Sample 10 Metonymy Sample 11 Reproducing Sample 12 Mechanical Site Sample 13 Analogy Reproducing Sample 14 Mechanical Sample 15 *p < .05 **p < .01 ***p < .001

N 60 60 60 60 60 60 60 60 60 60 60 60 60 60 60

M 5.657 5.388 5.418 5.275 4.788 4.670 5.507 5.378 3.707 4.767 3.610 4.185 3.850 3.823 4.733

Std. F value p value 1.1623 11.923 .000*** 1.0688 1.2778 1.0099 1.2061 1.4180 1.3435 1.1845 1.4677 1.6748 1.3583 1.2548 1.3655 1.5138 1.4880

Ranking 1 4 3 6 7 10 2 5 14 9 15 11 12 13 8

For all ambient media advertisement samples, media and design technique interactions significantly affected the level of message comprehension (F value = 11.923, p = .000 < .05). The mean value of Sample 1 (Metaphor + Site) was higher than that of Sample 7 (Allegory + Site), Sample 3 (Metaphor + Mechanical), Sample 2 (Metaphor + Reproducing), and Sample 8 (Allegory + Reproducing); the mean values

Design Techniques of Ambient Media Advertisements

37

presented the following order: M = 5.657 > M = 5.507 > M = 5.418 > M = 5.388 > M = 5.378. The results indicated that the participants’ perception of advertisement messages with metaphor, allegory, and site media showed higher uniformity than their perception of advertisement messages with other samples (Table 7).

5 Conclusion In the present study, the ambient media advertisements collected were analyzed and summarized, and the effects of design techniques and advertising media employed by these ambient media advertisements on the message comprehension level were determined. This study provided the following findings: (1) Five design techniques: The design techniques identified in this study were metaphor, similes, allegory, metonymy, and analogy. These techniques can provoke the imagination of people. The emotional reaction of people toward the advertisement was achieved by the stimulation of the senses through the media, which draw the public’s attention and awaken feelings through the design technique of hidden messages. The interactions of these two factors (media and design technique) can affect the message comprehension level of the advertisement, because people echo with the effects generated in them when they came in contact with the information. (2) Conveying messages through the three media: Site media communicate the message content through language and eliciting certain behaviors in people at the scene. Reproducing media create new topics by altering images and communicating through the visual sense. Mechanical media create special effects by employing mechanical techniques and technology to overcome time and space limits as well as by stimulating multiple senses. (3) Among various media, for all the design techniques, site media provided the highest message comprehension. Communication achieved through language and eliciting certain behaviors in people at the scene renders site media the most effective for conveying advertisement messages. Among various design techniques under three media, the metaphor design technique provided the highest message comprehension. In metaphor, the carrier can represent the tenor, and the similitudes and differences in the two objects can only be obtained through comprehensive thinking. The use of metaphor in advertisement design provokes people’s comprehensive thinking to understand the meaning of the advertisement and thus comprehend the message. (4) Regarding media and design technique interactions, the combination of metaphor and site media resulted in optimal message comprehension. Advertisers prefer that advertisements are seen at the adequate time and in the appropriate space for people to interact with the advertisement of their interest. When the advertisement is presented in the image form and the description is consistent or inconsistent with general understanding, the audience will be simulated to seek an answer. Overall, by combining experiences with a series of measures for persuasion, advertising and marketing will improve message comprehension and advertisement values and effectively influence the purchasing intentions of people.

38

Y. Hsu et al.

References 1. Barnes, J.: Creating a difference with ambient media. Admap 34, 46–49 (1999) 2. Shankar, A., Horton, B.: Ambient media: advertising’s new media opportunity? Int. J. Advertising 18(3), 305–321 (2015) 3. Hutter, K., Hoffmann, S.: Guerrilla marketing: the nature of the concept and propositions for further research. Asian J. Mark. 5(2), 39–54 (2011) 4. Lugmayr, A., et al.: Semantic ambient media—an introduction. Multimedia Tools Appl. 44 (3), 337–359 (2009) 5. Hutter, K.: Unusual location and unexpected execution in advertising: A content analysis and test of effectiveness in ambient advertisements. J. Mark. Commun. 21(1), 33–47 (2015) 6. Pieters, R., Warlop, L., Wedel, M.: Breaking through the clutter: benefits of advertisement originality and familiarity for brand attention and memory. Manage. Sci. 48(6), 765–781 (2002) 7. Hutter, K., Hoffmann, S.: Surprise, Surprise. Ambient media as promotion tool for retailers. J. Retail. 90(1), 93–110 (2014) 8. Hutter, K., Hoffmann, S.: Carrotmob and anti-consumption: same motives but different willingness to make sacrifices? J. Macromark. 33(3), 217–231 (2013) 9. Gambetti, R.C., Graffigna, G.: The concept of engagement: a systematic analysis of the ongoing marketing debate. Int. J. Mark. Res. 52(6), 801–826 (2010) 10. Cook, G.: The Discourse of Advertising. Psychology Press, London (2001) 11. Gamson, W.A., et al.: Media images and the social construction of reality. Ann. Rev. Sociol. 18(1), 373–393 (1992) 12. Ogilvy, D., Horgan, P.: Confessions of an Advertising Man. Atheneum, New York (1963) 13. Lim, C.H., Martin, T.G., Kwak, D.H.: Examining television consumers of mixed martial arts: the relationship among risk taking, emotion, attitude, and actual sport-mediaconsumption behavior. Int. J. Sport Commun. 3(1), 49–63 (2010) 14. Ang, S.H., Low, S.Y.: Exploring the dimensions of ad creativity. Psychol. Mark. 17(10), 835–854 (2000) 15. Trevino, L.K.: Ethical decision making in organizations: a person-situation interactionist model. Acad. Manage. Rev. 11(3), 601–617 (1986) 16. Wimberly, B.T., et al.: Structure of the 30S ribosomal subunit. Nature 407(6802), 327 (2000) 17. Jeong, S.H.: Visual metaphor in advertising: is the persuasive effect attributable to visual argumentation or metaphorical rhetoric? J. Mark. Commun. 14(1), 59–73 (2008) 18. Booth, W.C.: The Rhetoric of Fiction. University of Chicago Press, Chicago (2010) 19. Radden, G., Kövecses, Z.: Towards a theory of metonymy. Metonymy Lang. Thought 4, 17– 60 (1999) 20. Gentner, D.: Structure-mapping: a theoretical framework for analogy. Cogn. Sci. 7(2), 155– 170 (1983) 21. McLuhan, M., et al.: Jak rozumět médiím: extenze člověka. Odeon, Praha (1991) 22. Sherwood, M., Nicholson, M., Marjoribanks, T.: Controlling the message and the medium? The impact of sports organisations’ digital and social channels on media access. Digit. Journalism 5(5), 513–531 (2017) 23. Fiske, S.T., Taylor, S.E.: Social Cognition. Mcgraw-Hill Book Company, New York (1991) 24. Thøgersen, J.: Direct experience and the strength of the personal norm–behavior relationship. Psychol. Mark. 19(10), 881–893 (2002)

Design Techniques of Ambient Media Advertisements

39

25. Graeff, T.R.: Consumption situations and the effects of brand image on consumers’ brand evaluations. Psychol. Mark. 14(1), 49–70 (1997) 26. Holak, S.L., Lehmann, D.R.: Purchase intentions and the dimensions of innovation: an exploratory model. J. Prod. Innov. Manage 7(1), 59–73 (1990) 27. Bodnar, A., Corbett, R., Nekrasovski, D.: AROMA: ambient awareness through olfaction in a messaging application. In: Proceedings of the 6th International Conference on Multimodal Interfaces. ACM (2004) 28. Stepanova, E.V., Strube, M.J.: Attractiveness as a function of skin tone and facial features: evidence from categorization studies. J. Gen. Psychol. 145(1), 1–20 (2018)

How to Define the Passenger’s Hazard Perception Level by Combining Subjective and Objective Measures? Qianjing Hu1, Gang Guo1(&), Huaji Wang2, Mingqing Tang1, Chen Lv3, and Dongpu Cao4 1

College of Automotive Engineering, Chongqing University, Chongqing, China [email protected], [email protected] 2 AVL Powertrain UK Ltd, Coventry, UK [email protected] 3 School of Mechanical and Aerospace Engineering and School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, Singapore [email protected] 4 Mechanical and Mechatronics Engineering, University of Waterloo, Waterloo, Canada [email protected]

Abstract. With the prosperous growing of intelligent technology, the automobile industry is developing towards an unmanned and intelligent generation. However, making a car drive in a human-like way is still a great challenge to engineers, which slows the adoption of such cars in a mass scale. Considering human perception capabilities during the design process might be a solution to this problem, and hazard perception is of great important. The aim of the study is to propose a theoretical framework for exploring the relationship between driving environment and human perceived hazards to give suggestions on designing human-like vehicle controllers. Correlating objective environmental factors with subjective hazard ratings. Human-in-the-loop experiments were carried out on a high-fidelity driving simulator. 27 driving scenarios were designed and implemented for 14 participants to acquire their subjective hazard ratings, while objective measurements were also recorded. Specifically, by using the proposed methodology hazard perception level of passenger can be measured by using the subjective parameters adopted during the experiment, which stands for the subjective assessment of passenger’s hazard perception. To find what dynamic parameters having a significant correlation with hazard perception level, a seven scale of subjective hazard perception was defined from –3 to 3, namely over dangerous, dangerous, a little dangerous, normal, a little cautious, cautious, overcautious. Objective measures of environment include the velocity and acceleration of the subjective car and the distance between the subjective car and obstacle. A 3*3*3 mixed ANOVA was carried out in our study in order to find out the correlation between the environment and subjective assessment. The finding shows us that the interaction of velocity and acceleration and distance has a strong correlation with the passenger’s hazard perception. The others show no correlation be-tween hazard perception assessment. © Springer Nature Switzerland AG 2019 C. Stephanidis (Ed.): HCII 2019, LNCS 11786, pp. 40–53, 2019. https://doi.org/10.1007/978-3-030-30033-3_4

How to Define the Passenger’s Hazard Perception Level

41

This research is with great potential to improve the development of Advanced Driver Assistant System (ADAS) and intelligent automobile from a human perspective. The findings can also be applied to the design of vehicle controllers to improve the passenger’s comfort by knowing that on what condition or what kind of control the passenger will feel dangerous and uncomfortable. Keywords: Passenger Driving simulator


Hazard perception
Subjective
Objective


1 Introduction Road safety is achieved when the ‘driver-road-vehicle’ system works well. Among these three elements, the driver, without any doubt, may best affect the proper functioning of the interaction [1]. However, with the increasing development of internet, the automobile is becoming more and more intelligent and driverless. Every participant in car paly a same important role in the future ‘driver-road-vehicle’ system. There are many studies showed that hazard perception is a critical factor to accidents. According to Horswill and McKenna [2], among the different components of driving skill, only hazard perception has been correlated with traffic-accident involvement across a number of studies [3–5]. With the increasing attention on road accident, hazard perception is becoming an increasingly significant research hotpot. The assessment of hazard perception is often subjective because the perception is highly individual, depending on personal experiences with accidents and potential rewards of risk-taking [6]. Therefore, it is reasonable to measure the passenger’s subjective risk perception level. The objective situation can be defined as the subject car’s kinematic parameters (acceleration, speed and distance between the subjective car and obstacle on the road.). Moreover, the methods only consider subjective judgments with experiences, but no specific trip. Thus there almost no such study combining the subjective rating, objective dynamic parameter together. The aim of the study is to find out the correlation between passenger’s hazard perception and objective environment. Based on study, we can learn about under what kind of condition, the passenger will feel dangerous and uncomfortable. We can adjust the parameters of a car to control the driving behavior in cased of that the passenger is involved in a situation where he or she think is dangerous. It will make a big difference if we adopted this to the Advanced Driver Assistant Systems (ADAS). The aim of the experiment is to find out what is the significant objective environment factor that effect the passenger’s hazard perception level based on both objective and subjective parameters. Compared with other literature studies, we retain that the proposed methodology has some advantages, the primarily advantage is that we transfer our focus on passenger’s hazard perception not the driver, which could be very different from the other existed study. In addition, our proposed methodology combined both objective parameters and subjective judgment of the hazard perception level. Numerous studies have focused on the driver’s hazard perception. Few of the researches focused on the passengers and their hazard perception, even though passengers are also the car user and should be considered as the same importance as driver.

42

Q. Hu et al.

A brief review about hazard perception would be reported in the next section. After the short introduction, we proposed a literature review in the following section about the hazard perception. Then, we depict our method combing both the subjective and objective measures of hazard perception. A brief description including participants, 27 scenarios, instruments used for collecting data about the objective and subjective judgments of hazard perception will be introduced in detail. Data analysis is described later. Finally, a conclusive section is reported.

2 Literature Review The literature search was conducted using Google Scholar, because this search engine adopts full-text search and has broad coverage [7–9]. Searches were using the following key words: hazard perception”, “situation awareness”, “hazard aware-ness”, “risk perception”. Hazard perception (HP) has been defined differently by many researchers [10]. Wilde [11] and Mackenna [12] firstly defined hazard perception as the ability to anticipate dangerous situations on the road ahead. While Mills et al. [13] described hazard perception as the ability to read the road. Later on Horswill and McKenna [2] added that hazard perception may be regarded as situation awareness for hazardous, dangerous situations and McKenna, Horswill, and Alexander [12] appended the ability anticipating forthcoming events to the definition. David Crundall [14] said hazard perception (HP) is the process of detecting, evaluating and responding to dangerous events on the road, which have a high likelihood of leading to a collision. One increasingly common description is ‘the ability to predict dangerous situations on the road [2, 15]. The literature about the hazard perception almost studied on the driver’s hazard perception. Even though there are little few research studied on the passenger’s hazard perception, we can still learn something useful to the hazard perception study. Research on driver’s hazard perception can be divided into 3 types: (1). the relation between hazard perception and driving experience; (2). the relation between age, gender, nationality and hazard perception; (3). Hazard perception test. The first type of research focus on whether the driver’s driving experience has influence on the hazard perception or not and how the experience affects hazard perception according to references [14, 16–21] relevant to some research about experience and hazard perception. They try to figure out whether or not the driving experiences have influence on the hazard perception and how can we discriminate the driving strategy depend on the driver’s experience. The other type about hazard perception is about gender, age or nationality. References [16, 20, 22, 23] study on the relationships between the age and hazard perception to find how the age plays an effects in terms of the ability of hazard perception and what’s the differences between different kind of people at different age. References [24, 25] pay their attention on the nationality. That is whether the culture or environment will affect their hazard perception ability or not and how it plays. Meanwhile, there are also some researchers [19, 21, 26–28] who focused on the hazard perception test (HPT), trying to find out the importance of HPT on driver at different period of age or experiences.

How to Define the Passenger’s Hazard Perception Level

43

Part 1 is mainly talking about what and how the researchers do with the experiment environment where can be broadly divided into four categories: (1). Using a static pictures; (2). Using a traffic clips; (3). PC-based prototype driving-simulator; (4). Real road driving. There are three categories strategies of evaluating hazard perception ability in the study: (1). Questionnaire; (2). Using a button, which will be pressed when, the participant percept the hazard; (3). Thinking aloud (or Verbal Protocol Analysis); Using the static pictures and traffic clips can be considered the traditional ways used in hazard perception study. There are a lot of studies based on the traditional ways [14, 16–21, 24, 26]. Recently, the simulator becomes more and more popular in the hazard perception research like reference [23] using it to fulfill the study. With the advantage of allowing researchers to evaluate a driver’s ability by parameters such as angles of steering wheel, accelerator, and brake force, driving simulation tests are often used to ascertain driving skills [29–33]. Therefore, it is not only more approaching to reality than desktop neuropsychological tests, but also they are cheaper and more efficient than evaluations of on-road driving test [30]. However, there are some researchers [22, 34] trying to move their experiments to real road, which would be more dangerous and difficult. Considering the convenience, high efficiency and safety of the participant during the experiment, we decided to use a driving simulator to achieve our goal. A questionnaire is often used to measure the driving experience and the selfassessment of hazard-perception skills [23, 24, 27, 35–38] of the participants. The questionnaire can involve some demographic question and some other question like the frequency they drive, the score to measure how dangerous some traffic scene or something else. Specially, the questionnaire is to find out where, which, what is the hazard [24]. Another way used in hazard perception study is using a press button [16, 17, 20] or using the touch screen [21, 26] or computer mouse [19, 39–41]. The participant can press button or touch the screen or click the mouse when perceive the hazard, which can be collected as response data for researchers. It is a novelty approach called ‘Think aloud’ (or Verbal Protocol Analysis) used in reference [22]. Participants give continual commentary when they are driving in the real road. In addition, reference [1] combined the objective (like self-assessment) and subjective (speed and acceleration) measure to define the accident risk level. These novelty approaches are very important to improve our study of hazard perception. In the study about hazard perception, variables about response are widely used, like response time, average response time, response latency, response sensitivity, miss, false, false alarm. In addition, eye tracking is the most used assistant way in the experiment. Also, some studies use some biological indicator to describe the participants’ performance, such as palmar sweating response (PSR), electrodermal activity (EDA) in Reference [23]. There are a several of methods used to analysis the experiment data. ANOVA is obvious one of the most popular method used to find out the correlation between factors and hazard perception, which can be proved in References [16, 17, 21] et al. Meanwhile, chi-square analysis is widely used, in particular, to process data in categorization task [17]. Some studies [17, 21] applied logistic regression to fulfill predictive purpose. In their study, the participants watching the clips involve different kind of hazard would have different performance. Then, logistic regression was applied to divide drivers into two groups: novice or experienced,

44

Q. Hu et al.

depending on the variables the researchers defined. Moreover, there are some other methods are used, like Poisson regression [26], Paired t-tests [17], Leximancer software [22], Fisher’s exact test [17], Cronbach’s alpha [25, 26] and Pearson product–moment correlation coefficient [23]. Generally, in the literature, there are many subjects on hazard perception, which were divided into three types before. However, few of them combining the subjective parameters and objective parameters together, neither the study focus on passengers’ hazard perception level. However, Eboli [1] defined the accident risk level by introducing a novel methodology, which combines the objective and subjective measures of driving style. The aim of the study is to find out what parameter standing for the objective environment has a strong correlation with the passenger’s hazard perception rating through the methodology, which combines both subjective and objective parameters. By the use of the proposed methodology, the hazard perception level of the passenger can be tested by using the subjective parameters collected during the experiment. Our study defined a seven scale of passenger’s subjective hazard perception (over dangerous, dangerous, a little dangerous, normal, a little cautious, cautious, overcautious), and try to find the dynamic parameters which have significant correlation with hazard perception level. It is believed that the research can give a great contribution to improving the Advanced Driver Assistant System (ADAS) and the development of the intelligent automobile. What’s more, the study can make a big difference in ADAS to improve the passenger’s comfort when they are in a road trip if knowing that on what condition or what kind of control the passenger will feel dangerous and uncomfortable.

3 Method We design a 3  3  3 mixed experiment depending on three factors, velocity, acceleration and distance. The simulator will be drove in a controlled condition, where the velocity, acceleration and distance is pre-designed. The experimenter will drive at a specific speed Vs (30 km/h, 70 km/h, 110 km/h) and complete a braking task when there is an obstacle appearing in front of the road. By giving a sign to tell the driver where to start braking at the scenario, the acceleration will be controlled at a constant (–1 m/s2, –3 m/s2, –6 m/s2). The distance between the subjective car and the obstacle is settled at d = (1 + d’)  d0, d0 = v2/2a and d’ = (5%, 10%, 15%). Three velocities represent different speed level (low, normal, high) of the car. As the same, three accelerations standing for three kind of driving style (aggressive, normal, cautious) to stop the car. It also goes for three distance. Participants are required to give a hazard perception rating to describe the own feeling and judgement about the braking task. Each participant is required to complete a questionnaire including a rating scale, which is a subjective self-assessment questionnaire of perceived hazard. Participants express a level from –3 (extremely dangerous) to +3 (extremely cautious). Before the experiment, a hypothesis was proposed that the interaction of velocity, acceleration and distance has a strong relation with the passenger’s hazard perception.

How to Define the Passenger’s Hazard Perception Level

3.1

45

Participants

The participant sample is made up of 14 licensed student drivers and they all are between 23 and 26 years old. Even though they are qualified to be a driver, but during the experiment they are just told to act like a passenger siting on copilot seat and no need to drive. All of the participants were recruited among university students to be told to know very well the objectives of our research and the proposed methodologies. To make sure the participants to be representative the age, gender and driving experience of participants should be random. 3.2

Simulator and Questionnaire

To create the virtual driving environment and conduct the experiment, a fixed-base fidelity-driving simulator is used (Fig. 1). The driving simulator is made up of a cockpit, which is equipped with all necessary control systems similar to a real car. The graphics system includes three 42″projectors displays with a front screen resolution of 1920  1080 dpi and the two side screens resolution of 1360  768 dpi. The displays are situated around the cockpit and provide a 150° horizontal and 40° vertical perspective. The scenarios are presented at a rate of 60 frames per second. Speedometer, rear- and side-view mirror information is visible on the center and side LCD screens. In addition, the driving simulator is equipped a 3D sound system, which can provide a rich audio environment with the sound of the engine, wind and tires. Vehicle vibrations are also simulated via a bass speaker under the driver seat. Steering wheel, gas pedal, brake pedal and car seats are all taken from the vehicle of Ford.

Fig. 1. Fixed-base driving simulator used in the experiment

In order to find the correlation combining the objective and subjective measures. Objective data was pre-designed and controlled by experimenter. For the objective data, we can get it through a questionnaire (Table 1). Each participant in our study is required to finish a questionnaire that includes a rating scale, which is a subjective selfassessment about perceived hazard during the experiment. Specifically, participants is asked to make a self-evaluation of the hazard perception level by expressing a level according to a numerical rating scale about a pair of adjectives representing from overdangerous to overcautious. We decided to use a Semantic Difference Scale (SDS) to

46

Q. Hu et al.

quantify the location which associated with the distance from the center to the corresponding “0” value. The extreme position (near to the adjectives) has the highest value, while the position near to the center have the lowest value. The sign (positive or negative) is associated with the values that identifying the distance from the center for indicating the direction of the distance: it is a convention that the positive sign is on the right, and the negative sign is on the left. Nevertheless, both the extreme position stands for a bad performance: the left means too dangerous under that condition; the right means too cautious which is also considered as a bad performance. There is no doubt that over-aggressive or aggressive behavior will make us feel dangerous. However, it is acknowledged now that driving too cautious also have a terrible influence on the traffic environment and make the passenger feel nervous. Participants express a level from –3 (over dangerous) to +3 (overcautious). Table 1. Subjective hazard perception rating scale Hazard perception rating Over dangerous and uncomfortable –3 Dangerous and uncomfortable –2 Just feel a little dangerous and uncomfortable –1 Normal and comfortable 0 A little cautious and uncomfortable 1 Cautious and uncomfortable 2 Overcautious and uncomfortable 3

By combining the two kind of data: the objective data controlled by simulator the subjective judgment from participants. 3.3

Driving Scenarios

The survey pre-designed the experiment scenarios (Table 2). The main task is a braking task, which there is the situation where the experimenter found out the object car and began to brake in one way until the subjective car stopped. Depending on velocity, acceleration, and distance, we created four scenarios including one training scenario and three main task scenarios to stand for different kind of situation. Each main task scenario is made up of nine trips, and in each end of a trip, there will be a van stopped still in front of the lane as an obstacle. When the subjective car driving at a giving condition, the hazard in our study was pre-created. To make sure the car was drove at one of 27 given scenarios. At first, there will be a speed signal to mind the driver who drive the simulate car to keep the speed as the study asked (30 km/h, 70 km/h, 110 km/h). Then after driving about 200 m later, there will be a sign for driver to start braking. By means of the simulator, when the car was braking, the acceleration of the subject was pre-set at a constant value (–1 m/s2, –3 m/s2, –6 m/s2). Therefore, it was actually a uniform deceleration. The braking signal was placed in front of the van at a calculated distance, which is 5%, 10% and 15%

How to Define the Passenger’s Hazard Perception Level

47

Table 2. Pre-designed 27 experiment scenarios Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario Scenario

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

Vs 30 30 30 70 70 70 100 100 100 30 30 30 70 70 70 100 100 100 30 30 30 70 70 70 100 100 100

a –1 –1 –1 –1 –1 –1 –1 –1 –1 –3 –3 –3 –3 –3 –3 –3 –3 –3 –6 –6 –6 –6 –6 –6 –6 –6 –6

d’ 5% 5% 5% 10% 10% 10% 15% 15% 15% 5% 5% 5% 10% 10% 10% 15% 15% 15% 5% 5% 5% 10% 10% 10% 15% 15% 15%

longer than the uniform deceleration distance of the subject car. The training scenario was built to get the participants warmed up before the main test began, which was almost like the main task scenario, with the number of obstacle and the acceleration different from main scenario. In the training scenario, the number of van is 5, and acceleration was not pre-designed and controlled exactly. However, the subject car will be brake in three way (overaggressive, normal, and overcautious) randomly to simulate our main task. What’s more, the journey on the two lane suburban way is about 15 km. 3.4

Procedure

Fifteen participants (12 men and 3 women) were tested in 27 scenarios lasting approximately 1.5 h. A consent was obtained from each participant after the study

48

Q. Hu et al.

being described in detail. Our experimental procedure included a training session and main task session. Participant would be asked to finish the main task after being trained. Before the training session participant was asked to read the instruction. Then the experimenter would guide the participant to fulfill the training session. Each time the experimenter finished one training task, the participants were required to give a score to describe the own feeling and judgement about the braking task. Next, the participants would be asked to fulfill the 27 main tasks as a passenger. In each scenario, subject car would be drove by the same driver on the road in a given order. Once driver finished a driving scenario, the participant should give the score of the hazard perception each time. Between each session, participants can had a rest about one minute. The procedure would repeat for 27 times until the main tasks are finished (Fig. 2).

Fig. 2. Experiment scene

3.5

Data Collection

3.5.1 Variables In our study, we aim to find the correlation between the passenger’s hazard perception and objective parameters (velocity, acceleration and distance). Hazard perception rating of each session has been treated as an exogenous variable, and three factors are treated as endogenous variable. The independent variables in the experiment is velocity, acceleration, and distance, which respectively has three levels. 3.6

Design

With the specific aim of finding the correlation between hazard perception rate and objective environment factors the experiment is conducted. A 3  3  3 mixed design is used. The between groups factors are the velocity, acceleration and distance between objective car and obstacle when the braking task is finished. The groups factor are different level, such as velocity (low: 30 km/h, normal: 70 km/h, high: 110 km/h), acceleration (overcautious: –1 m/s2, normal: –3 m/s2, overaggressive: –6 m/s2).

How to Define the Passenger’s Hazard Perception Level

49

To determine whether there is any relationship between the participant’s hazard perception rating and the objective environment factors (velocity, acceleration, distance), we carry out a 3  3  3 ANOVA with velocity (30 km/h, 70 km/h, 110 km/h) and acceleration (low, normal, high) and distance (5%, 10%, 15%) as independent variables, hazard perception rating as the dependent variable.

4 Data Analyze As is shown in Table 3, we found out that the interaction of velocity, acceleration and distance shows significant influence on hazard perception score, F(8,351) = 2.324, P = 0.019 < 0.05, that is, the interaction of two independent variables has different influence on the dependent variable at different levels of the third dependent variable. Therefore, it’s necessary for us to have a simple two-way interaction test to find out whether the two-factor interactions have significant effect on HP score or not. Table 3. Test of between subjects effects Source df Mean square F p Velocity 2 120.495 40.703 0.05. In addition, when velocity is 100 km/h and acceleration is –2 m/s2, the result of pairwise comparisons shows that The correlation between 15% group and HP rating is significantly higher than that in 5% group, p < 0.001. It also shows that there has significant difference between 10% group and 5% group, p = 0.012 < 0.05. However, there is no sig-indicant difference between 15% group and 10% group, p = 0.119 > 0.05. Table 5. Result of Pairwise Comparisons v

a

(I)d

(J)d

Mean difference (I–J)

Std. error

Sigb

High

High

Max

Mid Min Max Min Max Mid Mid Min Max Min Max Mid Mid Min Max Min Max Mid

.286 1.357* –.286 1.071* –1.357* –1.071 .714 1.714* –.714 1.000* –1.714* –1.000* .071 .214 –.071 .143 –.214 –.143

.346 .346 .346 .346 .346 .346 .346 .346 .346 .346 .346 .346 .346 .346 .346 .346 .346 .346

1.000

Hypertension No Yes Type II Diabetes No Yes =>200mg/dl Urinary Protein No (- & -+) Not include -+

n/N

0.270

n/N

0.039

0.012

0.476

0.015

0.476 57/240 8/31

23.8 25.8

59/260 6/10

22.7 60.0

0.678

0.015

0.678 30/260 1/11

11.5 10.0

Table 2b presents the unadjusted associations of independent variables with hypertension, diabetes, and proteinuria among rural participants. Both hypertension and diabetes were significantly associated with age group (P < 0.001). The percentage of participants with hypertension was significantly higher among those with overweight or obesity (47%) than among those without overweight or obesity (24%) (P < 0.001), among those with diabetes (49%) than among those without diabetes (35%) (P < 0.001), and among those with proteinuria (35%) than among those without proteinuria (25%) (P = 0.026). The percentage of participants with diabetes was significantly higher among those with hypertension (23%) than among those without hypertension (14%) (P < 0.001) and among those with proteinuria (13%) than among those without proteinuria (5%) (P < 0.004). The percentage of participants with proteinuria was significantly higher among those with diabetes (55%) than among those without diabetes (29%) (P < 0.004) and among those with hypertension (38%) than among those without hypertension (27%) (P < 0.026).

606

F. Yokota and A. Ahmed

Table 2b. Unadjusted association of independent variables with hypertension, diabetes, and proteinuria among voluntary community participants at Bheramara sub-district in rural Bangladesh Hypertension (N = 2,411) Items

n/N

%

P for Diff. 0.34

Proteinuria (N = 440)

Diabetes (N = 2,539) n/N

%

n/N

%

Sex

N = 2411

Male

541/1462

37.0

257/1574

16.3

73/254

28.7

Female

360/949

37.9

191/965

19.8

66/186

35.5

Age groups

N = 2411

15–39 years

264/993

26.6

103/1013

10.2

61/190

32.1

40–49 years

231/568

40.7

116/602

19.3

30/106

28.3

50–59 years

241/505

47.7

130/531

24.5

24/72

33.3

≥60 years

165/345

47.8

99/393

25.2

24/72

33.3

Pulse rate

N = 2388

Normal

756/2099

36.0

322/1892

17.0

101/351

28.8

Abnormal

140/289

48.4

50/247

20.2

24/57

42.1

Overweight/obesity

N = 536