Enhanced Trustworthiness and End User Acceptance of Conditionally Automated Vehicles in the Transition Period [1st ed.] 9783030608606, 9783030608613

Table of contents :
Front Matter ....Pages i-ix
How Should TrustVehicles Behave? Trustworthiness, User Acceptance and Expectations (Lisa-Marie Schicker, Daniel Watzenig, Ahu Ece Hartavi, Micaela Troglia)....Pages 1-23
Bridging Gaps for the Adoption of Automated Vehicles—BRAVE Aspects for TrustVehicles—Development of Innovative HMIs to Increase Acceptance (Clemens Kraetsch, Gabriella Eriksson, Niklas Strand, Florent Anon, Jan-Paul Leuteritz, Bernhard Schrauth)....Pages 25-43
How Should TrustVehicles interACT? (Mikko Tarkiainen, Merja Penttinen, Kimmo Kauvo, Ersun Sözen, Marc Wilbrink, Marc Kaup)....Pages 45-67
Reliable Sense-Plan-Act Approaches for TrustVehicles (Ahu Ece Hartavi, Tarek Kabbani, Pouria Sarhadi, Abhishek Shah Alias Sangani, Johan Zaya, Stephanie Maroun et al.)....Pages 69-85
TrustVehicle Verification Procedure (Bernhard Hillbrand, Pamela Innerwinkler, Georg Stettinger, Ahu Ece Hartavi, Kemal Rodoplu, Ali Demir et al.)....Pages 87-106
Assessment Concept for TrustVehicles (Philipp Clement, Herbert Danzinger, Philipp Quinz, Bernhard Hillbrand, Ahu Ece Hartavi, Kübra Zehra Kasikci)....Pages 107-128

Citation preview

Lecture Notes in Intelligent Transportation and Infrastructure Series Editor: Janusz Kacprzyk

Daniel Watzenig Lisa-Marie Schicker   Editors

Enhanced Trustworthiness and End User Acceptance of Conditionally Automated Vehicles in the Transition Period

Lecture Notes in Intelligent Transportation and Infrastructure Series Editor Janusz Kacprzyk, Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland

The series “Lecture Notes in Intelligent Transportation and Infrastructure” (LNITI) publishes new developments and advances in the various areas of intelligent transportation and infrastructure. The intent is to cover the theory, applications, and perspectives on the state-of-the-art and future developments relevant to topics such as intelligent transportation systems, smart mobility, urban logistics, smart grids, critical infrastructure, smart architecture, smart citizens, intelligent governance, smart architecture and construction design, as well as green and sustainable urban structures. The series contains monographs, conference proceedings, edited volumes, lecture notes and textbooks. Of particular value to both the contributors and the readership are the short publication timeframe and the world-wide distribution, which enable wide and rapid dissemination of high-quality research output.

More information about this series at http://www.springer.com/series/15991

Daniel Watzenig Lisa-Marie Schicker •

Editors

Enhanced Trustworthiness and End User Acceptance of Conditionally Automated Vehicles in the Transition Period

123

Editors Daniel Watzenig Virtual Vehicle Research GmbH Graz, Steiermark, Austria

Lisa-Marie Schicker Virtual Vehicle Research GmbH Graz, Steiermark, Austria

ISSN 2523-3440 ISSN 2523-3459 (electronic) Lecture Notes in Intelligent Transportation and Infrastructure ISBN 978-3-030-60860-6 ISBN 978-3-030-60861-3 (eBook) https://doi.org/10.1007/978-3-030-60861-3 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

Automated vehicles will be accepted by customers and society only when they will be deemed easy-to-use and fully reliable and safe regarding the planned manoeuvres and their execution. A key challenge is to ensure safe vehicles handling with reduced driver attention. Especially for level 3 automated driving systems, an effective interaction between the driver and the automated vehicle plays an important role. To act in harmony with driver expectations, these systems should be engineered following a user-centric approach. User acceptance is particularly important for the design of, driver interfaces that will facilitate the transitions between human and automated driving. Moreover, the automated driving systems should be resilient to both system and driver failures and guarantee sufficient reliability and robustness in each and every situation in real world traffic. The introduction of automated vehicles into the existing traffic poses specific issues regarding safety, in particular during the transition period where there will be interactions with other vehicles (of any degree of automation or none) and other traffic participants such as pedestrians or cyclists. This topic was taken up by the Horizon2020 “Automated Road Transport” call in 2016. Out of the call entitled “Safety and end-user acceptance aspects of road automation in the transition period”, three projects were funded, each dealing with a different angle on how to increase trust in conditionally automated vehicles: TrustVehicle TrustVehicle aims at advancing technical solutions for automated driving to better assess critical situations in mixed traffic scenarios and even under harsh environmental conditions, hence increasing safety far beyond the current levels. The project follows a user-centric approach and provides solutions which significantly increase reliability and trustworthiness of automated vehicles and contribute to end-user acceptance. BRAVE BRAVE’s approach assumes that the launch of automated vehicles on public roads will only be successful if a user centric approach is used where the technical aspects go hand in hand in compliance with societal values, user acceptance, behavioural intentions, road safety, social, economic, legal and ethical considerations.

v

vi

Preface

interACT The interACT project works towards the safe integration of AVs into mixed traffic environments. In order to do so, interACT analyses todays’ human-human interaction strategies, and implement and evaluate solutions for safe, cooperative, and intuitive interactions between AVs and both their on-board driver and other traffic participants. Together, these three projects form the H2020-ART-04 cluster and are part of the ECSEL Mobility.E Lighthouse Initiative that supports the roadmap towards safe, electric, automated/autonomous and connected smart mobility. The main research activities and results achieved in TrustVehicle are presented in this book, complemented by results from BRAVE and interACT. The overall structure of the book is presented in the picture below.

Driving Simulator

Expectations

Interaction concepts

Driver Monitoring

Simulation Proving Ground

Demonstrators KPIs

End-user acceptance

e q u ir m e nt

De

velo p m en

Ve

t

rifi c a ti o n

Ch

apter 5/6

s

Re

Scenarios/use cases Ch

C h a pter 1

a pter 2/3/

As

4

sess m en

t

Interaction Behaviour Objectives

Sense-Plan-Act

Simulation Proving Ground Driving Simulator

Sensor and perception concepts ADAS functions (planning and control)

The chapters build upon each other to draw a comprehensive picture, starting from the collection of the requirements for end user trust. It continues with the development of interaction and driver monitoring concepts as well as reliable sense-plan-act approaches and goes further to verification and assessment on three levels: in simulation, on the driving simulator and on test tracks. These steps should pave our way to trustworthy and reliable automated vehicles—so called TrustVehicles. The present book on trustworthiness and end user acceptance of conditionally automated vehicles provides an overview of current and emerging technological challenges. We hope that the reader will be inspired by the different chapters selected.

Preface

vii

Finally, we would like to express our sincere appreciation and gratitude to all authors and co-authors, who made the publication of this book possible. Graz, Austria August 2020

Daniel Watzenig Lisa-Marie Schicker

Contents

How Should TrustVehicles Behave? Trustworthiness, User Acceptance and Expectations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lisa-Marie Schicker, Daniel Watzenig, Ahu Ece Hartavi, and Micaela Troglia

1

Bridging Gaps for the Adoption of Automated Vehicles—BRAVE Aspects for TrustVehicles—Development of Innovative HMIs to Increase Acceptance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Clemens Kraetsch, Gabriella Eriksson, Niklas Strand, Florent Anon, Jan-Paul Leuteritz, and Bernhard Schrauth

25

How Should TrustVehicles interACT? . . . . . . . . . . . . . . . . . . . . . . . . . . Mikko Tarkiainen, Merja Penttinen, Kimmo Kauvo, Ersun Sözen, Marc Wilbrink, and Marc Kaup

45

Reliable Sense-Plan-Act Approaches for TrustVehicles . . . . . . . . . . . . . . Ahu Ece Hartavi, Tarek Kabbani, Pouria Sarhadi, Abhishek Shah Alias Sangani, Johan Zaya, Stephanie Maroun, Mona Saleh, Srinath Shanmugam, Ashok Krishna, Jibin Ou, Ersun Sözen, Eren Aydemir, and Samia Ahiad

69

TrustVehicle Verification Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bernhard Hillbrand, Pamela Innerwinkler, Georg Stettinger, Ahu Ece Hartavi, Kemal Rodoplu, Ali Demir, Talat Büyükakin, Ersun Sözen, Eren Aydemir, Philipp Clement, Johan Zaya, Sami Sahimäki, and Mikko Tarkiainen

87

Assessment Concept for TrustVehicles . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Philipp Clement, Herbert Danzinger, Philipp Quinz, Bernhard Hillbrand, Ahu Ece Hartavi, and Kübra Zehra Kasikci

ix

How Should TrustVehicles Behave? Trustworthiness, User Acceptance and Expectations Lisa-Marie Schicker, Daniel Watzenig, Ahu Ece Hartavi, and Micaela Troglia

Abstract To make a L3AD vehicle trustworthy and make the user feel comfortable using such a vehicle is crucial for the adoption of automated vehicles. Such an automated vehicle cannot be just a vehicle it has to be a TrustVehicle. Therefore, the TrustVehicle project presents an approach to define how a vehicle should behave in order to be a TrustVehicle by collecting requirements in the form of user concerns and expectations as well as expert opinions. Based on these results, Key Performance Indicators are defined to measure trustworthy vehicle behaviour divided in human, technology and vehicle category. Finally, the defined trustworthy vehicle behaviour needs to be benchmarked. This is done by a variety of use cases, dealing with complex scenarios and different demonstrator vehicles to reflect diverse aspects of complex traffic situations and conditions. Keywords User expectations · Trustworthiness · Conditionally automated driving · L3AD · Key performance indicators

L.-M. Schicker (B) · D. Watzenig Virtual Vehicle Research GmbH, Graz, Austria e-mail: [email protected] D. Watzenig e-mail: [email protected] A. E. Hartavi University of Surrey, Guildford, UK e-mail: [email protected] M. Troglia CISC Semiconductor GmbH, Klagenfurt, Austria e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Watzenig and L. Schicker (eds.), Enhanced Trustworthiness and End User Acceptance of Conditionally Automated Vehicles in the Transition Period, Lecture Notes in Intelligent Transportation and Infrastructure, https://doi.org/10.1007/978-3-030-60861-3_1

1

2

L.-M. Schicker et al.

1 Introduction Automated vehicle technology has the potential to be a game changer on the roads, altering the face of driving as we experience it today. Many benefits are expected ranging from improved safety, reduced congestion and lower stress for car occupants to social inclusion, lower emissions and better road utilization due to optimal integration of private and public transport [1]. Nevertheless, all technological developments are “useless” as long as the end user does not trust and accept automated vehicles. This applies especially for conditionally automated vehicles (SAE Level 3) [2] in mixed traffic scenarios that include various road users of various degrees of automation. Here, the driver is supposed to take over control when the vehicle requests it, leaving a lot of uncertainties for the driver regarding these take overs. The challenge is to turn a vehicle into a TrustVehicle—a vehicle which the end user can trust. To meet this challenge, the first step is collecting requirements (see Fig. 1) to answer the question on how vehicles should behave in order to gain the end users’ trust.

Objectives Scenarios/use cases

KPIs

e q u ir m e nt s

Demonstrators

Re

Behaviour C h a pter 1

Expectations End-user acceptance

Interaction

Fig. 1 TrustVehicle requirements

How Should TrustVehicles Behave? Trustworthiness …

3

Fig. 2 Complex situations for automated vehicles

Of course, to gain the end-user’s trust broadly speaking, the technologies have to work reliably and predictably. The average end user will not deal with the technological features in detail but has to rely upon that they work and detect the surroundings properly. Therefore, end user uncertainty about whether the systems work reliably in all situations and under all conditions is understandable as the examples in Fig. 2 show. Such situations and conditions dominated by uncertainty easily occur in • mixed traffic scenarios, where automatically and humanly driven vehicles have to co-exist on a conventional road infrastructure. • urban areas with a complex traffic environment characterized by narrow streets, many different road users ranging from all kinds of vehicles to Vulnerable Road Users (VRUs) like cyclists and pedestrians. • harsh and ambiguous environmental conditions: – weather conditions: heavy rain, fog, snow, etc. – bad road conditions: mud, constructions areas etc. [3]. Another uncertainty aspect refers to SAE Level 3 automated driving (L3AD) systems, where the automated driving system performs all aspects of the dynamic driving task within its specified Operational Design Domain (ODD) with the expectations that the human driver will take over control if requested and caused by exiting its ODD. With intuitive Human Machine Interfaces (HMIs) this transition needs to be made safe and stress-less for the driver. To avoid uncertainties for the end user under such situations and conditions as well as to convince them of TrustVehicles, all kinds of requirements have been collected, starting with the definition of a TrustVehicle according to the opinion of end users as well as automated driving experts. The result is presented in Sect. 2. After defining what is expected from a TrustVehicle, the question about how to measure trustworthy behaviour is answered in Sect. 3 by translating the results from the user studies described in Sect. 2 into three dimensions of Key Performance Indicators (KPIs): the human, technological and vehicle dimension. Finally, Sect. 4 presents the TrustVehicle approach on how to benchmark trustworthy vehicle behaviour by choosing a variety of use cases, scenarios and demonstrators to reflect diverse aspects

4

L.-M. Schicker et al.

of complex traffic situations and conditions. These steps shown in Sects. 2–4 go hand in hand with the following TrustVehicle objectives: Objective 1: Systematic identification of critical road scenarios for the currently available Automated Driving (AD) systems, with special focus on the uncertainty associated with the behaviour of other road users and the sensor fusion system of the ego vehicle (covered in Sect. 4). Objective 2: Controllers and sensor fusion systems capable of dealing with complex, uncertain and variable road scenarios, for enhanced road safety (covered in Sect. 4 and Chap. 4). Objective 3: Development and demonstration of intuitive human-machine interfaces (HMIs) for the safe management of the transition between fully automated driving (SAE Level 5) mode and human driving, taking into account user acceptance and gender-specific aspects (covered in Sect. 4 and Chaps. 2 and 3). Objective 4: Development and demonstration of new tools for the cost- and time-effective assessment of vehicle and driver behaviour in complex mixed-traffic scenarios (reflected in Sect. 3 and Chap. 6). Objective 5: Evaluation of L3AD functions and vehicle tailoring accordingly (covered in Sect. 3 and Chaps. 5 and 6).

2 What Is a TrustVehicle? Examination of User Expectations and Acceptance Since end-user acceptance and trust is the key for the successful implementation of automated vehicles, in a first step the public opinion on autonomous and selfdriving vehicles has to be determined. To understand the future users’ acceptance of automated vehicles, a quantitative non-experimental research that uses a survey approach and describes the trends for users was adopted. All in all, three surveys have been conducted and are presented below.

2.1 User Concerns & Expectations of AD Objectives The aim of this first questionnaire (“Q-I” in the following) was to collect information on the general opinion and issues regarding automated vehicles. It was not related to specific scenarios. The questionnaire started with an explanation of the level of automation, according to 2014 SAE international. For every level of automation, the participants were asked regarding their level of concern with each level of automation. There were questions on benefits and issues regarding completely self-driving vehicles. Respondents were also asked to rank topics on laws, costs, and safety.

How Should TrustVehicles Behave? Trustworthiness …

5

The final part of the questionnaire aimed to collected some basic background information on gender, age, education, employment, the most often used vehicle, and offered the opportunity to add comments. The objectives Q1 were: (i) to explore differences in concerns related to automated vehicles and the extent to which participants expressed positive and negative concerns related to AD, and (ii) to examine whether individual behaviours and characteristics could significantly predict acceptance and purchasing intentions regarding automated vehicles (AVs). The Q-I objectives ultimately served TrustVehicle’s focus in defining requirements for and expectations about AVs by the public. Participants characteristics The sample of Q-I consisted of 128 participants of whom 66% identified as male and 33% identified as female. Participants were from 24 different countries and the majority was between 18 and 69 years old. Most participants had completed a higher level of education: 50.8% of respondents had a master’s degree, 24% a bachelor’s degree and 18% a doctoral degree. 85% participants were employed fulltime and 7% were employed part-time. The three most representative job sectors of the respondents were engineering, the automotive sector, and research. 98% of the sample indicated being familiar with autonomous or self-driving vehicles prior to participating in this questionnaire. Results • General concerns: for Level 5 Automated Driving men were more concerned than women. Men and women were most concerned about “riding in a vehicle with no driver controls available” and “fully self-driving vehicle in the cities.” • Mixed-traffic scenarios where automated vehicles of any automation level share the road with non-automated vehicles were considered unsafe. • Unexpected situations were associated with an elevated level of concern. • Automated public transport: Women declared being “moderately concerned” about “public transportation such as buses that are completely self-driving”. • Non-concerning scenarios: the scenarios that were most often selected as “non concerning” were “Fully self-driving vehicles on motorway” and “Self-driving vehicles moving by themselves from one location to another while unoccupied”. • Expected benefits of completely self -driving vehicles: while not reaching statistical significance, the findings still suggest that saving time by using self-driving vehicles is not considered an important advantage. However, 15.3% of all male participants indicated that they would use their extra time for working. • The price of autonomous vehicles seems not to receive little consideration as a negative or positive feature by participants.

6

L.-M. Schicker et al.

2.2 User Concerns and Expectations About L3AD Objectives The second survey (“Q-II”) focused, in contrast to Q-I, specifically on L3AD. The same questions (expectations and concerns related to automated vehicles and nonsensitive demographic information) were developed as for Q-I but only explicitly pertaining to L3AD. The objectives of Q-II were (i) to explore differences in concern between levels of automation and the extent to which participants expressed positive and negative concerns specifically related to L3AD, and (ii) to examine whether individual behaviours and characteristics could significantly predict acceptance and purchasing intentions regarding L3AD. The results of this questionnaire were compared to the findings of Q-I to determine whether there are significant differences between expectations towards fully automated driving and conditionally or level 3 automated driving. These objectives ultimately serve TrustVehicle’s focus in defining user requirements for and expectations about L3AD by the public. Participants characteristics The sample of Q-II consisted of 424 participants of whom 67% identified as male and 32% identified as female, while 1% preferred not to specify their gender. Participants were predominantly from 6 different countries (Austria, Italy, Finland, Sweden, United Kingdom, and Turkey), a small fraction were from other countries such as France, the United Arab Emirates and India. The age of participants ranged between 18 and 70 years, with the majority of the sample under the age of 40 (68%). The majority of the sample had some form of higher education, with only 10% having secondary level education or below as the highest academic achievement. 47% of participants had a bachelor’s degree or higher. 63% of the participants were employed full-time and 18% were employed part-time. Results • Time frame in which participants would buy Level 3 Automated Vehicles (L3AVs): According to the results of the analysis in general, individuals would not consider purchasing a L3AV straight away. There are no differences in gender with respect to when individuals consider buying a L3AV. However, it is important to note that age plays a role in the timeframe with younger individuals willing to buy L3AV sooner than older participants. • Potential benefits of L3AVs: The “benefit” that was considered the most likely to occur was better mobility for disabled and elderly individuals, followed by fewer crashes with other vehicles, and reduced severity of crashes. Males rated the benefits of L3AV higher than females consistently.

How Should TrustVehicles Behave? Trustworthiness …

7

Below the ranks of items in order of highest to lowest of the given alternatives, demonstrating what participants thought of as being the most likely benefit arising from L3AV are given: – – – – – –

Better mobility chances for elderly and disabled people Fewer crashes with other vehicles Reduced severity of crashes Better fuel economy Lower vehicle emissions Fewer crashes with pedestrians and bicycles

• General concerns with travelling in a L3AV: Participants were asked how concerned they were to travel in a L3AV in general. Roughly 70% were not concerned at all, or only slightly concerned, while roughly 30% were moderately or very concerned. Females consistently rated their concerns regarding L3AV higher than men. It is also important to note that in general, concerns regarding L3AV increased steadily with age and the higher the education of participants, the higher they rated their concerns. The concern that was rated the highest was system security from hackers, followed by the L3AV getting confused by unexpected situations, and data privacy concerns (such as location tracking). These results imply, broadly speaking, that the most concerning aspects of L3AV for the general population, are security, the system’s ability to cope with unexpected situations and data privacy. Below the ranks of concerns in order of highest to lowest, for L3AVs are given: – – – – – – – –

System security (from hackers) Self-driving vehicles getting confused by unexpected situations Data privacy (location and destination tracking) Interacting with non-self-driving vehicles Safety consequences of equipment failure or system failure System performance in poor weather Interacting with pedestrians and bicyclists Legal liability for “drivers”/owners

• Potential concerns of L3AV in different scenarios on roads: The concern that was considered rated the highest was taxis being L3AV, followed by buses being L3AV, and commercial vehicles such as trucks being L3AV. The older an individual the more concerned they are with L3AV in residential areas, in cities, and on motorways. Below the mean ranks of items listed from highest to lowest, demonstrating what participants thought of as being the most concerning, are given: – Taxis that are Level 3 – Public transportation such as buses that are Level 3 – Commercial vehicles such as heavy trucks or semi-trailer trucks that are Level 3 – Level 3 vehicles driving in residential streets

8

L.-M. Schicker et al.

– – – –

Level 3 vehicles on rural roads Level 3 vehicles in the cities Level 3 vehicles moving by themselves from one location to another Level 3 vehicles on motorways

Interestingly, the scenarios that involve trusting other people with L3AV are those that were rated as having most cause for concern in scenarios. Public transport, and commercial vehicles were also rated as more concerning than scenarios in which the general population might be operating the L3AV, meaning that trust is a major factor. In terms of real-world implications, results have shown that the opinions of the participants regarding L3AVs are extremely mixed. Basically, people do not trust the technology because they do not have experience with it.

2.3 L3AD KPIs According to Experts Objectives The third survey (Q-III) has been developed to identify the most important Key Performance Indicators (KPIs) for L3AD vehicles. At first, a relatively small group of experts were asked to propose performance indicators that can reflect the performance and user acceptance of the L3AD. More than 120 performance indicators were collected. Once the collection process was completed, Q-III was designed to filter the most important KPIs out of this collection. The targeted respondents were experts of automated driving systems. This questionnaire also aimed to determine KPIs for four different L3AD vehicle types and two automated driving systems technologies, namely: passenger vehicles, light commercial vehicles, trucks, buses, time of flight cameras (ToF), and sensors. The collected KPIs related to specific expert fields: Human Machine Interface (HMI), Driver Expectations, Sensor and Camera, and Vehicle Manufacturers. HMI, Driver Expectations, and Vehicle Manufacturers were separately ranked for passenger vehicle, light commercial vehicles, buses and trucks by experts as can be seen in Fig. 3. Each survey respectively included 31 (HMI), 30 (Driver Expectations), 42 (Vehicle Manufacturers), 12 (Sensor), 10 (Camera) KPIs. Experts were asked to rank on 10-point Likert-style scales (1–10, not importanthighly important). Each expert ranked the KPIs on a scale based on his/her field of expertise. Consensus level was defined a priori as 60% which means the percentage of responding experts scoring a KPI as 8, 9 or 10. The resulting KPIs of Q-III are presented in detail in Sect. 3. Participants characteristics This study included answers from 231 experts. Most of the experts were males (76.6%), aged between 18 and 39 (76.2%) and had an education level of a bachelor’s, master’s, or doctoral (91.4%) degree. Experts from different fields responded to the

How Should TrustVehicles Behave? Trustworthiness …

9

Fig. 3 Breakdown of the survey

different versions of the survey according to their field of expertise. 24% of experts were from the automotive sensor/camera field, 24% from automotive software, and 22% of from vehicle manufacturers. Experts were predominantly from 6 different countries (Austria, Italy, Finland, Sweden, United Kingdom, and Turkey). Respondents from 7 different fields of expertise took part in this questionnaire, which addressed four areas: HMI, Driver Expectations, Sensor and Camera, and Vehicle manufacturers. An overview on the fields of expertise is given in Fig. 4. Results According to the experts, the use of KPIs for L3AD should: (i) guarantee an appropriate degree of quality of the proposed HW/SW considering a human-centric approach, (ii) reveal the effectiveness of the used techniques/technologies, (iii) provide standards for establishing comparisons, and (iv) to allow the re-planning of objectives and the decision-making process to be improved considering the customer’s perception. Hence, the research questions (RQs) are defined as: RQ1: What are the technical KPIs needed to be considered for the L3AV manufacturers, and technology providers? RQ2: What are the non-technical KPIs that should be considered in the heart of the design, to achieve high human centred L3AVs?

10

L.-M. Schicker et al.

Fig. 4 Overview on the experts’ fields of expertise across the four areas of Q-III

3 How to Measure Trustworthy Vehicle Behaviour According to the results of the questionnaire described in Sect. 2.3, three dimensions of KPIs to measure trust and end user acceptance and hence trustworthy vehicle behaviour have been classified, as can be seen in Fig. 5.

Fig. 5 Classification of the TrustVehicle key performance indicators

How Should TrustVehicles Behave? Trustworthiness …

11

One the one hand the human or end user dimension is crucial, but to expect the end user to trust in a vehicle, the suppliers’ and the vehicle manufacturers’ trust and acceptance also play an important role.

3.1 Human Dimension KPIs: Expectations from Users of the Technology After reviewing the literature [4, 5] and conducting the surveys, two underlying themes emerged in the human dimension KPIs: user perceptions and feelings, and user workload. The following paragraphs discuss each of these themes with the relevant KPIs. User Perceptions and Feelings Perceptions and feelings of future automated vehicle users are crucial for designing effective automated driving systems. Particularly perceptions on trust, safety, stress and frustration, traveling comfort, and control, are found to be important factors that affect the extent to which users accept autonomous vehicles. The KPIs with the highest importance for vehicle design according to experts starting with the most important ones, were as follows: • Perceived feelings of safety: The degree to which the user perceives feelings of safety. • Reliability: The degree to which an automated vehicle is trustworthy and operating consistently well. • Expressions of Trust: The degree to which the user expresses trust in the automated vehicle. • Perceived control: The degree to which the user perceives control over the automated vehicle. • Perceived travelling comfort: The degree to which the user perceives comfort while travelling in the automated vehicle. • Availability of L3AD within limited traffic scenarios—highway roads. • Availability of L3AD within all traffic scenarios. • Availability of L3AD within limited traffic scenarios—urban roads. • Availability of L3AD during harsh weather conditions. • Usefulness: The degree to which the user perceives that the L3AD function is useful for her/him. • L3AD Interaction with other road users. • Braking comfort: System reaction times for braking in emergency situations. User workload Another central theme for L3AD is the extent to which the user needs to exercise effort in the driving process. Although automated vehicles are intended to reduce

12

L.-M. Schicker et al.

the “workload” of users, in some cases such as when manual control needs to be re-taken by the user, they can suddenly demand too much from the user, resulting in a likelihood for human errors. For the TrustVehicle project, four main specific KPIs related to user workload were defined according to experts: • Driver mental workload: The degree to which the user needs to use mental faculties in the automated driving process. • Requirement of attention and concentration: The degree to which the attention or concentration are required from the user in the automated driving process. • Reduction of driver fatigue and alertness in autonomous mode. • Multitasking: Engaging in multiple tasks, including those unrelated to driving, while still taking part in automated driving. Regarding user perception and feelings, the main KPIs for passenger cars, light commercial vehicles, buses and trucks were quite similar. The most important KIPs are safety, reliability and trust. In addition, for passenger cars and trucks, the availability of L3AD on highways is more of importance than for the other vehicle classes. User workload is very important for trucks and buses, so to say for professional drivers, whereas for passenger car and light commercial vehicle drivers this plays a rather minor role.

3.2 Technology Dimension KPIs: Expectations from Suppliers of the Technology Human-Machine Interactions In L3AD the driver has to take over control when the system requests it. Here the socalled Human-Machine-Interaction comes into operation to ensure a smooth, stressfree and safe takeover process [6, 7]. The following KPIs were ranked as most important by the experts: • Comprehensibility of take over request: The degree to which the user can understand when he/she has to take over the vehicle control. • Control transition request with visual warning: Visual warnings that accompany requests for control transitions in automated vehicles (measured by take-over time). • Control transition request with audial warning: Audial warnings that accompany requests for control transitions in automated vehicles (measured by take-over time). • Comprehensibility of user interface: The degree to which the user can comprehend the user interface of the automated vehicle. • Driver intention & readiness: The extent to which the driver intentions towards driving behaviour are in line with the intentions of the AD system.

How Should TrustVehicles Behave? Trustworthiness …

13

• Driving mode traceability: The degree to which the user can understand the choice of current driving mode (manual/transition/automated) of the automated vehicle. • Driver attention focusing: The degree to which the user can understand when and where to put attention in each driving mode (manual/transition/automated) of the automated vehicle. Sensing Systems Besides HMI, the main technology components, relevant for TrustVehicle are sensors [8, 9]. The main KPIs in this category according to experts are listed below: • • • • • • • • • • • • • • • • •

Frequency of sensor malfunctioning per 100 km Sensor accuracy: The deviation from the true range expressed in percent of range. Sensor robustness Sensor weather condition robustness: The deviation from the true range expressed in percent of range, under different weather conditions. Sensor range: The range, which the sensor can reliably measure. Sensor field of view: The extent of observed world seen at any given moment by the sensor. Sensor temperature robustness: The deviation from the true range of temperature expressed in percent of range, under different temperatures. Sensor resolution Sensor sensitivity to dirt Frequency of ToF camera malfunctioning per 100 km ToF camera weather condition robustness: The deviation from the true range of sensor reading expressed in percent of range, under different weather conditions. ToF camera environmental light and temperature robustness: The deviation from the true range of camera reading expressed in percent of range, under different temperatures and environmental light conditions. ToF camera accuracy: The deviation from the true range of camera reading expressed in percent of range. ToF camera resolution ToF camera field of view: The extent to which the detects the environment at any given moment. ToF camera range: The range, which the ToF camera can reliably measure. The targeted range for usage inside the car is 2 m and for outside the car is up to 4 m. ToF camera frame rate: The received frames per second.

3.3 Vehicle Dimension KPIs: Expectations from Vehicle Manufacturers The last dimension reflects the expectations of vehicle manufacturers and what they consider as being important to measure how trustworthy a vehicle behaves for L3AD [10]. Their expectations can be measured by the following KPIs:

14

L.-M. Schicker et al.

• Number of accidents in autonomous driving mode per 100 million km • System reaction times for braking in emergency situations • Number of instances where the situation is not correctly handled by vehicle in autonomous mode per 1000 km • Number of traffic violations in autonomous mode per 1000 km • Number of emergency stops per 1000 km • Number of conflicts encountered where time-to-collision (TTC) is less than a predetermined threshold • Malfunctioning of automated driving functions (number of events per 100 km) The main KPIs do not differ between the four vehicle classes with the number of accidents in autonomous driving mode per 100 million km and the system reaction times for braking in emergency situations being the most important ones.

3.4 Summary With these three dimensions, the most important L3AD KPIs for the four vehicle classes have been defined. In a next step, they are used as the foundation for the specific TrustVehicle L3AD use cases to realize trustworthy vehicle behaviour even in scenarios that have the potential to cause uncertainties for the end user.

4 How to Benchmark Trustworthy Vehicle Behaviour 4.1 Use Cases and Scenarios To benchmark trustworthy vehicle behaviour testing and demonstrating is indispensable. However, doing enough testing (many test kilometres) on proving grounds and public roads for highly automated vehicles (L3 and above) is economically challenging. This is due to the fact that highly automated vehicles have to fulfil very high functional safety requirements which cannot be proven by conventional road tests with reasonable effort is not possible (high complexity, vast number of scenarios). In the TrustVehicle project, a scenario-based testing approach was chosen instead of distance-based validation. Therefore, the verification and validation process is composed of three steps: (1) Virtual tests and simulation, (2) driving simulators and (3) test tracks. The approach started, as shown in Sects. 2 and 3 with the collection of AV user requirements, expectations and related KPIs. This step was followed by the development of scenarios. The TrustVehicle use-cases are defined not only to enhance the behavioural capability, but also to demonstrate the trustworthiness of the vehicles in a large variety of conditions and situations, for example under harsh weather conditions, and in mixed traffic situations. These use cases are presented in this section.

How Should TrustVehicles Behave? Trustworthiness …

15

The following steps aim to answer the questions of how the gaps for the adoption of automated vehicles should be bridged and how TrustVehicles should interact. This is done by the development of driving interaction and driver monitoring concepts and solutions (as described in Chaps. 2, 3 and 4) on the one hand and by the development of reliable sense-plan-act approaches based on the requirements previously defined in this chapter on the other hand. Then the final steps are dedicated to determining verification procedures and assessment concepts for these developments on three levels, namely in simulation, on the driving simulator and on test tracks including user acceptance testing. To build a solid basis for development, verification and assessment use cases have been defined to enhance the trustworthiness and weather-independence of conditionally automated vehicles in mixed traffic scenarios. They are selected to improve the behavioural capabilities of the vehicles and to avoid crashes in critical scenarios in urban environment, thereby increasing trust even in critical situations. In the context of the TrustVehicle project, critical scenarios are defined as scenarios especially dedicated to be handled with L3AD functions. Such scenarios typically involve VRUs or rough weather conditions, such as heavy rain or fog. In order to solve issues evolving from such scenarios, the scenarios in each use case are further divided between two classes: • critical scenarios solvable through enhanced automated driving controllers which build the basis for reliable sense-plan-act approaches. • critical scenarios solvable through a safe hand-over process to the driver which builds the basis for interaction and driver monitoring concepts. When speaking about enhanced driving controllers, apart from the controllers themselves we are also talking about enhanced sensor systems (e.g. self-cleaning sensors) and sensor fusion. Scenarios solvable through enhanced L3AD systems (controllers and sensors) typically involve light disturbances of the sensors (e.g. dust, light rain) or common traffic scenarios that do not require intuitive reactions of the driver. In these cases, sensor redundancy or sensor monitoring, and cleaning can enhance the input to the L3AD function and the function itself can be improved using the enhanced input. These improvements can solve critical scenarios but are dependent on the limitations of the considered L3AD function. According to SAE standards the functionality of L3AD systems is limited to well-defined ODDs, which can be either geographic, roadway-specific, environmental, traffic- or speed-specific and/or temporal limitations. The lack of sufficient consideration of the respective system limits can cause severe accidents and therefore should not be neglected when speaking of critical scenarios. In scenarios when the L3AD function meets its system limits, the situation can only be resolved by a safe hand-over process to the driver. The hand-over process shall be clearly defined and intuitive so that the human driver is able to take over vehicle control timely and seamlessly, even in a possibly stressful situation. For both categories it is crucial to test a wide set of critical scenarios so that the Level 3 functionality can be sufficiently validated.

16

L.-M. Schicker et al.

Therefore, the TrustVehicle scenarios comprise a wide range of critical driving scenarios to cover the large variety of critical situations and conditions that influence the trust and end-user acceptance benchmarked for the four vehicle classes.

4.2 Passenger Vehicle Use Case In automated driving one important factor is the occurrence of failures and how to deal with them during operation. At SAE Level 2 if a failure occurs, either from hardware or software side, the fall-back is the driver, who has to reclaim vehicle control immediately. For L3AD the system has to be able to cope with failures at least for some time until the driver is able to take over vehicle control. To ensure that the system is able to cope with failures, sensor monitoring is essential. Sensor Monitoring For automated driving scenarios with passenger vehicles, the focus will be on degraded sensor functionality and sensor data output. In order to conduct safe automated vehicle manoeuvres, vehicle controllers are highly dependent on trustworthy data from the vehicle sensors. The sensor output must ensure that lines, objects, VRUs and other potential obstacles within the driving path are detected and reported. However, ensuring a 100% detection rate is unlikely and much of this due to external disturbance and not necessarily due to sensor performance. Degradation of the sensor output can result in false detections, late detections or even missed detections which may lead to crashes. The degradation of sensor output also impacts the availability of the intended function, i.e. the automated driving function is availably much less than expected. The use cases for potential sensor output degradation are split into scenarios listed below and the sensor set-up is illustrated in Fig. 6. Scenario 1: Functional behaviour—sensor health and communication quality Scenario 2: Environmental conditions—harsh weather that may reduce sensor visibility.

Fig. 6 Sensor monitoring concept

How Should TrustVehicles Behave? Trustworthiness …

17

4.3 Truck Use Case Automated driving for vehicle reverse parking is becoming a widespread application for many OEMs. However, due to kinematic and geometric constraints, reverse parking of an articulated vehicle, especially a semi-trailer regarding its size, introduces a new problem which is addressed by the following two TrustVehicle truck reverse parking scenarios. Docking Station A loading dock is a recessed bay in a building, where trucks are loaded and unloaded. Loading docks are commonly found at manufacturing plants, warehouses and other industrial buildings. They are the primary location of the movement of product, coming in and out of a facility. Reverse parking is a frequent activity of a truck driver going in and out of a docking station. In a busy environment the parking process needs special care considering other vehicles and pedestrians in the field. The autonomous reverse parking scenario starts with the selection of the available docking platform and initial positioning of the vehicle. With the initiation of the self-parking feature, the truck will approach the desired position giving proper actuation and perception of the environment and any dangerous situation without any driver intervention. An example of such a docking station is shown below in Fig. 7. Construction Site Especially the construction sites along crowded city roads require extensive effort of the drivers accessing the site (see the example in Fig. 8). Complex manoeuvring with special attention to other vehicles and pedestrians is needed. The autonomous reverse approach feature will primarily detect the situation of the road and manoeuvre possibilities. This scenario calls for a simultaneous perception of the situation as well as the ability to update the manoeuvre accordingly. The manoeuvre will finish with the entry of the truck in the site and the final approach to the loading position. Possible handover cases will be considered for the scenario regarding changing road restrictions and immediate attention to the danger present in situations.

Fig. 7 Docking station reverse parking

18

L.-M. Schicker et al.

Fig. 8 Construction site reverse parking

4.4 Electric City Bus Use Case The electric bus use case was motivated due to some difficulties in positioning a city bus with pantograph correctly under charging poles in daily operation. Furthermore, this kind of use case for automated driving with electric city bus has not been studied or the results have not been published before. With this demonstration of automated positioning accurately under a charging pole, it is expected to achieve several benefits such as time savings in daily bus operations and decreased burden and stress for bus drivers. Automated Driving to Electric Charging Point on a Bus Stop The electric bus under consideration should drive automatically towards electric charging points at the bus stop. In this scenario, the bus is approaching the bus stop and charging spot on manual driving mode and then it will be switched to an automated driving mode. The system will subsequently provide instructions for the driver. The automated driving system calculates the driving path and checks possible obstacles. Charging takes place automatically after the system has parked under the charging spot. After charging, the bus will automatically drive to the border of the automated driving area. When the bus is approaching the bus stop in manual driving mode, the system provides driving instructions (e.g. speed and distance to automated driving enabled area) to the driver. Then when the bus enters the automated driving area, the preconditions for automated driving are checked, e.g. current speed, position, heading and angle of the steering wheel as well as sensor capabilities and weather restrictions. Then the automated driving system calculates the driving path and checks possible obstacles. If automated driving can be activated, it is offered to the driver via the HMI. The driver activates the automated driving or acknowledges the transition from manual driving to automatic driving. The system is now in transition mode.

How Should TrustVehicles Behave? Trustworthiness …

19

Fig. 9 Bus final demonstration target area in Helsinki, Finland

It confirms that automated driving mode is activated (HMI). While driving in automated driving mode, the system only displays relevant information to the driver, e.g. the current location of the bus, planned driving path, obstacles, destination, and distance to the destination. At the destination (under the charging point) the system indicates to the driver when the bus has been parked and charging can be activated. Automated driving mode is deactivated (shown in the HMI). The demonstration area is shown in Fig. 9. Exceptions to the success scenario are observed when the automated driving mode cannot be activated when the bus is entering the area e.g. due to excessive speed of the bus, position of the steering angle or obstacles on the route.

4.5 Light Commercial Vehicle Use Case An automated door to door delivery concept is a strategic priority for light commercial vehicles. Light commercial vehicles undertake a high percentage of urban delivery with their dimensional advantage when the delivery mission does not require high payload capacities. Logistic companies place special focus on the automation of delivery and reducing their expenses. Two urban traffic scenarios are implemented to investigate and increase user acceptance of L3AD for light commercial vehicle customers. Manoeuvring in narrow streets For a deliverer, it is a very frequent task to drive through narrow streets in urban traffic and to park in crammed spaces to load or unload the light commercial vehicle (LCV). Since the LCV differs from a passenger car in size, the difficulty of driving one in some places is comprehensible. With these motives in mind, a use case was selected in which the LCV passing through a street with narrow gateways and parking to a delivery area requiring reverse loading or unloading operations. The model and measurements of such narrow roads are given in Fig. 10. During the autonomous drive operations, narrow street manoeuvres, which will be realized by controllers, should completed quicker than normal in operation and transitions should be smooth

20

L.-M. Schicker et al.

Fig. 10 Sensor set-up and narrow street scenario

enough to achieve user acceptance. In addition to that, safety should be maintained for the passengers, VRUs present and other vehicles in the environment. Automated Backing Manoeuvre Besides driving in narrow urban streets, reverse parking in a delivery area for loading or unloading operations is part of the daily life for a delivery person. The aim is to cover low speed automated backing manoeuvres of LCVs, minimize the number of parking manoeuvre and minimize approaching tolerances.

4.6 Driver Identification Use Case Besides scenarios characterized by one of the four vehicle classes, TrustVehicle also chose a use case that focuses stronger on monitoring the driver per se in different situations rather than focusing on a concrete driving situation or on a specific vehicle class. Research suggests that drivers who are occupied with a secondary task will take longer to assume control. To provide a better and safer takeover process experience, driver monitoring can provide additional information about the driver’s condition. Based on this information the vehicle controller can make decisions on e.g. the procedure of the takeover process or the point in time when a driver will be notified about the impending takeover [11]. The driver identification use case includes a person entering the vehicle and sitting down. After the person is seated, the HMI will request the driver identification. If the person is known to the vehicle, the driver’s user ID is passed to the vehicle HMI. If the person is unknown, the driver registration use case option is provided. A new user profile can be created. The ToF camera will be explored for driver monitoring

How Should TrustVehicles Behave? Trustworthiness …

21

and possible kinds of additional information will be evaluated. Explored kinds of additional information include heart rate monitoring, head pose tracking and driver identification. Driver identification A person sits down on the driver seat. The person has never used this vehicle before. The vehicle recognises that this is an unknown person. If the creation of a new user profile is allowed, the vehicle requests the creation of a new profile. The instructions are given visually. It starts adjusting the mirrors to the eye height of the person and trains the recognition model to be able to identify the person in the future. The purpose is to create a convenient use of vehicles shared by several persons, as seat position and mirror alignment need to be adjusted to the person in order to safely operate the vehicle in manual driving mode. Identification of known driver A person sits down on the driver seat. The person has used the vehicle before. The vehicle recognises the person and a visual confirmation is shown. It starts adjusting the seat, mirrors and other settings to the preferences of this person. The purpose is also to create a convenient use of vehicles shared by several persons, as seat position and mirror alignment need to be adjusted to the person in order to safely operate the vehicle in manual driving mode. Driver health monitoring A person sits on the driver seat. The ToF camera is either measuring the heart rate remotely on face or through contact with the finger of the person. To measure the pulse wave, the finger of the person will be used. The results will be visualised (see Fig. 11). Monitoring the heart rate and pulse wave gives additional information about the driver state and fitness to drive the vehicle and hence reduces accidents caused by driver health issues.

Fig. 11 Driver identification, monitoring and visualisation of the pulse wave analysis on the ToF camera

22

L.-M. Schicker et al.

5 Conclusion Trust and end-user acceptance are one of the main prerequisites for the successful introduction of L3AD. As the surveys conducted in TrustVehicle showed, the general public is extremely mixed regarding their opinion on L3ADs. They are not sure if they can trust in L3AD vehicles especially due to a lack of experience with this technology. Questions like “What would happen in case of unexpected situations?”, “What would be prioritised?” arise and cause uncertainty for the end-user. Here ethical questions play a major role in determining end-user acceptance. To be able to define and measure trustworthy vehicle behaviour in order to increase trust and end-user acceptance, three dimensions of KPIs have been classified via expert consultation. The first one is the human dimension dealing with expectations from the users themselves. They can be split into user perceptions/feelings and user workload. The second dimension is the technological dimension reflecting the expectations of technology suppliers with regards to HMI and sensor systems. KPIs taking the expectations of vehicle manufactures into account, the so-called vehicle dimension, is the third one. With these three dimensions, the most important L3AD KPIs for TrustVehicle have been determined. This is followed by the definition of use cases which demonstrate the trustworthiness of L3AD vehicles through a large variety of conditions like harsh weather conditions, critical situations like driving in narrow urban streets and through using four different vehicle classes. The diversity of use cases ranges from a passenger vehicle dealing with sensor faults and reduced sensor visibility due to bad weather, to truck-trailer reverse parking scenarios, an electric city bus driving autonomously to and from the electric charging point, via a LCV for door-to-door delivery in narrow urban streets, to driver identification and health monitoring. All these findings and requirements build the basis for a successful development, verification and assessment of TrustVehicles. Acknowledgements The project leading to this chapter has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 723324. The chapter was written at Virtual Vehicle Research GmbH in Graz and partially funded by the COMET K2—Competence Centers for Excellent Technologies Programme of the Federal Ministry for Transport, Innovation and Technology (bmvit), the Federal Ministry for Digital and Economic Affairs (bmdw), the Austrian Research Promotion Agency (FFG), the Province of Styria and the Styrian Business Promotion Agency (SFG).

References 1. Watzenig, D., Horn, M. (eds.): Automated Driving—Safer And More Efficient Future Driving, 1st edn. Springer (2016) 2. SAE Standard J0316: Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles (2018)

How Should TrustVehicles Behave? Trustworthiness …

23

3. Haboucha, C.J., Ishaq, R., Shiftan, Y.: User preferences regarding autonomous vehicles. Transp. Res. Part C: Emerg. Technol. 78, 37–49 (2017) 4. Lee, J.D., See, K.A.: Trust in automation: designing for appropriate reliance. Hum. Factors 46(1), 50–80 (2004) 5. Abraham, H., et al.: Autonomous vehicles, trust, and driving alternatives: a survey of consumer preferences. In: Transportation Research Board 96th Annual Meeting, Washington, DC, pp. 8– 12 (2017) 6. Griffiths, P., Gillespie, R.B.: Shared control between human and machine: Haptic display of automation during manual control of vehicle heading. In: 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, HAPTICS’04. Proceedings, pp. 358–366. IEEE 7. Hartavi, A.E., Ka¸sıkçı K.Z., Sorniotti A.: Evaluation of Driving Control Authority Transition Time in Level 3 Conditional Automated Driving, 9th International Automotive Technologies Congress, 2018, Turkey (2018) 8. Foix, S., Alenya, G., Torras, C.: Lock-in time-of-flight (ToF) cameras: a survey. IEEE Sens. J. 11(9), 1917–1926 (2011) 9. Winner, H., Hakuli, S., Wolf, G. (eds.): Handbuch Fahrerassistenzsysteme: Grundlagen. Springer, Komponenten und Systeme für aktive Sicherheit und Komfort (2011) 10. Ballarin, C., Zeilinger, M.: The Truck of the Future: Autonomous and Connected Driving at Daimler Trucks. SAE, Daimler AG (2017). (s.1) 11. Nahler, C., et al.: Exploring the usage of Time-of-Flight Cameras for contact and remote Photoplethysmography. Paper presented at 21st Euromicro Conference on Digital System Design, Prague, Czech Republic (2018)

Bridging Gaps for the Adoption of Automated Vehicles—BRAVE Aspects for TrustVehicles—Development of Innovative HMIs to Increase Acceptance Clemens Kraetsch , Gabriella Eriksson , Niklas Strand , Florent Anon , Jan-Paul Leuteritz , and Bernhard Schrauth Abstract Automated cars bring not only technical but also non-technical challenges that influence acceptance such as legal and ethical questions, social and economic implications, or road safety effects. In general, if technical innovations are in line with societal values and behavioural intentions, they can gain acceptance. Based on a literature review, issues that influence the acceptance of automated cars are identified. However, not only the macro aspects are important for acceptance, but also the design of the technology for potential users, especially the HMI design. As the division of tasks between humans and machines changes, the implementation of automated driving systems changes the task of the driver fundamentally, posing new challenges for HMI design. Uses cases involving vulnerable road users (VRUs) are the focus of the BRAVE project (EU-funded) when developing and testing HMIs. Research on the interaction between automated vehicles and VRUs is needed, not only to increase acceptance among road users but also to increase safety for VRUs. C. Kraetsch (B) · B. Schrauth Institut für Empirische Soziologie (IfeS) an der Friedrich-Alexander-Universität Erlangen-Nürnberg, Nürnberg, Germany e-mail: [email protected] B. Schrauth e-mail: [email protected] G. Eriksson · N. Strand VTI, the Swedish National Road and Transport Research Institute, Linköping, Sweden e-mail: [email protected] N. Strand e-mail: [email protected] F. Anon Mov’eo, Saint-Étienne-du-Rouvray, France e-mail: [email protected] J.-P. Leuteritz Fraunhofer IAO, Berlin, Germany e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Watzenig and L. Schicker (eds.), Enhanced Trustworthiness and End User Acceptance of Conditionally Automated Vehicles in the Transition Period, Lecture Notes in Intelligent Transportation and Infrastructure, https://doi.org/10.1007/978-3-030-60861-3_2

25

26

C. Kraetsch et al.

The development of interfaces and products must be carried out in an iterative process, to allow users to be involved at an early stage. Adjusting concepts in the beginning of the process is more cost-efficient compared to making changes later in the process and ensures a better adaptation of the developed technical solutions to the expectations of end-users, thus favouring the acceptance of automated vehicles. Keywords Acceptance · Societal requirements · Human-Machine Interfaces (HMIs) · Methodology

1 Introduction Vehicle automation can be regarded as a machine that executes a function that was previously performed by the human driver. Like in all fields affected by increasing levels of automation, the implementation of automated driving systems has major consequences for the human users. The division of tasks between humans and machines changes, for instance from executing functions to supervising the automated execution of functions. This creates new challenges for the design of Human-Machine Interfaces (HMI). Innovative HMIs are expected to favour acceptance of automated vehicles1 by the society. But there is neither a universal definition of acceptance nor a single approach. However, two main elements can be considered: societal acceptance, and user acceptance of certain technical solutions. This chapter will focus on these two different facets of acceptance: On the one hand, the general societal acceptance including social, economic, ethical, legal and road safety issues. On the other hand, user acceptance of certain technical solutions. To gain acceptance for the technology, the development of innovative technical solutions is required within the specific context of HMIs. These HMIs are related to internal (Driver monitoring, ADAS, etc.) and external interactions (communication with other road users). The key findings and methodologies presented here are based on a comprehensive literature review, but also on specific studies conducted within the framework of the BRAVE project (financed by the European Commission, grant agreement no. 723021). Involving partners from 5 countries, it considers the needs, expectations and specificities of all stakeholders for the development of suitable HMIs towards a better acceptance of automated vehicles.

1 Although

the research in BRAVE explicitly focuses on SAE level 3 automation, in the following the term "automated" is used non-specifically for SAE level 3 to 5 cars, unless the specific topic requires a distinction to be made.

Bridging Gaps for the Adoption of Automated Vehicles …

27

2 Societal Acceptance of the Automation of Cars: Social, Economic, Ethical, Legal and Road Safety Aspects For the term acceptance itself, there is no universal definition and at the same time a wide range of theoretical constructs [1]. Essentially, acceptance means a positive attitude towards something or someone involving an active element describable as “willingness to do something” [2, p. 622]. As stated before, acceptance can be considered to refer to the two aspects of general societal acceptance and user acceptance of certain technical solutions. In general, only when technical innovations are in line with societal values and behavioural intentions, they may gain acceptance of the public. Therefore, BRAVE deals — among others — with both of these two issues. In the following, at first aspects of acceptance regarding road safety and HMI are briefly discussed and then the main topics on societal acceptance, social, economic, ethical and legal issues, are elaborated. We will identify these factors by a literature review. Road safety, HMI and acceptance One of the great promises of the introduction of automated cars is the reduction of the number of accidents. This is likely, since a significant amount of accidents, depending on the studies, between 67 and 90% [3, 4] is due to human error and/or distracted drivers and/or drivers under the influence of alcohol or drugs. Acceptance will also depend, among other factors, on whether this safety promise can be kept. However, many studies show that there could be new safety problems that can reduce the expected increase in road safety [1, p. 31ff.]. One of the biggest safety problems for SAE Level 3 cars will be to manage the situation where the driver needs to take control quickly (because the system is not able to drive in automated mode or of a system failure). So, a major research question is how the internal HMI must be designed to keep the driver in the loop (fallback) and to reduce response times to hazardous situations requiring human intervention. Regarding the external HMI, other road users might feel that their safety is compromised by automated cars, for instance if the car does not communicate with them in an understandable way. To gain acceptance, innovative internal and external HMI approaches are needed. Insights from a review on survey studies Several surveys about the acceptance of automated cars have been carried out over the last years (see [5], for a review until 2017), but in many studies a direct connection to theoretical acceptance models remains vague [1]. Still, most often the focus of these surveys is on the individual level (“would one use or accept the technology?”), whereas societal aspects in these studies are rarely focused or addressed. Regarding automated cars “individual” acceptance is mostly measured by the potential intention to use or to purchase the new technology, i.e. the willingness to pay (more) for the new technology.

28

C. Kraetsch et al.

Although the surveys are often difficult to compare with each other (because, if any, of different theoretical approaches or divergent questions/operationalisations), they apparently all indicate that there are positive expectations as well as widespread concerns in the public. As in many surveys and in the focus group discussions conducted in BRAVE [6], most respondents expected automated cars to reduce the number of accidents and increase road safety. However, the findings also reveal major concerns about reliability and equipment or system failures [7–10]. In addition, two other non-technical concerns were stated: (1) Liability in case of a crash [7–9, 11] and (2) Privacy/data protection [7, 8]. A shortcoming in the current research literature is that they relate to acceptance from the user’s perspective. The views of other road users who are directly affected by automated vehicle technology has so far received little consideration [12]. However, their view of automated vehicles is as important for societal acceptance as that of (potential) users, because if other road users such as e.g. pedestrians or cyclists, feel threatened by the emergence of automated cars in their personal road safety, a broad introduction of automated vehicles will face severe frictions. Insights from scientific literature on social and economic aspects Societal acceptance may depend on the expected social and economic impacts of the deployment of automated cars. A lot of studies assume that the widespread introduction of automated vehicles will mainly have positive effects for the society. Among others, a reduced accident rate — and thus, reduced costs for repairing cars and infrastructure as well for the medical treatment of injured persons —, the ease of traffic congestions, and improved utilisation of existing road capacity [13, 14]. It is also expected that the mobility of persons currently excluded from individual road traffic (elderly and disabled people) can be improved significantly [15]. On the contrary, some assume negative impacts for society such as more traffic on the roads [16, 17] and therefore, adverse effects on the environment and road capacities [17]. For example, it is often assumed that — if there are fully automated or autonomous cars on the market — fewer people will be interested in owning a car and use car sharing instead. So, there may be fewer cars registered but these cars might cause more traffic, as autonomous cars are used much more often than conventional cars because of the possibility to use the time in the car for other activities than driving. At this stage the social and economic effects of the automation of cars on traffic volume are not well predictable [18] and so the influence of social and economic implications on the acceptance of automated cars. There are two issues that make predictions difficult: (1) It is not clear if and how people will change their travel behaviour. For example, emissions can either be halved or doubled depending on mobility behaviour [17]. (2) The effects will strongly depend on what level of automation the cars will have in the next years. Obviously, if there are only SAE level 3 cars in the next decade on the road, for many issues nothing will change fundamentally. As, for example, people who are not allowed to drive today, will still not be allowed to drive a car soon, and thus a positive effect on inclusion is missing.

Bridging Gaps for the Adoption of Automated Vehicles …

29

Insights from scientific discussions on ethical aspects Even if it may not seem obvious at first glance, many decisions by people in traffic are ethical, as they can affect the lives of others [19]. And a car in an automated driving mode must ‘make decisions’ that potentially have an impact on the lives of humans. Although automated driving touches on many ethical aspects [20, 21], there are two topics in particular that are being discussed by ethicists and gained public attention: (1) How should an automated car behave in a crash? (2) Should there be mandatory settings, or should personal ethics settings be allowed? The discussion about ethical aspects in the event of an accident is based on three assumptions: First, unavoidable accidents will continue to occur, even if only autonomous vehicles will persist on the road [22]. Second, the sensors can capture the surrounding environment very accurately and in detail, e.g. the car can detect how many persons are in the vicinity (and can also differentiate them according to other characteristics such as height). Third, with regard to the capacity of the computer hardware, the autonomous car can ‘predict’ the course and the consequences of an accident and therefore can choose between different alternatives. Ethicists discussing the problem of crash behaviour generally use one certain scenario based on an ethical thought experiment called the “trolley problem” [23]: The autonomous vehicle is involved in an unavoidable crash, it cannot be avoided to injure or even kill a person but it is possible to choose who will be harmed by the crash. The trolley problem and scenarios deduced from it describe a moral dilemma because no clear ethically or morally right answer can be given on how to behave in such a situation [20]. Whatever decision is made, harm to persons is unavoidable and there are good ethical reasons for one or the other behaviour. It would be very problematic if the algorithms allocated more safety risks to certain types of road users. Not to program anything for this scenario is likewise ethically unacceptable, if the ‘car’ due to the capacity of its sensors and computer hardware could analyse the consequences of the accident and has enough time to select from alternatives [19, 22]. This algorithm could be developed thoroughly long before a possible accident occurs. Yet, describing ethical problems by means of the trolley problem is not without criticism [22]. It is mainly criticized because a crash similar to the trolley-problem scenario is (a) very rare and thus irrelevant compared to the safety benefits gained by using automated cars [24] and (b) the trolley problem gets too much attention [25]. However, even if the situations similar to the trolley problem are very rare, “their emotional saliency is likely to give them broad public exposure and disproportional weight in individual and public decisions about” [26, p. 3] automated vehicles. If one now assumes that an ethical programming for the event of a crash is necessary, the question therefore arises who should set the ethical rules for the car behaviour. Should there be a mandatory ethics setting (MES) for all—it remains to be clarified which institution should be responsible for this (e.g. government, car manufacturers, ethics committees)—or should the car user have the right to choose a personal ethics setting (PES). Advocates of a PES insist that citizens should have the right to decide on ethical issues that concern them personally [27]. Advocates

30

C. Kraetsch et al.

of an MES reject the idea of PES, especially based on the following two arguments: Morally problematic settings could be chosen and an MES is best for society as a whole, since allowing occupants of an autonomous car to choose between different ethical settings would lead to socially unwanted outcomes [28]. Obviously, the way automated vehicles are ethically programmed and whether there should be an option for personal ethics settings (PES) will also determine their societal acceptance. And whether there should be the possibility for PES or not has an impact for the design of the HMI. Insights from scientific literature on legal aspects With regard to legal issues and as several acceptance studies indicate, there are two in particular, which probably attract the most public attention or concerns: liability/responsibility — who is liable in which case for a crash — and privacy/data protection — who has access to the data collected by the automated car. How these aspects are regulated by law will certainly have a significant influence on the acceptance of automated cars. However, many aspects of automated driving seem to be unclear until now, especially with regard to liability and responsibility [29, 30]. Who is responsible in the case of an accident: The manufacturer, the designer, the software developer, the owner of the car, the driver? Another example of legal ambiguity in this context is the question of when the driver, or more precisely the person behind the wheel, must take control. In Germany, for example, the Road Traffic Act stipulates that a car occupant must take control if she/he recognises, “or has to recognise on the basis of obvious circumstances, that the prerequisites are no longer available for the intended use of the highly or fully automated driving functions” [31, own translation]). But what are “obvious circumstances”? In addition to questions about liability, there are also questions about data privacy protection: What happens with the data collected and stored by automated vehicles? Who has access to the data? The possibilities for monitoring the driving paths of automated cars already exist. In addition, the internal human-machine interaction devices also collect a considerable amount of data, e.g. on the behaviour of the ‘driver’ and the vehicle occupants), too. The programming of automated cars must comply with data protection regulations, in the European Union with the General Data Protection Regulation (GDPR). The question is whether the EU’s data protection principles cover crucial aspects of autonomous cars. Conclusion Societal acceptance is influenced by ethical, social and legal aspects. The following Table 1 summarizes the topics that must be clarified for successful societal acceptance regarding automated cars and which can also influence HMI design. With regard to ethical aspects, reference can also be made to the recommendations of the ethics committee for automated and connected driving set up in Germany [20, 21].

Bridging Gaps for the Adoption of Automated Vehicles …

31

Table 1 Summary of issues and requirements to achieve acceptance General issues regarding social acceptance Regulation of liability in the event of an accident Adherence to data protection/privacy: Compliance with GDPR Transparency on ethical programming/No discrimination against certain groups of people (e.g. older people) Ensuring road safety benefit Ensuring a positive environmental impact General requirements for HMI design

Ensuring reliability Easy operation/comprehensibility of the displays Keeping the driver in the loop Comprehensible communication with other road users

3 Proposed Methodology to Develop and Test Innovative HMIs HMI in automated driving The interaction between humans and automated driving systems must be described using relevant constructs and definitions. A human-machine interface (HMI) can be viewed as a foundation for the machine’s functional interaction with the human [32]. There are usually two main components of an HMI, namely: (1) an input, and (2) an output device. Through the input device the user may communicate information and commands to the machine, while the output device allows the system to communicate information to the user. The output device typically uses visual, acoustical or haptic representations of information, or combinations of these, to communicate [33]. In driving, the driver can use e.g. pedals, a steering wheel, and various indicator switches, knobs, buttons and touch screens. Furthermore, interaction strategies incorporating speech, gesture or gaze control could be integrated as a means of communication between the driver and the car. The use of interaction means is related to the task the driver faces, a task heavily linked to decision making on (1) strategical, (2) tactical, and (3) operational levels. The strategical level of driving includes a feedforward loop with aspects of planning, evaluation of risks, setting goals and making route choices. At the tactical level, the driver monitors the traffic and decides on manoeuvres to be executed. At the operational level, the driver maintains continuous control and adapts the execution based on information in the environment. Driving as we know it to date, and as we imagine the future of driving, includes various support systems assisting the driver in performing the driving task. Advanced Driver Assistance Systems (ADAS) are technical support systems providing the driver with information, but also relieving the driver through automation of driving related tasks. [34]. The implementation of automated driving systems changes the

32

C. Kraetsch et al.

task of the driver fundamentally, posing new challenges for HMI design as the division of tasks between humans and machines changes. Although the driver is relieved of tasks, new tasks arise. One of the major new tasks results from the responsibility for monitoring the automation; this is a task that humans are ill-equipped to perform [34]. A result of increasing automation in cars is that drivers can shift their attention to non-driving related tasks to a larger extent while in the driver’s seat. Such tasks could for instance be social activities such as texting, or entertaining activities such as browsing the web, or even consuming literature or film. A plausible requirement for HMI arising from these new activities is to take into consideration that the driver is occupied with—in relation to the main driving task—competing activities. The focus is thus not solely on the new monitoring task, but on tasks that could be viewed as distracting, leading to a failure in monitoring the traffic and automation in an adequate manner [35]. Therefore, driver distraction remains a major challenge that must be addressed in the development of HMI concepts. This could be achieved by measuring distraction [36] and leading the driver back to the driving task when needed. The distracting task is likely related to the acceptance of automated driving: why would a customer buy or use an automated car, if it did not allow one to use the time spent traveling for other activities than driving related tasks? Key theoretical constructs related to automated driving HMIs A construct that subsumes the driver’s ability to capture relevant stimuli is situation awareness (SA). SA has been defined as “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future” (p. 97) [37]. This suggests that the driver must have awareness of what is happening around them to be able to pursue the driving task appropriately. Automated driving systems can affect the drivers’ involvement in the driving task and the situation regarding perception, monitoring, and mental effort. It is thus likely that it influences the drivers’ SA, i.e. increasing or decreasing driver SA as a result of their innate design. A subordinate dimension of SA is mode awareness. Mode awareness requires that the driver knows enough about the system’s state and behaviour to achieve an error-minimized interaction [38]. For example, the driver knows which driving mode is active and how tasks and responsibilities are currently distributed between the human user and automation. Another related construct explaining how drivers predict and understand the behaviour of the automated driving system is the mental model conceptualization [39]. To guide their interaction with a system, users form an internal mental representation of the system at hand. This representation is based on individual experiences (including experiences of related technology) and knowledge of the system. This implies that drivers have maintained their understanding of the system’s handling, behaviour, properties, states, and boundaries in a mental picture of the system, guiding them through their interaction. In order to operate the automated driving system in a safe and appropriate manner a “correct” mental model is required [40]. However, some issues might arise for the driver, since some modes of the system are invisible and therefore not interpretable by the driver. Also, it might be unclear for the

Bridging Gaps for the Adoption of Automated Vehicles …

33

driver which information the system has received through its sensors and how that in turn will affect the system’s behaviour. System transparency is a key to address this issue. A transparent system eases the formation of the mental model for the driver. Without transparency the driver’s mental model might become skewed, leading to undesired use of the system. In research on artificial intelligence (which applies also to automated vehicles), the term explainability is now used increasingly to describe complex systems that help the user generate a valid mental model of what the system does. Explainability is assumed to be a relevant factor for the user’s trust. With transparency the driver finds it easier “to track and to anticipate the behaviour of automated systems”, which corresponds to the mode awareness definition (p. 7) [41]. From this, the following guidelines for HMI design can be derived, namely that: (1) the current state of the system should always be clear, (2) it should be clear who is in control at the moment, and (3) what will be the allocation of the driving task in a close future. Consequently, explainability or system transparency are required for good mental models, which in turn are requirements of mode awareness. In addition, the construct of trust in the system has a major role for the interaction. Trust has been defined as “the attitude that an agent will help achieve an individual’s goals in a situation characterized by uncertainty and vulnerability” (p. 51) [42]. Without trust in the system the user is likely to reject the use of it [42]. However, there are some issues related to trust, e.g. inappropriate reliance on the system such as overreliance [34] can lead to misuse. This phenomenon occurs when a driver inappropriately trusts the automation, consequently, violates critical assumptions of the system at hand and through that uses the system in an unintended way. Usability assessment The usability concept “applies to all aspects of a system with which a human might interact” [43]. Therefore, in order to assess the interaction between drivers and automated driving systems it is a key concept. In ISO 9241-11:2018 (3.1.7) usability has been defined as the “extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use”. Effectiveness can be described as “accuracy and completeness with which users achieve specified goals” [ISO 9241-11:2018, 3.1.12], and efficiency as “resources expended in relation to the accuracy and completeness with which users achieve goals” [ISO 9241-11:2018, 3.1.13], while satisfaction is described as “freedom from discomfort and positive attitudes towards” [ISO 924111:2018, 3.1.14]. To assess the subjective and objective dimensions of usability and thus generate a comprehensive measurement, subjective ratings as well as objective behavioural data concerning task performance should be collected. Behavioural data, for immature technology not yet released on the market, can be obtained by observation of user performance in the interaction with prototypes and demonstrators. A key resource to try prototypes of automated driving systems is to implement them in user testing employing human-in-the-loop driving simulators.

34

C. Kraetsch et al.

Driver monitoring With automated driving, monitoring the state of the driver becomes important for two reasons: secondary tasks competing with the driving task are expected to increase, and compared to non-automated cars, a scenario becomes possible in which the vehicle keeps moving while the driver is sleeping or unconscious. The driver states to be monitored include e.g. intoxication, fatigue, and attention. Attentional states must be continuously monitored as they may change within a short time span. Traditionally driver monitoring systems have been based on contextual information derived from vehicle systems. Examples are corrections using the steering wheel, speed variations, or based on the interaction with on-board systems [44]. Today, as image capturing devices have decreased in size and cost, in combination with advances in processing capabilities in embedded solutions [45], such technologies are becoming popular to assess driver states, as has been identified by car manufacturers as a key innovation for cars [46]. Cameras and computer vision systems allow for, among others, face and eye detection [47], head pose analysis [48], and eye gaze analysis [49]. By monitoring and assessing this information, drivers’ attentional levels can be classified, which is, for example, crucial information in takeover situations where the control between the driver and the automated system is exchanged. The system can check the driver’s state before asking him or her to take control, or the system could take control and bring the vehicle to a safe halt in case the driver is incapacitated or not responding. Requirements and objectives for HMI and driver monitoring On a general level, the requirements and objectives for an HMI and driving monitoring concept under development within the BRAVE project are operationalized and coupled to a methodological approach so that they can be achieved. For a successful implementation of the HMI and driver monitoring, ethical, legal, and social implications (ELSI) are of importance. The theoretical background on development of HMI concepts together with ELSI are viewed as baseline requirements for the development of HMI concepts. See Table 2 for an overview of the requirements. In addition to the ELSI requirements, the following objectives for the HMI concept could be derived from the theoretical constructs (Table 3). Iterative User Centric HMI Development The development of interfaces and products is recommended to be carried out in an iterative process [43], because user-friendly interfaces need to be developed with feedback from users. In an iterative process, users can test and evaluate a first version of a concept. Problems and misconceptions that are identified at an early stage can be further corrected in the product development process. Input from users, feedback that users give or test results, can be used to drive the design process and improve the user interface, until a next version for users to test is ready. The process can be repeated as many times as needed until a final product is ready to meet defined success criteria. Involving users at an early stage and adjusting concepts in the beginning of the process is more cost efficient compared to making changes later in the process.

Bridging Gaps for the Adoption of Automated Vehicles …

35

Table 2 Overview of ELSI requirements Relevant requirements for BRAVE system Acceptance ✓ Should gain users’ and the public’s trust (e.g. through override possibilities) ✓ Enable users to spend their time according to their wishes ✓ Reassure users and the public regarding cyber security and data privacy ✓ Be priced to meet the public willingness to pay Road safety ✓ Signals from the system must allow the driver to respond in time ✓ Feedback from the system should be transparent ✓ Feedback should support the driver to stay in the loop ✓ Drivers should be reminded about safety related tasks ✓ Interaction principles should be adapted to the drivers’ current states, their characteristics (e.g. age, experience level), their engagement in secondary tasks and how that engagement affects response times Ethical ✓ Decision-making by the system should follow ethics experts’ advice ✓ Consider users’ perspectives to reflect their ethical values and preferences Legal ✓ Comply with respective national and European regulations on road safety ✓ Comply with respective national and European regulations concerning data gathering and data protection (i.e. the GDPR) Social and economical ✓ Consider the amount of people that are included or excluded ✓ Consider the effect on the labour market ✓ Consider how it affects congestion ✓ Consider how it affects the environment Table 3 Overview of HMI requirements derived from the literature Relevant requirements for BRAVE system ✓ The system should be usable, intuitively understandable, and easy to learn ✓ The HMI should support the driver to maintain mode awareness (i.e. clear and transparent status of the system and the current responsibility of the driver and the automated driving system) ✓ The HMI should support the driver to maintain his or her situation awareness while driving with the system ✓ The HMI is usable (and accepted) by a large group of users ✓ The HMI should have minimal differences in acceptance due to gender, age, and previous experience of technology ✓ The HMI promotes a safe driving behaviour (not encourage the driver to show risky behaviours) ✓ The HMI should assure that it provides information and support depending on the driver’s state ✓ The HMI should assure that the driver always knows the intention of the automated driving system and thereby anticipates its behaviour ✓ In critical situations, the HMI should support a driver to relocate his or her attention to the relevant risk factors

36

C. Kraetsch et al.

Fig. 1 User centred design approach based on ISO 9241-210:2015, 2019

The iterative process is widely used, and the approach is included in the standard ISO-9241-210, Fig. 1 shows the process as in ISO 9241-210:2015, 2019 (with some modifications by the authors). The standardized approach begins by first deciding to use the human-centred design process and plan its execution. The second stage may be any of steps 2 to 5, depending on the starting situation. In any case, the context of use needs to be identified. This includes e.g. environmental factors of use (lighting, noise, posture) and factors related to the user (time pressure, sleepiness) as well as to other actors (other occupants of the vehicle, other road users). The context also refers to the user’s tasks and to system specifications about how and when the system will need to operate. The system specification can be used to improve systems already in use. In the third stage, user requirements are described. They result from the combination of the context and the user’s background, e.g. existing knowledge about similar systems. Then, in the next step, a product solution is designed based on the requirements identified and specified at earlier stages. In this stage, a prototype can be developed. In the fifth step, the final version is tested and evaluated based on the requirements stated in earlier stages. In case the product does not fulfil the requirements, any of the previous steps may be repeated in order to update the prototype and test it again. The updated version of the diagram in ISO 9241-210:2019 stresses that it is not important which step of the iterative process (after the decision to use human centred design) is necessarily the first step, as long as evaluation results are used as a basis to repeat the relevant steps with updated concepts, prototypes, etc. Within the BRAVE project, the aim was to design a highly usable HMI concept in an iterative process, giving feedback to users and helping them to behave as stated in the manoeuvre recommendations. The success criteria used to evaluate whether the HMI concept met requirements were based on objective and subjective measures throughout the process.

Bridging Gaps for the Adoption of Automated Vehicles …

37

Levels of distraction and driver states Particularly at medium levels of automation (e.g. SAE level 3), keeping track of the driver’s state is of critical importance. There is automated driving, but the driver is required as a fallback solution once conditions for automated driving are no longer met. If for a reason the driver becomes unavailable for taking over the driving task, safety measures need to be taken. At high levels of automation, drivers are less involved in the driving task and can engage in other activities. Drivers are expected to engage in tasks other than driving [35] and therefore systems need to be adopted with consideration to different levels of driver distraction or fatigue. In automated driving, levels of distractions can be used to reveal behavioural patterns which reflect drivers’ needs, and HMIs can be designed accordingly. Levels of distraction can be studied in driving simulators where the driver can be monitored while driving at different levels of automation. Results from previous studies show that when a driver is engaged in a secondary task before taking over control of the vehicle, the driver’s reaction times are prolonged. The BABS distraction scale developed at Fraunhofer IAO [50], illustrated in Table 4, classifies distraction at three levels (low, middle, and high) and matches the corresponding driver states or behaviours with appropriate warnings for each level. HMI concepts should match a driver’s current state, so that the warning is more salient at higher levels of distractions. Automated systems should include driver monitoring via cameras to enable the system to detect and recognize the driver’s current state. A driving simulator experiment conducted in the BRAVE project showed that a situation-and-driver-state-adaptive HMI increases user satisfaction (whilst it does not increase perceived usefulness or trust in the automation) [51]. Such technology would make it possible to develop more flexible HMI concepts that gives warnings according to drivers’ current states that may change during a journey. Table 4 Driver’s state and warnings at different levels of distraction Level

Driver’s state or behaviour

Low

Driver is looking at the road or at on-board Appropriate light in the dashboard screens, but the hands are not placed at the steering wheel

Warning alarm

Middle Driver gets involved in non-driving related Low level sound alarm or low seat tasks: Talking to passengers or by phone; vibration Reading, texting or similar; Using mirror for make-up or alike; Others Middle Driver is falling asleep or responses incorrect or too slow to transition alert

Activate high level alarm state, initiate minimum risk manoeuvre

High

High level sound alarm or intense seat vibration and or initiate safety manoeuvre

Driver is asleep or non-responsive to transition alert

38

C. Kraetsch et al.

Testing Use Cases and scenarios The BRAVE project focuses on acceptance of automation among all road users, including vulnerable road users (VRUs). Therefore, use cases involving VRUs were focused when developing and testing HMIs. Research on the interaction between automated vehicles and VRUs is needed, not only to increase acceptance among road users but also to increase safety. Autonomous Emergency Braking (AEB) was the system to be tested. AEB is a safety system that applies the brakes automatically when an object is detected and does not need any input from the driver. To include VRUs, use cases were designed to involve the interaction between the AEB system and VRUs. Namely, the scenario would involve the vehicle approaching a VRU (pedestrian and cyclist). Two use cases with a low and high-risk scenario which include the AEB and VRU interaction were designed to drive the development of an HMI concept and driving monitoring system. (1) Use Case 1: The ego vehicle travels in the direction of the VRU and detects the VRU. The VRU moves in line with the ego vehicle’s trajectory. There is no actual danger for either the ego vehicle or the VRU and the scenario is therefore considered low risk. (2) Use Case 2: The ego vehicle travels in the direction of the VRU and detects the VRU. The VRU is predicted to cross paths with the ego vehicle. There is a possible danger if the ego vehicle is unable to react in time and the scenario is therefore considered high risk. Evaluation The development of a trustable automation system, as aimed at in the BRAVE project, requires two main types of evaluation studies as part of the design process based on ISO 9241-210:2015: (1) Usability evaluations in order to ensure that the HMI concept as a whole, or its single components, can be used effectively, efficiently, and to the satisfaction of the user, given the relevant context (which includes the use cases, e.g. protecting VRU). (2) Evaluations of trust in the system, or general acceptance. Depending on which element is tested, these two types are more or less compatible and may require separate test designs. In practice, it may, for instance, not be possible to collect performance data and subjective measures of trust about different systems in the same experiment. Specifically, comparing different HMIs requires to experience all of them in the same condition—and measuring performance requires to give the user the power to alter the outcome of the situation, thus hampering comparability. The BRAVE project has collected measurements that are relevant for the evaluation of usable and trustable vehicle automation. The measurements used in BRAVE are shown in Table 5, for a more extensive list of measures, see Society of Automotive Engineers [52].

Bridging Gaps for the Adoption of Automated Vehicles …

39

Table 5 Overview of measures for evaluation Objective measures Time Exposed Time to Collision (TET)

The duration over which the time to collision (TTC) is below an undesired threshold [s]

Time Integrated Time to Collision (TIT)

The time interval over which the TTC is less than some undesired threshold weighted by how far below that threshold the TTC is at each moment [s]

Distance Headway

The longitudinal distance travelled between two vehicles measured from the same common feature of both vehicles [m]

Time Headway

The time interval between two vehicles measured from the same common feature of both vehicles [s]

Mean absolute lateral position (MLP)

A measure of lane keeping accuracy [m]

Standard Deviation of Lane position (SDLP)

A measure of the variability in lane positioning [m]

Steering reaction time

The time between a first conscious steering input over a certain threshold [s]

Driver’s behaviour

E.g. gaze at relevant objects measure with eye tracking and interventions such as hitting the brake pedal

Driver’s reaction time

Measure of reaction time [s]

Subjective measures SHAPE Automation Trust Index (SATI)

A human factors technique to measure trust in automated systems covering reliability, accuracy, comprehensibility, confidence, liking, and robustness (7-point Likert scale; never—always)

Van Der Laan Technology Acceptance Scale

9 items (pairs of adjectives along 5-point scale) questionnaire to measure usefulness and satisfaction of the tested systems

System Usability Scale

10 items questionnaire to measure usability

Nasa Raw-Task Load

6-item survey to evaluate perceived workload

Arnett Inventory of Sensation Seeking (AISS) 20-item questionnaire to assess sensation seeking (along the subscales Novelty and Intensity) Mini Driver Behaviour Questionnaire (DBQ)

12-item questionnaire to assess number of self-reported aberrant driving behaviour in traffic

Driver Skill Inventory

13-item questionnaire to assess self-reported driving performance

Generic willingness to use technology scale

Trust in technology, confidence, etc. (continued)

40

C. Kraetsch et al.

Table 5 (continued) Usability measures Comprehensibility of HMI elements

e.g. pictograms

Different HMI elements variations Ability to discriminate different stimuli

e.g. warning sounds

Task performance (appropriate behaviour)

e.g. avoiding a collision

4 Conclusion To support the successful introduction of automated vehicles, research and development efforts must be conducted in different scientific fields and at different levels. Both, internal and external HMIs of automated vehicles need to be carefully designed and tested in order to increase user experience (including usability) and trust. Use cases need to be carefully selected to address relevant traffic situations and to foster potential benefits, such as the protection of VRUs. Evaluation in the technical development needs to target both usability and trust. At societal level, survey data are needed to identify key aspects that influence acceptance, not only by drivers or occupants of vehicles but also by other participants in road traffic. Finally, it has been stressed that questions regarding ethics, legal responsibility, and data privacy need to be answered at a societal level to lay the foundation for a successful introduction of AVs. This requires a broader, open public discussion. Acknowledgements The research work has been funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement Nº 723021.

References 1. Johnsen, A., Strand, N., Andersson, J., Patten, C., Kraetsch, C., Takman, J.: Literature review on the acceptance and road safety, ethical, legal, social and economic implications of automated vehicles, Deliverable 2.1 from the EU-project BRAVE. Materialien aus dem Institut für empirische Soziologie an der Friedrich-Alexander-Universität Erlangen-Nürnberg, Nürnberg (2018) 2. Fraedrich, E., Lenz, B.: Societal and individual acceptance of autonomous driving. In: Maurer, M., Gerdes, J.C., Lenz, B., Winner, H. (eds.) Autonomous driving: technical, legal and social aspects, pp. 621–639. Springer Open, Berlin (2016) 3. Sütfeld, L.R., Gast, R., König, P., Pipa, G.: Using virtual reality to assess ethical decisions in road traffic scenarios: applicability of value-of-life-based models and influences of time pressure. Front. Behav. Neurosci. 11, 1–13 (2017) 4. Singh, S.: Critical reasons for crashes investigated in the National Motor Vehicle Crash Causation Survey. (Traffic Safety Facts Crash Stats. Report No. DOT HS 812 115). National Highway Traffic Safety Administration, Washington, DC (2015) 5. Becker, F., Axhausen, K.W.: Literature review on surveys investigating the acceptance of automated vehicles. Transportation 44, 1293–1306 (2017)

Bridging Gaps for the Adoption of Automated Vehicles …

41

6. Schrauth, B., Funk, W., Kraetsch, C.: From the perspective of other road users: acceptance of automated cars. In: Proceedings of 8th Transport Research Arena TRA 2020, April 27–30, 2020, Helsinki (in press) 7. Schoettle, B., Sivak, M.: A Survey of public opinion about autonomous and self-driving vehicles in the U.S., the U.K., and Australia (Report No. UMTRI-2014-21) (2014) 8. Kyriakidis, M., Happee, R., de Winter, J.C.: Public opinion on automated driving: results of an international questionnaire among 5000 respondents. Transp. Res. Part F: Traffic Psychol. Behav. 32, 127–140 (2015) 9. Cunningham, M.L., Regan, M.A., Horberry, T., Weeratunga, K., Dixit, V.: Public opinion about automated vehicles in Australia: results from a large-scale national survey. Transp. Res. Part A: Policy Practice 129, 1–18 (2019) 10. Shabanpour, R., Golshani, N., Shamshiripour, A., Mohammadian, A.: Eliciting preferences for adoption of fully automated vehicles using best-worst analysis. Transp. Res. Part C: Emerg. Technol. 93, 463–478 (2018) 11. Piao, J., McDonald, M., Hounsell, N., Graindorge, M., Graindorge, T., Malhene, N.: Public views towards implementation of automated vehicles in urban areas. Transp. Res. Procedia 14, 2168–2177 (2016) 12. Saleh, K., Hossny, M., Nahavandi, S.: Towards trusted autonomous vehicles from vulnerable road users perspective. In: 2017 Annual IEEE International Systems Conference (SysCon). Montreal, Canada (2017) 13. Alessandrini, A., Campagna, A., Delle Site, P., Filippi, F., Persia, L.: Automated vehicles and the rethinking of mobility and cities. Transp. Res. Procedia 5, 145–160 (2015) 14. Shi, L., Prevedouros, P.: Autonomous and connected cars: HCM estimates for freeways with various market penetration rates. Transp. Res. Procedia 15, 389–402 (2016) 15. Ohnemus, M., Perl, A.: Shared autonomous vehicles: catalyst of new mobility for the last mile. Built Environ. 42(4), 589–602 (2016) 16. Harper, C., Hendrickson, C., Mangones, S., Samaras, C.: Estimating potential increases in travel with autonomous vehicles for the non-driving, elderly and people with travel-restrictive medical conditions. Transp. Res. Part C: Emerg. Technol. 72, 1–9 (2016) 17. Wadud, Z., MacKenzie, D., Leiby, P.: Help or hindrance? The travel, energy and carbon impacts of highly automated vehicles. Transp. Res. Part A: Policy Pract. 86, 1–18 (2016) 18. Pugnetti, C., Schläpfer, R.: Customer preferences and implicit tradeoffs in accident scenarios for self-driving vehicle algorithms. J. Risk Financ. Manage. 11, 28 (2018) 19. Lin, P.: Why ethics matters for autonomous cars. In: Maurer, M., Gerdes, C., Lenz, B., Winner, H. (eds.), Autonomous Driving. Technical, Legal and Social Aspects, pp. 69–85. Springer, Berlin, Deutschland (2015) 20. Ethics Commission “Automated and connected driving”.: Report (Extract). Retrieved January 28, 2020 from http://www.bmvi.de/SharedDocs/EN/Documents/G/ethic-commission-report. pdf?__blob=publicationFile (2017) 21. Lütge, C.: The German ethics code for automated and connected driving. Philos. Technol. 30, 547–558 (2017) 22. Nyholm, S., Smids, J.: The ethics of accident-algorithms for self-driving cars: an applied trolley problem? Ethical Theory Moral Pract. 19(5), 1275–1289 (2016) 23. Foot, P.: The problem of abortion and the doctrine of the double effect. Oxford Rev. 5, 1–7 (1967) 24. Goodall, N.J.: Away from trolley problems and toward risk management. Appl. Artif. Intell. 30(8), 810–821 (2016) 25. Bonnefon, J.-F., Shariff, A., Rahwan, I.: The trolley, the bull bar, and why engineers should care about the ethics of autonomous cars. Proc. IEEE 107(3), 502–504 (2019) 26. Bonnefon, J.-F., Shariff, A., Rahwan, I.: Autonomous vehicles need experimental ethics: are we ready for utilitarian cars? Retrieved January 28, 2020 from https://pdfs.semanticscholar. org/13d4/56d4c53d7b03b90ba59845a8f61b23b9f6e8.pdf (2015) 27. Millar, J.: Technology as moral proxy. IEEE Technol. Soc. Mag. 47–55 (2015)

42

C. Kraetsch et al.

28. Gogoll, J., Müller, J.F.: Autonomous cars: in favor of a mandatory ethics setting. Sci. Eng. Ethics 23(3), 681–700 (2017) 29. Taeihagh, A., Lim, H.S.M.: Governing autonomous vehicles: emerging responses for safety, liability, privacy, cybersecurity, and industry risks. Transp. Rev. 39(1), 103–128 (2018) 30. Bienzeisler, J., Cousin, C., Deschamps, V., Eberle, U., Feldle, J., Gail, J., … Tango, F.: Deliverable D2.3// Legal aspects on automated driving. AdaptIVe. Retrieved March 30, 2020 from: https://www.adaptive-ip.eu/index.php/deliverables_papers.html (2017) 31. Straßenverkehrsgesetz, § 1b Abs. 2, S. 2 in der Fassung vom 5. Dezember 2019. Retrieved March 30, 2020 from: https://www.gesetzte-im-internt.de/stvg/STVG.pdf 32. Bullinger, H.-J.: Ergonomie: Produkt- und Arbeitsplatzgestaltung. B.G. Teubner, Stuttgart (1994) 33. Bruder, R., Didier, M.: Gestaltung von Mensch-Maschine-Schnittstellen. In: Winner, H., Hakuli, S., Wolf, G. (eds.) Handbuch Fahrerassistenzsysteme - Grundlagen, Komponenten und Systeme für aktive Sicherheit und Komfort, pp. 314–324. Vieweg+Teubner, Wiesbaden (2012) 34. Parasuraman, R., Riley, V.: Humans and automation: use, misuse, disuse, abuse. Hum. Factors: J. Hum. Factors Ergon. Soc. 39(2), 230–253 (1997) 35. Reilhac, P., Millet, N., Hottelart, K.: Thinking intuitive driving automation, Road Vehicle Automation 2, In: Meyer, G., Beiker, S. (eds.), Lecture Notes in Mobility. Springer (2015) 36. Ganzhorn, M., Diederichs, F., Widlroither, H.: A holistic approach for measuring driver distraction and inattention. In: ITS 3rd International Conference on Driver Distraction and Inattention, DDI. Gothenburg, Sweden (2013) 37. Endsley, M.: Design and evaluation for situation awareness enhancement. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 32(2), pp. 97–101. SAGE Publications (1988) 38. Kolbig, M., & Müller, S.: Mode awareness im Fahrkontext: Eine theoretische Betrachtung. In E. Brandenburg, L. Doria, A. Gross, T. Günzler, & H. Smieszek. Grundlagen und Anwendungen der Mensch-Maschine-Interaktion. 10. Berliner Werkstatt Mensch-Maschine-Systeme, Oct. 10–12, Berlin, Germany. Universitätsverlag der TU Berlin: Berlin, Germany (2013) 39. Norman, D.: Some observations on mental models. In: Gentner, D., Stevens, A. (eds.) mental Models, pp. 7–14. Lawrence Erlbaum, Hillsdale, NJ (1983) 40. Arndt, S.: Evaluierung der Akzeptanz von Fahrerassistenzsystemen - Prüfung eines Modells zur Vorhersage des Kaufverhaltens von Endkunden. Ph.D. thesis, Dresden University (2011) 41. Sarter, N.B., Woods, D.D.: How in the world did we ever get into that mode? Mode error and awareness in supervisory control. Hum. Factors 37(1), 5–19 (1995) 42. Lee, J.D., See, K.A.: Trust in automation: designing for appropriate reliance. J. Hum. Factors Ergon. Soc. 46(1), 50–80 (2004) 43. Nielsen, J.: Usability Engineering. Morgan, Fremont, California (1993) 44. EuroNCAP.: https://www.euroncap.com/en/ratings-rewards/euro-ncap-advanced-rewards/ 2011-mercedes-benz-attention-assist/ (2011) 45. NVIDIA Embedded Computing.: https://developer.nvidia.com/embedded/jetson-embeddedplatform 46. GM Pressroom.: http://media.gm.com/media/us/en/cadillac/news.detail.html/content/Pages/ news/us/en/2017/apr/0410-supercruise.html (2017) 47. Jayswal, A.S., Modi, R.V.: Face and eye detection techniques for driver drowsiness detection. Int. Res. J. Eng. Technol. (IRJET) 4(04), 2508 (2017) 48. Fridman, L., Lee, J., Reimer, B., Victor, T.: ‘Owl’and ‘Lizard’: patterns of head pose and eye pose in driver gaze classification. IET Comput. Vision 10(4), 308–314 (2016) 49. Vicente, F., Huang, Z., Xiong, X., De la Torre, F., Zhang, W., Levi, D.: Driver gaze tracking and eyes off the road detection system. IEEE Trans. Intell. Transp. Syst. 16(4), 2014–2027 (2015) 50. Ganzhorn, M., Diederichs, J. P., & Höfer, M.: Eine bewerterbasierte Ablenkungsskala (BABS) zur objektiven Beurteilung von Unaufmerksamkeit. VDI-Berichte, (2205) (2013)

Bridging Gaps for the Adoption of Automated Vehicles …

43

51. Linde, B., & Leuteritz, J.-P.: Acceptance of adaptive HMIs in SAE level 3 automated cars (in preparation) 52. Society of Automotive Engineers.: Operational definitions of driving performance measures and statistics (2015)

How Should TrustVehicles interACT? Mikko Tarkiainen, Merja Penttinen, Kimmo Kauvo, Ersun Sözen, Marc Wilbrink, and Marc Kaup

Abstract One crucial factor when it comes to creating trust in automated vehicles is the interaction of these vehicles, especially in mixed traffic. The interaction can be divided into two main areas: the internal interaction of the vehicle with onboard users, based on internal human-machine interfaces (iHMI) and the external interaction between the vehicle and external traffic participants, based on external human-machine interfaces (eHMI). In this chapter, two different approaches focusing on SAE level 3 automated driving, are presented. The first approach, coming from the TrustVehicle project, deals with internal interactions of automated heavy vehicles driven by professional drivers. It shows the design, development and implementation process of the internal HMI for a bus and a truck. It focuses on the hand-over cases between manual and automated driving mode in predefined scenarios in urban areas. The second approach comes from the interACT project, where besides the internal HMI, a strong focus lies on the interaction with external traffic participants (eHMI) in mixed traffic scenarios in urban areas. It also shows the overall design, development, and implementation process of an HMI in a passenger car. M. Tarkiainen (B) · M. Penttinen · K. Kauvo VTT Technical Research Centre of Finland, Espoo, Finland e-mail: [email protected] M. Penttinen e-mail: [email protected] K. Kauvo e-mail: [email protected] E. Sözen Ford Otosan, Gölcük, Turkey e-mail: [email protected] M. Wilbrink (B) Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR), Cologne, Germany e-mail: [email protected] M. Kaup HELLA GmbH & Co. KGaA, Lippstadt, Germany e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Watzenig and L. Schicker (eds.), Enhanced Trustworthiness and End User Acceptance of Conditionally Automated Vehicles in the Transition Period, Lecture Notes in Intelligent Transportation and Infrastructure, https://doi.org/10.1007/978-3-030-60861-3_3

45

46

M. Tarkiainen et al.

Keywords HMI · External interaction · Internal interaction

1 TrustVehicle Case: HMI Design for Automated Driving of a Heavy Vehicle One of the objectives of the TrustVehicle project was to develop and demonstrate an intuitive Human Machine Interface (HMI) for the safe management of the transition phases between automated driving and manual driving in heavy-duty vehicles (an electric bus and a truck). One visible and critical part of Automated Driving (AD) development especially in Level 3 Automated Driving (L3AD) is the Human Machine Interface. Information presented on the HMI is central to the driver’s behaviour and therefore important to road safety. In this section, we look at the HMI design for automated driving of heavy-duty vehicles in predefined low speed manoeuvre scenarios in urban areas. The drivers are professional drivers, which introduces some variations compared to the HMI design for regular consumers or drivers. However, in order to get AD functionalities accepted by the users, they need to trust the system. In general, trust-building HMI should be a core element of Automated Driving development in all levels and for all users. The next section describes the HMI design process in the TrustVehicle project and the outline of the electric bus and truck scenarios as well as high level description of the HMI. Description that is more detailed can be found from the TrustVehicle project deliverables and publications. The following sections provide details of the HMI design principles and discussion of the selected topics focusing on HMI design issues for heavy-duty vehicles and their professional drivers.

1.1 HMI Design and Development Process HMI design and development were done in the TrustVehicle project as step-by-step process with four consecutive phases. 1. First, potentially applicable available HMI guidelines were examined. The HMI guidelines were utilised later in the HMI design phase until the final HMI installed in the test vehicles. 2. In the second phase, requirements for the HMI were listed. The list included HMI requirements, which are common to both bus and truck scenarios. These requirements were derived from the scenarios selected by the electric bus and truck Original Equipment Manufacturers (OEM) Linkker and Ford Otosan. Additional automated driving related requirements were collected from the literature. The requirements were finalised with user (driver) tasks analysis in selected scenarios, and the detailed analysis of the automated driving areas.

How Should TrustVehicles interACT?

47

3. The functionality of the automated driving and the HMI were first designed as mock-up prototypes of the HMI screen layouts. This was done as an iterative process with a multidisciplinary team including experts of HMI design, software development and user centered design and usability, as well as representatives of bus and truck companies familiar with the professional drivers and the driver tasks in the planned scenarios. 4. In the third phase, for the electric bus case, a fully functional HMI prototype was implemented with Elektrobit EB Guide HMI design tool [1]. A few iteration rounds were conducted with the HMI prototype, concentrating mainly on the information provided to the driver in each situation. For the truck, the HMI prototype was developed in Google’s Android Studio [2] and tested in both Hardware-in-the-Loop (HiL) and prototype vehicle. 5. Then in the last phase, the HMI prototype evaluation was done with the potential users (the bus/truck drivers) before HMI implementation and integration to the test vehicles.

1.2 Description of the Heavy Vehicle Automated Driving Scenarios The HMI development was focusing on an electric bus and a truck scenario. The similarities in the Linkker and Ford Otosan scenarios enabled common general HMI concept development. The HMI design was led by VTT together with Linkker for the bus, and Ford Otosan followed the same HMI design principles for their truck HMI. Both scenarios were targeting low-speed automated driving in predefined limited urban areas. Electric bus automated driving scenario and HMI In the electric bus scenario, the bus drives autonomously and precisely to a charging station located at the bus stop. This scenario was selected by Linkker because it has proven that there are some difficulties in docking an electric city bus pantograph to the charging station in daily operations, especially in charging stations where precise manoeuvres are needed just before the bus stop. Automated driving is expected to provide time savings and the possibility to decrease drivers’ burden and stress. However, the traffic environment around the bus stop is very challenging for automated driving. When the bus is approaching the bus stop, there can be other road users such as vehicles, cyclists and other Vulnerable Road Users (VRU) close to the intended AD path. In addition, there can be people approaching, or even running, towards the bus stop from various directions. The driver HMI consists of two touch screens: one for the Automated Driving application and the second one for the 360° camera view. In addition, the HMI uses audio warnings and optionally stereo voice prompts. Both the AD application HMI screen and 360° surround camera screen are mounted on the right side of the steering wheel in the driver’s cabin such that they are easily visible and accessible for the

48

M. Tarkiainen et al.

Fig. 1 Linkker automated driving demonstration area (map data: Google Earth) [3]

driver. Automated Driving mode is activated with the steering wheel mounted paddle shifters. The driver can take back the vehicle control to manual mode at any time by turning the steering wheel or touching the brake pedal. In the electric bus scenario, the bus is approaching the bus stop on manual driving mode. When the bus enters the “Approaching area” (around 100 meters before the bus stop (see Fig. 1), the preconditions for automated driving are checked (e.g. current speed, lateral position, heading, safe distance to a potential leading vehicle) as well as possible operational design domain (ODD) limitations such as weather restrictions. When the bus is in the “AD starting area” and all preconditions for automated driving are met, the automated driving can be activated. The driver turns the AD ON by pulling the paddle shifters. While on automated driving mode (in “Automated driving area”), the HMI shows only the relevant information (e.g. the current location of the bus on the planned driving path and distance to the charging station). At the end of the automated drive when the charging point has been successfully reached, the HMI indicates to the driver the final position (and possible position error) of the bus and if the pantograph can be raised for charging. After the charging has been started, more detailed information about the automated driving is available to the driver including possible events (e.g. alerts, take-overs requests with reasons). Truck automated driving scenario and HMI The truck scenario consisted of a back-parking manoeuvre in automated driving mode of an articulated vehicle (truck trailer combination). Automated reverse parking of a semi-trailer heavy commercial vehicle, due to the kinematic and geometric constrains, especially considering the size and weight of the vehicle, is a notable challenge to solve. This scenario was selected for the potential of increasing both safety and efficiency of trailer reverse parking within critical areas, such as docking stations of warehouses and construction site entrances, which are located within city outskirts. Also in the truck scenario, the AD areas were defined similarly to the bus scenario and correspond to three phases, as shown in Fig. 2. In Fig. 3 another example, a construction sites parking areas was shown. Similarly, to the docking area scenario, the truck and trailer combination needs to accomplish reverse parking. The same three phases, approach, transition and automated parking are also applicable within this area.

How Should TrustVehicles interACT?

49

Fig. 2 Simplified representation of AD phases in truck scenario (map data: Google Earth) [3]

Fig. 3 An example parking area in Istanbul, which is dedicated for the construction trucks (map data: Google Earth)

The truck HMI that consists of a rugged tablet was positioned in front of the original vehicle infotainment screen. Since vehicle’s infotainment system is also a touch screen that is designed with driver ergonomics requirements, the position is validated in terms of e.g. reach, and usability. Activation of the AD is done in

50

M. Tarkiainen et al.

combination of HMI touch screen and pedals. Cancellation of the AD can be done if driver applies any pedal or steering wheel inputs. Within the approach boundaries (Approaching area), the HMI displays the available parking slots and reports the system status. After approach, the system switches to the Transition phase, where the vehicle is still under manual control of the driver. If all requirements for AD are met, the driver is allowed to press the Park button and the HMI shows “Release Brake Pedal” while the automated parking system is activated. If the transition to the automated driving mode is successfully completed, an automated parking view (a bird’s eye view) is shown to driver. After the vehicle reaches its final position in the automated parking phase, the HMI displays details of the automated back parking (such as lateral offset, manoeuvre completion time) to inform the driver about system performance. Displaying the system capabilities to the driver is expected to facilitate driver’s understanding of automated system.

1.3 HMI Design Principles and Guidelines for Professional Drivers The following sections provide discussion of the selected topics that were encountered during the HMI design for heavy-duty vehicles and professional drivers. HMI design guidelines and requirements for AD HMI requirements in automotive industry are well known for manual driving. Several guidelines describe the good practice in HMI design and focus on minimizing the driver distraction (e.g. the ESoP 2008 guideline [4] and the TRL HMI checklist [5]). Prior to TrustVehicle project’s start, there were no HMI guidelines available for automated driving. However, in August 2018, the National Highway Traffic Safety Administration (NHTSA) published a Human Factors Design Guidance for Level 2 and Level 3 Automated Driving Concepts [6]. It provides guidance to HMI developers with the HMI design and human factors assessment of the driver performance and behaviour under Level 2 and Level 3 AD with collection of research results, references to relevant standards and best practises. In addition, today there are UNECE regulations already available for Level 2 systems such as Lane keeping systems and Advanced Emergency Braking Systems (AEBS). These regulations describe the requirements for audible and visible signals and shapes expressing the information or warning conditionals in general. In the future, automated driving HMI regulations will most likely be based on the existing regulations. From the user perspective, it is also good to build on the experience, best practises and standards of the existing HMIs of the Advanced Driver Assistance Systems (ADAS). As users (drivers) become familiar with existing ADAS functions the HMI used in these systems provides a fertile basis for the HMI development for the first automated driving functions.

How Should TrustVehicles interACT?

51

HMI design and driver interaction The available HMI guidelines and requirements guides the HMI design, also in automated driving mode. For example, the information provided via HMI must be easily understandable, and there should be no possibility of false interpretation in any driving condition. The driver needs to be well aware of what is the system status and the current driving mode and what is expected from him/her. In addition, the driver attention should be guided to the events that are most critical at a specific point in time. The driver needs to be well aware of the system capabilities and its limitations. In addition, Massachusetts Institute of Technology (MIT) has introduced a Humancentered Autonomous Vehicle (HCAV) concept that lists the principles of shared autonomy [7]. The main idea of the HCAV concept is to remove the boundary between human and machine by getting humans and automated driving systems to collaborate effectively. Also in the TrustVehicle project, the main design principle for bus and truck HMI has been to keep the driver “in the loop” and actively monitor the vehicle surroundings. Special attention was paid on the timing and the visual demand of the information. Therefore, icons and pictograms widely used in automotive industry and traffic (e.g. traffic signs) were utilised in the graphical user interface. In addition to this, there was a driver monitoring system installed in the bus cockpit, which provides unobtrusive and continuous monitoring of driver focus of attention (see the driver monitoring and the HMI adaptation chapters). HMI design for professional drivers In both bus and truck scenarios, the driver of the vehicle will be an experienced professional driver who has been be trained to use the systems in the vehicle, including automated driving functionalities. The driver will use the vehicle systems daily in his/her work and is expected to learn how the system operates and its possible limitations relatively quickly. The role of a professional driver can be different compared to a non-professional driver, who will use automated features of a vehicle only occasionally. According to the definition of Level 3 automation, the driver does not need to monitor the road when vehicle is in automated mode. However, in these selected slow speed AD-scenarios the automated drive is very short (less than 1 min). The driver has been employed to drive the vehicle, and currently there are no other tasks, which driver would need to do during the short automated drive. Therefore, it was decided that when driving in automated mode the driver should monitor the automated driving functionality and the traffic environment all the time, which makes this solution more like Level 2 automation. HMI design should also take into account the professional usage of the system. Alerts that annoy the user can have undesirable safety effects, as users are likely to disable the system. This may happen even more with professional drivers who utilise the system every day all day long.

52

M. Tarkiainen et al.

1.4 TrustVehicle Approach: Adaptive HMI Based on Driver Monitoring and Environmental Perception The HMI and driver interaction adaptation in the bus and truck scenarios were based on driver monitoring and vehicle environment perception inputs. Driver monitoring Driver inattention is known to cause accidents in traffic. The camera-based driver monitoring can be used for both drowsiness and distraction (attention) detection both in manual and Level 2 and 3 AD modes. Driver monitoring can be used to detect if it is safe to give vehicle control back to the driver and if the driver is actively monitoring the automated or hands-free driving. Driver monitoring technologies are going to become a key component for the deployment of Level 2 and Level 3 automation. Fridman et al. [8] has proposed a measure of driver “functional vigilance” that incorporates both the ability of the driver to detect critical signals and the implicit ability of the driver to self-regulate when and where to switch from the role of manually performing the task (manual driving) to the role of supervising the automation (automated driving). In the paper, they also propose the expanded functional vigilance framework (for Level 2 automated driving), which includes driver monitoring and coaching for detecting and managing the functional vigilance of the driver. The feedback loop allows the system to supervise the supervisor (driver) and provide a warning if a functional vigilance is decreased, or a driver attentional failure is observed, or if other deviations from reasonable driving (or driving monitoring) activity is detected, see Fig. 4. This kind of vigilance supervision can be e.g. a

Fig. 4 Driver monitoring and coaching framework (modified from [8])

How Should TrustVehicles interACT?

53

camera-based driver monitoring system [8]. The TrustVehicle HMI concept uses the similar driver monitoring based HMI adaptation (and automated driving) adaptation. Environment perception The output of the vehicle environment perception sensors was used for Automated Driving as well as for HMI adaptation and driving context awareness. The sensor suite in the electric bus and in the truck test vehicles were different but the outputs (after sensor fusion) were similar: accurate position and heading of the vehicle (and the truck trailer), position of the parking/stop place, relative position, speed, and heading of the detected obstacles, etc.

1.5 HMI Adaptation and Interaction L3AD is a complex level of AD where the Automated Driving System (ADS) performs the driving tasks, but the driver still needs to be ready to take control over the vehicle within a certain timeframe when requested by the system. This issue is especially crucial when driving at high speeds. In the electric bus and truck scenarios, which both are focusing only on slow speed automated driving, this was not so critical issue. In TrustVehicle bus and truck scenarios events during automated driving were handled with various methods depending on the severity of the case. Both environment perception as well as driver monitoring were used to estimate the criticality of each situation. For example, if there is a moving object (e.g. a pedestrian) on the planned driving path, the ADS (via HMI) can inform the driver to pay attention to the object, ADS could slow down the driving speed or event stop the vehicle and wait for the path to become clear. In other cases, the ADS may require the driver to take over the system and provide a timely warning like e.g. a countdown to the required takeover. In more critical situation, the ADS can stop the vehicle on its own lane or path, and request takeover. However, in this case, the maximum longitudinal deceleration limits for buses with possibly standing passengers must be taken into account. During automated drive, the hands of the driver are off the steering wheel. Driver intention to take over in automated driving mode was detected with additional camera in the roof of bus cabin. If the driver makes a sudden move with both hands towards the steering wheel (in order to grab the steering wheel), it can be interpreted as an intention to take over the vehicle control even before the steering wheel or pedals are touched. In this case, the automated driving speed can be reduced and the preparations to change to manual driving mode can be initiated, but the actual mode change would be made after the actual steering or pushing the pedals occurs.

54

M. Tarkiainen et al.

1.6 The Influence of HMI Designs on Trust and User Acceptance Intel has conducted one of the first studies focusing on trusting the automated driving and understanding the human-to-machine interfaces with real autonomous cars on the road. Although the study was quite limited, the results related to HMI design and trust were interesting. “How it works vs. proof it works” and “Awareness vs. too much information” were one of the discovered HMI trust tension points. Understanding how the technology functions and its full capabilities was paramount to the users. However, once users gained confidence in the system, they felt some of the alerts and communications might become bothersome and intrusive. They also did not want to be distracted by too much information. [9]. Similarly, Waymo has also developed their in-vehicle HMI for autonomous car passengers, which would build trust [10]. Waymo has tested and developed new ways to show on the passenger HMI that their cars are aware of the surroundings. As noted before, it is very important that the users of the AD systems have realistic expectations of the system capabilities and limitations. In both bus and truck scenarios, the feedback about automated driving provided in multiple levels, but only after the drive. During driving the visual information on the HMI was kept minimum to avoid driver distraction, also in automated driving mode. While vehicle is stopped, the driver has most likely some time to view more visually rich information. This information may include driver monitoring alerts (e.g. detected distractions), detected obstacles, and system take-over requests as well as the uncertainties and possible limitations of the environment perception during the automated drive. Even the system imperfections can be reported to the driver with textual and visual representations. For example, if there was a sudden take-over request during automated driving and the reason for that was unclear to the driver, the driver can check the details afterwards. Professional drivers could be also used to provide feedback about the recorded events to the automated driving system developers. For example, the driver could validate possible false-positive detections and provide real world data about what actually happened. Furthermore, this kind of open communication about the AD performance and limitations could build the trust and understanding of the AD [3]. The HMI alone cannot build the trust to automated driving if the ADS, which does the actual driving, does something other than what users are used to (manual driving). Users must understand behaviour of the AD before they can trust it. All automated vehicles should also behave logically in similar ways when safety is concerned. Therefore, IEEE has started a new working group (P2846), which aims to define a standard formal model for safety considerations in automated vehicle decision-making [11]. The purpose of this upcoming standard is to create a common definition of what it means for an automated vehicle to drive safely and at the same time balancing safety and practicability. This standardised approach may have a significant role in the future for user acceptance (and trust) for automated driving. HMI will have an important supporting role in building the trust.

How Should TrustVehicles interACT?

55

2 interACT Case: Interaction Strategies and HMI Design for AV in Urban Mixed Traffic The main objective of the interACT project was to enable the safe integration of Automated Vehicles (AV) into mixed traffic environments by designing, implementing and evaluating solutions for safe, cooperative and expectation-conforming interaction of the AV with both its on-board user and other human road users (HRU) (Fig. 5). The project focused on urban use cases [12] and observed how today’s humanhuman interactions work in specific traffic scenarios like intersections and parking spaces [13]. Establishing an efficient and safe AV-traffic participant’s interaction, it seems likely that central elements of the existing human-human interaction need to be replaced by technical means [14]. interACT deals mainly with the design and selection of concrete internal (iHMI) and external human-machine interfaces (eHMI) and the selection of the appropriate technology to transmit chosen messages in an adequate way [15]. In this section, the overall interACT HMI design process is described. One exemplary use case is produced to describe the development in an illustrative way. Finally,

Fig. 5 Challenge of the interACT project—from human-human interaction toward human–machine interaction with AVs in mixed traffic environments [12]

56

M. Tarkiainen et al.

the technical implementation of HMI solutions to transfer information regarding the vehicle’s intention to AV’s on-board user (via iHMI) and other human traffic participants (via eHMI) is explained.

2.1 HMI Design and Development Process To achieve the project goals, interACT follows a well-explored, generic HMI design process as a guideline to get solutions for the safe and efficient interaction between AVs, their on-board user and other human road users. Associated work was done in an iterative, user-centred design process to allow for improvements of the chosen design based on user feedback during the whole process. Figure 6 shows a this process followed within interACT [14, 15]. The design process itself was split into three main steps: (1) As a basis of the design process all interACT partners defined the project specific use cases. Derived from the use cases, relevant messages were identified and general interaction strategies were developed. (2) Next step was the design of HMI signals to transfer these messages in an appropriate way. (3) Third, the appropriate technologies to cover the eHMI requirements were selected. After the completion of the design process, the HMI technologies and the final interaction

Fig. 6 Working process on interACT HMI design [14]

How Should TrustVehicles interACT?

57

design were developed and realised by hardware prototypes. This working process sets the basis for realising the interaction strategies in the demonstrator vehicles.

2.2 HMI Messages and Signals Based on Exemplary Use Case 2.2.1

Selected Use Cases

In the near future, AVs will be deployed in mixed traffic. One key challenge is the safe and efficient interaction with other traffic participants (TPs). interACT contributes to this with specific interaction strategies in various traffic situations. However, as the traffic environment consists of a huge variety of traffic situations, it is essential to reduce this complexity in a first step. Regarding the use cases definition, interACT started with a project wide common approach to identify relevant use cases and scenarios including all industrial and academic consortium members [12]. Use cases and scenarios have been selected in a step-wise process of intensive discussions within the consortium. Starting with some open brain-storming discussions, the use cases were aggregated and rated by the partners against several criteria (such as relevance for safety, need for interaction behaviour, etc.) to agree on the most relevant ones. The consortium defined four “must-have” use cases which are of highest relevance for society. These use cases were defined to be covered by research and technical developments in all technical Work packages (WPs) in the project. Further, the “must-have” use cases were evaluated in simulations and the interACT demonstrator vehicles. The project consortium decided on the following “must-have” use cases: React to crossing non-motorised TPs at crossings without traffic lights; react to an ambiguous situation at an unsignalised intersection; react to non-motorised TP at a parking space; react to manually driven vehicles at a parking space. The ambiguity of the use case was a key element for interACT. The project focuses on use cases which imply an uncertainty for the involved traffic participants. In this use cases the flow of events is not fixed by regulation or environmental factors and traffic participants need to interact whit each other to solve the situation. A detailed description and documentation of the use cases can be found in [12]. Each use case started with simple scenarios to define preliminary interaction strategies in an environment with reduced complexity. Interaction strategies were extended to more complex, multi-agent scenarios during project lifetime.

2.2.2

Message Catalogue

Derived from the agreed-on use cases the interACT consortium started the process of collecting and selecting relevant messages for the interaction with other TPs [3].

58

M. Tarkiainen et al.

A Starting point was the development of a list of potential messages an AV can send. These messages were classified into different categories of information: vehicle driving mode, next vehicle manoeuvres, environmental perception, cooperation capability and additional lower priority messages. In total, 35 potential messages were rated regarding multiple criteria: Does the message increase trust in AVs? Does the message increase acceptance of AVs? Does the message lead to an optimisation of traffic flow? Does the message help to increase safety for other traffic participants? Is the message required by law? The rating was done on an eleven-point scale (from 0 to 10) beginning with not important (0) and most important (10). In a third stage, two design workshops were conducted to discuss the ratings and decide which messages should be included into a catalogue of potential messages sent by the AV. The consortium decided to exclude all messages with explicit commandbased messages such as “go first” due to liability and legal issues and potentially negative effects such commands could have on traffic safety (e.g. inattentive behaviour of pedestrians and as result, not perceiving other traffic participants). As a result, the consortium agreed on the messages used within the interACT project (Table 1). These messages were addressed with two general interaction strategies: perception-signalling as well as intention-signalling messages. These messages are then used for designing the interaction strategies of the AV for the selected scenarios and were described in detail in Sect. 2.2.3. Table 1 Selected interaction messages in interACT for preliminary interaction strategies [3]

Vehicle driving mode AV drives in automated mode Temporal indication (e.g. searching for parking slot) Next manoeuvre AV will turn AV turns AV will start moving AV starts moving Environmental perception AV has detected (one or more) other/specific TP Cooperation capability AV gives way Lower priority AV says “thank you” AV indicates “irritation” AV has technical problems

How Should TrustVehicles interACT?

2.2.3

59

General Interaction Strategies (Intention-Signalling, Perception-Signalling)

Taking the message catalogue as a basis, interACT developed interaction strategies for the safe and efficient interaction of AV with other road users and the on-board user. Interaction strategies are seen as a combination of implicit and explicit communication of the AV. However, for the classification of the interaction strategies only the differences in the explicit communication of the AV were considered [14]. The interACT partners developed the general interaction strategies based on the existing research results and technical considerations. Two general interaction strategies were defined. Perception-signalling design: This design variant is characterized by giving explicit information to other traffic participants that they were perceived by the AV. The main goal is to substitute or even to optimise information that is normally exchanged by interpreting eye contact or head rotation in human-human communication. For the internal communication to the on-board user, the same information regarding the perception of the environment is presented. Intentionsignalling design: The second design provides explicit information regarding the current vehicle manoeuvres. Further it communicates its future behaviour and/or the cooperation capability of the AV. The same design was used for the internal communication to the on-board user. While the perception-signalling design should work with directly addressed messages (only for certain traffic participants in traffic environment), the intentionsignalling design can work with addressed as well as non-addressed (for all traffic participants in traffic environment) messages. For documentation purposes interACT used sequence diagrams to visualize the flow of messages in a timely based manner and the information that is exchanges between involved agents and technical components (Fig. 7). All relevant entities in a certain scenario are presented on the left side of the sequence diagram. The time elapse is shown from left to right and the flow of information is illustrated by arrows from the source of information to the receiver of the information. Over the course of interACT these strategies were tested in multiple user studies from different project partners to identify the best interaction strategy in different urban scenarios. A detailed description of the conducted studies can be found in interACT-Deliverable 4.2 (D4.2) [15] and Schieben et al. [16]. Based on the outcomes of these studies the most appropriate strategies were selected and further refined for application in the interACT demonstrator vehicles.

2.3 eHMI Design, Implementation and Integration The development of HMI technologies started with the overall goal to transfer intention-signalling as well as perception-signalling messages in an appropriate way. In general, the visual channel was selected as the most promising communication channel, as the sender of visual messages is clearly identifiable by the light emitting

60

M. Tarkiainen et al.

Fig. 7 Example of a sequence diagram [1]

surface on the car body. Visual signals are also discussed in the specific legislative and policy fora such as United Nations Economic Commission for Europe (UNECE), non-governmental organisations like the Groupe de Travail Bruxelles (GTB) or the Society of Automotive Engineers (SAE). Furthermore, interACT wants to create continuity, because for decades, vehicles have been sending information about their status or their next manoeuvres by means of light signals. The signals do not necessarily have to be intuitive for themselves, but history shows that they are very simple, clear and learnable. Acoustic signals can play an additional role in critical situations to avoid or minimise the impact of collisions, for meeting accessibility criteria for visually impaired groups, or for further improving the salience of the sent signal [15]. Depending on interaction strategy and messages that should be transferred, the AV can communicate information that refer to the vehicle itself, as well as information that refers to certain TPs. Information that refers to the vehicle itself can be communicated to every TP. If information is addressed to one certain HRU, it can be an option to direct the visual information only to that addressed person. The interACT project decided to have a closer look on both possibilities. This is the main reason why two different kinds of eHMI are developed and investigated during project duration—a 360° LED Light Band and a so-called Directed Signal Lamp. In addition to overall and specific requirements from [17], eHMI development pursues the target to keep the impact on the demonstrator car bodies as small as possible. Thus, integrability plays a main role right from the start of design. Figure 8 shows targeted appearance of the selected eHMI elements exemplarily on a BMW i3s demonstrator vehicle.

How Should TrustVehicles interACT?

61

Fig. 8 eHMI_1 | 360° light band (left); eHMI_2 | directed signal lamp (right)

eHMI_1 | 360° Light Band is a direct light-emitting ring around the vehicle, which can be illuminated completely or segmentally. Core components of the Light Band concept are blue-green light emitting diodes (LEDs) with a dominant wavelength of ≥495 nm. This so far non-restricted blue-green or turquoise or cyan colour, was chosen in alignment with recommendations from SAE and GTB dealing with the overall questions on vehicle lighting. Figure 9 illustrates a sectional view of the Light Band as modification of series car body. An additional view (red square) illustrates the optical concept in detail for one exemplary LED. In general, LEDs are almost predestined for applications like this, because they are established as robust, efficient, monochromatic light sources in

Fig. 9 Mechanical and optical design of LED light band concept [6]

62

M. Tarkiainen et al.

automotive signal lamps. In the present concept, all LEDs as part of the Light Band have their main radiation direction to the ground. The light is reflected, redirected and scattered by a diffuse reflector, before it passes the outer lens. On the one hand a homogeneous overall appearance across the length of the Light Band can be realised using multiple LEDs. On the other hand any LED is switchable or dimmable for itself to realize segmented or even animated signal designs [18]. eHMI_2 | Directed Signal Lamp idea was born while dealing with the question about if and how addressed messages could be conveyed only to specific relevant HRUs. A conventional signal lamp can be used to signalise e.g. a certain vehicle state or a next manoeuvre. Those messages refer to the vehicle itself, while it is useful that the light signals are visible for any other TP. As soon as a light signal can only be seen by one traffic participant, to whom a message refers, a light signal could contain messages that refer to the related traffic participant. One option is a personalized detection information. Every detected traffic participant, regarded as relevant for the AV, sees a light signal that lets her/him know that s/he has been detected and that the vehicle is aware of her/him, while other, possibly not detected or not relevant traffic participants, would not see a light signal at all [18, 19]. This leads to avoidance of distraction, miscommunication and overstimulation for not relevant TPs [14]. The technical conception of the Directed Signal Lamp module can be obtained by combining a light source, an addressable imaging element and optics. The most promising optical concept for near term realization of a Directed Signal Lamp is an imaging lens system—Fig. 10. Four lenses ➃ image a display ➂ over an angle of 70°. By imaging 320 columns of a display, 320 separate light channels can be realized. An elliptical reflector ➁ with its two focal points reflects the emitted light of an LED light source ➀ to the display ➂. Figure 11 visualizes the mechanical implementation and integration of the Directed Signal Lamp from different perspectives.

Fig. 10 Optical design of directed signal lamp concept [18]

How Should TrustVehicles interACT?

63

Fig. 11 Mechanical design of integrated directed signal lamp [6]

A sectional view shows the main components known from the concept and already its mounting position behind the windshield. The LCD is illustrated as a “purple plane” in these graphics. In the end both eHMI solutions have been fully integrated into demonstrator vehicles—Fig. 12 [18]. The intention-signalling interaction design, developed in interACT, uses an animated Light Band to communicate the intention of the AV. The frequency of the pulsing Light Band communicates the intention to give way (slow pulsing) or that the AV will start moving or even starts moving (fast pulsing). The Directed Signal Lamp was not used to communicate the AV’s intention. For the perception-signalling design, specific segments on the Light Band are illuminated directly pointing to the position of the interacting traffic participant(s). With the use of the Directed Signal Lamp, a detection light is sent in the direction of the other traffic participant(s) that is only visible for the relevant interaction partner. A combination of the Light Band and

Fig. 12 interACT demonstrator vehicles BMW i3s and JEEP renegade equipped with eHMI [20]

64

M. Tarkiainen et al.

the Directed Signal Lamp uses the Light Band for all intention-signalling communication and the Directed Signal lamp for all perception-signalling communication [16].

2.4 IHMI Design, Implementation and Integration In parallel to the eHMI development, an interaction strategy for the AV internal HMI was created as well. These interaction strategies should be easy to learn and in line with the eHMI signal designs. Providing as much consistency as possible, interACT uses the same interaction approaches for eHMI and iHMI (intention- and perception-signalling signals). Therefore an additional display (automation display) in the demonstrator vehicle was used to show information regarding the behaviour of the AV to the on-board user. Consistent with the information on the eHMI the iHMI provides intention- and perception-signalling information via symbolic and textual elements. Consequently, the perception-signalling approach displays the perception of other traffic participants while the intention-signalling communication focuses on the next manoeuvre of the AV. The design for these interaction strategies are described in detail in D4.2 [15] and D4.3 [18]. The automation display was integrated in the demonstrator vehicle of project partner CRF (JEEP Renegade). The display is positioned on the right-hand side of the on-board user in the height of the navigation display (Fig. 13).

Fig. 13 Integration of automation display in the CRF demonstrator vehicle

How Should TrustVehicles interACT?

65

3 Conclusions Two different approaches regarding driver interactions from two projects are presented in this chapter. The first part discusses the heavy-duty vehicle automated driving HMI design process in TrustVehicle project and HMI aspects. The professional drivers of buses and trucks differ from the regular (passenger car) drivers and this should be taken into account when designing the first automated driving functionalities and HMI for heavy-duty vehicles. The role of the driver should be clear and the expertise of the professional drivers should be utilised. Driver monitoring is a must in L2 and L3AD and could be also used more for ADAS functions to verify driver’s attention (and vigilance). The end-users (in this case the professional drivers) should be taken into the HMI design process from the beginning. In TrustVehicle project, the professional drivers were involved in the HMI design via OEMs and evaluation of the HMI with the bus and truck drivers was done with a fully functional prototype. Overall feedback to the HMI prototypes was very positive from both bus and truck drivers and the main principles of the HMI design were accepted including simplified HMI while driving (and details provided after the automated drive), driver attention monitoring, driver supervising AD, and driver providing feedback to the AD system developers. The second part of his chapter uses an exemplary use case to illustrate how the interACT partners have moved from a need for additional communication of an AV with other road users and on-board users to interaction strategies in future mixed traffic. This step-wise work was done in an iterative, user-centred design process to allow for improvements of the chosen design based on user feedback during the whole process. Finally, an intention-based and a perception-based interaction design and the possible combination of both were developed and implemented for empirical studies of the project partners. In the course of this, two eHMI technologies—a 360° Light Band and a Directed Signal Lamp—were developed, implemented and integrated into demonstrator vehicles with the aim of testing different interaction and signal designs against each other on test tracks and in the real world. The iHMI focus was at the same time on keeping the on-board user informed about the AV’s next driving maneuvers and its perception of environment in order to increase the feeling of safety, trust and acceptance and at the same time on avoiding unnecessary overtuning of the AV signal by the on-board user (non-active driver). Acknowledgements The research work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 723324 and No 723395.

66

M. Tarkiainen et al.

References 1. Elektrobit: EB Guide: HMI development tool for graphical, voice, and touch UX. Available from: https://www.elektrobit.com/ebguide/ (2019) [cited 15.1.2020] 2. Google: Android Studio Preview | Android Developers. Available from: https://developer.and roid.com/studio/preview (2019) [cited 15.1.2020] 3. Tarkiainen, M., Penttinen, M., Sahimäki, S., Sözen, E., Kauvo, K.: Automated driving and HMI design for city bus and truck with professional drivers. TP1798. Paper presented at 13th ITS European Congress, Brainport Eindhoven, Netherlands, 2019 4. ESoP: On Safe and Efficient In-Vehicle Information and Communication Systems: Update of the European Statement of Principles on Human-Machine Interface. Commission of the European Communities (2008) 5. Stevens, A., Cynk, S., Beesley, R., Limited, T.R.L.: Revision of the Checklist for the Assessment of In-Vehicle Information Systems. IHS, Bracknell, UK (2011) 6. Campbell, J.L., Brown, J.L., Graving, J.S., Richard, C.M., Lichty, M.G., Bacon, L.P., … Sanquist, T.: Human Factors Design Guidance for Level 2 and Level 3 Automated Driving Concepts (Report No. DOT HS 812 555). National Highway Traffic Safety Administration, Washington, DC (2018, August) 7. Fridman, A.: Human-Centered Autonomous Vehicle Systems: Principles of Effective Shared Autonomy. CoRR, abs/1810.01835. Available from: https://arxiv.org/abs/1810.01835 (2018) [cited 7.1.2020] 8. Fridman, L., Brown, D.E., Kindelsberger, J., Angell, L., Mehler, B., Reimer, B.: Human Side of Tesla Autopilot: Exploration of Functional Vigilance in Real-World HumanMachine Collaboration. Available from: https://hcai.mit.edu/human-side-of-tesla-autopilot/ (2019) [cited 15.1.2020] 9. Intel: When Will You Be Ready to Get in a Driverless Car? Intel Newsroom, Aug 24, 2017. Available from: https://newsroom.intel.com/editorials/when-will-you-be-ready-get-driverlesscar/ (2017) [cited 15.1.2020] 10. Google Design: Trusting Driverless Cars - How Waymo Designed an Experience that Reassures Riders Every Step of the Way. 03/13/2019. Available from: https://design.google/library/tru sting-driverless-cars/ (2019) [cited 15.1.2020] 11. IEEE: IEEE 2846 WG - Welcome to IEEE P2846! Available from: https://sagroups.ieee.org/ 2846/ (2019) [cited 15.1.2020] 12. Wilbrink, M., Schieben, A., Markowski, R., Weber, F., Gehb, T., Ruenz, J., Tango, F., Kaup, M., Willrodt, J.-H., Portouli, V., Merat, N., Madigan, R., Markkula, G., Romano, R., Fox, C., Althoff, M., Söntges, S., Dietrich, A.: interACT D1.1 - Definition of interACT Use Cases and Scenarios (2017) 13. Dietrich, A., Bengler, K., Portouli, E., Ruenz, J., Wu, J., Merat, N., Madigan, R., Lee, Y.M., Markkula, G., Giles, O., Fox, C., Camara, F.: interACT D2.1 - Preliminary Description of Psychological Models on Human-Human Interaction in Traffic (2018) 14. Wilbrink, M., Schieben, A., Kaup, M., Willrodt, J.-H., Weber, F., Lee, Y.M., Markkula, G., Romano, R., Merat, N.: interACT D4.1 - Preliminary Interaction Strategies for the interACT Automated Vehicles (2019) 15. Weber, F., Sorokin, L., Schmidt, E., Schieben, A., Wilbrink, M., Kettwich, C., Dodiya, J., Oehl, M., Kaup, M., Markkula, G., Lee, Y.M., Madigan, R., Markkula, G., Romano, R., Merat, N.: interACT D4.2 - Final Interaction Strategies for the interACT Automated Vehicles (2019) 16. Schieben, A., Wilbrink, M., Kettwich, C., Dodiya, J., Weber, F., Sorokin, L., Lee, Y.-M., Madigan, R., Markkula, G., Merat, N., Dietrich, A., Kaup, M.: Testing external HMI designs for automated vehicles – an overview on user study results from the EU project interACT. 9. Tagung Automatisiertes Fahren. Lehrstuhl für Fahrzeugtechnik mit TÜV SÜD Akademie, München, Germany (2019) 17. Drakoulis, R., Drainakis, G., Portouli, E., Tango, F., Kaup, M.: interACT D1.2 - Requirements, System Architecture and Interfaces for Software Modules (2017)

How Should TrustVehicles interACT?

67

18. Kaup, M., Willrodt, J.-H., Schieben, A., Wilbrink, M.: interACT D4.3 - Final Design and HMI Solutions for the Interaction of AVs With User On-Board and Other Traffic Participants Ready For Final Implementation (2019) 19. Dietrich, A., et al.: Projection-based external human machine interfaces – enabling interaction between automated vehicles and pedestrians. In: DSC 2018 Europe VR, Driving Simulation & Virtual Reality Conference & Exhibition, Antibes, France, pp. 43–50 (2018) 20. Drainakis, G., Bolovinou, A., Tango, F., Borrello, G., Markowski, R., Ruenz, J., Boehm, M., Pek, C., Kaup, M.: interACT D5.2 - Interaction Function Integration. Demonstrator Final Version (2019)

Reliable Sense-Plan-Act Approaches for TrustVehicles Ahu Ece Hartavi, Tarek Kabbani, Pouria Sarhadi, Abhishek Shah Alias Sangani, Johan Zaya, Stephanie Maroun, Mona Saleh, Srinath Shanmugam, Ashok Krishna, Jibin Ou, Ersun Sözen, Eren Aydemir, and Samia Ahiad

Abstract Autonomous Vehicles (AVs) have the potential to enhance the road safety, while reducing congestion, and GHG emissions. It is expected that vehicles with the different level of autonomy (Level 2+) will hit the road in the near future. This chapter discusses briefly the state of art of the three main building blocks of the automated driving (AD), to develop a reliable, safe, and trustable AV. In this framework, AV sensors, planning, and tracking controller techniques are reviewed. Moreover, the

A. E. Hartavi (B) · T. Kabbani · P. Sarhadi · A. Shah Alias Sangani University of Surrey, Guildford, UK e-mail: [email protected] T. Kabbani e-mail: [email protected] P. Sarhadi e-mail: [email protected] A. Shah Alias Sangani e-mail: [email protected] J. Zaya · S. Maroun · M. Saleh · S. Shanmugam · A. Krishna · J. Ou Volvo Cars, Gothenburg, Sweden e-mail: [email protected] S. Maroun e-mail: [email protected] E. Sözen · E. Aydemir Ford Otosan, Istanbul, Turkey e-mail: [email protected] E. Aydemir e-mail: [email protected] S. Ahiad Valeo Vision, Paris, Écouflant, France e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Watzenig and L. Schicker (eds.), Enhanced Trustworthiness and End User Acceptance of Conditionally Automated Vehicles in the Transition Period, Lecture Notes in Intelligent Transportation and Infrastructure, https://doi.org/10.1007/978-3-030-60861-3_4

69

70

A. E. Hartavi et al.

challenges, some promising technologies (e.g. self-cleaning sensors), technical solutions, and shortfalls that need to be alleviated are described considering different vehicle classes ranging from passenger up to heavy duty vehicle. Keywords Autonomous driving · Sensor · Trajectory planning · Trajectory controller

1 Introduction The idea of automated driving is not new. In fact, it dates back to 1920s, when technological developments in aviation and radio engineering prompted the world’s first autopilot, the airplane stabilizer, and then the first driverless car [1–3]. First remotecontrolled vehicle appeared in the United States in 1925, leading to the development of the modern cruise control in 1948, which transformed into cruise control only 20 years later [2, 4, 5]. The 2000s hit off with many leaps such as Nissan’s introduction of a lane departure warning system, Toyota’s pre-crash mitigation, and of course, Google’s Driverless Car’s ground-breaking appearance [4–6]. Since then, numerous promising trials took place in highway, and complex urban environment by many companies and research organizations. Although much progress has been made in a relatively little time, there are still barriers, in terms of safety, liability, usability, and acceptability, preventing a general and safe adoption of automated driving beyond technical issues [7–10]. The following chapters will discuss the 3 key building blocks of AVs, which are sensor technologies, planners, and controllers.

2 Reliable Sensor Solutions for Autonomous Vehicles Perception of the environment is the key to safe AD. Therefore, reliable and robust sensing capability is a prerequisite for an AD system to understand its environment, to locate and to move. In fast evolving AV market, various types of sensors (RADAR, LiDAR, camera, IMU, GNSS, RTK, etc.), along with extremely high precision maps (HD maps), gaining noticeable attraction. Each type of sensor has its own advantages and disadvantages. To precisely define the environment different number and types of sensors are used and fused to combine the advantages of each sensor. The number of sensors change based on the level of autonomy, and average number can rise from 24 (SAE Level 3) up to 32 (SAE Level 5). While using multiple sensors extends the sensing capability, it introduces challenges in terms of data fusion [11].

Reliable Sense-Plan-Act Approaches for TrustVehicles

71

2.1 High Definition (HD) Maps HD maps are regarded as sensors for AD systems. It provides lane level geo information to the vehicle to localize precisely, extract the drivable area, while providing a longer horizon to the path planning unit [12, 13]. Specifically, its special perception and prediction capabilities (e.g. inclusion of lane markings, traffic signs etc.) provides highly accurate representation of the road. This feature enables the vehicle know, where each road element is with a very high precision regardless of the weather conditions. Today, the challenge regarding the HD maps is mainly due to the need of updating them to reflect the changes on the road. To do that, sensor observation data [14] of multiple vehicles, driving at the same road, should be gathered to validate and update the HD maps, which is a very costly process. Moreover, several aspects of producing highly reliable HD maps and map distribution system need more clarification, specifically in terms of ASIL compliance [15].

2.2 Radio Detection and Ranging (RADAR) Radar is a type of sensor, which uses radio waves to determine the distance, angle, or velocity of objects. It is used widely in various applications, before transferred to automotive industry. Figure 1 illustrates basic operating principle of the RADAR. The sensor sends electromagnetic waves, where the energy reflected identifies an object. From the timing of the reflected wave, the position of an object can be determined. The frequency shift of the signal determines the velocity and identifies if it is a stationary or moving object [16]. It is used to detect road actors and measure their distance and speed in relation to the vehicle in real time. The Frequency Modulated Continuous Wave (FMCW) RADAR is the commonly used type due to the ability to send continuous waves, combines high resolution in range and depth, and detect more than one object using the beat signal at the same scan as well as its scalable architecture (Fig. 2) [17]. It functions by transmitting frequency modulated continuous wave that is generated

Fig. 1 Block diagram of an elementary form of a RADAR [18]

72

A. E. Hartavi et al.

Fig. 2 FMCW RADAR signal: a Block diagram. b Waveform of transmitted, received and beat signal [17, 18]

using the frequency synthesizer. The power amplifier is used to amplify to travel through a long distance, while the transmitter antenna is used to transmit the signal. Once the signal hits a target, then it is received by the receiving antenna. Then the mixer subtracts the incoming signal from the original signal to give the beat signal, which is then processed to generate detection [18], Fig. 2b. RADAR can execute standalone functionalities (e.g. Blind Spot Information, Cross-Traffic Alert, etc.), which are already exist in some of the modern vehicles. Moreover, its data can be fused with other sensors’ data to provide additional AV functionalities. Even though RADAR is a century long invention, still some technical challenges remain such as insensitive to environmental conditions (e.g. fog, rain etc.), sensitivity to object’s size, distance, reflection angle, and transmission power. Moreover, RADAR cannot detect the shape of the object, therefore, to meet these challenges, engineers are combining the inputs of different sensors, to provide required data.

2.3 Light Detection and Ranging (LiDAR) LiDAR is a relatively new sensor technology with respect to RADAR. It also uses reflection principle to identify the objects. However, while RADAR uses radio waves for detection, LiDAR uses high speed laser beams. LiDAR sends out rapid pulses of laser light, and then processes the amount of time each pulse takes to return. Accordingly, the distance and direction of the pulse hits are recorded as a point of data. By repetition of the process LiDAR renders the environment in the form of point clouds, where each point corresponds to a laser beam emitted and reflected from objects in the field of view of the sensor. Velodyne HDL-64E, as an example, can log as much as 2.2 million points per second. The 360° coverage LiDAR sensor generated point cloud gives a 3D representation of the environment without need for post-processing, however, it is required to identify and classify traffic objects and lane marks as well as drivable area (Fig. 3). The process of converting point cloud data to 3D image consists of several steps [19].

Reliable Sense-Plan-Act Approaches for TrustVehicles

73

Fig. 3 Velodyne high definition real-time 3D LiDAR point cloud data view [19]

The first step is the segmentation, where the ground plane and obstacles are separated within the data points. RANdom SAmple Consensus [20] is one of the well know techniques used for segmentation due to it’ faster convergence. The second step is organization of the data points into clusters, with respect to some similarity among data points. The third step is shape extraction, which is used to generate descriptors to extract shapes related to objects on the road, road markings and barriers, which could be used in tracking algorithms and other perception based functions. LiDAR is most commonly used in AVs to enhance sensing range and to support other sensor technologies as well as providing redundancy. Today, most LiDAR manufacturers that are close to market use pulsed lasers at a fixed wavelength. Downside of this approach is not being able to directly infer relative velocity of detection points from a single measurement. Hence, a coherent LiDAR technology, which is using FMCW similar to RADAR is currently entering to the market. Even though, LiDAR has no loss of performance in low light conditions, it can be affected by strong background illumination from the sun. Furthermore, it is not robust to harsh weather conditions (e.g. droplets, surface contamination). This can be partially addressed by adapting certain properties of the outgoing laser pulses. Other challenge regarding this that can be addressed is the risk of interference with other LiDARs using the same wavelength as well as possible interference from background solar illumination [21].

2.4 Cameras Cameras are passive sensors that absorb light to detect and recognize the objects. The basic camera concept is the pinhole type, wherein the light reflection from an object passes through a barrier on to a film. It then saves in an image format as shown in Fig. 4 [22]. The captured images can be used to recognize vehicle’s own position as well as to detect objects around. In recent AV research, camera images and artificial intelligence (AI) based algorithms are used together for object classification to describe the state of items of interest, such as type of object, vehicle direction and time to collision (TTC), etc.

74

A. E. Hartavi et al.

Fig. 4 Pinhole camera concept [23]

[23–25]. The fact that AI based algorithms use the pixels in order to detect the objects is one main fall, since it is extremely related to the imager sensor resolution of the camera. Also, cameras are advantageous to estimate objects in the lateral direction, but when it comes for the longitudinal direction has a worst estimation and this direction is very dependent on the camera calibration. One of the main challenges of cameras, is to have higher resolution imager sensor in a relatively small camera. Another challenge is integration of these camera systems into the vehicle. Advanced Driver Assistant Systems such as Adaptive Cruise Control (ACC), Forward Collision Warning (FCW) or Auto Emergency Braking (AEB), utilizes camera input together with other in sensors to actuate and prevent a potential accident. Today, different camera technologies are either commercially available or under development, some of which are; stereo, Time-of-Flight (ToF) (direct, indirect), and Eagle-eyed vision cameras [26]. Specifically, ToF is a very promising technology, which can be defined as a depth perception technology providing distance information by measuring the travel time of emitted light. In a recent study, the CamBoard pico flexx (Fig. 5) camera based on ToF technology for reliable face recognition for an AV is explored [27]. In the study not only the depth data but also the amplitude and noise is considered using Convolutional Neural Network (CNN) in the framework of the H2020 TrustVehicle project [28]. In a respective paper, the correlation between heart rate variability (HRV) and pulse rate variability (PRV) extraced from contact/remote photoplethysmography (PPG) on a ToF camera is explored for driver monitoring purpose [29]. Fig. 5 Camboard pico flexx [29]

Reliable Sense-Plan-Act Approaches for TrustVehicles

75

2.5 Global Navigation Satellite System (GNSS) Originally satellite navigation system has been started for military use in the 1960 s. It has found tremendous commercial use in the last few decades due to the increased demand of location-based services, one of which is AVs. Today, modern HD maps and mapping techniques have been achieved the required accuracy [30–32] to localize AV on the map and perceive its position relative to the environment using other sensor inputs. However, for absolute positioning and time, the only source to offer of globally consistent, precise position and time for a vehicle is GNSS. Modern high precision GNSS (HP-GNSS) provides lane determination and act as a standard reference for autonomous systems. Moreover, one of the unique features of it is unaffected by climatic conditions, issues of range, field of view etc. Whilst GNSS can be seen as a complementary AV sensor for extreme weather conditions, on the other hand side it suffers from outages and limited in blocked sky view, especially under tunnels, bridges, indoor parking areas. Under these circumstances, vision and range sensors can be used as a complementary for the GNSS. Apart from that a market ready GNSS through the availability of an affordable and scalable receiver hardware. Today several mass market multi-frequency, multiconstellation GNSS chipsets are available [33–36]. Most of these are in various stages of developments for being able to fulfil the ISO 26262 function safety requirements [37].

2.6 Ultrasonic Sensors It is the far most inexpensive sensor type for the AVs. These sensors are predominantly used in the parking applications ranging from assisted parking up to fully autonomous parking. Ultrasonic sensors work according to the sonic altimeter principle with which, for example, bats orientate themselves. The sensors send out short ultrasonic impulses with a frequency of ~48 kHz by using the diaphragm, which are reflected by barriers. The diaphragm then relaxes for some duration, receives the reflected sound from the barriers and vibrates. These vibrations are outputted by the piezo-ceramic element as analog signal, it is then amplified and converted to a digital signal. The echo signals are registered by the sensors and are evaluated by a central control unit [38, 39]. The reliability of these sensors are in the close range environment (~6 m) makes them a standalone system a complementary solution for the AVs.

2.7 Future Research Trends in Sensing Camera, RADAR, LiDAR, GNSS, HD Maps, and ultrasonic sensors are not the only sensors for AVs, there are also other sources, such as Inertial Measurement

76

A. E. Hartavi et al.

Fig. 6 Valeo supervision and cleaning functions

Unit, Vehicle-to-Vehicle, and Vehicle-to-Infrastructure communication. Bringing all of these data sources and the growing complexity of the designs opens the door for new challenges (e.g., computational burden) as well as additional risks that need to be undertaken, such as safety of the intended functionality and cybersecurity. Apart from that in recent research studies was shown that health monitoring of the sensors is also quite crucial for improving the sensor reliability. It has become a key component that enables the detection of when and which part of the system reaches its limits of operation or malfunctions. While trying to identify that not only internal factors (e.g., hardware, software faults) but also external factors such as environmental conditions (e.g., heavy rain and soiling) needs to be considered. Unfortunately, the current sensor technology is inadequate to operate reliably under different weather conditions and harsh environments since the dirt or salt blocks the required world state information (Fig. 6). Within the TrustVehicle project, a wide range of strategies to ensure the AD safety, keeping the reliability and availability of the sensor at a higher level is investigated both from the safety point of view as well as increase the trust and acceptability of the end-users. In addition to the above-mentioned challenges regarding reliable sensing, it is important to note that the vehicle type can also introduce additional challenges. Specifically, automation of the articulated vehicle is a quite challenging technology due to the complexity introduced by the vehicle itself (e.g. vehicle length, height, articulation). The challenges start from the selection and identification of the number of sensors required which is higher than passenger vehicles (PV). Even though a spinning LiDAR with 360° field of view (FOV) can be mount on a PV roof, due to the height of can not be mounted on the roof of the heavy commercial vehicle. Moreover, increased number of sensors bring greater complexity to the sensor fusion algorithms together with the need of wider bandwidth. Also wiring harness becomes increasingly challenging and complex not only due to longer cable lengths but also due to delays in the signals. Even further, in an articulated vehicle, truck sensor’s FOV can be obstructed via trailer or vice-versa. Therefore, during the sensor selection phase not only the road actors but also vehicle itself state need to be considered based on the hitch angle.The aforementioned issues together with the integration interface

Reliable Sense-Plan-Act Approaches for TrustVehicles

77

related issues are also studied in the TrustVehicle project for Ford F-Max Level 3 Autonomous Truck [40, 41].

3 Trajectory Planning Algorithms Generation of an obstacle free, smooth path/trajectory is one of the key technology enabler for AVs. It is usually divided into two sub-problems, Path and Trajectory Planning (TP). The former one usually considers the environment, in which the obstacles are static, while the latter mostly deals with dynamic unstructured environment. Even though, path and TP terms are sometimes used interchangeably, they are quite different [42, 43]. In the literature there are various TP techniques [44, 47]. In [48, 49] model based control technique is used to generate a reference trajectory. The studies consider either static environment or rather complex and has a big computational burden for on-line implementation. In [50], space search-based technique is used to generate a trajectory for a vehicle addressing only high-velocity maneuvers. Another study has proposed quintic polynomial approximation for an overtaking scenario, which is dependent on a priori calculated initial path [51]. Despite that some sampling and grid search-based planning methods account for systems with higher degrees, most of them are computationally heavy to implement in real-time due to their complex nature [52]. Some of which are Probabilistic Road Map (PRM), A* and Rapidly-exploring Random Tree (RRT) [53, 54]. The dynamic window (DW) is also one of the emerging TP technique originally developed for holonomic robots, which is then transferred to road vehicles [55]. In [56] it is applied to a vehicle considering the constraints due to vehicle kinematics, and operating design domain. The window serves in limiting the space of admissible velocities, that is fed to the vehicle, before the subsequent sub-interval. The technique has the local minima and equilibrium point limitations, which is overcome by a global approach technique that uses a navigation function proposed in [57]. Temporal logic (TL) is another TP technique, which is an extension of propositional logic and consists of set of rules and symbols that can capture temporal relations [58, 59]. It was originally introduced in [60]. It has been broadly used in motion planning as a specification language to express desired vehicle behavior. Due to the lack of decision making it was exploited to a method that synthesis a correct-by-construction controller. There are several solutions exists for the correctby-construction controller synthesis. In [61] Real Temporal Logic (RTL) is defined as specifications for the desired motion of a robot considering the controller error and the kinematic model. Then, the product is made between the specification’s automaton and one of the robot transition system resulting in a third automaton that accepts paths only respecting the specifications. In [62] nonlinear system is considered in the controller synthesis using a symbolic model. In [63] syntheses were made with sensor-based TL planning for an agent and a cooperating multi–agent system However, those approaches are either complex or have a high computational burden.

78

A. E. Hartavi et al.

They may also lead to a state explosion problem, while not automatically finding the receding horizon invariant for the application. To reduce time, and complexity, the receding horizon TL planning has been introduced that automatically synthesizes the control algorithm while generating a feasible trajectory regardless of its operating environment [64]. Results have shown the robustness of the system and the capability of handling violations of the environmental assumptions. The technique was then used to develop a toolbox called TuLiP (Temporal Logic Planning) [65, 66]. In [67–69] the method is applied for more complex and realistic AD scenarios. The limitations of the technique rely on that task specifications are expressed in linear temporal logic (LTL) as well as in parameters, which is found by a priori off-line. It is also important to note that there are other tools such as BluSTL for automatically generating controllers from specifications written in Signal Temporal Logic (STL) [70]. Another TP technique used in the recent AV studies is an artificial potential field (aPF). It was first proposed for an autonomous robot guidance system in [71]. The main principle is to generate attraction and repulsion forces, within the environment to guide the vehicle to the destination point. Since the path is the result of interaction between forces, problem is converted into an optimum field configuration search problem, rather than direct construction of an optimum path. The main advantage of aPF is its capability of generating feasible collision-free trajectories in real-time. This is because it does not involve expensive and lengthy computations. However, the optimality of the path is not guaranteed [73]. Different approaches were investigated to design the potential field [72]. In [74], aPF has been used to generate vehicle reference based on elliptical shape potential field for a collision-free trajectory. The effectiveness of the method has been shown for a curved road scenario with static and dynamic obstacles. Another study implemented minimum potential field concept to use less computation time, while overcoming main limitations of aPF [76]. Other studies incorporated potential field forces into the controller to avoid collision, while considering vehicle stability [77, 78]. In [79] potential field is used to generate a trajectory reference, where reference model is not given explicitly. In brief, different approaches are developed/used for trajectory planning of AVs [80]. Even though, different techniques exist they are mainly developed for rigid (non-articulated) vehicles. In the TrustVehicle project, aPF is developed for an articulated vehicle for forward and reverse parking scenario in a very limited semi-structured environment [81, 82]. The reference generator is also simple and flexible to the model of wind and other uncertainties (e.g. friction), which in turn make the solution more robust as in [83]. Moreover, look-ahead and safety distances are integrated into the algorithm to enhance the reliability of the technique. Simulation analysis highlights the smoothness of the trajectory based on the total potential field mesh shown in Fig. 7, which enhances the trust, comfort and acceptance of the end-users. In conclusion, an acceptable TP technique should run in real-time while producing optimal, and feasible obstacle-free trajectory with limited vehicle and environment information. Moreover, before the implementation phase proposed technique need to be evaluated according to completeness, complexity, robustness, optimality and

Reliable Sense-Plan-Act Approaches for TrustVehicles

79

Fig. 7 a Artificial potential field mesh of environment model for reverse parking scenario. b Level 3 articulated truck [81, 82]

flexibility [85]. Therefore, TP for road vehicles is a complex and challenging task and it becomes even more challenging for articulated vehicles operating in a dynamic unstructured environment.

4 Acting Acting is the third enabler of the AV that computes the control signal to precisely track the specified path/trajectory. It can be decomposed into two layers. A highlevel controller that performs the tracking, whereas low-level controller generates the actuation signals that need to be sent to the vehicle. Since the high-level controller is generally being considered as the primary research topic for the AVs, the main focus of this section is devoted to the tracking algorithms [44–85]. Requirements of tracking controller are defined based on the TP as well as the available data. In path following (PF), there is a tendency to separate the longitudinal and lateral controller, whereas in trajectory tracking (TT) combined control structure is preferred [44, 86]. This may cause lower degrees of freedom in the input variables than the system outputs, which has been identified in [87]. Therefore, either use of additional information is required or trajectory should be transferred into a path to be able to implement the PF techniques [88–90]. In the literature comprehensive comparison of TT algorithms is presented for the rigid vehicles [44, 85, 88]. Also dynamics and control of articulated vehicles (e.g. truck and trailer, car-like vehicles etc.) have been extensively studied since 1970s [91–100, 107]. In [95] authors extensively reviewed vehicle handling dynamics and stability control, while in [96] control strategies for backward motion are studied to identify the challenges. The initial studies on the control side mainly focused on the development of driver assistance techniques, whereas the later ones focused more on the development of path and trajectory tracking controller for the AVs. Several patents have been granted as well [108–115]. Preliminary research studies on PT for articulated vehicles mainly considered onaxle vehicles, and try to utilize exact linearization techniques to solve the nonlinear

80

A. E. Hartavi et al.

motion control problem [101, 102]. In [93] the automated steering system design considering the truck-trailer stability is discussed, whereas in [94] instability in backward motion is studied extensively, and linear systems theory is used to develop an optimal regulator controller and validated on an off-axle vehicle in simulation environment. However, despite there is a considerable amount of theoretical publications written in the field, there is only few experimental studies published [98, 103–106]. The challenges associated with articulated vehicle mentioned in the literature are: (i) Unstable dynamics in reverse maneuvers [103], (ii) Non minimum phase reactions [100], (iii) Non-collocated actuator problem in non-steerable trailers [116], and (iv) Jack-knifing [117, 118], as well as other control problems such as actuator amplitude saturation, and rate limit [103, 119], which become even more critical due to large lateral maneuverers in a limited parking area. Furthermore, it is important to note that the requirements of an automated parking algorithm are different from an ordinary path/trajectory controller design. For example, tracking errors become less important than the final errors for the position and heading of the vehicle during the assessment of the algorithm [120]. Therefore, some authors named it as “Point to Point Motion” controller instead of a PF or TT [121]. It is also important to note that, most of the tested scenarios for validating the tracking algorithms exists in the literature are: straight, circle, sinusoidal or eight (infinity) shaped curves. Although their combinations can generate most of the paths, in limited parking spaces, more complicated maneuvers are needed. In conclusion, despite the importance and value of the available research in the field, further research studies need to be carried out on the control of articulated vehicles. Considering the complexity and challenging nature of the articulated vehicles, and the gap in the field in the TrustVehicle project not only rigid, but also articulated heavy duty vehicle is considered for a narrow parking scenario, which is even a very challenging task for an expert human driver [122].

5 Conclusion Autonomous vehicles can be described by three key technology pillars: Sense, Plan and Act. To design a successful AV, acceptance of end user, together with the reliability of the system needs to be enhanced. To this context, in this study the current and prospective technologies and techniques are reviewed briefly to highlight the shortfalls regarding the real-world implementation and different vehicle class. Also, importance of vehicle dynamics is addressed briefly from the planning and control perspective, which can push the limits of the AV technology together with a reliable sensing system. Acknowledgements The project leading to this chapter has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 723324.

Reliable Sense-Plan-Act Approaches for TrustVehicles

81

This book chapter is the collective work of a very large number of colleagues from very different industries, which would not have been possible without their invaluable contributions. Therefore, I would like to thank all the authors who helped us in writing.

References 1. Phantom Auto’ will tour city (1926). “The milwaukee sentinel” 2. Kröger, F.: Automated driving in its social, historical and cultural contexts. In: Autonomous Driving, pp. 41–68. Springer, Berlin, Heidelberg (2016) 3. “Driverless auto, guided by radio, navigates street”, The Washington Herald, p. 5 (6 August 1921) 4. Rosenzweig, J., Bartl, M.: A review and analysis of literature on autonomous driving. E-J. Making-Innovation (2015) 5. Bain & Company: Projected size of the global market for autonomous driving features from 2016 to 2025 (in billion U.S. dollars). In: Statista - The Statistics Portal (n.d.) 6. Saporito, B.: The 50 best inventions of 2010. Time Magazine (2010) 7. Fagnant, D.J., Kockelman, K.: Preparing a nation for autonomous vehicles: opportunities, barriers and policy recommendations. Transp. Res. Part A Policy Pract. 77, 167–181 (2015) 8. Saffarian, M., de Winter, J.C., Happee, R.: Automated driving: human-factors issues and design solutions. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 56, no. 1, pp. 2296–2300. Sage Publications, Sage CA, Los Angeles, CA (2012) 9. Kyriakidis, M., Happee, R., de Winter, J.C.: Public opinion on automated driving: results of an international questionnaire among 5000 respondents. Transp. Res. Part F Traffic Psychol. Behav. 32, 127–140 (2015) 10. Lu, Z., de Winter, J.C.: A review and framework of control authority transitions in automated driving. Proc. Manuf. 3, 2510–2517 (2015) 11. Global Autonomous Driving Market Outlook: “Global automotive & transportation research team at Frost & Sullivan” (2018) 12. TomTom: HD Maps with RoadDNA: High Definition Map with Sensor-Agnostic Localization (2018) 13. Levinson, J., Montemerlo, M., Thrun, S.: Map-based precision vehicle localization in urban environments. In: Robotics: Science and Systems III (2007) 14. Extending the vision of automated vehicles with HD Maps and ADASIS 15. ISO: Road Vehicles – Functional Safety [Norm]. ISO 26262, ISO, Geneva, Switzerland (2011) 16. Skolnik, M.: Introduction to radar systems (Chap. 1.5). In: History of Radar Development. McGraw-Hill book company, INC., New York (1979) 17. Ju, Y., Jin, Y., Lee, J.: Design and implementation of a 24 ghz fmcw radar system for automotive applications. In: International Radar Conference (2014) 18. Sundaresan, S., Anjana, C., Zacharia, T., Gandhiraj, R.: Real time implementation of fmcw radar for target detection using GNU radio and USRP. In: IEEE ICCSP Conference (2015) 19. Velodyne: Velodyne HDL-64E. Online: Accessed 2019 20. Tittmann, P., Shafii, S., Hartsough, B., Hamann, B.: Tree detection, delineation, and measurement from LiDAR point clouds using RANSAC (2011) 21. Li, Y., Ibanez-Guzman, J.: LiDar for autonomous driving: the principles, challenges, and trends for automotive lidar and perception systems. IEEE Signal Process. Mag. 37(4), 50–61 (2020) 22. Hata, K., Savarese, S.: Camera Models. Online: Accessed October 2019 23. Kilicarslan, M., Zheng, J.Y.: Predict vehicle collision by ttc from motion using a single video camera. In: IEEE Transactions on Intelligent Transportation Systems, p. 85 (2019) 24. Klette, R.: Concise Computer Vision. Springer-Verlag, London (2014)

82

A. E. Hartavi et al.

25. Klette, R.: Keypoints and Descriptors (2014) 26. Future Markets Magazine: 3D Cameras in Autonomous Vehicles. Future Markets Magazine (n.d.) 27. Nahler, C., Feldhofer, B., Ruether, M., Holweg, G., Druml, N.: Exploring the usage of time-offlight cameras for contact and remote photoplethysmography. In: 21st Euromicro Conference on Digital System Design (DSD), Prague, pp. 433–441 (2018) 28. Trustvehicle.eu: “Trustvehicle” (2018) 29. Pasinetti, S., Hassan, M., Eberhardt, J., Lancini, M., Docchio, F., Sansoni, G.: Performance analysis of the PMD Camboard Picoflexx time-of-flight camera for markerless motion capture applications. In: IEEE Transactions on Instrumentation and Measurement, pp. 1–16 (2019) 30. The Sanborn Map Company: Sanborn HD maps for autonomous driving. Colorado Springs, CO (2017) 31. Dannehy, M.: TomTom 3D maps: beyond automotive. Tech. Rep. (2016) 32. Slovick, M.: Autonomous cars and overcoming the inadequacies of road mapping (2019) 33. Cozzens, T.: NovAtel test drives STMicroelectronics’ Teseo APP and Teseo V chipset (2018) 34. de Groot, L., Infante, E., Jokinen, A., Kruger, B., Norman, L.: Precise positioning for automotive with mass market GNSS chipsets. In: Proceedings of the 31st International Technical Meeting of The Satellite Division of the Institute of Navigation, Miami, FL, pp. 596–610 (2018) 35. Cozzens, T.: U-BloX F9 designed for high-precision market (2018) 36. Qualcomm: Trimble and Qualcomm establish alliance to produce high accuracy positioning solutions for connected vehicles (2019) 37. Fabio, P., di Grazi, D., Avellone, G., Serrano, L., Kruger, B.: Innovation: multi-band GNSS with embedded functional safety for the automotive market 38. Bosch-Mobility-Solutions: Ultrasonic Sensor (2020) 39. Shahian Jahromi, B.: Ultrasonic sensors in self-driving cars. Medium (2019) 40. Sözen, E., Bretagnol, F., Nahler, C., Rodoplu, K., Tarkiainen, M., Sahimäki, S., Zaya, J.: D6.1: “Sensor technology integration”. TrustVehicle (2019) 41. Tarkianen, M., Kauvo, K., Sözen, E.: D6.2: HMI integration. TrustVehicle (2019) 42. Ghallab, M., Nau, D., Traverso, P.: Automated planning: theory and practice. Elsevier (2004) 43. González, D., Pérez, J., Milanés, V., Nashashibi, F.: A review of motion planning techniques for automated vehicles. IEEE Trans. Intell. Transp. Syst. 17(4), 1135–1145 (2015) 44. Paden, B., Cap, M., Yong, S.Z., Yershov, D., Frazzoli, E.: A survey of motion planning and control techniques for self-driving urban vehicles. IEEE Trans. Intell. Veh. 1(1), 33–55 (2016) 45. Katrakazas, C., Quddus, M., Chen, W.H., Deka, L.: Real-time motion planning methods for autonomous on-road driving: state-of-the-art and future research directions. Transp. Res. Part C Emerg. Technol. 60, 416–442 (2015) 46. Li, X., Sun, Z., Cao, D., He, Z., Zhu, Q.: Real-time trajectory planning for autonomous urban driving: framework, algorithms, and verifications. IEEE/ASME Trans. Mechatron. 21(2), 740–753 (2015) 47. Gasparetto, A., Boscariol, P., Lanzutti, A., Vidoni, R.: Path planning and trajectory planning algorithms: a general overview. In: Motion and Operation Planning of Robotic Systems, pp. 3–27. Springer, Cham (2015) 48. Hattori, Y., Ono, E., Hosoe, S.: Optimum vehicle trajectory control for obstacle avoidance problem. IEEE/ASME Trans. Mechatron. 11(5), 507–512 (2006) 49. Anderson, S.J., Peters, S.C., Pilutti, T.E., Iagnemma, K.: An optimal-control based framework for trajectory planning, threat assessment, and semi-autonomous control of passenger vehicles in hazard avoidance scenarios. Int. J. Veh. Auton. Syst. 8(2–4), 190–216 (2010) 50. Spenko, M., Kuroda, Y., Dubowsky, S., Iagnemma, K.: Hazard avoidance for high–speed mobile robots in rough terrain. J. Field Rob. 23(5), 311–331 (2006) 51. Shamir, T.: How should an autonomous vehicle overtake a slower moving vehicle: design and analysis of an optimal trajectory. IEEE Trans. Autom. Control 49(4), 607–610 (2004) 52. Pepy, R., Lambert, A., Mounier, H.: Path planning using a dynamic vehicle model. In: Information and Communication Technologies, vol. 1, pp. 781–786 (2006)

Reliable Sense-Plan-Act Approaches for TrustVehicles

83

53. Elbanhawi, M., Simic, M.: Sampling-based robot motion planning: a review. IEEE Access 2, 56–77 (2014) 54. Noreen, I., Khan, A., Habib, Z.: Optimal path planning using RRT* based approaches: a survey and future directions. Int. J. Adv. Comput. Sci. Appl. 7(11), 97–107 (2016) 55. Brock, O., Khatib, O.: High-speed navigation using the global dynamic window approach. In: Proceedings of IEEE International Conference on Robotics and Automation, vol. 1, pp. 341– 346 (1999) 56. Ögren, P., Leonard, N.E.: A convergent dynamic window approach to obstacle avoidance. In: IEEE Transactions on Robotics, vol. 21, no. 2, pp. 188–195 (2005) 57. De Lima, D.A., Pereira, G.A.S.: Navigation of an autonomous car using vector fields and the dynamic window approach. J. Control Autom. Electr. Syst. 24(1–2), 106–116 (2013) 58. Emerson, E. A.: Temporal and modal logic. In: Handbook of Theoretical Computer Science, Volume B: Formal Models and Sematics (B), vol. 995, no. 1072, pp. 5 (1990) 59. Manna, Z., Pnueli, A.: The Temporal Logic of Reactive and Concurrent Systems: Specification. Springer Science & Business Media (2012) 60. Pnueli, A.: The temporal logic of programs. In: 18th Annual Symposium on Foundations of Computer Science, pp. 46–57 (1977) 61. Fainekos, G.E., Girard, A., Kress-Gazit, H., Pappas, G.J.: Temporal logic motion planning for dynamic robots. Automatica 45(2), 343–352 (2009) 62. Pola, G., Girard, A., Tabuada, P.: Approximately bisimilar symbolic models for nonlinear control systems. Automatica 44(10), 2508–2516 (2008) 63. Kress-Gazit, H., Fainekos, G.E., Pappas, G.J.: Temporal-logic-based reactive mission and motion planning. In: sIEEE transactions on robotics, vol. 25, no. 6, pp. 1370–1381 64. Wongpiromsarn, T., Topcu, U., Murray, R.M.: Receding horizon temporal logic planning for dynamical systems. In: Proceedings of the 48th IEEE Conference on Decision and Control, 2009 held jointly with the 2009 28th Chinese Control Conference. CDC/CCC 2009, pp. 5997– 6004 (2009) 65. Filippidis, I., Dathathri, S., Livingston, S.C., Ozay, N., Murray, R.M.: Control design for hybrid systems with TuLiP: the temporal logic planning toolbox. In: IEEE Conference on Control Applications (CCA), pp. 1030–1041 (2016) 66. Wongpiromsarn, T., Topcu, U., Ozay, N., Xu, H., Murray, R.M.: TuLiP: a software toolbox for receding horizon temporal logic planning. In: Proceedings of the 14th International Conference on Hybrid Systems: Computation and Control, pp. 313–314 (2011) 67. Wongpiromsarn, T.: Formal Methods for Design and Verification of Embedded Control Systems: Application to an Autonomous Vehicle. California Institute of Technology (2010) 68. Kabbani, T., Di Gennaro, S.: Automatic synthesis via TuLip of an autonomous vehicle controller ensuring collision avoidance. In IEEE Conference on Decision and Control (CDC), pp. 6699–6704. IEEE (2018) 69. Wongpiromsarn, T., Topcu, U., Murray, R.M.: Receding horizon temporal logic planning. IEEE Trans. Autom. Control 57(11), 2817–2830 (2012) 70. Donzé, A., Raman, V., Frehse, G., Althoff, M.: BluSTL: controller synthesis from signal temporal logic specifications. In: ARCH@ CPSWeek, pp. 160–168 (2015) 71. Khatib, O.: Real-time obstacle avoidance for manipulators and mobile robots. Int. J. Robot. Res. 5(1), 90–98 (1986) 72. Cosío, F.A., Castañeda, M.P.: Autonomous robot navigation using adaptive potential fields. Math. Comput. Model. 40(9), 1141–1156 (2004) 73. Lee, L.F.: Decentralized motion planning within an artificial potential framework (APF) for cooperative payload transport by multi-robot collectives. Doctoral Dissertation, State University of New York at Buffalo (2004) 74. Kabbani, T., Lúa, C.A., Di Gennaro, S.: Vehicle reference generator for collision-free paths. Int. J. Control Autom. Syst. 17(1), 181–192 (2019) 75. Tee, K.P., Ge, S.S., Tay, E.H.: Barrier lyapunov functions for the control of output-constrained nonlinear systems. Automatica 45(4), 918–927 (2009)

84

A. E. Hartavi et al.

76. Pozna, C., Troester, F., Precup, R.E., Tar, J.K., Preitl, S.: On the design of an obstacle avoiding trajectory: method and simulation. Math. Comput. in Simul. 79(7), 2211–2226 (2009) 77. Gerdes, J.C., Rossetter, E.J.: A unified approach to driver assistance systems based on artificial potential fields’. ASME J. Dyn. Syst. Meas. Control 123(3), 431–438 (2001) 78. Shimoda, S., Kuroda, Y., Iagnemma, K.: Potential field navigation of high speed unmanned ground vehicles on uneven terrain. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation, pp. 2828–2833 (2005) 79. Ji, J., Khajepour, A., Melek, W.W., Huang, Y.: Path planning and tracking for vehicle collision avoidance based on model predictive control with multi-constraints. IEEE Trans. Veh. Technol. 66(2), 952–964 (2017) 80. Deittert, M., Mathews, G.M.. Trajectory planning. WIPO Patent WO2012098372A1, 26 Jul 2012 81. Cooper, J.: Individual project: development and implementation of a trajectory planning algorithm for an autonomous articulated vehicle in urban scenarios (2019) 82. Kabbani, T., Sarhadi, P., Hartavi Karcı, A., Ozan, B., Sozen, E., Hillbrand, B., Schicker, L.: D3.6: report on trajectory planning algorithm flow-chart and algorithm input-output specifications. Trustvehicle (2019) 83. Luders, B.D., Sugel, I., How, J.P.: Robust trajectory planning for autonomous parafoils under wind uncertainty. In: AIAA Infotech@ Aerospace (I@ A) Conference, p. 4584 (2013) 84. Yang, H.I., Zhao, Y.J.: Trajectory planning for autonomous aerospace vehicles amid known obstacles and conflicts. J. Guid. Control Dyn. 27(6), 997–1008 (2004) 85. Amer, N.H., Zamzuri, H., Hudha, K., Kadir, Z.A.: Modelling and control strategies in path tracking control for autonomous ground vehicles: a review of state of the art and challenges. J. Intell. Rob. Syst. 86(2), 225–254 (2017) 86. Ljungqvist, O.: On motion planning and control for truck and trailer systems. PhD Thesis, Department of Electrical Engineering, Linköping University (2019) 87. Kanayama, Y., Kimura, Y., Miyazaki, F., Noguchi, T.: A stable tracking control method for an autonomous mobile robot. In: Proceedings Presented at the IEEE International Conference on Robotics and Automation, Cincinnati, OH, USA (1990) 88. Calzolari, D., Schürmann, B., Althoff, M.: Comparison of trajectory tracking controllers for autonomous vehicles. In: 20th International Conference on Intelligent Transportation Systems (ITSC) Presented at the IEEE (2017) 89. Kayacan, E., Chowdhary, G.: Tracking error learning control for precise mobile robot path tracking in outdoor environment. J. Intell. Rob. Syst. 95(3–4), 975–986 (2019) 90. Heilmeier, A., Wischnewski, A., Hermansdorfer, L., Betz, J., Lienkamp. M., Lohmann, B.: Minimum curvature trajectory planning and control for an autonomous race car. In: Vehicle System Dynamics, pp. 1–31 (2019) 91. Jindra, F.: Lateral stability of a three-unit trailer train. Ing. Arch. 17(4), 209–225 (1963) 92. Schmid, I.: Engineering approach to truck and tractor train stability. Automobile Eng. 53(3), 96–101 (1968) 93. Saito, Y.: On the stability and followability of the semi-trailer vehicle with an automatic steering system. Veh. Syst. Dyn. 8(2–3), 206–211 (1979) 94. Kageyama, I., Saito, Y.: On the backward maneuverability of articulated vehicles. Vehicle Syst. Dyn. 14(1–3), 42–46 (1985/06/01) 95. Kang, X., Deng, W.: Vehicle trailer handling dynamics and stability control an engineering review. Presented at the Vehicle Dynamics & Simulation, Detroit, Michigan, 16–19 April 2007 96. David, J., Manivannan, P.V.: Control of truck-trailer mobile robots: a survey. Intell. Serv. Robot. 7(4), 245–258 (2014) 97. Kayacan, E., Saeys, W., Ramon, H., Belta, C., Peschel, J.M.: Experimental validation of linear and nonlinear mpc on an articulated unmanned ground vehicle. IEEE/ASME Trans. Mechatron. 23(5), 2023–2030 (2018) 98. Ljungqvist, O., Evestedt, N., Axehill, D., Cirillo, M., Pettersson, H.: A path planning and path-following control framework for a general 2-trailer with a car-like tractor. J. Field Robot. 36(8), 1345–1377 (2019)

Reliable Sense-Plan-Act Approaches for TrustVehicles

85

99. Marques, F., Lucas, B., Marcos, B., Silva, M.M., Terra, M.H., GrassiJunior, V.: Robust pathfollowing control for articulated heavy-duty vehicles. Control Eng. Pract. 85, 246–256 (2019) 100. Michalek, M.M.: Agile maneuvering with intelligent articulated vehicles: a control perspective. IFAC-Papers OnLine 75(2), 458–473 (2019) 101. Sampei, M., Tamura, T., Itoh, T., Nakamichi, M.: Path tracking control of trailer-like mobile robot. IEEE International Workshop on Intelligent Robots and Systems ’91, Osaka, Japan, Japan, 3–5 Nov 1991 102. Sampei, M., Tamura, T., Kobayashi, T., Shibui, N.: Arbitrary path tracking control of articulated vehicles using nonlinear control theory. IEEE Trans. Control Syst. Technol. 3(1), 125–131 (1995) 103. Pradalier, C., Usher, K.: Robust trajectory tracking for a reversing tractor trailer. J. Field Robot. 25(6–7), 378–399 (2008) 104. Werling, M., Reinisch, P., Heidingsfeld, M., Gresser, K.: Reversing the general one-trailer system asymptotic curvature stabilization and path tracking. IEEE Trans. Intell. Transp. Syst. 15(2), 627–636 (2014) 105. Leng, Z., Minor, M.: A simple tractor-trailer backing control law for path following. Presented at the IEEE/RSJ international conference on intelligent robots and systems (2010) 106. Rimmer, A.J., Cebon, D.: Implementation of reversing control on a doubly articulated vehicle. J. Dyn. Syst. Meas. Control 139(6), 061011-1–061011-9 (2017) 107. Michałek, M.: Non-minimum-phase property of N-trailer kinematics resulting from off-axle interconnections. Int. J. Control 86(4), 740–758 (2013) 108. Deng, W., Lee, Y. H.: Anti-jackknife control for vehicle-trailer backing up using rear-wheel steer control. United States Patent US6854557B1, 15 Aug 2005 109. Lee, Y.H., Deng, W., Chin, Y.K., Lin, W.: Vehicle-trailer backing up system using active front steer. United States Patent US7154385B2, 16 Dec 2006 110. Rupp, M.Y., Smit, D.D., Trombley, R.A., Pilutti, T.E., Lavoie, E.M., Shim, T.: Managing jackknife enabling conditions during backing of a trailer by reducing speed of a vehicle backing the trailer. United States Patent US8755984B2, 17 June 2014 111. Goswami, A., Chiu, J.: Reverse drive assist for long wheelbase dual axle trailers. United States Patent US9393996B2, 19 Jul 2016 112. Pilutti, T.E. , Trombley, R.A., Hafner, M., Rupp, M.Y., Rhode, D.S., Recker, D.A.: Control for trailer backup assist system. United States Patent US9493187B2, 15 Nov 2016 113. Hafner, M., Pilutti, T.E.: Trajectory planner for a trailer backup assist system. United States Patent US9708000B2, 18 Jul 2017 114. Christos, K., Lavoie, E.M.: Jackknife detection for vehicle reversing a trailer. United States Patent US20170008560A1, 20 Feb 2018 115. Lavoie, E.M., Trombley, R.A., Pilutti, T.E., Rolfes, N.A., Hafner, M.: Trailer curvature control and mode management with powertrain and brake support. United States Patent US9623859B2, 18 April 2018 116. Wie, B., Bernstein, D.D.: Benchmark problems for robust control design. J. Guid. Control Dyn. 15(5), 1057–1059 (1992) 117. Chiu, J., Goswami, A.: The critical hitch angle for jackknife avoidance during slow backing up of vehicle–trailer systems. Vehicle Syst. Dyn. 52(7), 992–1015 (2014) 118. Trigella, A.S., Rothhamela, M., Pauwelussenb, J., Kural, K.: Advanced vehicle dynamics of heavy trucks with the perspective of road safety. Vehicle Syst. Dyn. 55(10), 1572–1617 (2017) 119. Sarhadi, P., Khosravi, A., Bijani, V.: Identification of nonlinear actuators with time delay and rate saturation using meta-heuristic optimization algorithms. Proceedings of the Institution of Mechanical Engineers, Part I: J. Syst. Control Eng. 229(9), 808–817 (2015) 120. Liu, W., Li, Z., Li, L., Wang, F.Y.: Parking like a human: a direct trajectory planning solution. IEEE Trans. Intell. Transp. Syst. 18(12), 3388–3397 (2017) 121. Luca, A.D., Oriolo, G., Samson, C.: “Feedback Control of a Nonholonomic Car-Like Robot” Robot Motion Planning and Control. Springer, Berlin, Heidelberg (1998) 122. Kabbani, T., Sarhadi, P., Sozen, E., Kinav, E., Hartavi, A.E.: Level 3 Autonomous ReverseParking Algorithms for a Heavy-Duty Truck (2020)

TrustVehicle Verification Procedure Bernhard Hillbrand, Pamela Innerwinkler, Georg Stettinger, Ahu Ece Hartavi, Kemal Rodoplu, Ali Demir, Talat Büyükakin, Ersun Sözen, Eren Aydemir, Philipp Clement, Johan Zaya, Sami Sahimäki, and Mikko Tarkiainen

Abstract Automated driving is supposed to alleviate drivers’ workloads and improve traffic efficiency and road safety. Still, in order to achieve these goals, it is crucial to test the automated systems thoroughly. Since pure testing on test tracks B. Hillbrand (B) · P. Innerwinkler · G. Stettinger Virtual Vehicle Research GmbH, Inffeldgasse 21A, 8010 Graz, Austria e-mail: [email protected] P. Innerwinkler e-mail: [email protected] G. Stettinger e-mail: [email protected] A. E. Hartavi University of Surrey, Guildford, UK e-mail: [email protected] K. Rodoplu · A. Demir · T. Büyükakin Tofa¸s Turk Otomobil Fabrikasi Anonim Sirketi, ˙Istanbul, Turkey e-mail: [email protected] A. Demir e-mail: [email protected] T. Büyükakin e-mail: [email protected] E. Sözen · E. Aydemir Ford Otosan, Istanbul, Turkey e-mail: [email protected] E. Aydemir e-mail: [email protected] P. Clement AVL List GmbH, Graz, Austria e-mail: [email protected] J. Zaya Volvo Car Corporation, Gothenburg, Sweden e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Watzenig and L. Schicker (eds.), Enhanced Trustworthiness and End User Acceptance of Conditionally Automated Vehicles in the Transition Period, Lecture Notes in Intelligent Transportation and Infrastructure, https://doi.org/10.1007/978-3-030-60861-3_5

87

88

B. Hillbrand et al.

and real roads is highly time consuming and still cannot cover merely all critical scenarios, usage of simulation and driving simulators can be a way to speed up the testing process. The TrustVehicle project aims at improving SAE Level3 automated driving (L3AD) functionalities, especially in critical situations and under harsh weather conditions. Therefore, co-simulations, driving simulator and real-world tests are used to test the trustworthiness and availability of the developed functionalities. The use cases therein cover exact low-speed maneuvering on construction sites and in urban environments, as well as medium speed applications for rural roads, where also various types of vehicles (truck-trailers, light commercial vehicles, electric buses and passenger cars) are investigated. In addition to safety and reliability considerations, the presented approach also allows for end-user feedback already in the development process. Keywords Level 3 automated driving · Reliability · Driver-centric approach · Co-simulation · Driving simulator

1 Introduction TrustVehicle applies three different methods to verify the work in this project. Each of these methods has a separate work package dedicated to it (see Fig. 1): • Simulation (WP3) Fig. 1 Verification methods in TrustVehicle

S. Sahimäki Linkker OY, Villähde, Finland e-mail: [email protected] M. Tarkiainen Teknologian Tutkimuskeskus VTT OY, Espoo, Finland e-mail: [email protected]

TrustVehicle Verification Procedure

89

• Driving Simulator (WP5) • Test Track/Real Roads Demonstration (WP6) Each method has its advantages and deficiencies and combined they complete each other very well. Simulations have the capability to test a wide range of scenarios and use cases. Furthermore, it is possible to make and test parameter adjustments without spending much money on built-up demonstrators and testing facilities. Another advantage of simulation is repeatability. The same conditions lead to 100% same result, which is not true for real-life testing. The focus of this method is on functional development. The Driving Simulator on the other hand is a good way to determine user acceptance and get feedback on the behavior and settings of driving functions. The Test Track/Field tests are necessary to verify the simulation models, determine user acceptance and test the robustness of the demonstrators under real environmental influences, such as harsh weather and road conditions. The following sections explain the work that has been done in the TrustVehicle project for each of these methods. Section 2 gives a short introduction into co-simulation and the co-simulation tool AVL Model.CONNECTTM. Also, three simulation use cases will be described. Detailed information about the use cases can be found in Chap. 1 of this book. In Sect. 3 the focus is on the driving simulator and the scenarios that have been designed for the driving simulator study. In Sect. 4 the third TrustVehicle verification method, test track/field tests, and its use cases are described. Finally, a short wrap-up is given in the conclusion section.

2 Co-simulation Approach In the past few decades, numerous specific simulation tools have been established in the automotive industry. However, these tools typically specialize on individual areas of expertise. There is very limited support for a heterogeneous simulation environment. The goal of “co-simulation” is to overcome this limitation and to merge the challenges from different areas. Common co-simulation platforms are often limited to a single area of expertise (e.g. the design of a thermal management system with a heterogeneous tool landscape). Such platforms typically address problems from a very specific, restricted dynamic range, which means that the different models exhibit similar time behavior. However, the development of modern, mechatronic systems requires a much broader approach. The interactions between sub-systems from different areas have to be taken into account through a suitable interconnection of the parts. The coupling of existing (specific) simulation programs (and the models implemented therein) from different areas of expertise represents a promising approach for the simulation of the complete system. With the introduction of co-simulation in the development process the task of developing complex mechatronic systems can be solved in a very efficient way.

90

B. Hillbrand et al.

The task of a co-simulation platform is to take into account the complex interactions of the various simulation models in a suitable and correct way. The platform has to enable the precise co-working of different simulation tools.

2.1 Model.CONNECT The co-simulation platform Model.CONNECTTM [1] enables setting up and executing entire mechatronic system simulations that are composed of subsystem and component models from multiple model authoring environments. Either standardized interfaces (e.g. Functional Mockup Interface, FMI) or specific interfaces to simulation tools can be used to integrate models into the framework. There is already a wide range of specific interfaces available for the inclusion of many well-known simulation tools, especially concerning tools common in the automotive industry. A list of supported tools can be found in the Model.CONNECTTM user manual. A huge advantage of Model.CONNECTTM is that it supports the user in organizing system model variants. Different configurations of the system under investigation as well as different testing scenarios and testing environments can be managed. For coupling of the models two options are available. Both options can also be combined if needed. The first option is model integration based on models that are provided as executable libraries. In this case (FMI for Co-Simulation or Model Exchange, as well as compiled MATLAB/Simulink models) the model configurations can be executed in one process. The second option is tool-coupling based on the ICOS technology. ICOS is a distributed co-simulation platform included in Model.CONNECTTM with a wide variety of supported simulation tools, industry-leading co-simulation algorithms and the possibility to connect real-time systems to the co-simulation [2, 3]. Interacting via inter-process communication, all tool-interface elements are running as individual processes. Apart from local co-simulation, Model.CONNECTTM also allows the connection of different operating systems on distributed resources. A running remote server on each host is required to perform a distributed co-simulation, i.e. connecting different simulation tools on different host computers, where every one of them may have their individual operating system. Co-simulation was used in the three use cases descript in the following sections.

2.2 Truck Use Case In this first use case, two scenarios for trucks with trailers are considered: parking in a docking station and backing in a construction site. For both scenarios it is a common practice that the driver needs several maneuvers to bring the trailer in the correct position, either to park correctly to the dedicated slot in the docking station, or to

TrustVehicle Verification Procedure

91

bring the truck in position on the construction site. While the main concern when considering the docking station scenario is the time spent to position the trailer, the problem with construction sites is also the surrounding traffic and other road users, such as pedestrians. The aim of this use case is to make backing scenarios with truck-trailer combinations more efficient. The vehicle shall be positioned precisely, and the maneuver should become less time-consuming. A key-point in this context is also user acceptance, since the truck driver shall feel comfortable using the driving function performing the backing maneuver. Therefore, an HMI guaranteeing smooth transitions from manual to automated driving and vice versa has to be considered and the automated function has to be reliable and robust. The system has to perform in various environmental conditions. In the docking station, different weather conditions must be taken into account. In case of construction sites, the sensors additionally have to deal with dirt and dust covering the sensors so that the sensor availability is decreased. Sensor robustness and availability consequently form another KPI (Key Performance Indicator). The high-level architecture of the simulation model is shown in Fig. 2. The environment block represents the surrounding obstacles and boundaries of the use case, whereas the vehicle model defines the dynamics of the vehicle considering the tire dynamics. The perception model is used for the identification, and interpretation of multisensors information in order to provide the Path Planner with the required inputs. The model gathers information from all sensors and performs a data fusion to give required information regarding the environment. The Trajectory Planner generates a reference trajectory for the vehicle to follow. This model takes the inputs of surroundings for the perception model and calculates a safe and adequate path considering the vehicle dynamics. The Trajectory Controller takes the path and gives the required longitudinal and lateral control signals to follow the path. Path planning and tracking are MATLAB/Simulink functions. An interface is defined between each model block to adapt the data transferred between the models.

Fig. 2 Subsystem overview of the Ford Otosan use case

92

B. Hillbrand et al.

Finally, a Handover Switch block is dedicated to monitor potential faults and malfunctioning of other blocks. This block switches the control from the driver to the Automated Driving System (ADS) and vice versa.

2.2.1

Model Verification

For parametrization and correlation studies, the IPG Truckmaker and Simulink interface is the main platform for the Truck use case (in Fig. 3 the F-Max tractor unit is shown in IPG Truckmaker). The purpose of this model is to create a virtual environment, which has a representative vehicle, to test the trajectory planner and controllers not only performing according to the bicycle model of the truck but also including vehicle dynamics. With IPG-Truckmaker capabilities, the developed simulation model has many detailed parameters included or present from generic vehicle models. It reduces the time spent for actual vehicle testing by reducing controller calibration time. The parametrized model is validated using correlation data from real-life trucks as well as installed environment sensors, where focus is on parameters related to lateral control since longitudinal behavior was already previously considered in other projects. In a later stage, truck-trailer combinations will also be correlated with reallife measurements. The correlation tests include constant steering maneuvers, sine steering and lane change maneuvers.

2.3 Light Commercial Vehicle Use Case Automated door to door delivery concepts belong to the most trending topics for commercial vehicles. Light commercial vehicles (LCV) have a big portion of urban delivery missions. Tofa¸s implemented several urban traffic scenarios to investigate and increase user acceptance of L3AD for light commercial vehicle Fig. 3 IPG Truckmaker with correlated F-max tractor unit in the front

TrustVehicle Verification Procedure

93

Fig. 4 Subsystem overview of the LCV use case

customers. According to the received feedbacks from LCV customers two critical urban road scenarios have come forward. The first scenario contains backward and forward approaching to dedicated points for loading/delivering and automatically departing/approaching to pre-defined wireless charging stations. The other one contains fine-tuned low speed maneuvers for narrow street delivery duties with complex traffic (vehicle + pedestrian + parked vehicles). Hand over situations will be implemented for high density urban traffic scenarios. The dedicated scenarios aim to increase acceptance of L3AD LCVs by covering some critical urban traffic issues. Increasing efficiency for urban delivery duties and reducing driver efforts are other aims for the commercial user’s perspective. Maneuvers which will be realized by controller models shall be maintained within less time than normal operation and shall be smooth enough to fulfill user acceptance. Figure 4 shows the simulation block diagram of the TF use case subsystems. The vehicle is represented virtually by its vehicle dynamic parametrization in the dSPACE ASM environment model. This model is active with a soft controller in the Matlab/Simulink platform. The perception model is created within a virtual simulation platform and provides the required signals to the trajectory interface.

2.3.1

Vehicle Sensors

To be able to obtain a reliable perception model for eLCVs (electric LCVs), a detailed analysis on different sensors has been performed. According to an analysis, it was decided to use a combination of LIDAR, Mono Camera and Ultrasonic on the eLCV mule vehicle (see Table 1 and Fig. 5).

94

B. Hillbrand et al.

Table 1 Sensors used for the TF MiL model

Fig. 5 Sensor localization and field of view representation of the Tofa¸s (TF) mule vehicle

2.3.2

Sensor Fusion

Fusing different types of sensor data results in higher accuracy than using each sensor data individually. Sensor fusion increases robustness, confidence and reliability. Since different sensors have different coverage, sensor fusion results in extended sensory coverage and improved resolution. In the TF use case scenarios, low-level

TrustVehicle Verification Procedure

95

Fig. 6 Sensor fusion block diagram

sensor fusion is used to combine different sources of raw data to obtain new data that is richer than the original raw data. It is important for the LCV use case scenarios to detect edges and surrounding dynamic and static objects. There are lots of deep learning architectures such as YOLO, R-CNN, Faster RCNN, Mask R-CNN etc. but all of them are searching for objects in the image. The searching process makes these algorithms slow. The probability of the proposed areas to be an object rely on the training session of these architectures. So, these networks might try to classify even background as object. In the LCV use case, a new method by fusing LiDAR with camera is proposed to overcome this problem (Fig. 6). Added depth information by LiDAR is used to detect object locations. Then these locations are directly fed to the classifier network which produces faster results than object detection architectures. This way not only the speed of the algorithm, but also the robustness and accuracy of object detection and classification are improved. To validate the used sensor fusion, predefined scenarios are used. An example is given in Fig. 7.

2.4 Passenger Car Use Case In order to conduct safe automated vehicle manoeuvres, vehicle controllers are highly dependent on trustworthy data from the vehicle sensors. The sensor output must ensure that lines, objects and Vulnerable Road Users (VRUs) within the driving path are detected and reported. However, 100% detection rate is unlikely, often due to external disturbance and not necessarily sensor performance. Degradation of the sensor output can result in false detections, late detections or even missed detections. The degradation of sensor output also impacts the availability of the automated function. By monitoring system faults, the trustworthiness of sensor output can be increased and therefore the performance advantage of each sensor in different driving scenarios and environmental conditions can be exploited.

96

B. Hillbrand et al.

Fig. 7 TF sensor fusion validation. Scenarios: Case 1—turning right with +30° steering angle, no braking. Case 2—braking, no turning. Case 3—turning left with −60° steering angle, no braking

• Fault Scenario 1: Sensor health and communication quality Diagnostics of components, such as reporting heartbeat or signal quality, is already vital part of sensor implementation. The Sensor Monitoring Module can use diagnostic information as a confidence value to measure the trustworthiness of the sensor outputs. • Fault Scenario 2: Environmental conditions The availability of vehicle sensors is dependent on environmental conditions and different sensors react differently depending on these conditions. Conditions such as harsh weather or reduced visibility can highly degrade the sensor, which has to be reported to the Sensor Monitoring System. The KPIs for the passenger car use case are mainly related to the improvement of the availability of the used L3AD system in various environmental conditions. It is intended to decrease the number of take-over scenarios and therefore improve the user acceptance. For detailed information about the KPIs see Chap. 1 of this book. In [4] the verification and validation methods of the TrustVehicle project are descripted using only the passenger car use case.

TrustVehicle Verification Procedure

97

Fig. 8 Passenger car use case simulation setup

In the simulation performed on the driving simulator the sensor fusion and automated driving functions were provided by Virtual Vehicle (VIF) (see Fig. 8). Both are replaced for the real-world testing in the Volvo XC90 demonstrator vehicle by Volvo internal functions. For the passenger car use case on the driving simulator, a multi-sensor multiobject tracking system from VIF, based on radar, LiDAR and camera sensors was used. The Automated Driving Function developed at VIF is a SAE Level 3 driving function, relieving the driver of longitudinal as well as lateral control responsibilities including adaptive cruise control (ACC), lane keeping and lane change. It is based on the work described in [5] and adapted for the passenger use case in TrustVehicle. In order to achieve a fail-operational architecture, two sets of functionalities are executed in parallel (see Fig. 8). The primary driving function, including full longitudinal and lateral control in the ego vehicle’s own lane as well as the adjacent lanes (lane change functionality), is the Trajectory Planner. A combination of ACC and lane keeping assistant (LKA) serves as the hot standby. In case the sensor monitoring system detects a failure in the sensor system that affects the availability of specific functionalities (in this case lane change), the switch position changes and signals from ACC/LKA instead of from the Trajectory Planner are sent to the Trajectory Tracking. The driving function model is only one part of the Model.CONNECTTM cosimulation setup. In Fig. 9 all components of the Simulation setup for the passenger car use case are shown. The VTD block represents the connection to the environment simulation tool Vires Virtual Test Drive (VTD). This tool is working on a separate computer. The green block with the white car is the VSM (Vehicle Simulation Model) block. It simulates the vehicle dynamics in a detailed way. Since it uses another coordinate system as VTD a transformation block is connected between them. VSM is also connected to a small block that controls the function of the driving simulator motion platform.

98

B. Hillbrand et al.

Fig. 9 Model.CONNECT™ setup for the driving simulator study

The other blocks are for the functionality of the Time-of-Flight camera for driver monitoring (the top left subsystem block), for collecting data for AVL Drive (bottom) and some blocks that provide constant signals.

3 Driving Simulator During the project two studies on the driving simulator were conducted. A small one to test the hardware, scenarios and the testing method and a bigger one. In this second study 55 participants from different age groups took part. All about the driving simulator itself as well as the details of the study will be part of Chap. 6. In this chapter the focus will be on the scenarios.

3.1 Scenarios Ten scenarios were presented to the participants of the study. These scenarios should be diverse and represent several critical situations. Thus, it was possible to build up a large data set. In order to avoid too long time in the simulator for the participants,

TrustVehicle Verification Procedure

99

each scenario should have a maximum duration of 2 min. The ten scenarios consist of four different maps. The remaining six are variations with different controllers or parameters. Scenario 1: “Sensor Cleaning” The focus of the first scenario was on sensor degradation and cleaning. The ego car drives on a street with two lanes both in the same direction. The weather is cloudy, and the asphalt is wet. The focus in this scenario is degradation of a sensor due to dirt that is caused by a car passing by on the left lane. The participant receives a message on the dashboard that the lane change functionality is temporarily not available. Therefore, the car has to brake behind a slower car in front of the ego car. After 15 s the participant is informed that the sensor is cleaned, and the ego car performs the lane change to overtake the slower car. This scenario was used two times with different controller settings. First a comfortable controller with a moderate acceleration rate, where the vehicle starts to break earlier and keeps a bigger distance to the vehicle in front of the ego car. Also, the takeover process starts earlier. The second controller is a sportier controller, where the acceleration rate is double the rate of the comfortable controller, the vehicle doesn’t break as early and the distance to the vehicle in front of the ego car while following is smaller. The goal of this was to see which controller and therefore which parameters are preferred by the participants to adjust the setting of the driving functions. Scenario 2: “Reduced Visibility” The second scenario is about the reduction of L3 functionality due to harsh weather conditions. The ego car drives through a village with a speed limit of 50 km/h. It’s a street with one lane on each side and oncoming traffic. At the begin it rains only lightly with good visibility but then the rain gets heavy and the visibility is reduced. Therefore, the speed of the ego car is adjusted according to the visual range of the sensor. The road is a straight road. The roadside is a mixture of rural and urban environment (village/small town) with houses but also green areas. A speed limit sign (50 km/h) is displayed at the beginning of the scenario. Some oncoming traffic is included to make the scene feel more realistic. This scenario was also used two times with the two different controller settings mentioned above. Scenario 3: “Pedestrian Crossing” The scenario takes place on a single-way road in a city with buildings on both sides. On this road, there is a parking area all the way on the left side with a width of 2.5 m. The driving lane starts with 5 m and narrows to 2.5 m, when the parking space on the right side starts. The right parking space also has a width of 2.5 m. The length of the narrow driving lane is 100 m. On both sides of the street there is a pedestrian walkway next to the parking area. Cars are parked on the parking spaces in a parallel

100

B. Hillbrand et al.

Fig. 10 Scenario 3 from a bird’s eye view

arrangement. These cars are not uniformly parked and are different in type. The weather is clear, and the road conditions are dry asphalt. For different variations of the scenario the deceleration and the breaking point of the Ego-Vehicle at position B (see Fig. 10) is changed. In the first variation the Ego-Vehicle brakes early and decelerates down to 4 kph. In the second variation the Ego-Vehicle brakes very late and decreases its speed down to 0 kph. The third variation is the same as the first sequence. Scenario 4: “Construction Site on the Highway” In this scenario the Ego vehicle is approaching a construction site and therefore has to change the lane and drive by the obstacle. At the end of the construction site the Ego vehicle is changing the lane back again. This scenario has three variations. First it is performed with good weather conditions, the second variation is the same with bad weather and in the third variation suddenly a van is cutting in in front of the Ego vehicle (happening in B pictured in Fig. 11).

4 Proving Ground/Field Tests Representing the real-life traffic scenarios are one of the most critical methods used by automotive industry for verification and validation processes. It becomes more essential if verified or validated attributes are related to the safety measures. Since the main purpose of the TrustVehicle project is to develop L3AD autonomous functions which are focusing to enable more safe and reliable products in the end, validation of these developed functions with the traffic scenarios have a vital importance.

TrustVehicle Verification Procedure

101

Fig. 11 Scenario overview of the highway with construction site

4.1 Truck Demonstrator 4.1.1

Demonstration Setup

Ford Otosan is developing a L3AD reverse parking truck and trailer combination in TrustVehicle project and this reverse parking feature will be demonstrated in Ford Otosan’s Test Track which is located in Eskisehir, Turkey [6].

4.1.2

Scenarios

Ford Otosan L3AD reverse parking scenario is founded on the urban construction site entry and parking difficulties with an articulated vehicle, in this case a truck and semi-trailer combination. For representation of the construction scenarios, 3 different construction areas are examined with Google Earth Satellite photos. It is decided to use an approximate version of the examined construction areas in terms of perimeter distances and total area available for the parking maneuver.

4.1.3

Construction Site Traffic Event 1

A Ford Vehicle parks to the dedicated parking slot. An arbitrary projection of an area similar to the vehicle parks at the entrances of the construction areas will be marked on the Ford Otosan proving ground, as can be seen in Fig. 12.

102

B. Hillbrand et al.

Fig. 12 TV FO construction site projection on the proving ground

4.1.4

Construction Site Traffic Event 2

A FO Vehicle Parks to the available parking slot by selection. Available parking slots are in different positions and according to the static (parked) other vehicles, which are other construction trucks. Demonstration content is similar to the traffic event in terms of the utilization of the proving ground.

4.1.5

TrustVehicle KPI Satisfaction

With the selected traffic scenarios, Ford Otosan aims to satisfy the TrustVehicle KPI’s listed below. • Driver Expectations KPIs for Truck: Trust • HMI KPIs for Trucks: Take over request understanding, Indication of the current mode, Information in case of deactivation in AD mode.

4.2 Passenger Car Demonstrator 4.2.1

Demonstration Setup

Volvo Cars’ use case, using a passenger vehicle, is based on degradation and its effect on the overall function availability. Volvo’s tests will take place in two different environments—Volvo’s proving ground in Hällered and in real traffic conditions. Two different demonstrator setups will be used for these environments:

TrustVehicle Verification Procedure

103

(a) Proving ground The test location is Volvo Cars multipurpose test facility. If the environmental conditions are not met on the day of testing, the conditions can be manually simulated, e.g. watering the driving path for road spray conditions and using a water spray system for the sensors for rain conditions. Scenarios in the test environment will focus on how well each sensor performs in different environmental conditions and thereafter how well the complete perception system, i.e. sensor fusion software, performs in the same environmental conditions. The test in this scenario will be carried out according to the EuroNCAP scenario “Car-to-Pedestrian Nearside Child”. Both the vehicle and the pedestrian in the test scenario will be operated by robotic installations that control the vehicle actuators such as acceleration, steering and braking. (b) Field Tests Scenarios in the real environment will focus on how well the complete perception system, i.e. sensor fusion software, performs in real traffic situations in different environmental conditions. The test scenario will be carried out on a driving route outside the central parts of Gothenburg, Sweden. This route is specifically chosen to reduce complex situations such as oncoming vehicle, pedestrians and traffic lights. The vehicle will be operated by professional test drivers employed and educated to operate test by Volvo Cars. The vehicle will be equipped with the complete system solutions including sensors and cleaning systems; however, the vehicle will be fully controlled by the driver.

4.2.2

Scenarios

The scenarios of the passenger vehicle focus on the sensor degradation due to harsh weather conditions. The weather conditions that will be included in the scenarios are derived from the list of environmental conditions that Volvo Cars targets when developing components and new vehicles like light/moderate/heavy rain and snow as well as wet and light snow-covered roads. The passenger vehicle for this use case will be a Volvo XC90. Sensor and sensor cleaning solutions will be integrated in order to test the performance complete system solution in the different weather conditions. The performance evaluation will be based on measuring the perception output from the sensor, i.e. the ability to detect lines, pedestrians and vehicles etc. in the different environmental conditions. The conclusions related to the KPI of overall function availability will thereafter be based on to what extent the cleaning solutions will be able to reduce sensor degradation during different environmental conditions.

104

4.2.3

B. Hillbrand et al.

TrustVehicle KPI Satisfaction

With the selected traffic scenarios, VCC aims to satisfy the TrustVehicle KPI’s listed below. • Critical road scenarios—harsh weather conditions (assessment of the automated driving functionalities for non-crossable objects during degraded sensor performance) • Controllers and Sensor Fusion Systems—Safety (sensor performance during critical scenarios), availability (function availability wrt. sensor performance), reliability (weather condition robustness) • Evaluation of L3AD functions and vehicle tailoring accordingly—Trust and Perceived Safety (function performance during critical scenarios).

4.3 Bus Demonstrator 4.3.1

Demonstrator Setup

In this use case an electric bus is used. The bus is approaching the combined bus stop/charging station at the end of the line on manual driving mode. When the bus enters the automated driving area, the preconditions for automated driving are checked and the automated driving system calculates the driving path, checks possible obstacles such as other vehicles and pedestrians and ensures the driver’s ability and availability. If automated driving can be activated, the driver can switch the automated driving ON. While driving in automated driving mode, the system monitors the traffic environment as well as the driver and shows to the driver only relevant information. At the bus stop (under the charging station) the system indicates when the bus has been parked and the charging can be activated, and automated driving mode is deactivated.

4.3.2

Scenarios

(a) Demonstration in real environment The target area is a busy bus stop in Helsinki (see Fig. 13). The arrangement for closing the area only for automated driving testing is very difficult to organise safely. Following features could be demonstrated: • Successful case: the bus driving mode is changed (while the bus is moving) from manual to automated and the bus is driven automatically and accurately to the combined bus stop/charging station – Functionality of the safe HMI (b) Demonstration at the VTT testing area

TrustVehicle Verification Procedure

105

Fig. 13 Linkker automated driving target area

The layout of the approaching street and the bus stop has been matched with the real environment. The exact layout dimensions of the target area have been replicated at the VTT testing area to enable efficient system development. In the test area, the curb on the side of the street includes a similar bend before the bus stop. The curb stone has been replicated with wooden timber, which match to the height of the curb stones used in the real environment. In this closed area, diverse features could be demonstrated: • Behaviour of automated driving and HMI (including driver monitoring) in events such as: • Safe handling of non-critical obstacle event • Safe handling of driver distraction combined with non-critical obstacle event • Safe handling of critical obstacle event leading to Take-Over-Request. 4.3.3

TrustVehicle KPI Satisfaction

With the selected traffic scenarios, VTT/Linkker aim to satisfy the TrustVehicle KPI’s listed below. • • • •

Smoothness—Passenger ride quality Trust Safety HMI KPI’s: Ease of use, Comprehensibility of user interface, Take over request understanding, User satisfaction.

5 Conclusion The full verification and validation procedure used in the TrustVehicle project is described. Basis for every V&V procedure is the thorough specification of requirements and KPIs. This was done by both experts and end-users, using multiple rounds of questionnaires. The second stage is testing in simulation. Using real-world data to parameterize the developed models this stage is intended to reduce the amount of real-world testing needed to cover every critical driving situation. To add the driver

106

B. Hillbrand et al.

into the mix, a testing stage on the driving simulator was performed. Especially for the sensor monitoring system developed in the described use case, end-user feedback is essential and can be obtained at an early stage in the development process. Finally, real-world testing is performed. Realistic traffic scenarios are tested on real roads and are supplemented by testing on proving grounds, where harsh weather conditions and specific critical driving scenarios, which cannot be staged on real roads, can be constructed. Acknowledgements The project leading to this chapter has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 723324. The chapter was written at Virtual Vehicle Research GmbH in Graz and partially funded by the COMET K2—Competence Centers for Excellent Technologies Programme of the Federal Ministry for Transport, Innovation and Technology (bmvit), the Federal Ministry for Digital and Economic Affairs (bmdw), the Austrian Research Promotion Agency (FFG), the Province of Styria and the Styrian Business Promotion Agency (SFG).

References 1. Model.CONNECTTM. AVL’s open model integration and co-simulation platform. (https://www. avl.com/-/model-connect), R2020a. Accessed 4 Feb 2020 2. Thek, N., et al.: Co-simulation under hard-real-time conditions. Paper presented at NAFEMS world congress 2013, Salzburg, Austria 3. Stettinger, G., et al.: Extending co-simulation to the real-time domain. Paper presented at SAE symposium 2013, Stuttgart, Germany 4. Hillbrand, B. et al.: Validation and verification procedure for automated driving functions using the example of the TrustVehicle project. In: Meyer G., Michelmann J. (eds.) Advanced Microsystems for the Automotive Applications. Springer (2020) 5. Nestlinger, G. et al.: Virtual concept development on the example of a motorway chauffeur, control strategies for advanced driver assistance systems and autonomous driving functions. In: Waschl H., Kolmanovsky I., Willems F. (eds.) Control Strategies for Advanced Driver Assistance Systems and Autonomous Driving Functions. Lecture Notes in Control and Information Sciences, vol. 476. Springer, Cham (2019) 6. Tarkiainen M., Penttinen M., Sahimäki S., Sözen E., Kauvo K.: Automated driving and HMI design for city bus and truck with professional drivers. TP1798. Paper presented at 13th ITS European Congress, Brainport Eindhoven, Netherlands (2019)

Assessment Concept for TrustVehicles Philipp Clement, Herbert Danzinger, Philipp Quinz, Bernhard Hillbrand, Ahu Ece Hartavi, and Kübra Zehra Kasikci

Abstract The present chapter discusses the ability of a moving driver simulator to assess the trust of drivers in conditionally automated driving functions. This process includes extracting the right scenarios, the method for creating objective assessment criteria, and the technical solution of a simulator setup. The linking of different sensors and hardware interfaces, as well as time synchronisation, are discussed. Limitations of a driver simulator are analysed. The examined study of 60 participants, including the testing method and its strengths and weaknesses, is described and critically reflected. Keywords Driver simulator · Scenario extraction · Objective assessment tool · Human measurement · Biosensors · Simulator study · Automated driving · Conditionally automation · Assessment of trust

P. Clement (B) · H. Danzinger · P. Quinz AVL List GmbH, Graz, Austria e-mail: [email protected] H. Danzinger e-mail: [email protected] P. Quinz e-mail: [email protected] B. Hillbrand Virtual Vehicle Research GmbH, Graz, Austria e-mail: [email protected] A. E. Hartavi · K. Z. Kasikci University of Surrey, Guildford, UK e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Watzenig and L. Schicker (eds.), Enhanced Trustworthiness and End User Acceptance of Conditionally Automated Vehicles in the Transition Period, Lecture Notes in Intelligent Transportation and Infrastructure, https://doi.org/10.1007/978-3-030-60861-3_6

107

108

P. Clement et al.

1 Introduction How can you measure trust in conditionally automated vehicles? One answer is: Exposing human drivers to critical driving situations, where automated driving (AD) functions (partly) take control and assessing their experience by using an appropriate method. This method of providing “input” to the driver and measuring the “output” is called Driver-in-the-Loop (DiL). To ensure both safe and stable conditions during the experiment, studies are usually conducted in a driving simulator rather than in a real vehicle. In the funded project TrustVehicle [1], AVL’s advanced ADAS driving simulator (Fig. 1) has been used. It is mounted on a hexapod base and thus can reproduce the original vehicle kinematics, contributing significantly to the impression of sitting in a real vehicle. Experiments involving human drivers have two constraints: (1) The limited availability of the test subjects, together with the limited duration of the experiment. Thus, it requires a careful selection of the scenarios to be tested. The methodology of selecting relevant scenarios is described in detail in this chapter. (2) The sensitivity of the recorded data, especially when using biometric sensors to measure the physiological response of the drivers. Proper test design is required, which matches the simulator setup with the scenarios to be tested, as well as with the research questions. This setup includes the selection of sensors for the biometric questions, the preparation of questionnaires for subjective feedback and integration of the simulation output. The coupling of the simulation with the selected vehicle/cockpit, the setup of the recording and the time synchronization of all input signals and availability for the post-processing and data analytics. Depending on the amount and the nature of the recorded data,

Fig. 1 AVL’s advanced ADAS driving simulator used during the TrustVehicle experiments

Assessment Concept for TrustVehicles

109

the data storage might become a crucial point. In the context of TrustVehicle, the European General Data Protection Regulation (GDPR) [2] has been applied. The limitations of a DiL are hardware constraints in the movement (acceleration and distance) and the quality of the simulation models in the co-simulation framework. This framework consists of the vehicle, the driving function, the Humanmachine-interface (HMI) and the driving behaviour models (smoothness and realism of the simulation). In addition, an adverse effect might be the driver’s awareness of interacting in a safe environment. This might influence the results of assessments on a DiL and can affect both critical and safety-relevant manoeuvres. The benefit of being on a DiL is that especially these manoeuvres can safely be tested in such an environment with the highest reproducibility without high risks of harming the drivers. The study used ten scenarios, each participant had to pass. Therefor each participant spent about one hour in the simulator, depending on the time they spent on answering the questionnaires. For the assessment of the manoeuvres, AVL-DRIVE™ AD [3]—the evaluation tool for comfort, safety, validation and verification—was extended to support the scenarios defined for the study.

2 Critical Scenarios and Testing Possibilities for Conditionally Automated Vehicles In the context of TrustVehicle, critical scenarios are defined as scenarios, which are especially delicate to handle by Level 3 Autonomous Driving (L3AD) functions [4]. Such scenarios typically involve vulnerable road users (VRUs) or rough weather conditions, such as heavy rain or fog. In order to solve problems evolving from such scenarios, they are further divided into two classes: (i) critical scenarios solvable through enhanced automated driving controllers; (ii) critical scenarios solvable through a safe hand-over process to the driver. Enhanced automated driving controllers, apart from the controllers themselves are also including enhanced sensor systems (e.g. self-cleaning sensors) and sensor fusion. Scenarios solvable through enhanced L3AD systems (controllers and sensors) typically involve light disturbances of the sensors (e.g. dust, light rain) or common traffic scenarios that do not require intuitive reactions of the driver. In these cases, sensor redundancy or sensor monitoring, and cleaning can enhance the input to the L3AD function. Therefore, the functions itself can be improved using the enhanced input. These improvements can solve critical scenarios but are dependent on the limitations of the considered L3AD function. According to SAE standards, the functionality of L3AD systems is limited to well-defined operational design domains (ODD), which can be either geographic, roadway specific, environmental, traffic or speed specific and/or temporal limitations. SAE International, formally known as Society of Automotive Engineers, provides industrial standards with worldwide acceptance. [5] The lack of sufficient consideration of the respective system limits

110

P. Clement et al.

can cause severe accidents (e.g. the fatal accident involving a Tesla [6]) and, therefore, should not be neglected when speaking of critical scenarios. In scenarios where a L3AD function meets its system limits, the situation can only be resolved by a safe hand-over process to the driver. The hand-over process shall be clearly defined and intuitive so that the human driver is able to take over the vehicle control seamlessly, even in a possibly stressful situation. For both categories, it is crucial to test a comprehensive set of critical scenarios, so the level 3 functionality can be validated.

2.1 The Determination Process of Critical Scenarios for Trusted Conditionally Automated Vehicles The absence of a well-established methodology for identifying and assessing critical scenarios is recognized as a significant challenge in regard to the validation of automated vehicles [7]. As a result, most research effort is placed on developing validation methods and procedures. The aim of TrustVehicle is to contribute to this existing gap. Mainly focusing on the development of a standard process that can be used to determine critical transition scenarios that on the one hand focus exclusively on L3AD, and on the other hand encompass four different vehicle types ranging from passenger vehicle up to commercial vehicles (van, bus, truck). The sudden and relatively fast increase in the popularity of automated vehicles has led to a lot of different struggles to the all fields of vehicle development. Before moving on to outline the development of the TrustVehicle process, all related terminology will be defined.

2.2 Critical Transition Scenario Taxonomy Where possible, the SAE-J3016 standards are adhered, as they have been developed by major stakeholders and are generally accepted as conventional [1]. General technical terms on the scenario development process have been adapted to the context of level 3 automated driving and automation-initiated vehicle to driver transition. The definition of the following terms are well described in the following references [5, 7–10] and illustrated within Fig. 2: (1) Automated Driving Systems (ADS), (2) Dynamic Driving Task (DDT), (3) Operational Design Domain (ODD), (4) TakeOver (TO), (5) System Limit, (6) Time Window, (7) Take-Over Time (TOT), (8) Fall Back. To generate a set of tests for the evaluation of trust with participants in a study, the following terms are used to describe the link between the single tests. Use Case Cluster Use Case Cluster is a group of similar automated driving system-related use cases (Fig. 3).

Assessment Concept for TrustVehicles

111

Fig. 2 Illustration of automated driving system-related terms

Fig. 3 Illustration of technical terms on the TrustVehicle scenario development process

Use Case Use Case is a functional description for the conventional or automated driving system and its desired behaviour for specific usage (Fig. 3). Scenario A scenario is defined as a subset of all the possible variations of a Use Case and describes precisely one of them. This is also known as a test case (Fig. 3). Scene/Sequence A snapshot of the automated driving system environment which contains all actors, conditions and the relationships between them. It is a sequence in time of a Scenario (Fig. 3).

112

P. Clement et al.

2.3 Development of Scenarios Statistical analysis has shown that 97% of all the accidents are due to human error. Hence, the goal of higher levels of automation is to enhance road safety while improving driving comfort [11]. Highly automated vehicles have to fulfil very high functional safety requirements (e.g. random hardware failure rates