Advances in Service and Industrial Robotics: RAAD 2021 3030752585, 9783030752583

Table of contents :
Preface
Organization
Program Chair
Program Committee
Contents
Mechanical Design and Kinematics
Design and Analysis of 2 DOF Elbow Prosthesis
1 Introduction
2 Design Methodology of the Prosthesis
3 Actuator’s Prosthesis Selection
4 Design of the Transmissions
4.1 Design of the Worm Drive
4.2 Design of the Pinion Crown Transmission
5 Simulation and Results
6 Conclusions
7 Acknowledgement
References
Design and Preliminary Testing of a Novel Semi-automatic Saffron Harvesting Device
1 Introduction
2 Functional Analysis and Design Requirements
3 Conceptual Design
4 Dimensional and Mechanical Design
4.1 3D Modeling
4.2 Cam and Leaf-Spring Design
5 Preliminary Experimental Test
5.1 3D Printing and Assembly
5.2 Preliminary Experimental Validation
6 Conclusion
References
The Stiffness Model of the Compliant Positioning Mechanism with Watt’s Linkage
1 Introduction
2 XY Positioning Stage
2.1 Stiffness Model
2.2 FEM Analysis and Comparison of Results
3 Conclusion
References
Workspace and Singularity Zones Analysis of a Robotic System for Biosamples Aliquoting
1 Introduction
2 Robotic System Model Design
3 Algorithm Synthesis for Workspace Determination
4 Simulation Results
5 Conclusion
References
Preliminary Observations for Functional Design of a Mobile Robotic Manipulator
1 Introduction
2 Robot Mobility
3 Task Oriented Optimization
4 Conclusions
References
Development of a Low-Cost Force Sensory System for Force Control via Small Grippers of Cooperative Mobile Robots Used for Fabric Manipulation
1 Introduction
2 Problem Statement
3 The Force Sensory System (FSS) and the Control Approach
3.1 Development and Experimental Testing of the FSS
3.2 Control Approach in a Leader-Follower (L-F) Formation for Fabric Transfer
4 Experimental Validation
5 Conclusions
References
Industry Robotics
Extended State Automata for Intuitive Robot Programming
1 Introduction
2 Related Work
3 Automata Model
4 Programming System
5 Evaluation
6 Conclusion
References
Identification of a UR5 Collaborative Robot Dynamic Parameters
1 Introduction
2 Mathematical Model of the UR5 Collaborative Robot
2.1 Regressor Reduction
2.2 Excitation Trajectory
3 Results
3.1 Algorithm Validation
3.2 Effect of Temperature on Dynamic Parameters
4 Conclusions
References
Assessment of Universal Robot Force Control and External Force Compliance Device for Surface Treatment
1 Introduction
2 Setup
3 Test Scenarios
3.1 Ramp Profile
3.2 Straight Profile
4 Test Evaluation
5 Force Control Analysis
6 Conclusion
References
Kinematic and Dynamic Assessment of Trunk Exoskeleton
1 Introduction
2 Materials and Methods
2.1 Exoskeleton
2.2 Experimental Tests
3 Results and Discussion
4 Conclusion
References
Development of a Franka Emika Cobot Simulator Platform (CSP) Dedicated to Medical Applications
1 Introduction
2 Kinematic Modeling
3 Dynamic Modeling
4 Conclusion
References
Quality Inspection of Household Machines Using Collaborative Robot
1 Introduction
2 Robot Inspection Cell
3 Torque Sensing Problems
4 Typical Inspection Tasks
4.1 Button Pressing Test
4.2 Button Turn Test
5 Conclusions
References
Learning and Vision
Accelerating Robot Reinforcement Learning with Accumulation of Knowledge
1 Introduction
2 Accumulation of Knowledge Using Neural Network
3 Trajectory Encoding and Learning
4 Experimental Evaluation
4.1 Experimental Setup
4.2 Task
4.3 Methodology
5 Results
6 Conclusion
References
Detection of Fungus in Gladiolus Fields Using a Quadcopter
1 Introduction
2 Related Work
3 Proposed Methodology
3.1 Dataset Collection
3.2 Frame Extraction and Ground Truth Generation
3.3 Image Processing and Plants Extraction
3.4 Learning and Classification
4 Experiments and Results
5 Conclusion
References
Angular Dependency of the Zero Moment Point
1 Introduction
2 The Zero Moment Point
2.1 The Standard Cartesian Definition
2.2 The Angular Definition
3 Simulations
4 Discussion
5 Conclusion
References
Marker-Based Automatic Dataset Collection for Robotic Vision System
1 Introduction
2 Motivation and Experimental Setup
3 Workflow of the Robotic Vision Module (RVM)
3.1 Algorithm for Marker-Based Automated Dataset Collection and Object Recognition
3.2 Workflow of the RVM
4 Experiments
5 Conclusion and Discussion
References
Rehabilitation Robotics and Exoskeletons
Design of a Cable-Driven Parallel Robot for the Cervical Spine Motion Training
1 Introduction
2 Head Joints Range of Motion
3 Design of the End-Effector and CDPR Structure Selection
4 Kinetostatic Model of the CDPR
5 Results and Discussions
6 Conclusion
References
Design of a Cable-Driven Robot for Elbow and Wrist Rehabilitation
1 Introduction
2 Upper Limb Motion and Requirements
3 Cable Driven Device for Upper Limb Motion Assistance
4 Simulation and Performances Evaluation
5 Lab Prototype and Tests
6 Conclusion
References
Controllable Passive Transtibial Prosthesis - Design
1 Introduction
2 Prosthesis Design
2.1 General Presentation
2.2 Foot Construction
2.3 Universal Ankle Joint
2.4 Magnetorheological Stop-Valve
2.5 Sensorial and Control System
3 FEM Analyses of Main Elements
4 Conclusions
References
A Study of Feasibility for a Novel Cable-Driven Upper-Limb Exoskeleton with Magnetic Balancing
1 Introduction
2 Requirements for the Proposed Device
2.1 Desired Motions
2.2 Conceptual Design of a Novel Cable-Driven Upper-Limb Exercising Device
3 Energy Storage and Gravity Compensation System
4 Integration of the Magnetic-Spring System
5 Static Analysis
6 Conclusions
References
Robotic-Assisted Platform for Spinal Surgery: A Preliminary Study
1 Introduction
2 Experimental Setup
3 Preliminary Results
4 Conclusion
References
Using a Robot Calibration Approach Toward Fitting a Human Arm Model
1 Introduction
2 Materials and Methods
3 Results
4 Discussions
5 Conclusions
References
Feasibility Study of a Passive Pneumatic Exoskeleton for Upper Limbs Based on a McKibben Artificial Muscle
1 Introduction
2 Materials and Methods
2.1 Original Design of the Exoskeleton and PAM Modeling
2.2 Cam-Based Design of the Transmission System
3 Results and Discussion
4 Conclusion
References
Control
Adapting Behaviour of Socially Interactive Robot Based on Text Sentiment
1 Introduction
2 State-of-the-Art
3 Implementation
3.1 Textual Communication
3.2 Sentiment Analysis
3.3 Workflow
4 Experimentation and Evaluation
5 Conclusion
References
Modular Real-Time System for Upper-Body Motion Imitation on Humanoid Robot Talos
1 Introduction
2 Humanoid Robot Talos
2.1 Software Architecture
2.2 Body Tracking
3 Imitation Process
3.1 Joint Calculation
3.2 Safety Processing
4 Experimental Evaluation
5 Conclusions
References
Cloud-Enabled Bi-manual Collaborative Robot with Enhanced Versatility for Customized Production
1 Introduction
2 Progress Beyond State-of-the-Arts
3 Mechanical Designs
4 Intelligent Control Architecture
5 Use Case
6 Conclusion
References
Multi-objective Trajectory Tracking Optimization for Robots with Elastic Joints
1 Introduction
2 Problem Statement
3 Proposed Approach
3.1 Motion Control and Planning
3.2 Multibody Dynamics Simulation
3.3 Control Optimization Using Genetic Algorithms
4 Simulation Results
5 Conclusion
References
Stabilization of Mobile Weapon Aiming
1 Introduction
2 Vector Bases
3 Initial Orientation
4 Incremental Change of Orientation
5 Discrete Orientation Changes
6 Geo Coordinates Stabilization
7 Conclusion
References
Using the Digital Twin Concept to Design and Validate a Robotized Bin Picking Process
1 Introduction
2 Application Description
3 Collision Free Handling Strategy
4 RobotStudio Implementation
5 Conclusions and Future Work
References
Preliminary Study of Self-adaptive Constrained Output Iterative Learning Control for Robotic Manipulators
1 Introduction
2 Problem Formulation
3 Self-adaptive Constrained Output ILC
4 Simulation Results
5 Conclusion
References
Author Index

Citation preview

Mechanisms and Machine Science

Saïd Zeghloul Med Amine Laribi Juan Sandoval   Editors

Advances in Service and Industrial Robotics RAAD 2021

Mechanisms and Machine Science Volume 102

Series Editor Marco Ceccarelli , Department of Industrial Engineering, University of Rome Tor Vergata, Roma, Italy Advisory Editors Sunil K. Agrawal, Department of Mechanical Engineering, Columbia University, New York, USA Burkhard Corves, RWTH Aachen University, Aachen, Germany Victor Glazunov, Mechanical Engineering Research Institut, Moscow, Russia Alfonso Hernández, University of the Basque Country, Bilbao, Spain Tian Huang, Tianjin University, Tianjin, China Juan Carlos Jauregui Correa, Universidad Autonoma de Queretaro, Queretaro, Mexico Yukio Takeda, Tokyo Institute of Technology, Tokyo, Japan

This book series establishes a well-defined forum for monographs, edited Books, and proceedings on mechanical engineering with particular emphasis on MMS (Mechanism and Machine Science). The final goal is the publication of research that shows the development of mechanical engineering and particularly MMS in all technical aspects, even in very recent assessments. Published works share an approach by which technical details and formulation are discussed, and discuss modern formalisms with the aim to circulate research and technical achievements for use in professional, research, academic, and teaching activities. This technical approach is an essential characteristic of the series. By discussing technical details and formulations in terms of modern formalisms, the possibility is created not only to show technical developments but also to explain achievements for technical teaching and research activity today and for the future. The book series is intended to collect technical views on developments of the broad field of MMS in a unique frame that can be seen in its totality as an Encyclopaedia of MMS but with the additional purpose of archiving and teaching MMS achievements. Therefore, the book series will be of use not only for researchers and teachers in Mechanical Engineering but also for professionals and students for their formation and future work. The series is promoted under the auspices of International Federation for the Promotion of Mechanism and Machine Science (IFToMM). Prospective authors and editors can contact Mr. Pierpaolo Riva (publishing editor, Springer) at: [email protected] Indexed by SCOPUS and Google Scholar.

More information about this series at http://www.springer.com/series/8779

Saïd Zeghloul Med Amine Laribi Juan Sandoval •

•

Editors

Advances in Service and Industrial Robotics RAAD 2021

123

Editors Saïd Zeghloul University of Poitiers SP2MI - Site du Futuroscope Poitiers Cedex 9, France

Med Amine Laribi University of Poitiers SP2MI - Site du Futuroscope Poitiers Cedex 9, France

Juan Sandoval University of Poitiers SP2MI - Site du Futuroscope Poitiers Cedex 9, France

ISSN 2211-0984 ISSN 2211-0992 (electronic) Mechanisms and Machine Science ISBN 978-3-030-75258-3 ISBN 978-3-030-75259-0 (eBook) https://doi.org/10.1007/978-3-030-75259-0 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

The 30th International Conference on Robotics in Alpe-Adria-Danube Region, RAAD 2021, is held in the Futuroscope Technopole, the CNAM-IFMI building of the University of Poitiers, France, June 21–23, 2021. The conference offers an international and cooperative platform to exchange and discuss recent research results and future trends about robotic technology, engineering, and science. Researchers are from 15 countries, mostly affiliated to the Alpe-Adria-Danube Region, as well as researchers from extra-European countries. This year, researchers from Mexico, USA, Taiwan, and Tunisia will participate in the conference. RAAD represents an event on robotics that is organized yearly by a community of robotic specialists in the Alpe-Adria-Danube Region of Europe. The first event was held in 1992, organized by the Jozef Stefan Institute, Ljubljana, Slovenia. It was named the 1st International Meeting on Robotics in Alpe-Adria Region. In the following years, the event was named International Workshop on Robotics in Alpe-Adria Region (RAA) until 1996, when the denomination was changed into International Conference on Robotics in Alpe-Adria-Danube Region (RAAD). According to its tradition, RAAD 2021 covers all major areas of research, development, and innovation in robotics, including new applications and trends related to new theories, advanced design of robot mechanics and control architectures, and development of intelligent robotic applications. In relation to these topics, we are very proud that two renowned keynote speakers present the current state as well as future innovations in their main research areas. They are Carl Nelson from University of Nebraska-Lincoln (Perspectives on 15 years of medical robotics research) and Kosta Jovanović from University of Belgrade (Robotic Innovations in Support of the Healthcare Workers against COVID-19 – DIHHERO perspective). After a careful and accurate peer review process, with at least three reviews for each paper, 30 papers, authored by researchers from the RAAD community but also from other European and extra-European countries, were considered suitable for publication in this book. The evaluation process has considered the relevance, novelty, and clarity of the papers which guaranteed the high-quality level of this v

vi

Preface

collection. We would like to thank all the reviewers for allocating their valuable time and effort for their outstanding work in reviewing the assigned papers. We are also very grateful to all the authors for their effort and excellent work to improve their papers. According to the planned conference sessions, this book is organized in five chapters, covering the major research areas of the conference: I. II. III. IV. V.

Mechanical design and kinematics Industry robotics Learning and vision Rehabilitation robotics and exoskeletons Control

We thank the University of Poitiers, in particular the Fundamental and Applied Science Faculty, for its availability to host the RAAD 2021 event. We also acknowledge the support of the International Federation for the Promotion of Mechanism and Machine Science (IFToMM, http://iftomm.net/). The conference received generous support from local sponsors, namely the University of Poitiers, the Grand Poitiers, and the Nouvelle-Aquitaine region, which were crucial to make this conference possible. Unfortunately due to the COVID-19 pandemic situation and the related travel restrictions and social distancing requirements, the in-person meeting is very limited and both in-person and virtual attendance were combined during RAAD 2021. This year, as well as the last year, has been very affected by the COVID-19 pandemic. Thus, some papers that were not presented from the previous edition have been included to the technical program to be presented with the planned 30 papers of RAAD 2021. We thank the publisher Springer and its editorial staff for accepting and helping in the publication of this proceedings volume within the book series on Mechanism and Machine Science (MMS). May 2021

Saïd Zeghloul Med Amine Laribi Juan Sandoval

Organization

Program Chair Med Amine Laribi

Université de Poitiers, France

Program Committee Juan Sebastián Sandoval Arévalo Carlo Ferraresi Alessandro Gasparetto Marco Ceccarelli Giuseppe Carbone

Université de Poitiers, France Politecnico di Torino, Italy University of Udine, Italy University of Rome Tor Vergata, Italy University of Calabria, Italy

vii

Contents

Mechanical Design and Kinematics Design and Analysis of 2 DOF Elbow Prosthesis . . . . . . . . . . . . . . . . . . R. Hernández-Cerero, C. R. Torres San Miguel, J. A. Leal-Naranjo, and M. Ceccarelli Design and Preliminary Testing of a Novel Semi-automatic Saffron Harvesting Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alessandro Denarda, Andrea Manuello Bertetto, Doina Pisla, and Giuseppe Carbone The Stiffness Model of the Compliant Positioning Mechanism with Watt’s Linkage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jaroslav Hricko and Stefan Havlik Workspace and Singularity Zones Analysis of a Robotic System for Biosamples Aliquoting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dmitry Malyshev, Larisa Rybak, Giuseppe Carbone, Tatiana Semenenko, and Anna Nozdracheva Preliminary Observations for Functional Design of a Mobile Robotic Manipulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Luca Carbonari, Luigi Tagliavini, Andrea Botta, Paride Cavallone, and Giuseppe Quaglia Development of a Low-Cost Force Sensory System for Force Control via Small Grippers of Cooperative Mobile Robots Used for Fabric Manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ioannis D. Dadiotis, John S. Sakellariou, and Panagiotis N. Koustoumpardis

3

13

23

31

39

47

ix

x

Contents

Industry Robotics Extended State Automata for Intuitive Robot Programming . . . . . . . . . Lukas Sauer and Dominik Henrich

61

Identification of a UR5 Collaborative Robot Dynamic Parameters . . . . . Andrea Raviola, Andrea De Martin, Roberto Guida, Stefano Pastorelli, Stefano Mauro, and Massimo Sorli

69

Assessment of Universal Robot Force Control and External Force Compliance Device for Surface Treatment . . . . . . . . . . . . . . . . . . . . . . . Stefan Gadringer, Hubert Gattringer, and Andreas Mueller Kinematic and Dynamic Assessment of Trunk Exoskeleton . . . . . . . . . . Elisa Panero, Margherita Segagliari, Stefano Pastorelli, and Laura Gastaldi Development of a Franka Emika Cobot Simulator Platform (CSP) Dedicated to Medical Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Amir Trabelsi, Juan Sandoval, Moncef Ghiss, and Med Amine Laribi

78 86

95

Quality Inspection of Household Machines Using Collaborative Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 Leon Žlajpah and Samo Gazvoda Learning and Vision Accelerating Robot Reinforcement Learning with Accumulation of Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Zvezdan Lončarević and Andrej Gams Detection of Fungus in Gladiolus Fields Using a Quadcopter . . . . . . . . . 127 Muhammad Hassan Zaheer, Syed Atif Mehdi, Hannan Ejaz Keen, and Karsten Berns Angular Dependency of the Zero Moment Point . . . . . . . . . . . . . . . . . . 135 Tilen Brecelj and Tadej Petrič Marker-Based Automatic Dataset Collection for Robotic Vision System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 Denis Chikurtev and Kaloyan Yovchev Rehabilitation Robotics and Exoskeletons Design of a Cable-Driven Parallel Robot for the Cervical Spine Motion Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Alizée Koszulinski, Eliot Stantinat, Léa Ollivier, Juan Sandoval, and Med Amine Laribi

Contents

xi

Design of a Cable-Driven Robot for Elbow and Wrist Rehabilitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 Axel Fort, Octavie Somoza Salgado, Med Amine Laribi, Juan Sandoval, and Marco Ceccarelli Controllable Passive Transtibial Prosthesis - Design . . . . . . . . . . . . . . . 176 Ionel Cristian Vladu, Florina Cristian Pană, Cristian Copiluși, Iulian Petrișor, Ileana Vladu, and Nicu George Bîzdoacă A Study of Feasibility for a Novel Cable-Driven Upper-Limb Exoskeleton with Magnetic Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . 184 Jhon F. Rodríguez-León, Daniele Cafola, Brandon Suarez, Eduardo Castillo-Castañeda, and Giuseppe Carbone Robotic-Assisted Platform for Spinal Surgery: A Preliminary Study . . . 192 Térence Essomba, Med Amine Laribi, Juan Sandoval, Chieh-Tsai Wu, and Saïd Zeghloul Using a Robot Calibration Approach Toward Fitting a Human Arm Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 Valerio Cornagliotto, Elisa Digo, and Stefano Pastorelli Feasibility Study of a Passive Pneumatic Exoskeleton for Upper Limbs Based on a McKibben Artificial Muscle . . . . . . . . . . . . . . . . . . . . . . . . . 208 Stefania Magnetti Gisolo, Giovanni Gerardo Muscolo, Maria Paterna, Carlo De Benedictis, and Carlo Ferraresi Control Adapting Behaviour of Socially Interactive Robot Based on Text Sentiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 Sarwar Hussain Paplu, Mohammed Nazrul Islam Arif, and Karsten Berns Modular Real-Time System for Upper-Body Motion Imitation on Humanoid Robot Talos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 Kristina Savevska, Mihael Simonič, and Aleš Ude Cloud-Enabled Bi-manual Collaborative Robot with Enhanced Versatility for Customized Production . . . . . . . . . . . . . . . . . . . . . . . . . . 240 Aleksandar Rodić, Jovan Šumarac, Ilija Stevanović, and Miloš Jovanović Multi-objective Trajectory Tracking Optimization for Robots with Elastic Joints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 Werner Ainhauser, Johannes Gerstmayr, and Andrea Giusti Stabilization of Mobile Weapon Aiming . . . . . . . . . . . . . . . . . . . . . . . . . 259 Karol Dobrovodský and Pavel Andris

xii

Contents

Using the Digital Twin Concept to Design and Validate a Robotized Bin Picking Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266 Silviu Răileanu, Florin Anton, Theodor Borangiu, and Silvia Anton Preliminary Study of Self-adaptive Constrained Output Iterative Learning Control for Robotic Manipulators . . . . . . . . . . . . . . . . . . . . . . 275 Kaloyan Yovchev and Lyubomira Miteva Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

Mechanical Design and Kinematics

Design and Analysis of 2 DOF Elbow Prosthesis R. Hernández-Cerero1 , C. R. Torres San Miguel1(B) , J. A. Leal-Naranjo2 , and M. Ceccarelli3 1 Escuela Superior de Ingeniería Mecánica y Eléctrica, Instituto Politécnico Nacional, SEPI.

Unidad Zacatenco, 07738 Mexico City, Mexico [email protected] 2 School of Engineering, University of Liverpool, Brownlow Hill, Liverpool L69 3GH, UK 3 Laboratory of Robot Mechatronics, Department of Industrial Engineering, Italy University of Rome Tor Vergata, 00133 Rome, Italy

Abstract. This paper presents a 2 DOF elbow prosthesis design and analysis, which includes developing a worm gear transmission, adapting a torsional spring for the flexo-extension movement, and a crown pinion transmission with two torsional springs for the pronosupination movement. A set of dynamic simulations was conducted to determine how feasible the proposed mechanism is and the torque needed for daily activities’ performance. The results show the physical and kinematic characteristics of the designed system. Keywords: Biomechanics · Prosthetic design · Flexible mechanism · Elbow prosthesis · Torsional springs · Elbow joint

1 Introduction The principal challenges faced by superior limb prosthetic researchers and designers prosthetic devices feel that the replacement limb is a natural part of the patient. Researchers have another challenge that summarises a critical component with a viable approach for an anthropomorphic robotic arm. They deal with this obstacle to adapt the actuators that provide the strength, power, and tension comparable to those of the musculoskeletal system [1, 5]. Examples include the DEKA arm [2], the RIC arm [3] and the others mentioned in Table 1. Table 1. Arm prosthesis models. Name

DOF

Movements

Weight (kg)

Driving

Torque (Nm)

Actuator

Imitation Transhumeral prosthesis [1]

2

Flexo-extension, prono-supination

1.2

Worm ring

10.58

Dc

Pneumatic Transhumeral Prosthesis [6]

1

Flexo-extension

2

Pneumatic

30

Pneumatics

(continued) © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 3–12, 2021. https://doi.org/10.1007/978-3-030-75259-0_1

4

R. Hernández-Cerero et al. Table 1. (continued)

Name

DOF

Movements

Weight (kg)

Driving

Torque (Nm)

Actuator

MoBio Transhumeral [2]

2

Flexo-extension, prono-supination

3.2

Harmonic gears

12

Dc

Contreras Elbow [1]

1

Flexo-extension

1.7

Gear train

2.4

Servo motors

CINVESTAV Prosthetic Elbow [2]

2

Flexo-extension, prono-supination

1.05

2 actuators

4

Dc

Multi-DoF transhumeral [6]

1

Flexo-extension

2.5

Harmonic gears

16

Dc

Nowadays, different intelligent actuators are based on lightweight materials. These have made a significant echo in mechanisms, robotics, aerospace, biomechanics and biomedical [4, 5]. Among all these new types of actuators, it can find springs [6], which are helpful to the different applications that their forms can provide [7]. They can be found in different types as compression, extension, and torsion, depending on their application. These can provide specific displacements [8]. Torsion springs display torque or rotational force. Their ends are attached to other components, and when they rotate around the centre of the spring, they always try to return to their original position [9]. Also, torsion springs are subjected to bending stress than torsional stress [10]. Like all springs, they store and release energy, in the particular case of torsional springs, the energy is angular, and they can keep a mechanism static in a particular position [11]. This type of spring is subjected to tight winding; it can also have a pitch between turns to reduce friction, and depending on the application is essential to design winding direction by clockwise or counterclockwise [12]. As far as is known, very few research works have been developed related to the design of prosthetic devices whose design is based on the implementation of those actuators such as torsional springs. This argument leads to implementing a methodology to propose an elbow prosthesis design inspired by a flexible joint.

2 Design Methodology of the Prosthesis This work has been directed in part from the path laid down by the methodology proposed by Ozkaya and Nordin, used for the formulation of solutions to problems focused on biomechanics, which is illustrated in Fig. 1 [13].

Design and Analysis of 2 DOF Elbow Prosthesis

5

Fig. 1. Design methodology.

The design is divided into two parts. A hinge-type mechanism proposed for pronosupination is used as a functional arc of 80° in flexion-extension, providing one degree of freedom. It is used a worm drive and torsion spring in a free configuration (Fig. 2a-2b). The forearm’s pronosupination movement highlights 25°, using a pinion-crown and two torsional springs located at 90° to replicate the functional arch (Fig. 2c-2d).

Fig. 2. Elbow Prosthesis, a) Flexion movement, b) extension movement, c) supination movement, d) pronation movement.

The flexo-extension and pronosupination are based on gear and torsional springs, increasing load capacity, accumulating, and returning potential energy, and decreasing the DC torque. The complete design is made up of the pieces shown in Fig. 3 that is the conjuntion of the 2 mechanisms.

6

R. Hernández-Cerero et al.

Fig. 3. Mechanisms parts.

3 Actuator’s Prosthesis Selection The elbow torque is determinate by a mass ubicated in the hand-palm of 1 kg. A free body diagram was made, shown in Fig. 4, where the external forces interact with the designed prosthesis. The maximum weight “Pb” for the forearm is 1.12 kg, and “Pd” is the weight of 1 kg for the mass that will be moving by design. The reactions generated in point “A” are obtained, as shown in Fig. 5. It is calculated with Eq. 1, and the equilibrium equations of a rigid body in two dimensions.

Fig. 4. Elbow torque diagram

The spring’s material selection and design are based on the parameters such as inner diameter, outer diameter, and coil, but the first parameter to calculate this spring is to obtain the spring constant (M) using Eq. 1, where the force is the load, and the displacement is presented by (θ). M =F ∗θ

(1)

Design and Analysis of 2 DOF Elbow Prosthesis

7

Then, Eq. 2 can estimate the constant of the spring (K), where the parameter to use as a variable is a moment obtained in Eq. 1 and the displacement of one arm of the spring represented by (θ). K=

M θ

(2)

The wire diameter (d) is determined by Eq. 3 in where (S) establishes the deformation based on the material selections.  3 32 ∗ M d= (3) π ∗S The spring variables are determined considering its modulus of elasticity. In this case, the material selected is steel wire known as ASTM 228, whose elasticity is 2070 MPa. Table 2 shows torsional spring parameters for the flexo-extension movement, and Table 3 shows torsional spring parameters for the pronosupination movement [5], where the most critical variables are the (S) the deformation, (C) Spring index, the inner (Di) and the outer diameters (Do), the diameter of the wire (d), the number of turns (Nb), which is composed by the number of active turns (Na) and the number of external turns (Ne) meanwhile, the dependent variables are: the elastic spring constant (K). The moment (M) expressed in Nmm/° is relevant along with the angular displacement (θ) to the result of the simulation, where it is necessary to specify that the free-angle for flexo-extension spring is 160° and for pronosupination springs are 90° in Fig. 5 is shown the spring that was also calculated the displacement reached.

Table 2. Flexo-extension spring results

Table 3. Pronosupination springs results

Flexo-extension spring

Pronosupination springs

K

27 Nmm/°

K

d

2.6 mm

d

0.5 mm

0.5 mm

Di

53.04 mm

Di

7.32 mm

6.2 mm

Do

58.02 mm

Do

7.32 mm

6.2 mm

D

55.6 mm

D

6.82 mm

5.7 mm

C

21

C

13.64

11.4

Na

5.8

Na

12.92

12.92

Ne

1.14

Ne

0.42

0.42

Nb

4.6

Nb

13.3

13.3

S

1142 Mpa

S

10131.4 Mpa

10324.4 Mpa

1.56 Nmm/°

2.35 Nmm/°

8

R. Hernández-Cerero et al.

Fig. 5. a) Spring for flexo-extension b) Springs for pronosupination and range of movements

4 Design of the Transmissions 4.1 Design of the Worm Drive The transmission is composed of a worm drive. The procedure to determine the transmission design is Fecos gears. The idea is to propose a design whose power transmission system is internal and lets it perform flexion-extension. The prosthesis configuration is assimilated in this design as a hinge-type mechanism, thus providing a degree of freedom. This proposed design will be attached to the user’s stump through a socket and a harness that allows carry it. It is also intended that this model’s weight does not exceed 1 kg. After the calculus, it was determined design parameters shown in Fig. 6.

Fig. 6. Worm drive parameters

4.2 Design of the Pinion Crown Transmission Otherwise, in the reference point of the pronosupination movement, it has been sought the functional arch of 75° in pronation and 50° in supination, it was implemented a pair

Design and Analysis of 2 DOF Elbow Prosthesis

9

of torsional springs with a configuration of 90° each one, where the responsible spring for reproducing the supination has a turn counterclockwise. Meanwhile, the spring used to reproduce the pronation has clockwise turned; the mechanism consists of a pinion crown transmission where the pinion has 10 teeth, and the crown has 25 teeth, and the Fecos gears procedure was used to calculate these transmission gears. As it was shown in Fig. 7, where the complete pronosupination mechanism is visble.

Fig. 7. Pronosupination mechanism design

5 Simulation and Results The elbow is a hinge/pivot joint formed by the ulna’s proximal ends and the radius and the humerus’s distal end. The elbow is not a single joint. This means that the latter is a complex joint made up of three joints: ulna-humeral (or ulnar humeral), radio humeral and superior or proximal radio-ulnar. Functionally, it has been reasoned that the elbow is a single joint since everything is surrounded by a joint capsule in which the ligaments and synovial fluid are located. Two longitudinal axes have been assumed for study purposes, which form an angle with a variation of 0–20°, called the load angle. It has been considered that due to its complex bone anatomy, the elbow permits movement in two planes, which translates into 2 DOF, and these are Flexo-extension: This movement is performed by sliding and rolling articular surfaces, which are the surfaces of the humerus, radius and ulna, so its range or arc of movement is 0–150° and Pronosupination: Movement carried out through the radius’s sliding; it is a rotational movement around its longitudinal axis. It occurs in a mechanical association of the superior and inferior radioulnar joints, reaching values of 75° of pronation and 85° of supination. However, many daily activities are performed in a range of 100° in flex-extension and 50° in pronosupination. In terms of forces, the literature suggests that the humerusradial joint transmits 60% of the force. The compressive force is eight times the weight supported by the hand at 90°. In extension, in the elbow, there is a less mechanical advantage and less joint congruence. The parameters that must be considered for this design’s operation are a) the elbow is a hinge/pivot mechanism, b) the degrees of freedom, c) the displacements made by the joint, and d) the functional arches. The CAD design is based on a torsional spring supported by a cylindrical rack, and a worm drive generates the rotational movement, coupled with this assembly that gives the design action. This

10

R. Hernández-Cerero et al.

design based on a torsional spring increases forearm load capacity, and mechanical torque of the flexion and extension movement is reduced. Both mechanisms are shown in Fig. 8 where the authors present the final 3D model.

Fig. 8. 3D prosthesis model. A) Cross-Sections of flexo-extension, Cross-section of Pronosupination

It was determinate that the motor will provide 16 rpm. As a result, the force requirement of 2.0 N, considering the intervention of the torsional springs. The behaviour curves obtained through the Solidworks® show a set of simulations, where the spring was placed in different positions and with different positional points. Two experiments were carried out, in Fig. 9a the first graph does not consider the torsional spring and yielded a torque of 187 N mm, which translates into 0.18 Nm, with a contrast of 0.3 Nm proposed in the analytical one, in Fig. 9b refers to the experiment using the proposed spring with the previously calculated parameters expressed as 27 Nmm/° and the complementary angle of 160°. The spring’s behaviour can be appreciated, a torque of 125 Nmm and falls to a critical point at 32° because it is the point where the torque reaches zero. The second part of the stroke begins and shows that sustaining the proposed load’s mass requires a torque of 125 Nmm, which translates to 0. 12 Nm, showing a torque reduction of 33%. The hinge mechanism critical point is when the spring supports the load in extension, and the stress will appear when it passes to the flexion position. This is explicit in b) graphic of Fig. 9, where the torque line presents a bounce. The reason for this bounce is the performance of the spring in flexo-extension, where the spring reaches its limit of fatigue, meaning that the wire diameter is not the right one to perform its function or the number of turns is not enough to store and dissipate enough energy, so it performs the decrease of torque, but it does not support the effort of the performed function.

Fig. 9. Results from flexo-extension simulations: (a) Torque from the design without spring; (b) Torque with spring in flexo-extension

Design and Analysis of 2 DOF Elbow Prosthesis

11

6 Conclusions The innovation resides because this elbow prosthesis was thought to have 2 DOF, and the implementation of torsional springs is a fundamental part of its operation. In the case of the flexo-extension movement, it can be appreciated that it reduces the DC torque and supports the flexion load. Meanwhile, it can be visible the limit of the hinge mechanism designed in where the critical point is the position where the spring supports the load in extension, and the stress will appear when it passes to flexion position.

7 Acknowledgement The authors thankfulness to the Consejo Nacional de Ciencia y Tecnología (CONACyT), and the Instituto Politécnico Nacional for the support received in the 202110282 projects, as well EDI grant, all from to SIP/IPN.

References 1. Cowley, J., Resnik, L., Wilken, J., Smurr Walters, L., Gates, D.: Movement quality of conventional prostheses and the DEKA Arm during everyday tasks. Prosthet. Orthot. Int. 41(1), 3–40 (2017). https://doi.org/10.1177/0309364616631348 2. Leal-Naranjo, J.A., Ceccarelli, M., Miguel, C.R.T.S.: Mechanical design of a prosthetic human arm and its dynamic simulation. Adv. Intell. Syst. Comput. 540(November), 482–490 (2017). https://doi.org/10.1007/978-3-319-49058-8_52 3. Troncossi, M., Borghi, C., Chiossi, M., Davalli, A., Parenti-Castelli, V.: Development of a prosthesis shoulder mechanism for upper limb amputees: application of an original design methodology to optimize functionality and wearability. Med. Biol. Eng. Comput. 47(5), 523– 531 SPEC. ISS (2009). https://doi.org/10.1007/s11517-009-0465-9 4. Dietl, H.: Mechatronic integration in exoprosthetics. IFAC Proc. 33(26), 537–541 (2000). https://doi.org/10.1016/s1474-6670(17)39200-5 5. Solís-Santomé, A., et al.: Conceptual design and finite element method validation of a new type of self-locking hinge for deployable CubeSat solar panels. Adv. Mech. Eng. 11(1) (2019). https://doi.org/10.1177/1687814018823116 6. Rosyid, A., El-Khasawneh, B., Alazzam, A.: Gravity compensation of parallel kinematics mechanism with revolute joints using torsional springs. Mech. Based Des. Struct. Mach. 48(1), 27–47 (2020). https://doi.org/10.1080/15397734.2019.1619579 7. Klug, S., Von Stryk, O., Möhl, B.: Design and control mechanisms for a 3 DOF bionic manipulator. In: Proceedings First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, BioRob 2006, vol. 2006, pp. 450–454 (2006). https://doi.org/ 10.1109/biorob.2006.1639129 8. Norton, R.: Diseño de maquinaria sintesis y analisis de mecanismos. Worcester Polytechnic Institute Worcester, Massachusetts Revisión (2004) 9. Mott, R.L.: Diseño de Elementos de Máquinas (2006) 10. Paulista, U.E., Em, P.D.E.P., Biológicas, C.: DISEÑO EN INGENIERÍA MECÁNICA, Novena edi. McGRAW-HILL 11. Juan, P., Flores, A., No, P., Matemáticos, F.: “INGENIERIA Y TECNOLOGIAS AVANZADAS,” no. 0 (2020)

12

R. Hernández-Cerero et al.

12. Ortiz Domínguez, M., Cruz Avilés, A., Muños Sánchez, Y.: Diseño de máquinas, Ingenio y Concienc. Boletín Científico la Esc. Super. Ciudad Sahagún, vol. 5, no. 9 (2018). https://doi. org/10.29057/ess.v5i9.2904 13. Hernández-Acosta, M.A., Torres-San Miguel, C.R., Piña-Díaz, A.J., Paredes-Rojas, J.C., Aguilar-Peréz, L.A., Urriolagoitia-Sosa, G.: Numerical study of a customized transtibial prosthesis based on an analytical design under a flex-foot® variflex® architecture. Appl. Sci. 10(12) (2020). https://doi.org/10.3390/app10124275

Design and Preliminary Testing of a Novel Semi-automatic Saffron Harvesting Device Alessandro Denarda1 , Andrea Manuello Bertetto2 , Doina Pisla3 , and Giuseppe Carbone1(B) 1 DIMEG, Department of Mechanical, Energy and Management Engineering,

University of Calabria, 87036 Rende, Italy [email protected] 2 DIMEAS, Department of Mechanical and Aerospace Engineering, Politecnico di Torino, 10129 Turin, Italy 3 CESTER, Technical University of Cluj-Napoca, 400641 Cluj-Napoca, Romania

Abstract. This paper addresses the procedure for designing a novel mechatronic device for a semi-automatic saffron harvesting. The design process includes the kinematic analysis and synthesis of the proposed mechanism to fulfill a grasping and harvesting of saffron flower with a specific two-finger gripper design. Then, a cam mechanism and elastic spring elements are designed to provide a shaking movement that achieves the splitting of the flower from its steam and a consequent flower harvesting. This is followed by a suction system for collecting and storing the harvested flowers. A 3D printed prototype is reported and preliminarily tested for validating the engineering feasibility and effectiveness of the proposed design solution. Keywords: Mechatronic design · Saffron harvesting · Grasping

1 Introduction Saffron is an important edible spice, which is used in several regional and international meals [1]. The production of saffron requires a complex process, which is mostly handled by manual workers, who need to bend their back for a significant amount of time with high discomfort and pain. Multiple attempts have been made to automatize the harvesting process. Some examples are reported in [2–7], showing semi-automatic and automatic systems with focus at the design of a specific end-effector, which plays a key role in mechanical harvesting of horticultural products as mentioned for example in [8, 9]. In literature many studies are available on the design of the End-effectors design for robotics harvesting, but they are always designed for single-product applications [10–13]. This circumstance occurs again in mechanical harvesting of saffron, which is particularly challenging due to the high fragility of saffron flowers. This paper addresses the design of a novel mechatronic device for a semi-automatic saffron harvesting, which need to be simple and economical, with productivity comparable to a human operator, and consequently with low-costs, ergonomic, transportable, © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 13–22, 2021. https://doi.org/10.1007/978-3-030-75259-0_2

14

A. Denarda et al.

light, and autonomous, for upright position use. In particular, in the following Sects. 2 and 3 focus at the design requirements and conceptual design of proposed solution, respectively. Section 4 focuses at the design of a prototype. Finally, Sect. 5 shows the rapid prototyping and preliminary experimental validation to demonstrate the main functioning principles of the proposed device.

2 Functional Analysis and Design Requirements Table 1 reports a list of main requirements to be achieved with a proposal of solutions to fulfill them. The attached problem can be divided into two distinct parts. Namely, there should be a mechanical element (End-effector) that is able to make the cut on the stem without damaging the leaves, and an auxiliary device capable of allowing the movement and storage of the flowers. Once the flower has been separated from the plant, the second device will take the object from the work area and store it. Thanks to the great lightness of saffron flowers, it has already been seen how they are easily achievable through a suitable suction system. Placing the suction mouth near the work area of the mechanical element, the cut flower can be grasped by the air flow and transported inside the storage chamber. Finally, the operator must be able to directly manage the devices, as well as some parameters such as the power delivery of the aspirator. This presupposes the implementation of a small control unit, easily reachable by the operator’s fingers. Table 1. Functional analysis of the proposed device. What to do

How to do

Include the flower

Without damaging the surrounding ones

Stem cutting

The leaves must not be damaged.

Suction system

Flowers moving

Cut flower must be gripped without excessive stress.

Flowers storage

No compression during transport.

Control unit

Units managing

Operator must be able to manage the device parameters

End-effector

The main steps that will be followed to complete the entire design phase, and consequently define the tool, it can be summarized starting from the distinct definition of the two detachment systems and aspiration, up to the elements that will allow their interaction.

3 Conceptual Design The proposed device can be imagined like a backpack brush cutter, with a moving part that can be associated with a modern garden blower. The entire system could be composed of two distinct units, connected however to each other through flexible elements. A first element, transported manually, will have the shape and size suitable to allow the operator

Design and Preliminary Testing of a Novel Semi-automatic

15

to reach the flower without bending the back. At the lower end there will be both the End-effector and the mouth of the suction system. Through a flexible rubber tube, the cut flowers will be able to reach the storage area, and both the suction system and the tank could be housed in a shoulder-mounted structure. Total weight must not exceed 5 kg The device must also have a fully electric power supply system, which can be powered by a battery, capable to guarantee the autonomy for the entire harvesting phase, and to be protected from external agents like rain. Finally, the system must have an average production cost, to be accessible by small and medium-sized enterprises, which are the principal producers of saffron. Table 2 reports a functional analysis for the proposed device summarizing the main expected characteristics. Table 2. Functional analysis of the proposed device. Value

Notes

2–5 s/flower

Positioning, cutting, suction

Shape

Gardening tool

Manual unit and shoulder-mounted element

Weight

3–5 kg

Total weight

Implementation

Electrical

For suction system and end-effector

Power supply

18–40 V

Rechargeable electric battery

Autonomy

3–6 h

Half a working day

Materials

Aluminum – Plastic

Maybe rubber, fabric, or steel

Harvest time

Temperature

18 °C–28 °C

Room temperature

Work environment

Open field

Moisture, organic materials, rain

Production cost

300–500

Average cost range

A first constructive hypothesis could be represented by a simplification of the kinematics of the two-finger gripper presented in [7]. Two specular fingers with a suitably modeled terminal profile approach as the two elements are rotated, until they are close enough to guarantee the grip of the aerial parts of the plant. Once the rotation movement has reached the end of its stroke, and the flower will be correctly grasped, the gripping elements must rotate in order to subject the flower stem to a stress of traction. The simplest constructive choice can be represented by the exploitation of an elastic element, which releases all the accumulated energy at the appropriate time. As for the suction system, the power used must be contained within a predetermined range, since excessive stresses along the path could damage the product, and at the same time a too weak air flow could not guarantee the correct capture of the flower. Consequently, the sizing of the vacuum pumps will be carried out starting from a simplified geometric model of the flower, to estimate its floating speed. Subsequently, once the suction duct has been sized, it will be possible to evaluate the total head losses along the entire path, and therefore to trace the required pump model.

16

A. Denarda et al.

4 Dimensional and Mechanical Design 4.1 3D Modeling The sizing of the gripping elements with the relative handling was carried out based on the existing solution. Two mirror fingers, like those used in [7], are placed at the ends of two transmission shafts integral with each other by means of two gear wheels of the same module and pitch circumference. The surface of the two fingers has been modeled in such a way that during the rotation phase they remain at a fixed distance of 1.5 mm. A wheelbase regulation system allows to adapt this distance between the two grasping surfaces at the variability of stem diameter. The housing for the grasping elements consists of a front plastic element with two holes for the passage of the fingers, and a rear plate, which presents a revolute joint on the back side that will be used to create the hinge to generate the traction of the flower. A series of leaf springs are directly realized from an element in aluminum, that had also function of houses. A plastic external body was created to protect the End-effector internal components from external environment, essentially composed of two elements, which can be inserted one on top of the other. The external body of the End-effector will also be used to connect the instrument to the maneuver rod. The suction hood has been modeled as a separate element, connected to the external body by a hinge, and positioned exactly above the two gripping elements. On the back side, an adjustable length tie rod has been inserted. Figure 1 shows 3D CAD models with the main assembly for the proposed End-effector.

Fig. 1. Assembly for End-effector: a) Internal part assembly; b) External part assembly

Modeling of suction system was made starting from the geometry and dimensions of the main elements, such as vacuum pumps and batteries. the suction system can be considered composed of two distinct bodies: a head, inside which all the electronic elements will be assembled, as well as the outlet of the suction duct, and a storage compartment, which will represent the real tank of the system under consideration, with a capacity of 14 L. The two elements, which can be easily assembled by means of two toggle closures, will form the entire shoulder-mounted structure.

Design and Preliminary Testing of a Novel Semi-automatic

17

Gear Sizing. The distance between the central axis of the two shafts must be equal to 33.60 mm, and this leads us to define the primitive radius of the two mirror gears, equal to 16.8 mm. Two gears with number of teeth and module respectively equal to 48 and 0.7 were chosen. Considering the operation of the latter over a 180° arc as a data plate, to avoid positioning errors or too long times, it was decided to insert an additional gear that would allow to obtain a transmission ratio τ = 0.625 between shaft and motor, which will have a primitive radius of 10.5 mm, and a number of teeth equal to 36, and will also be integral with one of the two transmission shafts. The pinion connected directly to the servomotor will act on this last element. In this way it will be possible to limit the maximum travel of the servo to an arc of 112.5◦ , obtaining a rotation of 180◦ on the two fingers. Motor Sizing. The first servomotor can be sized starting from the simple equation of equilibrium to the rotation of the moving elements. Neglecting the former, the magnitude of the moments of inertia, evaluated once the 3D design phase has been completed, was found to be sufficiently low not to constitute a relevant factor for the choice of the servomotor. As for the sizing of the second servomotor, the force value imposed during the design phase of the elastic charging system was equal to 50 N, and consequently the choice of the servo can only fall on a model that is able to guarantee the necessary torque value, equal to 0.3 Nm. Then, it will suffice choose a pair of servomotors of the same model, which reflect the specifications imposed. Figure 2 shows 3D CAD models with the main assembly for the proposed Portable Collecting Device.

Fig. 2. Portable Collecting Device; a) front view; b) back view.

4.2 Cam and Leaf-Spring Design Starting from the strength or displacement value necessary to stress the stem of the flowers until they break, it would be possible to sizing the leaf springs. Unfortunately,

18

A. Denarda et al.

no data in this regard was available in the literature, and an experimental determination was impracticable. Consequently, it has been necessary to start from presumed values, hypothesized based on the information collected from the in-depth study of the plant in question and of the existing systems. A loading force of 50 N was chosen as a reference value, which was sufficient to generate an arrow of 1 mm on the free end. To ensure that the Bernoulli theory on the bent beam was respected, the best option was to insert several leaf springs, arranged in parallel, and to be considered as a single elastic body. From the definition of stiffness of the bent beam and through the expression of the arrow (Eq. 1), it was possible to go back through iterative calculations to the number of sheets needed, as well as to the values of thickness, length and width of the single sheet, considering how material an aluminum alloy that had an elastic modulus of 69000 Mpa The result led to the definition of a number of 5 springs, arranged in parallel, and with geometric dimensions equal to 3 × 15.2 × 90 mm as summarized in the 3D CAD model shown in Fig. 3.

Fig. 3. Aluminum element with leaf spring.

F =n

3EI y L3

(1)

The traction tension will be applied by a nylon thread, through an innovative cam system modeled appropriately for the proposed purpose (Fig. 4). With this solution, one end of the nylon thread will be connected to the main plate of the grip block, while the other will be rigidly connected to the structural case. In this way the wire, which at rest will be in a vertical position, will always have its axis tangent to the profile of the cam. Carrying out a first rotation of the cam of 90° counterclockwise, it is able to put the wire in traction for the time necessary to perform the gripping operation, then allowing its instantaneous release once traveled the next 90°. Once the trigger has been made, the servo will be able to return to its initial position without involving the wire in any way, indeed returning the cam to the condition of being able to re-engage it if the next operation is to be performed.

Design and Preliminary Testing of a Novel Semi-automatic

19

Fig. 4. Cam system: a) CAD model; b) wire hooking in the first handling phase; c) wire traction and leaf-spring loading; d) wire snap and elastic energy releasing.

5 Preliminary Experimental Test 5.1 3D Printing and Assembly Through 3D printing technique, a prototype of the End-effector was created by CAD models created during design phase. As a raw material, polylactic acid (PLA) was used for 3D printing such as proposed in [14]. Components such as servomotors, elastic plates, or connection systems such as screws, nuts and inserts, were found through the purchase, if the accidental availability was not present. 5.2 Preliminary Experimental Validation Much of the instrument’s efficiency strongly depends on the good calibration of the bending movement that the End-effector performs once the two fingers have grasped the stem of the flower. To investigate the dynamics of this movement in greater depth, paying particular attention to the accelerations reached following the release of the traction wire, it was decided to carry out experimental measurements on the prototype using an accelerometer. The FXLN8371Q analog triaxial accelerometer, manufactured by Freescale Semiconductor, Inc, with a selected sensitivity range of ±2 g was used as the sensor. The instrument was positioned on the front side of the End-effector (Fig. 5), in such a way as to obtain the x and y axes lying on the plane parallel to the front structural plate, with the outgoing z axis.

Fig. 5. Final prototype and accelerometer positioning.

20

A. Denarda et al.

Fig. 6. Plot of y-axis acceleration.

A series of measurements were carried out by putting the End-effector into operation, over a time interval that embraced the entire movement phase of the two servomotors. In this way it was possible to obtain the data relating to the accelerations that develop on the gripping system, following traction with consequent release of the wire by the second servomotor.

Fig. 7. Preliminary field tests on wild saffron flowers: a) approaching to a saffron flower; b) harvesting a saffron flower.

Important results were obtained on y-axis, Fig. 6, where it can be observed an acceleration peak that exceeds the value of ±2 g. This data is representative of the instant in which the cam releases the traction wire, allowing the gripping system to snap upwards. Preliminary field tests were performed on wild saffron flowers, Fig. 7, highlighting the need for a pre-industrial prototype, with optimized specifications and features compared to the current one laboratory version. Further experimental activities will be planned in future to systematically validate the effectiveness of the proposed device in terms of

Design and Preliminary Testing of a Novel Semi-automatic

21

grasping features and for achieving a performance optimization such as proposed in [15– 17]. Further field tests will be specifically planned to validate the achievable productivity rates with the proposed device.

6 Conclusion In this paper the design with the consequent realization of a semi-automatic system for the saffron flowers harvesting were described. In particular, in Sect. 2 is presented a functional analysis of the device that will be proposed, starting from a list of main tasks to be achieved by the system. In Sect. 3 the problem is divided into two distinct part, concerning respectively the End-effector for cut the saffron flowers and the shouldermounted suction system for collecting and storage them. In Sect. 4, the design phases led to an End-effector with two fingers, characterized by a mechanism capable of performing a rapid snap-folding movement on the gripping system, thanks to five leaf springs, preloaded by a nylon thread put in traction by an innovative cam system. Section 5 shows experimental validation of proposed device, carried out by an accelerometer, and preliminary test on field, which confirm conceptual design. Further experimental activities will be planned in future to validate the achievable productivity rates with the proposed device.

References 1. Diaz-Marta, A., Arghittu, G.L., Astrka, A., et al.: White Book Saffron in Europe Problems and Strategies for Improving the Quality and Strengthen Competitiveness. INTERREG IIIC (2006) 2. Ruggiu, M., Bertetto, A.M.: A mechanical device for harvesting crocus sativus (saffron) flowers. Appl. Eng. Agric. Am. Soc. Agric. Biol. Eng. 22(4), 491–498 (2006) 3. Antonelli, M.G., Auriti, L., Zobel, P.B., Raparelli, T.: Development of a new harvesting module for saffron flower detachment. Rom. Rev. Precis. Mech. Opt. Mechatron. 39, 163–168 (2011) 4. Bertetto, A.M., Ricciu, R.: Mechanization in Harvesting Saffron: An Opportunity for Economic Development in Sardinia. Advances in Business-Related Scientific Research Conference, Olbia (2012) 5. Asimopoulos, N., Parisses, C., Smyrnaios, A., Germanidis, N.: Autonomous vehicle for saffron harvesting. In: 6th International Conference on Information and Communication Technologies in Agriculture, Food and Environment, Corfu (2013) 6. Gambella, F., Paschino, F., Bertetto, A.M.: Perspective in mechanization of saffron (Crocus sativus L.). Int. J. Mech. Control 14(2), 3–8 (2013) 7. Bertetto, A.M., Ricciu, R., Badas, M.G: A mechanical saffron flower harvesting system. Meccanica 49, 2785–2796 (2014) 8. Russo, M., Ceccarelli, M., Corves, B., et al.: Design and test of a gripper prototype for horticulture products. Robot. Comput.-Integr. Manuf. 44, 266–275 (2017) 9. Carbone, G., González, A.: A numerical simulation of the grasp operation by LARM hand IV: a three finger robotic hand. Robot. Comput.-Integr. Manuf. 27(2), 450–459 (2011) 10. Rodriguez, F., Moreno, J.C., Sanchez, J.A., Berenguel, M.: Grasping in Agriculture: Stateof-the-Art and Main Characteristic. Grasping in Robotics, pp. 385–409 (2013)

22

A. Denarda et al.

11. Li, Z.G., Liu, Z., Li, P.P.: Study on the collision mechanical properties of tomatoes gripped by harvesting robot. Afr. J. Biotechnol. 8(24), 7000–7007 (2009) 12. Dimeas, F., Sako, D.V., Moulinanitis, V.C., Aspragathos, N.A.: Design and fuzzy control of a robotic gripper for efficient strawberry harvesting. Robotica 33(5), 1085–1098 (2015) 13. Van Hented, E.J., Hemming, L., Van Tuijl, B.A.J., et al.: An autonomous robot for harvesting cucumbers in greenhouses. Auton. Robots 13(3), 241–258 (2002) 14. Cafolla, D., Ceccarelli, M., Wang, M.F., Carbone, G.: 3D printing for feasibility check of mechanism design. Int. J. Mech. Control 17(1), 3–12 (2016) 15. Yao, S., Ceccarelli, M., Carbone, G., Dong, Z.: Grasp configuration planning for a low-cost and easy-operation underactuated three-fingered robot hand. Mech. Mach. Theory 129, 51–69 (2018) 16. Yao, S., Ceccarelli, M., Carbone, G., Zhan, Q., Lu, Z.: Analysis and optimal design of an underactuated finger mechanism for LARM hand. Front. Mech. Eng. 6(3), 332–343 (2011) 17. Carbone, G., Ceccarelli, M.: Experimental tests on feasible operation of a finger mechanism in the LARM hand. Mech. Based Des. Struct. Mach. 36(1), 1–13 (2008)

The Stiffness Model of the Compliant Positioning Mechanism with Watt’s Linkage Jaroslav Hricko(B)

and Stefan Havlik

Institute of Informatics, Slovak Academy of Sciences, Banská Bystrica, Slovakia {hricko,havlik}@savbb.sk

Abstract. The conversion of the kinematic structures into micro-robotic devices enables to achieve devices with high performance in the small usually monolithic body. One example of a micro-robotic two degree of freedom positioning device that works on the principle of compact compliant mechanisms, and is inspired by Watt’s linkage is presented. However, such devices require a complex design process, where the stiffness model provides the necessary basic element in the evaluation/calculation of the compliant mechanism performance. The stiffness model of the proposed XY micro-positioning device is executed, where the conditions for dimensioning of the flexure hinges and actuators are formulated too. In the conclusion of the paper are compared results from described stiffness model with results from FEM analysis. Keywords: Compact compliant mechanisms · Straight-line mechanisms · Positioning stage · Watt’s linkage · Stiffness modeling

1 Introduction The design of the micro-robotic devices that utilize elastic deformation to produce the desired motion is based on the creation and evaluation of the mathematical models. These design methods usually use approaches based on the principles of Model Based Design and design methodology for mechatronic systems VDI2206 [1, 2]. In the case of compact compliant devices, the initial design process is usually based on the engineering intuitive approach. It results, that the common kinematical structures have been converted into their elastic replacements. Such an approach replaces the kinematical pairs by the flexure hinges or compliant mechanisms e.g. universal joint is replaced by two-axis flexure joint or notch U joint, and prismatic kinematical pair is usually replaced by parallelogram or four-bar-notch block [3], and etc. The movement of the compliant structures is round because there are built only from an elastic version of revolute joints. Consequently that the inspiration by straight-line kinematics structures seems to be a proper approach to reach the straight-line output motion of a micro-robotic device. For instance, the parallel movement of microgripper fingers is required. The design of a gripper with straight-line jaw motion of fingers may be desired for gripping soft objects such as cells, gels, and assemblies of nanostructures © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 23–30, 2021. https://doi.org/10.1007/978-3-030-75259-0_3

24

J. Hricko and S. Havlik

such as carbon nanotubes. This gripper utilizes Hoeken’s linkage with connected parallelogram for removing rotation of output point [4]. A similar approach for the design of robot end-effector is presented in [5], where a modified Scott Russell straight-line mechanism was utilized. The design of the XY positioning device based on Roberts’s linkage with a working range larger than 12 mm in each axis is proposed in [6], such device has the parasitic motion of less than 1.7% of the device motion stroke. Another example is the miniature manipulator consists of a molded pantograph mechanism, which is composed of large-deflective hinges and links is elaborated in [7]. Such a device enables the work in the area under 50 × 40 mm since the device dimensions are up to 100 mm. One of the simplest straight-line mechanisms is Watt’s linkage that consists only of three rigid links that are connected by revolute joints. Such kinematic structure has chosen as suitable to build a compliant XY positioning stage. It is expected, that such design enables to work as a smart device. For instance as positioning stage for positioning of the objects in a small range with high positioning accuracy and with frequency up to a hundred motion cycles per second, e.g. for positioning of the tool of the micro engraving machine that is used in jewelry, other possible applications are described in [16]. Or as a sensor of the external contact reaction force on the central square table (for measurement of forces in z-axis they should be added strain gauges situated on the arms) [15]. The FEA analysis has been used in the initial design phase (see Fig. 1), where the revolute joints have been replaced by flexure hinges. By some modification has been achieved structure that meets desired conditions. By careful stiffness analysis that is described in this paper could be calculated parameters of flexure hinges, actuators, and the whole performance of the proposed positioning device.

Fig. 1. The initial design phase (first iteration) of compliant positioning device based on Watt’s linkage (left); Sketch of the proposed XY positioning stage – one from the latest design iteration (right).

2 XY Positioning Stage The new design of XY positioning stage based on the combination of the lever mechanism, that is utilized as a mechanical amplifier of input displacement of the piezoelectric stack (PZT) and the Watt’s linkage what is one of the most known straight-line

The Stiffness Model of the Compliant Positioning Mechanism

25

mechanism, where its output movement is approximate straight-line. The initial design supposes the output movement in the range ±0.5 mm. The material of the whole device will be plastic; it will be depended on the production of prototypes. In the case of water cutting technology, the Teflon (PTFE) material is considered, in the case of 3D printing technology, nylon (polyamide, PA), polypropylene (PP), or Polylactic acid (PLA) can be considered too. With the chosen manufacturing technology are connected some restrictions, one of the main is the minimal thickness of the flexure hinges, in this case, it is 0.5 mm (for 3D printing is expected print nozzle with diameter 0.25 mm). For the demanded output motion are expected dimensions up to 25 × 25 mm, for bigger output movement it is expected increasing of such dimensions with appropriate scale. The sketch of the proposed mechanism is shown in Fig. 1. As is seen the symmetry of the arms has been adopted. Therefore device requires the application of two PZT actuators for movement on one axis. 2.1 Stiffness Model The design of compact compliant devices requires a complex approach focused on mathematical modeling of proposed devices. One of the most important indicators of their performance is a calculation of mechanical stiffness. This one enables the calculation of deflections (see Eq. (1)), arisen stresses (see Eq. (16)), and dynamical properties [9] too. For the calculation of the mechanism stiffness model, it is necessary to calculate the stiffness of the main building block – flexure hinge. There is a number of research works that solve such a problem [8–12]. Between most known belong utilized of Castigliano’s theorems what is well done adopted by Lobontiu [9]. Another approach is based on the FEM analysis and follow-up processing of results to corresponding equations [12]. One of the promising approaches is a derivation of equations related to dimensionless stiffness [10, 12]. The stiffness model can be derived effectively by the matrix method under the assumption of the validity of Hooke’s law. The relationship between an acting load and resulting deformation (deflections, bending) is expressed as u = CF → F = Ku

(1)

where u is a deflection vector u = [ux , uy , uz , θx , θy , θz ]T (in case of in-plane deflections u = [ux , uy , θz ]T ). The vector of external acting load is F = [Fx , Fy , Fz , Mx , My , Mz ]T and the C or K (C = K−1 ) are compliance matrix or stiffness matrix, respectively. The components of the compliance/stiffness matrices are marked as C ij , where index i represents arising deflections/bending in corresponding axes and index j express acting load that deflection cause. To calculate the compliance/stiffness of the mechanism built from several flexure hinges it is necessary to consider its configuration. It can be in serial, parallel, or combined configuration. In each flexible element is located local coordinate system, to which is calculated compliance/stiffness (e.g. to the point A, see Fig. 3 right).

26

J. Hricko and S. Havlik

For calculation between local coordinate systems, the transformation matrix Tij is in the form   Rij −Rij Pij (2) Tij = 0 Rij where is Rij is rotation matrix between coordinate systems and Pij is translation matrix between coordinate systems i and j. The compliance of serial configuration expressed to S (see Fig. 2) coordinate system is [13, 14] CS =



 T TjS Cj TjS

(3)

n

where Cj is compliance matrix located in j-th coordinate system. A similar equation expresses the stiffness for the parallel configuration of the compliant mechanism [13, 14]. KP =



 T TjP Kj TjP

(4)

n

yS

S

yP

P

xS

Tj+1P

xP

zS

TjS yj+1

Tj+1S

zP

T jP

xj+1 zj+1

yj

yj+1

yj

zj xj

zj

xj

zj+1

xj+1

Fig. 2. An example of compliant mechanism configuration: serial (left) and parallel (right).

Because the symmetry of the proposed mechanism was adopted in the design process, only compliance/stiffness of one leg should be calculated (it means between actuator and point on the output plate, see Fig. 3). The dimensions from Fig. 3 are l 1 = 2r; l2 = l 5 − t/2; l 3 = 2r + t; l 4 = l 5 ; l 6 = 3r. The one type of flexure hinge with equal dimensions is used in the whole mechanism, which means that only one compliance matrix of the hinge is calculated and based on its rotation is transformed and used in the next calculations. The calculation of compliance between local coordinate systems located in points in O1 and in O2 is calculated as follow. The rotation between such coordinate systems is α 12 = π /2. The translation vector between O1 and O2 is p12 = [l1 , −l 3 /2, 0]T , the compliance matrix for in-plane deflections and rotations of the flexure hinge located in O1 is ⎛

C11

⎞ CyFy 0 −CyMz = ⎝ 0 CxFx 0 ⎠ −CyMz 0 CθzMz

(5)

The Stiffness Model of the Compliant Positioning Mechanism

27

Fy w yA

A

t

xA

Mz F x

r

y x

z

Fig. 3. Location of local coordinate systems with dimensions of proposed mechanism’s leg (left), right circular hinge, the place of acting load and location of the local coordinate system to which is calculated compliance Cii (right).

The rotation and translation matrixes are ⎛ ⎞ cos α12 − sin α12 0 R12 = ⎝ sin α12 cos α12 0 ⎠ 0 0 1

⎛

P12

⎞ 0 0 − l23 = ⎝ 0 0 −l1 ⎠ l3 2 l1 0

(6)

By substitution (6) into (2) and then to (3) is compliance matrix related to point O2 expressed as C12 = T12 C11 TT12  ⎞ l1 CθzMz CxFx + l12 CθzMz − 21 l1 2CyMz + l3 CθzMz  = ⎝ − 21 l1 2CyMz + l3 CθzMz CyFy + l3 CyMz + 41 l32 CθzMz − 21 2CyMz + l3 CθzMz ⎠  CθzMz − 21 2CyMz + l3 CθzMz l1 CθzMz (7) ⎛

C12

Considering the parallel configuration of the flexure hinges in points O1 and O2 the inverse (stiffness) matrix of C12 should be calculated. Therefore the compliance in point O2 is −1  −1 + C C122 = (K12 + K22 )−1 = C−1 12 22

(8)

Other transformations from O2 to H can be calculated by 

α23 = 0; p23

l3 = l1 + l3 , l2 + , 0 2

T ; C23 = T23 C122 TT23 + C33

α34 = 0; p34 = [l1 + l4 , 0, 0]T ; C34 = T34 C23 TT34 + C44 T  1 l3 + l5 α4H = π ; p4H = l3 − l5 , l1 + , 0 ; C4H = T4H C34 TT4H 2 2

(9) (10)

(11)

28

J. Hricko and S. Havlik

The legs “a” and “c” constitute one parallelogram therefore the compliance Cac can be calculated as −1  −1 (12) Cac = (2K4H )−1 = 2C4H Similarly the legs “b” and “d” are connected to the legs “a” and “c” parallel, what lead to −1  (13) CmechH = (2Kac )−1 = 2C−1 ac For the dimensioning of the actuator, the compliance/stiffness of the whole mechanism to the point where is actuator located should be calculated. The rotation angle is α HA = π, and translation vector pHA = [lHAx , l HAy , 0]T , where l HAx = 3l 1 + 2l 3 + l 4 − 0.5l 5 + l 6 , and l HAy = l 1 − l 2 + 0.5l5 . CHA = THA CmechH TTHA ; KHA = C−1 HA

(14)

Considering that the actuator cause only in one axis, only one component K xFx from stiffness matrix KHA , is required for the actuator dimensioning. Thanks to the knowledge of the compliance/stiffness of the whole mechanism it is possible to calculate all arisen deflections and rotations according to (1). Based on the kinematic analysis, it is necessary to verify the maximum arisen stress in the compliant joint where the maximum deformation (bending and displacement) is present. The maximum stress in flexure is depended on its shape, and acting load [9] σmax =

6ktb  kta Fx + 2 lFy + Mz wt wt

(15)

where k ta and k tb are the theoretical stress concentration factors. However by substituting (1) into (15) it is possible to achieve such dependence between arisen stress, stiffness and displacement or bending arisen in the mechanism σmax =

 6ktb  kta KxFx ux2 + 2 LKyFy + KyMz uy2 + LKyMz + KθzMz θ2 wt wt

(16)

2.2 FEM Analysis and Comparison of Results The FEM analysis has been used to verify the stiffness model and behavior of the proposed device. Considering that 3D printing technology will be adopted in the production of prototypes, the Ultimaker Nylon is used as the material of the whole device. Where Young’s modulus is 579 MPa and tensile stress at yield for 3D printed devices is 27.8 MPa. The characteristics dimensions of the proposed device are: the thickness of flexure hinge t = 0.5 mm, notch radius r = 0.8 mm, and the width of the device is w = 1 mm. The length of the middle plate l5 = 5 mm. The whole device is a square with an arm of 27.8 mm (include fixing holes) and according to simulations, the output movement is in the range ±0.485 mm and the maximum arisen stress is 27.4 MPa.

The Stiffness Model of the Compliant Positioning Mechanism

29

The compliance matrix of flexure hinge has been calculated according to Lobontiu [9] and next the compliance of the whole mechanism according to (13) is calculated. From the FEM analysis (see Fig. 4), the compliance matrix is given according to (1), where the input loads and output displacements are known. Considering that the moment around the z-axis does not act, the compliance matrix consists only of elements relevant to forces.

Fig. 4. Distribution of von Mises stress in the proposed device (only force F x acting) (left); 3D printed prototype in scale (right).

⎛

⎞   0.09395 −0.06884 −0.017767 0.08779 −0.00419 ⎜ ⎟ −3 −3 CmechH = 10 ⎝ −0.06884 0.08769 0.01294 ⎠; CmH _FEM = 10 −0.00419 0.08779 −0.017767 0.01294 34.1194

3 Conclusion An example of a two degree of freedom micro-positioning device that utilizes elastic deformation to achieve the desired movement is presented. Such a device combines Watt’s mechanism to produce the straight-line movement and the lever mechanism as a mechanical amplifier of the piezoelectric actuator stroke. To achieve the performance characteristics of the device the stiffness model has been developed and described. Its importance is shown in the dimensioning of the actuators and flexure hinges. At the end of the paper, the comparison of compliance matrices achieved by analytical approach and FEM analysis is presented. Considering the high differences of parasitic elements (C xFy ) the measurements on the prototype should be executed. Acknowledgment. This work was supported by the national scientific grant agency VEGA under project No.: 2/0155/19 – “Processing sensory data via Artificial Intelligence methods” and by project APVV-14-0076 – “MEMS structures based on load cell”.

References 1. Jiang, P., Zhou, Q., Shao, X.: Surrogate model-based engineering design and optimization. Springer Tracts in Mechanical Engineering (2020). https://doi.org/10.1007/978-98115-0731-1

30

J. Hricko and S. Havlik

2. VDI 2206: Design methodology for mechatronic systems. Entwurf, Beuth Verlag, Berlin (2004) 3. Machekposhti, D.F., Tolou, N., Herder, J.L.: A review on compliant joints and rigidbody constant velocity universal joints toward the design of compliant homokinetic couplings. J. Mech. Des. 137 (2015) https://doi.org/10.1115/1.4029318 4. Beroz, J., Awtar, S., Bedewy, M., Sameh, T., Hart, A.J.: Compliant microgripper with parallel straight-line jaw trajectory for nanostructure manipulation. In: Proceedings of 26th American Society of Precision Engineering Annual Meeting, Denver, CO (2011) 5. Zhu, J., Hao, G.: Design and test of a compact compliant gripper using the Scott-Russell mechanism. Arch. Civ. Mech. Eng. 20, 81 (2020). https://doi.org/10.1007/s43452-020-000 85-3 6. Wan, S., Xu, Q.: Design and analysis of a new compliant XY micropositioning stage based on Roberts mechanism. Mech. Mach. Theory 95, 125–139 (2016). https://doi.org/10.1016/j. mechmachtheory.2015.09.003 7. Ishii, Y., Thümmel, T., Horie, M.: Dynamic characteristic of miniature molding pantograph mechanisms for surface mount systems. Microsyst. Technol. 11, 991–996 (2005). https://doi. org/10.1007/s00542-005-0525-5 8. Yong, Y.K., Lu, T.-F.: Kinetostatic modeling of 3-RRR compliant micro-motion stages with flexure hinges. Mech. Mach. Theory 44(6), 1156–1175 (2009). https://doi.org/10.1016/j.mec hmachtheory.2008.09.005 9. Lobontiu, N.: Compliant Mechanisms: Design of Flexure Hinges. CRC Press, Boca Raton (2003) ISBN 0849313678 10. Wu, Y., Zhou, Z.: Design calculations for flexure hinges. Rev. Sci. Instrum. 73(8), 3101–3106 (2002) 11. Linß, S., Schorr, P., Zentner, L.: General design equations for the rotational stiffness, maximal angular deflection and rotational precision of various notch flexure hinges. Mech. Sci. 8, 29–49 (2017). https://doi.org/10.5194/ms-8-29-2017 12. Schotborgh, W.O., Kokkeler, F.G., Tragter, H., van Houten, F.J.: Dimensionless design graphs for flexure elements and a comparison between three flexure elements. Precision Eng. 29, 41–47 (2005) 13. Sciavicco, L., Siciliano, B.: Modeling and Control of Robot Manipulators. McGraw-Hill International Editions, Electrical Engineering Series (1996) ISBN 0-07-114726-8 14. Ottaviano, E., Carbone, G., Ceccarelli, M.: Workspace analysis and performances of a binary actuated parallel manipulator with flexural joints. J. Mech. Eng. Sci. 217, 313–330 (2003) 15. Havlik, S.: Design of smart robotic mechanisms. In: 2014 23rd International Conference on Robotics in Alpe-Adria-Danube Region (RAAD) (2014). https://doi.org/10.1109/raad.2014. 7002270 16. Howell, L.L., Magleby, S.P., Olsen, B.M.: Handbook of Compliant Mechanisms. John Wiley & Sons, Hoboken (2013)

Workspace and Singularity Zones Analysis of a Robotic System for Biosamples Aliquoting Dmitry Malyshev1(B) , Larisa Rybak1 , Giuseppe Carbone2 Tatiana Semenenko3 , and Anna Nozdracheva3

,

1 Belgorod State Technological University n.a. V.G. Shukhov, Kostyukova 46,

308012 Belgorod, Russia 2 University of Calabria, 87036 Rende, Calabria, Italy 3 Gamaleya National Research Center for Epidemiology and Microbiology, Moscow, Russia

Abstract. This paper discusses the use of optimization algorithms to solve the problem of determining the workspace for a new robot for biological material aliquoting. The proposed robot is based on a parallel Delta-like kinematic architecture. A method for approximating the set of solutions to a system of nonlinear inequalities describing constraints on the geometric parameters of the robot is considered. Algorithms for constructing the workspace are synthesized. The workspace of the robot is obtained, considering the singularity zones of the robot, using a method based on the analysis of the Jacobian matrix of the mechanism. The analysis of changes in the volume of the workspace, considering the constancy of the determinant of the Jacobian matrix, depending on the options for solving the ambiguous inverse kinematics of the robot. The simulation results are presented. Keywords: Approximation set · Parallel robot · Workspace · Non-uniform covering · Optimization · Singularity zones

1 Introduction Parallel robots are more and more applied every year in industry and medicine to perform various technological operations. Such robots have high productivity, have increased structural rigidity, and the accuracy of the operations performed. Medical biotechnology is one of the application areas of parallel robots. The compliance of the quality of bio-samples with high international standards for laboratory and clinical research can be achieved only by reducing the duration of preparation of initial samples, minimizing the number of manual operations, increasing the accuracy of the considered characteristics of samples and reducing their storage time at room temperature. The necessity to reduce the number of production processes using manual labor is also significant to prevent contamination of equipment and work surfaces. Currently, the main robotic systems for aliquoting are the PIRO robotic station manufactured by Dornier-LTF (Germany) [1], the Freedom EVO® series robotic laboratory station manufactured by TECAN (Switzerland) [2], an automatic dosing system (robot dispenser) Microlab STAR manufactured © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 31–38, 2021. https://doi.org/10.1007/978-3-030-75259-0_4

32

D. Malyshev et al.

by Hamilton [3]. A promising direction is the development of new types of robotic systems for biomaterial aliquoting, based on the application of parallel manipulators, and providing increased characteristics in terms of positioning accuracy of the end effector, structural rigidity and compact design.

2 Robotic System Model Design Let’s consider a structure of proposed robotic system for biomaterial aliquoting (Fig. 1a) within the framework of this study. It consists of two modules - a parallel manipulator based on the 4-DOF Delta mechanism, equipped with a dispensing head for performing basic liquid pipetting operations, and a module based on a serial collaborative robot for transporting racks with test tubes and plates with microtubes. An important characteristic of the robot is its workspace, within which the end effector should be located when performing technological operations. Finding the workspace of parallel robots is much more difficult than for serial robots, due to the peculiarities of structure, kinematics and dynamics. The study of the workspace of the delta robot was investigated earlier in [4–6]. Within the framework of the current one, it is proposed, in addition to the workspace, to determine the singularity zones of the mechanism using the method of non-uniform coverings. It can be classified as interval deterministic method. The application of interval analysis, including based on bisection, forming constraints in the form of a box, is considered in [7]. The development of the non-uniform coverings concept based on interval methods and its application for various robots is considered in [8–11]. The non-uniform coverings method is universal, applicable to any design of robots. It can also be modified and combined with other approaches to solving specific problems of multi criteria optimum design of manipulators [12].

Fig. 1. Robotic system for biomaterial aliquoting: a) 3D model: 1 - base, 2 - parallel delta robot, 3 - end effector, 4 - dispenser tip, 5 - Universal robot arm, 6 - movable base of the robot arm, 7 workspace, 8 - tube rack, 9 - tray for consumables, 10 - trolley, b) Delta robot scheme.

Workspace and Singularity Zones Analysis of a Robotic System

33

It also outperforms discretization [13] and Monte Carlo-based approaches [14]. The method allows to determine the boundary of the workspace, and internal workspace, in contrast to the approaches that determine only the boundary. Let’s consider the structure of a delta robot (Fig. 1b). The manipulator has, in addition to 3 DOF - translational movements along the X, Y and Z axes, an additional fourth DOF, which is the rotation of the end effector around the Z axis. The robot includes three RUU kinematic chains, that is, each of the chains uses a rotary drive hinge Ai for connection with a fixed base, and two gimbals - Ci for connection with a mobile platform and Bi for connecting two intermediate links. The fixed base and the movable platform are equilateral triangles. Due to the absence of the influence of the rotational degree of freedom on the shape and size of the workspace, as well as on singularity zones, further in the work, the point P, located in the center of the movable platform, will be considered as an end effector. The workspace constraints can be represented as a system of inequalities:  θi − θmax ≤ 0 , (1) θmin − θi ≤ 0 where θi are the angles of rotation in the drive joints, θmin and θmax are the minimum and maximum angles of rotation, respectively. Using the solution to the inverse kinematics described in [6], we obtain:  ⎛ ⎞ −Fi ± Ei2 + Fi2 + Gi2 ⎠, (2) θi = 2 tan−1 ⎝ Gi − Ei where Ei ,Fi and Gi are calculated as  a − 2c , F1 = 2zP d , E1 = 2d yP + √ 2 3   a − 2c a − 2c 2 − e2 , G1 = xP2 + yP2 + zP2 + + d 2 + 2yP √ √ 2 3 2 3   √ 2c − a 2c − a , F2 = 2zP d , E2 = −d + yP + √ 3 xP + 4 4 3      2c − a 2c − a 2c − a 2 2c − a 2 G2 = xP2 + yP2 + zP2 + − e2 , + yP + + d 2 + 2 xP √ √ 4 4 4 3 4 3   √ 2c − a 2c − a − yP − √ , F3 = 2zP d , 3 xP − E3 = d 4 4 3      2c − a 2c − a 2c − a 2 2c − a 2 − e2 , G3 = xP2 + yP2 + zP2 + − xP + + d 2 + 2 yP √ √ 4 4 4 3 4 3

(3)

Due to the ambiguity of the solution to the inverse kinematics for one coordinate of the point P, there are 2 possible angles of rotation in the drive rotary joint for each of the three kinematic chains. Therefore, there are 8 possible solutions to the inverse kinematics and a workspace can be determined for each. Let‘s consider the method proposed by C. Gosselin [15, 16], based on the analysis of the Jacobi matrix of the mechanism obtained by differentiating the constraint equations

34

D. Malyshev et al.

to determine the singularity zones. The determinant of the Jacobi matrix has the form ⎡

∂Θ1 ∂Θ1 ∂Θ1 ∂xP ∂yP ∂zP

⎢ ⎢ det(JA ) = ⎢ ∂Θ2 ⎣ ∂xP

∂Θ2 ∂Θ2 ∂yP ∂zP

∂Θ3 ∂Θ3 ∂Θ3 ∂xP ∂yP ∂zP

⎤ ⎥ ⎥ ⎥, ⎦

(4)

where Θi correspond to formulas (2) of angles θi . In view of the cumbersomeness of the formulas for each of the elements of the determinant, we present only one of them   2xp (2dzp +s1 ) 2xp s2    2 +  ∂Θ1 = ∂xP

   2 √ s2 −2d yp − 2c−a

x10 s2 −2d yp −

2 3



(2dzp +s1 )

2

2xp s2

 where s1 =

2c−a √ 2 3

,

(5)

+1

 2  √ 4d 2 zp2 + 4d 2 yp − 2c−a − s22 , 2 3

 s2 =

2c − a √ 2 3

2

 − 2yp

2c − a √ 2 3

− e2 + d 2 + xp2 + yp2 + zp2

Let us write the condition for the occurrence of singularity zones as det(JA ) = 0. Based on the presence of eight solutions to the inverse kinematics, there are also 8 possible Jacobi matrices and, accordingly, their determinants. It is required to ensure that the determinant of the Jacobi matrix is constant to determine the workspace without singularity zones. To do this, add one of the conditions to (1): det(JA ) < 0 or det(JA ) > 0, depending on the sign of the determinant in main part of the workspace.

3 Algorithm Synthesis for Workspace Determination The algorithm for approximating the workspace consists of solving the system of inequalities (1). Let’s write in general form: ⎧ g1 (x) ≤ 0, ⎪ ⎪ ⎨ ... (6) ⎪ ≤0 g (x) m ⎪ ⎩ ai ≤ xi ≤ bi , i = 1, .., n. For this problem, n = 3, since the system of inequalities (6) has three variables the moving platform’s coordinates x, y, z. In this case, the geometric parameters are set constant. The initial box Q, which includes the entire set of solutions X, is determined by the constraints of the intervals ai ≤ xi ≤ bi , i = 1, .., n. Consider an arbitrary box B. Let m(B) = maxi = 1,..,m minx∈B gi (x) and M (B) = maxi = 1,..,m maxx∈B gi (x). If m m(B) > 0 then B contains no possible points for system (6). The proposed algorithm

Workspace and Singularity Zones Analysis of a Robotic System

35

excludes such boxes. If M (B) ≤ 0, then each point of the box is a possible solution. Therefore, it can be added to the coverage as an interior box. If a box cannot be excluded, it is divided into two smaller boxes if its diameter is not less than the specified precision δ. Such boxes are added to the boundary approximation. The algorithm for determining the workspace, taking into account singularity zones, is similar and differs by a large number of inequalities in the system, as well as additional conditions when checking m(B) and M(B), taking into account the strict inequality for the determinant of the Jacobian matrix (4). The algorithm uses a system of equations instead of a system of inequalities to determine singularity zones. To find the set of its solutions, calculate M(B) is calculated as (B) = mini∈1,k maxx∈B gj (x), the list PI and there are no conditions for it. Condition for excluding boxes takes the form:  M (B) < 0 m(B) > 0 The proposed algorithms can be applied to other robots. It is easy enough to describe the workspace constraints by inequalities or equations. Moreover, the number of variables can be different. Link interference conditions can also be added. We use synthesized algorithms for numerical simulation. To implement the developed algorithms, a program was written in the C++ programming language. The bisection procedure can be implemented using standard tools such as MATLAB. However, a program developed in C++ allows additional control over multithreading to speed up calculations, use dynamic vectors of various structures, and export the results as a threedimensional model. Also, the transformation of a set of boxes has been implemented to reduce their number without decreasing accuracy. This is discussed in more detail in [17].

4 Simulation Results The workspace was modeled for the following parameters of the mechanism: a = 450 mm, c = 200 mm, d = 150 mm, e = 230 mm. The workspace is built without taking into account singularity zones, shown in Fig. 2. Figure 2a shows the entire workspace

Fig. 2. a) Volumetric workspace, b) in section in the XOZ plane,

36

D. Malyshev et al.

of the delta robot, Fig. 2b - half of the workspace with cut along the XOZ plane. The overall dimensions of the workspace are 535 × 535 × 382 mm. Next, we will define singularity zones. The simulation results are shown in Fig. 3. Figure 3a shows the singularity zones inside the workspace taking into account the Jacobi determinant when using “+” in solving the inverse kinematics (2), Fig. 3b - only singularity zones when using “−”, Fig. 3c - for all 8 possible combinations “+” and “−” in solving the inverse kinematics. The average computation time for the approximation accuracy δ = 2 mm and the grid dimension for calculating the 64 × 64 × 64 functions on a personal computer was 32 s. In the process of simulation singularity zones, it was determined that in most of the inner part of the workspace the positive sign of the determinant of the Jacobi matrix is observed, if for two or all three kinematic chains from two solutions of the inverse kinematics choose “−”. Negativity is observed in the opposite cases, that is, if “+” is selected for at least two circuits. Based on this, when defining the workspace, considering singularity zones, for four possible solutions we add the condition for the positive determinant det(JA ) > 0, and for the other four - the condition for the negativeness det(JA ) < 0. Table 1 summarizes the simulation results. Table 1. Comparison of workspace volumes. №

Sign in inverse kinematics

Sign of the Jacobi determinant inside the workspace

Volume of the workspace, Excluding singularity zones

Taking into account

×107 mm3

Volume reduction in %

θ1

θ2

θ3

1

+

+

+

−

5,13958

4,822

6,18

2

+

+

−

−

5,034

4,285

14,88

3

+

−

+

−

5,034

4,285

14,88

4

+

−

−

+

4,931

4,705

4,58

5

−

+

+

−

5,034

4,285

14,88

6

−

+

−

+

4,931

4,705

4,58

7

−

−

+

+

4,931

4,705

4,58

8

−

−

−

+

4,83093

4,449

7,91

As can be seen from the table, among the 8 cases for solving the inverse kinematics, 4 groups can be distinguished: where all the kinematic chains with “+”, all with “−”, one chain with “+” and two with “−”, as well as two with “+” and one with “−”. The first two groups have one case; the second two have three cases. Due to the symmetrical design of the robot, the volume of the workspace is the same for the case group. This is confirmed by the obtained volume values. The table shows that for cases 2, 3 and 5, as well as for cases 4, 6 and 7, the volume of the workspace is the same. In Fig. 4 shows the workspace for case 1 and 8.

Workspace and Singularity Zones Analysis of a Robotic System

37

Fig. 3. Areas corresponding to singularity zones: a) with “+” in solving the inverse kinematics, b) with “−”, c) with all possible combinations.

Fig. 4. Workspace taking into account singularity zones: a) for case 1, b) for case 8.

The volume of the workspace, considering singularity zones, decreased by 4.58– 14.88%, depending on the combination of the solution of the inverse kinematics. In addition to singularity zones, the shape and volume of the workspace is directly affected by the intersection of the robot links with the platforms and with each other.

5 Conclusion The article discusses a parallel delta robot with rotary driven kinematic pairs. A numerical method has been developed for determining the workspace of the robot, based on the concept of non-uniform coverings, which allows taking into account the singularity zones of the mechanism. The volume of the workspace, taking into account singularity zones, decreased in comparison with the original workspace by 4.58–14.88%, depending on the combination of the solution of the inverse kinematics. This fact must be taken into account for designing robots.

38

D. Malyshev et al.

Acknowledgements. This work was supported by the state assignment of Ministry of Science and Higher Education of the Russian Federation under Grant FZWN-2020-0017.

References 1. https://www.pharmaceutical-networking.com/wp-content/uploads/2014/11/PIRO-the-qua lity-Pipetting-robot.pdf Accessed 31 Jan 2021 2. https://lifesciences.tecan.com/products/liquid_handling_and_automation/freedom_evo_ser ies?p=tab–2 - Freedom EVO® series. Accessed 31 Jan 2021 3. https://www.hamiltoncompany.com/automated-liquid-handling/platforms/microlab-star Accessed 31 Jan 2021 4. Laribi, M.A., Romdhane, L., Zeghloul, S.: Advanced synthesis of the DELTA parallel robot for a specified workspace. In: Wu, H. (ed.) Towards New Applications. IntechOpen (2008) 5. Curcio, E.M., Carbone, G.: Design of a novel robot for upper limb rehabilitation. In: Rauter, G., Cattin, P.C., Zam, A., Riener, R., Carbone, G., Pisla, D. (eds.) New Trends in Medical and Service Robotics. Mechanisms and Machine Science, pp. 12–20. Springer, Cham (2021) 6. Williams II, R.L.: The delta parallel robot: kinematics solutions. Internet Publication www. ohio.edu/people/williar4/html/pdf/DeltaKin.pdf 7. Merlet, J.: Interval analysis and robotics. Robot. Res. 66, 147–156 (2010) 8. Evtushenko, Y., Posypkin, M., Rybak, L., Turkin, A.: Approximating a solution set of nonlinear inequalities. J. Glob. Optim. 7, 129–145 (2018) 9. Malyshev, D., Posypkin, M., Rybak, L., Usov, A.: Approaches to the determination of the working area of parallel robots and the analysis of their geometric characteristics. Eng. Trans. 67(3), 333–345 (2019) 10. Evtushenko, Y., Posypkin, M., Turkin, A., Rybak, L.: The non-uniform covering approach to manipulator workspace assessment. In: Proceedings of the 2017 IEEE Russia Section Young Researchers in Electrical and Electronic Engineering Conference, ElConRus, IEEE, pp. 386–389 (2017) 11. Rybak, L., Posypkin, M., Turkin, A.: Method for approximating the workspace of the parallel robot. Int. J. Pharm. Technol. 8(4), 25045–25055 (2016) 12. Ceccarelli, M., Carbone, G., Ottaviano, E.: Multi criteria optimum design of manipulators. Bull. Pol. Acad. Sci. Tech. Sci. 53(1), 9–18 (2019) 13. Lara-Molina, F., Dumur, D.: A fuzzy approach for the kinematic reliability assessment of robotic manipulators. Robotica 1–15 (2021). https://doi.org/10.1017/s0263574721000187 14. Zhou, Y., Niu, J., Liu, Z., Zhang, F.: A novel numerical approach for workspace determination of parallel mechanisms. J. Mech. Sci. Technol. 31(6), 3005–3015 (2017) 15. Gosselin, C., Angeles, J.: Singularity analysis of closed-loop kinematic chains. IEEE Trans. Robot. Autom. 6(3), 281–290 (1990) 16. Gherman, B., Birlescu, I., Plitea, N., Carbone, G., Tarnita, D., Pisla, D.: On the singularityfree workspace of a parallel robot for lower-limb rehabilitation’. Proc. Rom. Acad. Ser. A Math. Phys. Tech. Sci. Inf. Sci. 20(4), 383–391 (2019) 17. Rybak, L., Malyshev, D., Gaponenko, E.: Optimization algorithm for approximating the solutions set of nonlinear inequalities systems in the problem of determining the robot workspace. Adv. Optim. Appl. 1340, 27–37 (2021)

Preliminary Observations for Functional Design of a Mobile Robotic Manipulator Luca Carbonari(B) , Luigi Tagliavini, Andrea Botta, Paride Cavallone, and Giuseppe Quaglia Politecnico di Torino, 10129 Torino, Italy {luca.carbonari,luigi.tagliavini,andrea.botta,paride.cavallone, giuseppe.quaglia}@polito.it

Abstract. This paper presents the first steps approached for the functional design of a mobile robotic manipulator conceived for home assistance. The main characteristic of the mobile platform is that of owning an innovative architecture conceived to enhance the dynamics performance offered by the present-day solutions for omni-directional planar motions. The mobile robot, named Paquitop.arm, is aimed at working in a domestic unstructured environment and it has been designed to mainly perform monitoring activities. The variety of tasks to whom it can be devoted significantly increases if an on-board manipulation ability is installed. Due to that, it is worth investigating what aspects could be to be considered while designing such a mobile manipulator. The manuscript approaches a simplified kinematics analysis to define the most relevant parameters necessary for implementation of a commercial robot arm on the customized mobile platform Paquitop. Keywords: Personal assistance robot manipulator · Collaborative task

1

· Mobile robot · Mobile

Introduction

The ageing society is a very well known scenario [1] which kept the attention of the robotics research community for the potential wide field of applications, especially for home and hospital personal assistance [2,3]. Mobile robots in particular play a fundamental role, for they represent the most natural manner to distribute services with an as low as possible impact on the users life. The research in the field of mobile robotics produced many relevant results in the last years, to the point that almost any kinematic scheme and actuation layout was already analysed [4]. At present, two main classes of machines seem to have reached the maturity level for being industrially exploited: full mobile robots with omniwheels, and differential drive platforms. Pros and cons of both of them are well known and are mainly related to their mobility: the former can take advantage of high manoeuvrability, but are poorly effective on non-structured environment and uneven grounds; the latter are more versatile in this sense, yet they provide only two degrees of freedom motions. Among the investigated applications, c The Author(s), under exclusive license to Springer Nature Switzerland AG 2021  S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 39–46, 2021. https://doi.org/10.1007/978-3-030-75259-0_5

40

L. Carbonari et al.

Fig. 1. The Paquitop mobile platform provided with the six degrees of freedom KINOVA Gen3 Lite collaborative manipulator.

mobile manipulation is rising an ever-growing attention [5,6], especially since the development of collaborative robotic arms [7,8]. Such manipulators, in fact, fit almost perfectly the requirements for mobile purposes: low weight, moderate payload, and compatibility with the presence of humans in the workspace made them extremely attractive to both research and industrial communities. Even in this field, a lot has been said in almost every field of application, such as industrial [9], assistive [10], and medical [11] applications. The most studied aspects involve collision avoidance strategies for safety [12], and control algorithms for dexterous manipulations [13]. In such crowded scenario, the researchers at Politecnico di Torino proposed in the recent past a novel mobile robot, named Paquitop, provided with full on-plane mobility [14]. The leading idea was that of designing it with a modular structure, eventually provided with a manipulation capability depending on specific requests. The platform is suspended on four wheels: two off-centred passive castor wheels and two driven wheels which are also provided with a steering degree of freedom. Such layout leads many mechanical design challenges, already tackled in [15,16]. A first prototype of the robot is at present in construction, and the manuscript presents the first steps moved towards a further version of the robot, Paquito.arm, with manipulation ability provided by a collaborative robot arm implemented over the Paquito main module, as shown in Fig. 1. The choice of a commercial collaborative manipulator was made by looking for a convenient compromise of cost, performance, and weight. The final cost of the whole project, in fact, is an aspect not to be overlooked, since the expected users are private persons interested in a help for simple yet needful daily tasks, such as to carry or collect small objects, or even to open doors on behalf of unable persons (such as elders who need walkers on a daily basis). Therefore, the performance of the manipulator (which needs to be collaborative for obvious safety reasons) takes a back seat with respect to the success of the entire project. Highlevel over actuated robots were then discarded a priori, although their extreme versatility should have brought substantial benefit either in terms of dexterity, or in terms of available payload. As a plus, however, low cost cobots generally

Preliminary Design of a Mobile Robotic Manipulator

41

Fig. 2. Kinematics schemes of Paquitop and KINOVA Gen3 Lite arm.

own a reduced weight with respect to over-actuated ones. The final choice fell on the KINOVA Gen3 Lite robot, a six degrees of freedom manipulator with on board controller. Such arm, designed for educational and mobile applications, owns a weight (∼5.4 kg ) compatible with the dimensions of Paquitop, with an overall reach (740 mm) sufficient for most part of the above-mentioned tasks. The joints architecture of the robot is shown in Fig. 2. The maximum payload is definitely moderate (0.5 kg), although it is in line with other commercial cobots of the same market sector.

2

Robot Mobility

To analyse the mobility of the robotic manipulator together with the mobile platform, the Jacobian matrix of the robot arm can be considered. The velocity of the end-effector of the arm can be depicted as the linear combination of the velocities offered by all the joints composing its kinematic chain, proportionally to the rate of the joints themselves. Thus, it is:    n ωee $i q˙i (1) = vee i=1

where ωee and vee are the angular and linear velocities of the end-effector, q˙i is the rate of the ith of the n joints, and $i is its unit screw. The unit screws of the six revolute joints are then composed as:   si $i = ee (2) pi × si being si the axis of the ith joint, and ee pi the position of any point of such axis with respect to the end-effector. In brief, the Jacobian matrix of the manipulator can be worked out as a function of the six unit screws of the arm joints as:   (3) J = $1 . . . $6

42

L. Carbonari et al.

This quite basic result gains relevance if the mobility of the Paquitop platform is considered together. Due to its particular architecture, Paquitop is provided with a full on-plane mobility and it can exhibit any velocity in the plane. In other words, by rearranging the wheels steering angles δwr and δwl , and the respective angular velocities θ˙wr and θ˙wl , the instantaneous center of rotation icr of the mobile robot can be placed anywhere, according to: ⎤ ⎡ −2 sin δwr sin δwl 2a ⎣ sin δwr + δwl ⎦ (4) icr = sin (δwl − δwr ) − It is worth noticing that such relation is obtained considering the fundamental constraint that relates the two steering angles together with the actuation velocities: θ˙wr sin δwr = θ˙wl sin δwl . It basically represents the rigid body constraint of the robot chassis that the wheels must fulfil when non-null steering angles are used. Otherwise, the position of icr is located along the common axis of the two actuated wheels depending on the difference of velocity of the wheels. Also, (4) shows that when δwr = δwl = 0 the platform loses its ability to perform angular velocities around the vertical axis. Except for this singular case, the instantaneous mobility can be described by a zero-pitch unit screw passing through the center of rotation icr . The consequent velocity of Paquitop is:   ωc (5) = $P γ˙ vc where γ˙ is the yaw rate of Paquitop main body, while $P is the aforementioned zero pitch unit screw, obtainable as:   s $P = ee P (6) icr × sP If pitch and roll rotations are disregarded, the axis of the screw is merely vertical:  T sP = 0 0 1 . The velocity of the mobile platform contributes to that of the end-effector like the remainder of the actual joints of the arm. The resulting Jacobian is now:   (7) J∗ = $1 $2 $3 $4 $5 $6 $P which is a quite typical shape for redundant manipulators with seven degrees of freedom. Such peculiarity lasts unless a singularity occurs, i.e. unless two or more unit screws become linearly dependent within J∗ . By construction, every articulated arm is affected by specific singular postures which have been widely investigated in the past and are well known in the field. Aside such configurations, others can be added by the presence of the mobile platform. In particular, it is worth considering the screw $1 of the first revolute joint of the arm, whose position and orientation are actually a design problem that should be carefully addressed to obtain the best possible performance from the system. Usually, the first joint of mobile manipulators is placed vertically, especially when redundant arms are used (as done, for example, by KUKA for the well known KMR iiwa). In the specific

Preliminary Design of a Mobile Robotic Manipulator

43

case of Paquitop, the use of a low cost six degrees of freedom arm can benefit from a different concept of coupling by maintaining the redundant platform-arm system as distant as possible from singularities by avoiding the overlapping of the two screws $1 and $P . To maintain such screws independent one can act in two directions: by keeping away the centre of rotation of the platform from s1 , or by providing the system with non parallel s1 and sP by placing the first joint of the robot arm with a fixed non-null orientation with respect to the vertical axis. The first solution moves the issue from mechanical design to motion planning and control of the manipulator. Therefore, it represents a possible limitation in the use of the robot. The second option, which is basically effortless from the point of view of realization, does not require any made on purpose actuation strategy and significantly improves the usability of the arm. From the point of view of position kinematics, in fact, dealing with a redundant manipulator instead of a classic one allows approaching in a more efficient way many issues typical of collaborative applications, such as that of obstacle avoidance: maintaining a redundant architecture provides the motion planner with more configurations exploitable for avoiding obstacles within the workspace. The optimal solution in this sense should be that of having s1 ⊥ sP . In that case, when the two axes intersect in icr , the three screws $1 , $2 , and $P form a spherical joint permitting the redundant platform-arm system the widest possible range of configurations for each point of the workspace. This simple concept is shown in Fig. 3 for four different arrangements of the arm over Paquitop. For the sake of simplicity, the representation considers a simple three degrees of freedom manipulator, comparable to a wrist-less robot arm. The links dimensions were set-up equal to those of the KINOVA Gen3 Lite. Figure 3(a) refers to parallel s1 and sP : whatever the yaw angle of the platform is, the arm can reach the target point with only one configuration (two, if the second assembly mode proper of the three axes manipulators is considered). Figure 3(d) shows the dual situation, when s1 ⊥ sP . In this case, the arm can advantage of infinite configurations by positioning the proximal and the distal links onto two conical surfaces. Such wide range of configurations can be exploited to avoid particular regions of the workspace, as in the case of the presence of obstacles. Figure 3(b) and Fig. 3(c) depict intermediate assets, with reduced conical surfaces with respect to Fig. 3(d).

3

Task Oriented Optimization

According to the observations of the previous section, the adoption of an horizontal configuration might seem the optimal possible solution. Actually, the obtainable manipulability is not the only aspect that must be kept into consideration, since the final performance of the system with respect to its tasks shall heavily depend on the choices made at this design step. The most relevant aspects to be considered are the arm behaviour within its workspace, and its reach with respect to both the mobile platform and the external environment. To such aim, the simple wrist-less kinematics used in the previous section was used again to figure out what the result could be of settling the robot arm over

44

L. Carbonari et al.

Fig. 3. Range of possible configurations for different inclinations of s1 : (a) s1 vertical (only one configuration possible), (d) s1 perpendicular to the vertical (complete cones of configurations); (b) and (c) depict intermediate configurations of 30◦ and 60◦ .

the Paquitop platform. In Fig. 4 such aspects are graphically represented by means of the 2-norm condition number of the Jacobian matrix:   (8) cn = σmin JJT /σmax JJT with σmin and σmax minimum and maximum eigenvalues of the matrix JJT : by definition such parameter is always bounded in the range 0 ≤ cn ≤ 1. Also, for a better representation, cn was normalized within the workspace with respect to the maximum value assumed. Doing so, cn vanishes when singularities occur, while it is cn = 1 in the point characterized by the higher kinematic performance. The maps of Fig. 4 can be useful to understand what is the effect of the orientation of the arm base. Figure 4(a) shows a non-tilted configuration: as visible, the lighter regions of the cn map (corresponding to higher values of the parameter) are located in front and behind Paquitop chassis. Due to the height of the system (∼30 cm), such configuration do not valorise the actual possibilities offered by the manipulator due to the expected eight of the robot targets. Its tasks, in fact, involve human-sized spaces and heights (planes of tables, chairs, etc.) for which an higher placement should be preferable. Unfortunately, the arm base cannot be indefinitely raised up, for the sake of machine compactness and for mass distribution issues. The set-up can be improved applying a fixed orientation, as in

Preliminary Design of a Mobile Robotic Manipulator

45

Fig. 4. Condition number of the Jacobian matrix around the mobile platform for two assembly configurations of the arm with inclination null and equal to 30◦ .

Fig. 4(b), where the first joint of the robot arm was rotated of 30◦ around the y axis of {c}. Such rotation, aside the advantages discussed in the previous section, moves the region characterized by higher values of cn in more useful portions of the workspace, allowing a more effective exploitation of the arm. Furthermore, the region adjacent to the robot back owns the higher values of cn close to the ground surface, improving the arm ability to manipulate objects lying on the ground.

4

Conclusions

The paper shows some preliminary observations useful for the installation of a commercial cobot on the customized mobile platform Paquitop. The platform, which owns itself a redundant set of actuators, can be exploited to enhance the mobility of the robotic arm by providing it with a further degree of freedom. This characteristic can be exploited to augment the arm space of configurations, which is a non-negligible feature for dexterity and obstacle avoidance. A basic mobility analysis has been proposed to prove the advantages that can be brought by a non-vertical orientation of the first axis of the robot arm. Also, such rotation can enhance the reach of the gripper with respect to the task of the arm-platform system, as shown in the last part of the document. In future, such considerations must be kept into practice by installing a Kinova Gen3 Lite manipulator on the Paquitop prototype, whose mechanical design was already presented by authors.

References 1. Lutz, W., Sanderson, W., Scherbov, S.: The coming acceleration of global population ageing. Nature 451(7179), 716 (2008) 2. Naotunna, I., Perera, C.J., Sandaruwan, C., Gopura, R., Lalitharatne, T.D.: Meal assistance robots: a review on current status, challenges and future directions. In: 2015 IEEE/SICE International Symposium on System Integration (SII), pp. 211–216. IEEE (2015)

46

L. Carbonari et al.

3. Smarr, C.A., Mitzner, T.L., Beer, J.M., Prakash, A., Chen, T.L., Kemp, C.C., Rogers, W.A.: Domestic robots for older adults: attitudes, preferences, and potential. Int. J. Soc. Robot. 6(2), 229–247 (2014) 4. Campion, G., Bastin, G., Dandrea-Novel, B.: Structural properties and classification of kinematic and dynamic models of wheeled mobile robots. IEEE Trans. Robot. Autom. 12(1), 47–62 (1996) 5. Han, Z., Allspaw, J., LeMasurier, G., Parrillo, J., Giger, D., Ahmadzadeh, S.R., Yanco, H.A.: Towards mobile multi-task manipulation in a confined and integrated environment with irregular objects. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 11025–11031. IEEE (2020) 6. Wang, C., Zhang, Q., Tian, Q., Li, S., Wang, X., Lane, D., Petillot, Y., Wang, S.: Learning mobile manipulation through deep reinforcement learning. Sensors 20(3), 939 (2020) 7. Buhl, J.F., Grønhøj, R., Jørgensen, J.K., Mateus, G., Pinto, D., Sørensen, J.K., Bøgh, S., Chrysostomou, D.: A dual-arm collaborative robot system for the smart factories of the future. Procedia Manuf. 38, 333–340 (2019) 8. Kim, W., Balatti, P., Lamon, E., Ajoudani, A.: MOCA-MAN: a mobile and reconfigurable collaborative robot assistant for conjoined human-robot actions. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 10191– 10197. IEEE (2020) 9. Zhang, Y., Liu, L., Huang, N., Radwin, R., Li, J.: From manual operation to collaborative robot assembly: an integrated model of productivity and ergonomic performance. IEEE Robot. Autom. Lett. 6(2), 895–902 (2021) 10. Zachiotis, G.A., Andrikopoulos, G., Gornez, R., Nakamura, K., Nikolakopoulos, G.: A survey on the application trends of home service robotics. In: 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1999–2006. IEEE (2018) 11. Chiriatti, G., Palmieri, G., Palpacelli, M.C.: Collaborative robotics for rehabilitation: a multibody model for kinematic and dynamic analysis. In: The International Conference of IFToMM ITALY, pp. 431–438. Springer, Cham (2020) 12. Scoccia, C., Palmieri, G., Palpacelli, M.C., Callegari, M.: A collision avoidance strategy for redundant manipulators in dynamically variable environments: on-line perturbations of off-line generated trajectories. Machines 9(2), 30 (2021) 13. Kim, W., Peternel, L., Lorenzini, M., Babiˇc, J., Ajoudani, A.: A human-robot collaboration framework for improving ergonomics during dexterous operation of power tools. Robot. Comput. Int. Manuf. 68, 10 p (2021) 14. Carbonari, L., Botta, A., Cavallone, P., Quaglia, G.: Functional design of a novel over-actuated mobile robotic platform for assistive tasks. Mech. Mach. Sci. 84, 380–389 (2020) 15. Carbonari, L., Botta, A., Cavallone, P., Tagliavini, L., Quaglia, G.: Dynamics characterization of Paquitop, a novel platform for robotized domestic applications. In: Proceedings of the International Mechanical Engineering Congress and Exposition, IMECE2020. ASME (2020) 16. Tagliavini, L., Botta, A., Cavallone, P., Carbonari, L., Quaglia, G.: On the suspension design of Paquitop, a novel service robot for home assistance applications. Machines 9(3), 52 (2021)

Development of a Low-Cost Force Sensory System for Force Control via Small Grippers of Cooperative Mobile Robots Used for Fabric Manipulation Ioannis D. Dadiotis, John S. Sakellariou, and Panagiotis N. Koustoumpardis(B) Mechanical Engineering and Aeronautics Department, University of Patras, 26504 Rio Patras, Greece [email protected], [email protected], [email protected]

Abstract. The development and installation of a low-cost force sensory system (FSS) on a relatively small mobile robot, as well as the design of a proper force controller for fabric manipulation by two cooperative mobile robots, is presented. The FSS consists of two half bridge strain gauge setups which are integrated into a special fabric gripper of a differential-drive wheeled mobile robot (WMR). The FSS is calibrated and experimentally tested through measurements of varying horizontal 2D forces applied on a fabric during its simultaneous manipulation by the FSS-WMR and another robot. The FSS-WMR is used as a follower in a leader-follower formation for the control of the applied forces on the fabric via two independent PID controllers with force feedback. The performance assessment of the FSS-WMR is achieved via experiments with two cooperative WMRs for the transfer of a fabric in straight and curvilinear trajectories. The results indicate that the FSS-WMR may lead to the effective manipulation of fabrics in planar surfaces using two mobile robots, retaining tensional forces within fabric limits, thus rendering the new low-cost FSS a promising option for such types of fabric manipulation. Keywords: Force sensors · Force control · Fabric manipulation · Mobile robots · PID controller

1 Introduction Robotization of fabrics manipulation, such as folding, unfolding, transferring and so on, is a highly challenging task with various difficulties. Dexterous fabric manipulation by mobile robots requires high capabilities of perception and cooperation. The mobile robots should be capable of processing the necessary information, control the fabric tension, and take proper decisions in real time for the precise, fast, and non-catastrophic handling of fabrics. Although robotic manipulators have been widely used for fabric handling tasks, the use of mobile robots with force perception and control for dexterous fabric manipulations © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 47–58, 2021. https://doi.org/10.1007/978-3-030-75259-0_6

48

I. D. Dadiotis et al.

in small surfaces (e.g. the table of a sewing machine) is almost unexplored. This is due to the relatively high cost of the required force/torque (F/T) sensors and constraints such as the availability of limited and low voltage source and the requirement for mobile robots of low weight and volume. In addition, the development of reliable and low-cost force sensors with proper dimensions for small mobile robots that handle fabrics, is an open and challenging research topic. There is a variety of leader-follower(s) (L-F) formations for transferring rigid objects via decentralized force control of mobile robots [1, 2], but their applicability to very flexible objects such as fabrics is very limited. Moreover, transferring and unfolding fabrics using a team of mobile robots based exclusively on visual feedback have been presented in studies [3, 4]. However, these tasks have been achieved without control of the applied forces to the fabrics so as to lie within some critical limits, and thus guarantying that fabrics remain intact after their manipulation. Furthermore, grippers of adequately small dimensions for mobile robots that may be used for fabric manipulation on surfaces with limited dimensions have been presented only in [5]. On the other hand, the use of F/T sensors with proper dimensions for relatively small mobile robots is an important issue due to the fact that oftentimes commercial sensors are quite big and heavy, mostly suitable for typical robot manipulators, while their cost is much higher than the cost of a mobile robot. Recently, high performance F/T sensors have been developed by Bota Systems and integrated in both aerial [6] and legged [7] mobile robots, yet their size and cost is still prohibitive for small lowcost mobile robots. A force sensory system (FSS) may consist of a variety of different sensing elements as presented in [8], such as force sensing resistors, and piezoelectric and capacitive elements of different sensing principles. The first two are suitable for tactile (grasping) force sensors, while they are not appropriate for measuring forces applied on a robot or on its gripper. Such measurements, especially for mobile robots, require low-cost sensors of small size with adequate sensitivity such as the widely used strain gauges. The aim of the present study is the development and installation of a new strain gauges based FSS for relatively small mobile robots, as well as the design of a proper force controller for fabric manipulation by two cooperative, leader-follower, mobile robots. The unique characteristics of the developed FSS and the force controller over the state of the art are: • The low-cost materials, the limited dimensions and the need for low power render the developed FSS along with the used gripper appropriate for many types of mobile robots with limited dimensions. • The developed force controller combined with the high sensitivity of the FSS, allow for L-F mobile robot formations for fabric transfers in small planar surfaces without undesired deformations. The FSS is installed on a unique small fabric gripper of a mobile robot that consists of a low cost 3-finger cams-followers mechanism actuated by a single motor [5]. The force controller is based on two PID and it is designed to retain fabric tension within certain limits (set by the user) during its manipulation by two cooperative mobile robots.

Development of a Low-Cost Force Sensory System for Force Control

49

Both, the FSS and the force controller, are experimentally assessed via the transfer of a fabric in straight and curvilinear trajectories by the two robots.

2 Problem Statement The specific problem that is investigated in this study may be stated as follows: Given a low-cost and relatively small (~25 × 20 × 10 cm) 3-wheel differential drive mobile robot (WMR) equipped with a special 3-finger gripper1 [5]; measure and control the applied forces to a fabric during its manipulation by two cooperative, L-F, mobile robots, which should be moved in a planar surface of limited dimensions. To this end, the development of a novel FSS and a force controller are based on the proper handling of the forces Fx , Fy (see Fig. 1), which are applied from the fabric to the follower’s gripper during the robots relative movement. These forces are of low magnitude up to 1.5 N with one, let’s say positive, direction due to the fact that the manipulated fabric may induce only tensile strength to the gripper. The force on axis Z is negligible due to the fabric handling on a planar surface and thus it is not included in the FSS and the force controller. The angle of the total force to the gripper may vary in the range between 0o and 180o as shown in Fig. 1. It is noted that the FSS is installed on the WMR, with the whole system abbreviated in the following as FSS-WMR, which is the follower in the L-F formation of the two cooperative mobile robots.

Fig. 1. The 3-finger gripper [5] (left), the small mobile robot (right), and the forces applied from the fabric to the gripper (right).

3 The Force Sensory System (FSS) and the Control Approach 3.1 Development and Experimental Testing of the FSS The Design. In a random grasping position, shown in Fig. 2, the grasped fabric exerts a force F on the middle finger of the gripper which can be analyzed in force components Fx and Fy on the horizontal plane XY. To compose the actual force F, the measured Fx and Fy must be available simultaneously. 1 Video with the 3-Finger Robotic Gripper: https://youtu.be/M2ms5rogc-o.

50

I. D. Dadiotis et al.

FXZ'

Fy

F

FX

50º FX FXX' Fy = Fyy'

Fig. 2. Exerted fabric force F to the gripper and coordinate system X Y Z on the middle finger.

The measurement of both Fx and Fy may be achieved via the deformation that each component induces to the middle finger of the gripper. For the development of the FSS we assumed that there is no coupling between the force components, this assumption is validated in Sect. 3. As shown in Fig. 2, the above force components can be analyzed in the finger coordinate system X Y Z in the components Fyy´ (corresponding to Fy ) and Fxz , Fxx (corresponding to Fx ). The Fyy , Fxx cause bending to the finger with respect to axes X and Y respectively, while the Fxz compresses the finger. The measurement of the different deformations which are induced by each of the Fx and Fy necessitates the installation of at least two different strain gauges setups. Therefore, four strain gauges are selected, which are installed on the finger’s perimeter (every 90o ), as shown in Fig. 3(a) and integrated into two half Wheatstone bridges (Fig. 3(b)). The use of half bridge setups is preferred against quarter bridge ones for increased sensitivity of the FSS as the induced forces are of small magnitudes (also see Sect. 2).

Fig. 3. The strain gauge setups (Ri : ith strain gauge resistance, R: constant resistances, Uo , U: input and output voltages, respectively).

Each half bridge is governed by the following equation: k U = (ε1 − ε2 ) Uo 4

(1)

where ε1 and ε2 are the deformations of the two catercorner strain gauges that are integrated in the setup, Uo and U are the input and output voltages, respectively, k is the constant strain gauge factor. According to the force analysis of Fig. 2, each of the ε1 and ε2 consists of one part caused by the finger’s bending and a second part caused by the finger’s compression.

Development of a Low-Cost Force Sensory System for Force Control

51

The parts due to bending have the same magnitude and opposite sign (as defined for two catercorner points on the circumference of a bending beam with cross-sectional section), while the parts due to compression are equal and are canceled according to Eq. (1), which thus becomes: k U = ·ε Uo 2

(2)

where ε is the magnitude of the deformation of each strain gauge due to the finger’s bending. The measurement which is performed by the first half bridge setup is due to the deformation caused by the bending of the finger about axis X , while the one through the second bridge setup is due to the bending of the finger about the axis Y . Thus, the FSS provides two different measurements, say X and Y, which correspond to the force components Fx and Fy . The measurement chain of the proposed FSS, shown in Fig. 4, is based on four Tokyo Instruments FLAB-3-350-11-1LJB-F strain gauges, two custom made half Wheatstone bridges, a RobotShop Load Cell/Wheatstone Shield Amplifier, which incorporates the double channel AD8426 amplifier, a Nucleo F334R8 microprocessor with integrated 12bit ADC and a PC for monitoring the measurements. All elements of the measurement chain (except the PC) are selected in order to be lightweight, small, easily integrated on the robot and powered by a 5 V battery. The achieved measurement force resolution is 0.1 N in the range 0–1.5 N, while the considered force direction angle resolution is 5o in the range 0o –180o (see Fig. 1). It should be noted that the original cylindrical shape (see Fig. 2) of the gripper’s middle finger increases the installation complexity of the four strain gauges. For this reason, the finger modified to a square cross section, as shown in the first photo of Fig. 5, without affecting the other parts of the gripper.

ε

Catercorner ΔR strain gauges

Wheatstone bridge (half)

Microprocessor V Amplifier

ε F

Catercorner ΔR strain gauges

Data output

mV

Gripper’s finger

Signal processing

Signal condi oning

Wireless

Sensing

ADC

Microcomputer

unit (PC)

Wheatstone bridge (half)

Visual display

mV Measurements X, Y

Fig. 4. The measurement chain of the FSS.

The appropriateness of the employed strain gauges to measure the induced deformations to the finger is verified via typical finite element analysis for the case where the total force that is applied to the finger is minimum and equal to 0.1 N with a direction that forms 45o angle with X and Y axes (also see Fig. 2). The results, shown in Fig. 5, indicate that the induced deformations at the four sides of the finger where the strain

52

I. D. Dadiotis et al.

Strain gauge posi ons side A

side B

side C

side D

Fig. 5. Finite element analysis for the case of minimum force magnitude (Solidworks Simulation).

gauges are installed, are higher than the minimum value (10−6 ) that may be measured by them. Experimental Testing. The FSS prototype is properly placed on a 3D printed model of the gripper made of Z-ABS and then all necessary parts are integrated into the robotic platform (see Fig. 6). 1 strain gauge for each side of the finger

WiFi module

Gripper

Half Wheatstone bridges

Amplifier shield

Microprocessor

Fig. 6. The 3D printed gripper and the mobile robotic platform with all necessary parts integrated.

Multiple experiments2 have been initially performed for testing the measurements of the developed FSS based on a sequence of different semi-static and dynamic movements of the grasped fabric with respect to the static robot. In the following, one of the experiments is presented to demonstrate that the performance specifications of Sect. 2 are met. The sequence of movements are (see some frames in Fig. 7): dynamic fabric stretching (force > 1 N) and loosening (1 → 2 → 3), anticlockwise 45o rotation of the loosened fabric (3 → 4), dynamic fabric stretching (force > 1 N) and loosening (4 → 5 → 6), 90o rotation of the loosened fabric, dynamic fabric stretching (force > 1 N) and loosening (→ 7), anticlockwise 45o rotation of the loosened fabric (→ 8). The raw (fraction of the microprocessor’s reference voltage 3.3 V) measurements X and Y via the FSS during the experiment are presented in Fig. 7. It is obvious that the FSS may achieve a clear distinction between the loosened and the stretched fabric through measurement X during the sequence of different fabric movements, and according to the ADC 12-bit 2 All experiments of the current work can be found on the video https://youtu.be/TkOTe_GqpTY.

Development of a Low-Cost Force Sensory System for Force Control

53

resolution, identifies 28 measurement points in the force range 0–0.2 N. Moreover, the fabric rotation is also evident through the measurement Y. Moreover, during the fabric movement (snapshots 1 → 2 → 3) and when the fabric force is applied on the X direction, there is no significant variation in measurement Y, implying that the assumption (Sect. 3.1) of the decoupling between the forces on axis X and Y is reasonable.

Υ X

7

8

Fig. 7. Indicative results from the FSS experimental testing and measurements X, Y.

Based on the above assumption, the calibration of the FSS is executed through matching the measurements X and Y as provided by the FSS, with the actual force components Fx and Fy , following the linear interpolation of the calibration points and using the Mecmesin Portable Force Indicator PFI-200 N GEB. 3.2 Control Approach in a Leader-Follower (L-F) Formation for Fabric Transfer The task of fabric transfer by the FSS-WMR and another WMR in an L-F formation based on the design of a force control approach is studied in this section. The FSSWMR is the follower where the controller is designed using force feedback from the FSS, while the leader is a “blind” robot following a predefined path. In the cooperative fabric transfer (see Fig. 8) the follower (F) may feel the force components Fx and Fy through the FSS and the controlled fabric manipulation is considered successful when the L-F formation is aligned as shown in right part of Fig. 8. Based on this, the reference force component Fyref should be 0 N, while the reference Fxref is set to 0.2 N to keep the fabric unwrinkled.

54

I. D. Dadiotis et al.

FSS-WMR WMR

Fig. 8. Force control approach in the leader-follower formation.

The magnitude of Fy that is applied to the follower depends on the relative angular rotation between the two robots and therefore it can be controlled via the properly selected angular velocity commands to the follower. Similarly, the magnitude of Fx can be controlled via proper speed commands to the follower. These commands are engaged in the FSS-WMR kinematics based on the solution of the inverse problem: ⎤⎡ ⎤ ⎡ ⎤ ⎡ l 1 v ϕ˙l ⎣ ⎦ = ⎣ d 2d ⎦⎣ ⎦ (3) l 1 ω ϕ˙r d − 2d where ϕ˙l , ϕ˙r are the angular velocity of the left and right driving wheel of the robot, respectively, d the diameter of the wheels, l the half distance between the two driving wheels of the robot, v the speed of the robot and ω the angular velocity of the robot. Due to technical limitations of the motors used in this study, the velocity of each wheel must always preserve a minimum value which is formulated as follows: ϕ˙l = 0.4 + fc (v) + g(ω) ϕ˙r = 0.4 + fc (v)−g(ω)

(4)

where ϕ˙l and ϕ˙r are normalized with respect to the corresponding maximum velocity, 0.4 is a constant that is added in order to set a minimum velocity to the motors and fc (v), g(ω) are factors determined by the corresponding robot speed and angular velocity so as fc (v) and g(ω) are bounded in the range [−0.1, 0.1]. Based on the above, a control approach which uses the force measurements of the FSS and consists of two PID controllers is developed, as shown in Fig. 9. In particular, the Steering PID controller is responsible for the control of the force component Fy and provides the follower with commands determined by the angular velocity ω, while the Speed PID controls the force component Fx and provides the follower with commands determined by the speed v. The raw measurements of X and Y are used for feedback, and therefore the values of Fxref , Fyref in the beginning of the task are matched to raw values of Xref and Yref according to the initial FSS calibration.

4 Experimental Validation The experimental validation of the developed FSS and the above control approach via the transfer of a fabric by two cooperative mobile robots is presented in this section. The L-F robot formation is assigned to transfer a small fabric (~20 × 20 cm) on a table (Fig. 10(a)) following: (i) a straight line, (ii) a clockwise curvilinear and, (iii) an

Development of a Low-Cost Force Sensory System for Force Control Y

ref

+

-

ey

Measurement Y Steering PID

g(ω) Inverse Kinema cs

Xref

e + -

x

55

Speed PID

Wheels’ Veloci es

Follower robot & FSS

Eq. (4) fc(v) Measurement X

Fig. 9. The FSS-WMR (follower) force control approach. The PID gains are determined via a trial and error procedure and are Kp = 15, Kd = 0.25, KI = 80 for the Speed PID and Kp = 25, Kd = 70, KI = 110 for the Steering PID while the control period is 55 ms.

anticlockwise curvilinear trajectory. Each robot has wireless communication through Wi-Fi network with a central PC that is responsible for the follower’s (FSS-WMR) control implementation, transmitting the controlled wheels velocities to both robots (see Fig. 10(b)).

Fig. 10. (a) The experimental setup with the FSS-WMR (follower) and a typical robot as a leader using a simple clamp gripper, and (b) schematic of the robots Wi-Fi connection with the PC.

Fig. 11. Transfer of the fabric by the two cooperative robots following: (a) the straight-line trajectory, (b) the clockwise curvilinear trajectory and, (c) the anti-clockwise curvilinear trajectory.

56

I. D. Dadiotis et al.

A sequence of snapshots during the experiments with the three trajectories is presented in Fig. 11. Figure 12 includes the measured force components applied to the follower in each considered trajectory, as well as the corresponding resultant forces applied to the fabric. Both, Fx and Fy force components are controlled successfully in all test cases, reaching magnitudes close to the set reference values very fast ( zcom , 0 < zzmp < zcom and zzmp < 0, respectively. (b) ¨ com . The sign of ϕzmp with respect to the sign of x

If the additional conditions pz = 0 and z¨com = 0, used for the derivation of the linear inverted pendulum model, are considered, Eq. (10) translates to Eq. (5). Condition (4) can be for a humanoid, oriented towards the positive direction of the x axis, also expressed with the angles of its heel and its toes with respect to the vertical line, passing through its COM, as ϕheel ≤ ϕzmp ≤ ϕtoes ,

(11)

where ϕheel = arctan

x

heel

− xcom 

zcom

and ϕtoes = arctan

x

toes

− xcom 

zcom

,

(12)

as shown in Fig. 4. If the humanoid is oriented in the opposite direction, ϕheel and ϕtoes must be in Eq. (12) replaced one with the other. In case when only one foot of the humanoid lies on the ground, both ϕheel and ϕtoes refer to the heel and the toes of the same foot, while if both feet of the humanoid lie on the ground, ϕheel and ϕtoes refer to the heel and the toes of the back and the front foot, respectively. If the condition (11) is fulfilled, the ZMP coincides with the CoP and the humanoid is balanced, while if the condition (11) is not fulfilled, the humanoid will flip either over its heel or over its toes.

Angular Dependency of the Zero Moment Point

141

Fig. 4. The angles of the locations of the heel and the toes of the foot of the humanoid with respect to its COM, as defined by Eqs. (12).

3

Simulations

To better understand and visualise the behaviour of ϕzmp , some calculations have been performed. Figure 5a shows the relation between x ¨com and ϕzmp for some discrete values of z¨com . It should be emphasised that z¨com is the actual acceleration of COM in the vertical direction. For the case when z¨com = 0, the vertical component of F zmp only balances the gravitational acceleration of the COM. But if z¨com = g, the COM is accelerated to the ground with the acceleration due to gravity and thus F zmp has no vertical component. The four red dots in Fig. 5a correspond to all four combinations of two different values of x ¨com , 5 m/s and 15 m/s, and two different values of z¨com , 5 m/s and −15 m/s, while their corresponding ZMP lines are shown in Fig. 5b. As z¨com > g for the dots 1 and 2, their corresponding ZMP angles lie in the range 0 > ϕzmp > −90◦ . On the other hand, z¨com < g for the dots 3 and 4 and therefore their corresponding ZMP angles lie in the range −90◦ > ϕzmp > −180.

4

Discussion

One of the drawbacks of defining the location of the ZMP with Eq. (3) or (5) is that in some situations the value of pgx may be very large or even infinite, and thus useless in practice. Moreover, in some cases, the forces accelerating the humanoid in the vertical direction cannot be neglected, so the ZMP cannot be determined using the widely used linear inverted pendulum model from Eq. (5).

142

T. Brecelj and T. Petriˇc

(a)

(b)

Fig. 5. (a) Calculations of ϕzmp for different values of x ¨com and z¨com . The four red dots represent four different values of ϕzmp obtained with the following parameters: dot 1: xCOM = 15 m/s2 , z¨COM = 5 m/s2 ), dot 3: (¨ xCOM = 5 m/s2 , z¨COM = 5 m/s2 ), dot 2: (¨ xCOM = 15 m/s2 , z¨COM = −15 m/s2 ). (¨ xCOM = 5 m/s2 , z¨COM = −15 m/s2 ) and dot 4: (¨ (b) Four ZMP lines corresponding to the four dots from (a).

All these scenarios are overcome with the angular definition of the ZMP. When ϕzmp approaches ±π/2, the ZMP does not need to lie at extremely large distances from the COM, but can be positioned at any point of the ZMP line, defined by Eq. (10). And in the extreme case, when ϕzmp = ±π/2, the ZMP is on the horizontal ZMP line. Furthermore, the ZMP is not restricted to the locations where 0 ≤ pz ≤ zcom , but it can also lie below the ground where pz < 0, or above the COM, where pz > zcom , as long as it lies on the ZMP line. Additionally, the angular definition of the ZMP also accounts for cases where the COM of the humanoid is accelerated either upward or downward. There is another advantage to expressing ZMP as ϕzmp instead of pgx . As long as ϕzmp is inside the support polygon, the equilibrium of the humanoid can be ensured by all reaction forces acting inside the support polygon and satisfying the force and torque equilibrium condition. This is particularly convenient when the humanoid is in equilibrium not only through its feet, but also, for example, through its arms or other support mechanisms whose reaction forces do not act on the ground, as in Fig. 6. The advantage of the angular expression of the ZMP can also be formulated the other way around: Knowing the value of ϕzmp , the reaction forces acting through the arms, legs, or other support mechanisms that provide force and torque balance to the humanoid must be applied at such angles as to ensure that ϕzmp is embedded in the support polygon.

Angular Dependency of the Zero Moment Point

143

Fig. 6. A humanoid with the force F com acting at its center of mass, balanced by two forces acting at its foot and its arm, F foot and F arm , respectively, and creating a support polygon, embedding the ZMP. For completeness, the gravitational force, F g , is also shown.

5

Conclusion

In this paper, derivations of both the standard and angular definitions of the ZMP are presented and discussed. It is shown that the angular definition allows the ZMP to be expressed more generally, since it allows the ZMP to be at any height, even below the ground or above the COM of the humanoid, and also accounts for cases where F zmp is horizontal. To help the reader understand the behaviour of ϕzmp with respect to the accelerations of the humanoid COM in the horizontal and vertical directions, some computer simulations are also shown. Finally, the advantages of the angular definition of ZMP are presented on a real example of a humanoid robot supported by both its foot and arm at different heights, where the standard definition of ZMP could not be used.

References 1. Atkeson, C.G., et al.: Using humanoid robots to study human behavior. IEEE Intell. Syst. Appl. 15(4), 46–56 (2000). https://doi.org/10.1109/5254.867912 2. Babiˇc, J., Lenarˇciˇc, J.: Optimization of biarticular gastrocnemius muscle in humanoid jumping robot simulation. Int. J. Humanoid Rob. 3, 219–234 (2006) 3. Ijspeert, A.J.: Central pattern generators for locomotion control in animals and robots: a review. Neural Netw. 21(4), 642 – 653 (2008). https://doi.org/10.1016/ j.neunet.2008.03.014. Robotics and Neuroscience

144

T. Brecelj and T. Petriˇc

4. Kajita, S., Tani, K.: Study of dynamic biped locomotion on rugged terrainderivation and application of the linear inverted pendulum mode. In: Proceedings. 1991 IEEE International Conference on Robotics and Automation, vol. 2, pp. 1405–1411 (1991). https://doi.org/10.1109/ROBOT.1991.131811 5. Lahajnar, L., Kos, A., Nemec, B.: Skiing robot - design, control, and navigation in unstructured environment. Robotica 27, 567–577 (2009). https://doi.org/10.1017/ S0263574708004955 ˇ 6. Petriˇc, T., Gams, A., Babiˇc, J., Zlajpah, L.: Reflexive stability control framework for humanoid robots. Auton. Rob. 34 (2013). https://doi.org/10.1007/s10514-0139329-0 ˇ 7. Petriˇc, T., Zlajpah, L., Garofalo, G., Ott, C.: Walking control using adaptive oscillators combined with dynamic movement primitives, January 2013 8. Righetti, L., Ijspeert, A.: Programmable central pattern generators: an application to biped locomotion control, vol. 2006, pp. 1585–1590, June 2006. https://doi.org/ 10.1109/ROBOT.2006.1641933 9. Shin, D., Sardellitti, I., Park, Y.L., Khatib, O., Cutkosky, M.: Design and control of a bio-inspired human-friendly robot. Int. J. Robot. Res. 29(5), 571–584 (2010). https://doi.org/10.1177/0278364909353956 10. Vukobratoviˇc, M., Juriˇci´c, D.: Contribution to the synthesis of biped gait. IEEE Trans. Bio-med. Eng. 16(1), 1–6 (1969) 11. Vukobratoviˇc, M., Borovac, B.: Zero-moment point - thirty five years of its life. I. J. Humanoid Rob. 1, 157–173 (2004). https://doi.org/10.1142/S0219843604000083

Marker-Based Automatic Dataset Collection for Robotic Vision System Denis Chikurtev1(B)

and Kaloyan Yovchev2

1 IICT, Bulgarian Academy of Sciences, Sofia, Bulgaria

[email protected] 2 Faculty of Mathematics and Informatics, Sofia University, Sofia, Bulgaria

[email protected]

Abstract. Nowadays, the robotics is making significant progress in terms of its application in all sectors of industry, economy and household. Computer vision algorithms and systems have made a major contribution to this progress. The article presents the architecture of a robotic vision module, which allows robots to detect and recognize objects. The module also is able to collect data for unknown objects and pretrain its neural network model. The module is based on algorithms for image processing, convolutional neural networks, augmented reality markers and machine learning. An algorithm for identifying new objects and self-learning of the vision module is presented. The obtained results show that the developed system successfully handles the task of recognizing objects and self-learn. Keywords: Automatic dataset collection · Augmented reality markers · Object recognition · Robotic vision

1 Introduction Service robots are designed to work in various environments, home or office, manufacturing facilities, outdoors, dangerous places, etc. [1]. Most of those robots are used indoors and they are designed to interact with people and objects that are located within their environment [2]. In order for this interaction to take place, robots must be able to recognize different objects and understand what is happening around them [3, 4]. To achieve this goal, various algorithms from the field of the computer vision, machine learning, and deep learning techniques are used [5]. In fact, the main technology is computer vision, and algorithms for recognizing, finding and classifying objects are based on machine learning. Object recognition is one of the main important concepts in the field of the service robotics [6]. Through the computer vision, the robots receive information about their surroundings, objects and people around them [7, 8]. Computer vision can be incorporated within the control system of the robots so that they are able to recognize objects or to detect objects within an image received from their color camera sensors [9]. While object recognition provides information about the objects themselves, object detection provides information about the location of the object in the image. These two concepts © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 145–153, 2021. https://doi.org/10.1007/978-3-030-75259-0_16

146

D. Chikurtev and K. Yovchev

in most cases work simultaneously. By detecting and recognizing objects in the data from the video camera attached to the robot, the robot can understand the position and orientation of the object relative to itself and interact with the object accordingly [10]. Nowadays, the main methods and algorithms used as part of computer vision are based on machine learning and deep learning techniques [11, 12]. These methods use mainly convolutional neural networks to train a system, and at the output present as a result the recognized objects and their positioning on the image, as well as the probability of authenticity of the recognized object [13]. Among the modern and most popular algorithms for object recognition and localization are YOLO (You Only Look Once), Single Shot Detector (SSD), Fast R-CNN, Faster R-CNN, Region-based Convolutional Neural Networks (R-CNN) [14–16]. These algorithms are so called detector algorithms and they can be classified to onestage object detectors (e.g., DSSD, RetinaNet, RefineDet) or two-stage object detectors (e.g., Mask RCNN, DetNet) [17, 18]. All these algorithms are based on the use of layers of convolutional neural networks, structured and tuned by different methodologies. The principle of using and applying any of the mentioned algorithms requires the implementation of the following steps: • • • •

collection of training dataset; setting training criteria; implementation of a training process; starting the trained system and object classifier.

As already stated, there are various well researched algorithms for implementation of the training process. But the first step of collection of a training dataset is crucial to the overall performance of the classifier. This is a time-consuming task which is supposed to be executed within the actual working environment of the robotic system. At the moment automated methods for dataset collection and training are not well researched. These two processes require a significant resource of time and calculations. This study aims to automate these processes and provide an autonomous or semi-autonomous subsystem that can easily collect new data and update its database and periodically retrain its object classifier. The idea is this method to be universal and independent of the used convolutional neural network architecture. In this research a novel algorithm is proposed which uses augmented reality markers (Fig. 1) in order to automate the dataset collection step.

Fig. 1. Augmented reality markers.

Further incorporating of this algorithm into the vision system of the robot will allow the robotic system to achieve greater autonomy and reduce the need of a robotic engineer for the actual programming and training of the object classifier. The regular user will be able to enrich and expand the knowledge of the robot for various objects, used in

Marker-Based Automatic Dataset Collection

147

their everyday life. Even if a new object appears in the home or office, after a certain number of encounters with this object, the robot will be able to automatically classify and recognize it. In addition, this method will improve the human-robot interaction, because it requires simple and minimal actions by the user to set the new object classes and types.

2 Motivation and Experimental Setup We will consider a service mobile robot with RGB camera that must recognize objects in its work area. If the objects are known in advance and are a fixed number, then a neural network can be applied for their classification. The problem with service robots is that they operate in a non-deterministic environment in which recognized objects can be changed (even visually) or new ones are added. Also, mobile service robots should operate in a user environment that will be specific to each different user (different lighting conditions, furniture colors, etc.). This will require individual training of the computer vision system for each integrated robotic system of this type. In this case, a standard approach should collect a sufficient amount of new data (photos of either new or changed objects) and perform a new neural network training. This is a laborious process. The purpose of this publication is to propose an algorithm by which this training can be performed by the user of the service robot without the intervention of a programmer (robot operator). This is possible because, in reality, the service robot can collect a large enough amount of data (photos) on the new objects while moving, but there must still be a way for it to know the ground truth of exactly which object was captured. For the purposes of the publication, we will limit the possible objects to those with small to medium sized, for example: cardboard boxes up to 1-L, different types of desserts, etc. This motivation leads to the workflow of the robotic vision module described in the following section.

3 Workflow of the Robotic Vision Module (RVM) The robotic vision module (RVM) proposed in this publication will implement an algorithm for marker-based automatic dataset collection. RVM is required to implement a convolutional neural network that can classify the various marked objects via RGB camera image input. Through the trained neural network, the RVM should also be able to recognize objects even if they are not marked. For the purposes of this study, the neural network architecture created by Simonyan and Zisserman [11] is chosen. With the described requirements the following algorithm can be formulated: 3.1 Algorithm for Marker-Based Automated Dataset Collection and Object Recognition 1. The necessary set of objects is created in the database of the RVM through a user interface. For the newly created objects, the robotic system will have to collect the necessary dataset automatically and afterwards it will start a new training of the

148

D. Chikurtev and K. Yovchev

neural network classifier. The user interface will require from the user to enter for each object a name and its proportions - height and width of the objects and then will generate a fixed size unique marker associated with a database entry which contains information about the corresponding object class. Those markers will be placed on the gravity center of the objects in order to be used for ground truth data in the training process. The markers must be read through an RGB camera in realtime. The AprilTags developed by APRIL Robotics laboratory are suitable for this purpose [19]. The computer vision module will be based on OpenCV and for this reason ArUco markers, which are an extension of the AprilTags that is integrated into the OpenCV library will be used [20]. 2. The user will need to attach the generated markers in the gravity center of the corresponding objects. In Fig. 2 are shown objects with attached markers. Since the video stream is two-dimensional and the objects are three-dimensional, the user will have to attach markers on each side of the objects in the corresponding center of gravity. If the sides of the objects are asymmetric, then the user will have to fill in data for each object side and generate a different tag for it.

Fig. 2. Objects with attached augmented reality markers.

3. Before the initial training of the neural network for any new class, the RVM must collect a sufficient number of input images. RVM is actively detecting the generated markers in the video stream of the camera during the movement of the robot and is collecting training data for the marked objects. When the ArUco marker is detected, the module will know where the center of the object is. In addition, the marker provides information about the orientation of the object. In the database, the user has recorded the proportions of the respective side of the object, which allows the module to cut from the video frame only the data for the specific object and add them to the set of data.

Marker-Based Automatic Dataset Collection

149

4. RVM periodically retrains the neural network. During its entire operation, the module collects data and expands its database of images of objects with which it can work. The RVM periodically checks for new data. If a certain number of new images for a given class is collected, then a retraining process is activated. This criterion can be changed according to the environment in which the robot is operating. This allows the training to be performed with always up-to-date data, which are collected in the actual working environment of the service robot. This algorithm determines the workflow of the automated dataset collection of the RVM. 3.2 Workflow of the RVM RVM monitors the video stream of the camera and performs simultaneously the following two actions: object recognition and marker detection. Object recognition is the more expensive operation from a computational point of view, and depends on the performance of the RVM hardware, so it is supposed to be executed less frequently and when the RVM is in idle state. When a marker is detected the RVM will do the following: 1. Depending on the proportions entered in the database, the image for the selected object should be cropped and resized. With this normalization, it should be reduced to a square of 96 × 96 pixels. There must be pixels in the normalized image that belong only to the subject. 2. The image is saved to the training data and will wait for further training. 3. The RVM will mark the founded objects based on the markers. When recognizing an object: 1. RVM takes the entire frame of the video stream and starts scanning it for objects. The scan must be performed in real time. This scan detects all potential connected components of pixels, which are then enclosed in a rectangle, reduced to a normalized shape of 96 by 96 pixels. Each image is checked through the neural network. 2. The RVM marks objects founded and successfully classified by the neural network. The design of the proposed RVM is shown in Fig. 3. It includes software modules, which are connected accordingly to the proposed requirements, algorithm and workflow. The final result of the system is camera frame with marked known objects.

150

D. Chikurtev and K. Yovchev

Fig. 3. Structure of the proposed Robot Vision Module.

4 Experiments A set of 6 objects was used for the experimental verification of the proposed approach for automated dataset collection and subsequent object recognition, three of which have the same shape but different color schemes. The selected size of the markers is 1.5 by 1.5 cm. It was chosen experimentally, after establishing that when normalizing the input image, markers with this size occupy an area that does not affect the recognition and classification of the objects extracted from the video stream. The size will vary with the overall size of the objects. The process of segmentation of the objects from the background in the frames of the video stream is done by resizing the frame to a width of 300 pixels and converting it to a gray scale image. This improves the performance of the segmentation process. A Gaussian Blur filter is then applied, and the resulting image is binarized by using the Otsu’s method. Then, with the OpenCV library, the closed contours in the binary image are detected. A mask with rectangular shape is mapped to each contour, through which the pixels of the object are extracted from the input color image. The extracted image is then normalized and given as an input to the neural network classifier. In Fig. 4 is shown a video frame from the RVM during the object recognition stage. On top right of the figure is shown the binarized video frame. In the conducted experiments it was found that the proposed approach is suitable for automated data collection and manages to successfully recognize objects in real time. The process of separating an object from the background is sensitive to the contrast in the colors of the objects and the background, and if the colors are similar, then the pixels of the objects cannot be determined correctly. Additional research should be performed on the process of segmenting objects from the background. The performance of the proposed system is measured only on CPU. We tested the performance on Intel® Core™ i7-4710HQ and Intel® Core™ i7-9750H CPUs. The

Marker-Based Automatic Dataset Collection

151

Fig. 4. Video frame with recognized objects.

frames per second for the first processor were within the range 3–15 and for the second processor was within the range 7–24 depending on the number of segmented objects. We plan to test the module on CUDA based GPU and Nvidia Jetson Nano board.

5 Conclusion and Discussion The proposed approach successfully automates the process of dataset collection and training a robotic vision system. It makes it accessible to ordinary users of the service robot, as it provides them with a user-friendly interface. The system combines the ability to use a neural network and markers used for augmented reality applications. This allows certain objects that are too small or difficult to distinguish from each other to be pre-marked. This is suitable for objects that are used over a long period of time, i.e., are not consumable items, such as glass cups. On the other hand, objects that are consumables such as a box of milk do not need to be marked every time, and the neural network will be relied on to recognize them. Future research may focus on optimizing the process of scanning the video stream, using data from robot distance sensors or depth sensor. This will allow the video stream to be divided into zones at equal distances from the robot’s camera and a separate scan to be performed in these smaller areas. Another way to improve productivity is through the use of specialized hardware. It is also possible to set zones in the navigation system of the mobile robot in which to perform the scan, and for each zone the set of possible objects to be known in advance. Acknowledgement. The research presented in this paper was supported by the Bulgarian National Science Fund under the contract # KP-06-M27/1 – 04.12.2018.

152

D. Chikurtev and K. Yovchev

The work was partially supported by the Bulgarian Ministry of Education and Science under the National Research Programme “Young scientists and postdoctoral students” approved by DCM # 577/17.08.2018.

References 1. Lu, V.N., Wirtz, J., Kunz, W., Paluch, S., Gruber, T., Martins, A., Patterson, P.: Service robots, customers and service employees: what can we learn from the academic literature and where are the gaps? J. Serv. Theory Pract. 30, 361–391 (2020) 2. Lee, C.Y., Lee, H., Hwang, I., Zhang, B.T.: Visual perception framework for an intelligent mobile robot. In: 2020 17th International Conference on Ubiquitous Robots (UR), pp. 612– 616. IEEE (2020) 3. Shakev, N., Ahmed, S., Popov, V., Topalov, A.: Recognition and following of dynamic targets by an omnidirectional mobile robot using a deep convolutional neural network. In: 2018 International Conference on Intelligent Systems (IS), pp. 589–594. IEEE (2018) 4. Chan, D., Riek, L.: Unseen salient object discovery for monocular robot vision. IEEE Robot. Autom. Lett. 5(2), 1484–1491 (2020) 5. Meyer, B.J., Drummond, T.: The importance of metric learning for robotic vision: open set recognition and active learning. In: 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, pp. 2924–2931 (2019) 6. Ivanov, A., Chivarov, N.: Methods for object recognition and classification for tele-controlled service robots. In: IOP Conference Series: Materials Science and Engineering, vol. 878, no. 1, p. 012005. IOP Publishing, Bristol (2020) 7. Eden, B., Rosman, B.: An overview of robot vision. In: 2019 Southern African Universities Power Engineering Conference/Robotics and Mechatronics/Pattern Recognition Association of South Africa (SAUPEC/RobMech/PRASA), Bloemfontein, South Africa, pp. 98–104 (2019). https://doi.org/10.1109/RoboMech.2019.8704781 8. Lan, G., Benito-Picazo, J., Roijers, D.M., Domínguez, E., Eiben, A.E.: Real-time robot vision on low-performance computing hardware. In: 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV), Singapore, pp. 1959–1965 (2018). https://doi. org/10.1109/ICARCV.2018.8581288 9. Lee, J., Wu, Y., Jhao, Y., Chen, L., Chen, H.: Development of mobile robot with vision inspection system and three-axis robot. In: 2018 3rd International Conference on Control and Robotics Engineering (ICCRE), Nagoya, pp. 6–10 (2018). https://doi.org/10.1109/ICCRE. 2018.8376424 10. Lin, H., Chen, Y., Chen, Y.: Robot vision to recognize both object and rotation for robot pickand-place operation. In: 2015 International Conference on Advanced Robotics and Intelligent Systems (ARIS), Taipei, pp. 1–6 (2015). https://doi.org/10.1109/ARIS.2015.7158364 11. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014) 12. Graziani, M., Eggel, I., Deligand, F., Bobák, M., Andrearczyk, V., Müller, H.: Breast histopathology with high-performance computing and deep learning. Comput. Inf. 39(4), 780–807 (2020) 13. Ahmed, N., Shahzad Asif, H.: Perceptual quality assessment of digital images using deep features. Comput. Inf. 39(3), 385–409 (2020) 14. Shafiee, M.J., Chywl, B., Li, F., Wong, A.: Fast YOLO: a fast you only look once system for real-time embedded object detection in video. arXiv preprint arXiv:1709.05943 (2017) 15. Shaohua, W., Goudos, S.: Faster R-CNN for multi-class fruit detection using a robotic vision system. Comput. Netw. 168, 107036 (2020). ISSN 1389-1286

Marker-Based Automatic Dataset Collection

153

16. Zhang, S., Wen, L., Bian, X., Lei, Z., Li, S.Z.: Single-shot refinement neural network for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4203–4212 (2018) 17. Li, Y., Ren, F.: Light-weight retinanet for object detection. arXiv:1905.10011 (2019) 18. Zhao, Q., Sheng, T., Wang, Y., Tang, Z., Chen, Y., Cai, L., Ling, H.: M2Det: a single-shot object detector based on multi-level feature pyramid network. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 9259–9266 (2019) 19. Wang, J., Olson, E.: AprilTag 2: efficient and robust fiducial detection. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4193–4198. IEEE (2016) 20. ArUco marker documentation. https://docs.opencv.org/3.4/d5/dae/tutorial_aruco_detection. html. Accessed 28 Jan 2021

Rehabilitation Robotics and Exoskeletons

Design of a Cable-Driven Parallel Robot for the Cervical Spine Motion Training Alizée Koszulinski, Eliot Stantinat, Léa Ollivier, Juan Sandoval, and Med Amine Laribi(B) Department of GMSC, Pprime Institute CNRS, ENSMA, UPR 3346, University of Poitiers, Poitiers, France {alizee.koszulinski,eliot.stantinat, lea.ollivier}@etu.univ-poitiers.fr, {juan.sandoval, med.amine.laribi}@univ-poitiers.fr

Abstract. In this paper, a novel device intended to assist therapists in rehabilitation of the cervical spine poly-joint is proposed. Firstly, rehabilitation motion of the cervical spine is analysed. The poly-joint amplitude and the surface swept by the head are studied and recorded using a motion capture system. Based on these measurements and design constraints, an orthosis is proposed to allow an adaptability to several subject and confer a sufficient support of the head when performing the movements. The adopted structure is chosen among several proposals using a survey. The preliminary architecture of a proposed cable-driven parallel robot (CDPR) is able to replicate all the movements. Moreover, the design of the end-effector as a universal joint simplifies the CDPR modelling by limiting the number of degrees of freedom (DoF) necessary to perform the various tasks. A kinetostatic model of the CDPR is conducted and the space statistically accessible inside its workspace is built. Keywords: Cervical spine motion · Rehabilitation movements · Motion capture system · Cable-driven parallel robot

1 Introduction The cervical spine injuries are not to be taken lightly, in fact, more than two out of three French people suffer from neck pain during their lifetime. And it is one of the leading causes of disability [1]. These pathologies can be caused by muscle imbalance, trauma, stress or degenerative disease. It is therefore essential to practice rehabilitation in order to limit pain and especially a risk of recurrence. The cervical spine is a brittle polyjoint of great complexity at 3 degrees of freedom (DoF). For this reason, rehabilitation is practiced by physiotherapists who have the mission to manipulate this poly-joint carefully and within specific joint limits [2, 3].

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 157–166, 2021. https://doi.org/10.1007/978-3-030-75259-0_17

158

A. Koszulinski et al.

Nowadays, robotic rehabilitation devices are mainly stationary devices composed of a robotic orthosis with the advantage of providing an active or passive compensation force giving a direct impact for rehabilitation. To illustrate these points, ETS MARSE [4] is a robot allowing the rehabilitation of the joints of the upper limbs; however, its 7-DoF complicate the human-machine coordination. We also find cable-driven robots like the STRING MAN [5], a robot specialized in the rehabilitation of the lower limbs using cables to limit the weight of the body while adjusting the required assistance. In rehabilitation devices specialized in the cervical spine, foam neck braces are mainly used, serving to relieve tensions and to limit pathologies [6, 7]. However, the immobilization of the joints promotes amyotrophy of the neck muscles. Finally, the ROBOTIC NECK BRACE is a mobile cervical spine rehabilitation robot weighing approximately 1 kg. It is based on the principle to promote the mobility of the cervical spine to allow rehabilitation of the concerned muscles. Nevertheless, the workspace may be a disadvantage in covering only 70% of the amplitude of the cervical spine poly-joint [8]. The use of a cable-driven is an interesting choice since besides its low cost, its large working space allows covering the entire movement of the cervical spine. Moreover, the precise kinetostatic behaviour as well as the creation of constraints only in traction, make the device safer and more reassuring for users. This paper is structured as follows: Sect. 2 details the characterization of cervical spine amplitude by a motion analysis study. Section 3 exposes the design of the endeffector and the choice of the structure. Section 4 specifies the robot modelling. Presentation and discussion of the results are provided in Sect. 5. The last section concludes this paper.

2 Head Joints Range of Motion In order to allow the rehabilitation of the cervical spine poly-joint, four movements have been identified. As shown in the Fig. 1, the first tree movements are flexion/extension, the right and left inclinations and the right and left axial rotation of the head. The fourth movement studied is a circular combined movement which corresponds to the full rotation of the head through the maximum flexion/extension positions and lateral inclinations to the right and left.

Fig. 1. Three out of the four studied movements [9].

Design of a Cable-Driven Parallel Robot

159

An experimental measurement campaign has been carried out, using a motion capture system in order to analyse the angular amplitudes of the studied poly-joint. The overall aforementioned movements have been recorded using a Qualisys motion capture system composed of 8 high-resolution infrared cameras and a set of 7 reflective markers fixed to specific anatomical locations as shown in Fig. 2. Four volunteers aged from 20 to 30 were invited to participate to the experimental measurements. Before beginning the movements, subjects had to sit in the back of a chair with their backs straight and glued to the backrest. The middle of their heel had to be aligned with the chair legs and their shoulders had to be relaxed. The subjects’ heads had to be vertical, the gaze into the distance and the hands positioned on the thighs. Once the subject was in a neutral position, a stretching and warm-up session was organized in order to warm up the muscles and allow the accuracy of the measurements. While recording movements, the volunteers completed 10 cycles for flexion/extension, right/left lateral inclinations and right/left rotations. Finally, subjects performed 5 cycles of circular combined motion by turning clockwise and then 5 more cycles anti-clockwise.

Fig. 2. Markers locations.

Fig. 3. Coordinate systems in initial position [10].

In order to characterize the range of motion of each subject, local coordinate systems of the head and thorax were defined using various markers placed on the head and trunk (Fig. 3). Based on the orientation of the head coordinate system, the x-axis corresponds to the flexion-extension axis, the y-axis represents the lateral inclinations axis and the z-axis is associated with the head rotation axis. The head rotation matrix as well as the surface scanned by this latter during the various movements were determined through an algorithm developed in MATLAB software. To obtain the surface scanned, the hypothesis that the centre of rotation of the head is located near the intervertebral joint C7-T1 [11, 12] was made. The resulting surface has therefore a spherical shape as illustrated in Fig. 4.

160

A. Koszulinski et al.

Fig. 4. Surface scanned by the head.

3 Design of the End-Effector and CDPR Structure Selection The end-effector is composed of two parts, namely an orthosis and inner/outer crowns. The orthosis is the flexible part of the end-effector, which allows an adaptability to several subjects (Fig. 5). Each crown and the orthosis are connected by revolute connections of orthogonal axis. The movements of the outer crown are then contained in a horizontal plan. Since the cables tie points of the robot are located on the outer crown, this design in universal joint equivalence greatly simplifies the modelling.

Fig. 5. End-effector.

For the structure, a 1-by-1-m frame allows to delimit a sufficient workspace to contain all the movements of the end-effector. In order to select the most suitable structure of the CDPR several criteria were chosen: the compactness, the rigidity, the need for additional materials, the ease of accessibility for the subject, the ease of assembly and the cost. To make the most objective choice possible, a survey was carried out among ten research teachers and master’s students in engineering. The evaluation of each criterion was carried out with a mark ranging from 1 to 5: a score of 5 corresponding to a very satisfactory assessment of the criterion. In order to give more weight to some criteria than others, they have been weighted according to their importance. The results of the study are shown in Table 1.

Design of a Cable-Driven Parallel Robot

161

Table 1. Evaluation of the different structure proposals. Structure

4-legged stand

Z stand

Corner stand with one leg

Corner stand with two legs

Ground

Ground

Tables

Tables

CAD model

Compactness

3,4

3,0

3,8

3,4

3,7

3,5

Stiffness

4,8

3,4

1,7

2,5

2,7

3,9

Additional equipment

1,7

2,3

4,3

2,0

4,2

2,3

Access

2,5

4,0

3,7

3,0

3,4

3,3

Installation

3,3

3,6

3,4

3,4

2,8

3,3

Cost

2,0

3,4

3,5

4,1

2,9

3,5

Total

3,0

3,2

3,2

3,1

3,2

3,4

Figure 6 shows the design of the adopted structure for the CDPR and the position of the subject under the frame. The compactness is optimal since it is held in a 1-by1-m frame. Rigidity is promoted by maintaining two sides of the frame. This frameholding configuration will simplify the installation of actuator modules later. Finally, the equipment made available allows the structure to be placed on two tables reducing the cost.

Fig. 6. Representation of the structure and a subject.

The outer crown of the end-effector is connected to the upper frame of the structure by four cables driven by servomotors. Orientable pulleys are used to direct these cables to the end-effector (Fig. 7).

162

A. Koszulinski et al.

As shown by the results of the motion capture, the neck is a complex poly-joint and its movements cause head rotation around the x, y and z axis as well as translations along the x and y axis. These movements require a 5-degree-of-freedom robot. However, the design of the cardan end-effector allows to support two free rotations at the level of the outer crown. Indeed, the rotations around the x and y axis are not transmitted to the outer crown due to the two revolute links. Thus, performing the different tasks requires only 3-DoF (two translations along the x and y axis and one rotation around the z axis): four cables are then necessary [13]. In addition, to perform the rotation of the head as much as possible by the robot, the cables are tied crossed at the end-effector.

Fig. 7. Diagram of the CDPR frame components.

The exit points of the orientable pulleys are included in the same plan as the one formed by the points of attachment. This simplifies the modelling of the CDPR, which therefore can be done in 2D.

4 Kinetostatic Model of the CDPR The aim of the modelling is to find, from a given set of the CDPR geometric parameters, the value of the cables tensions for a given position and orientation of the end-effector. These cable tensions must be minimized so that the motors provide a minimum torque in order to reduce the energy consumption. In addition, the tension of each cable must be kept positive as the cables can only pull and not push. Thus, each tension is limited between the minimum and maximum values defined as acceptable: the minimum tension is the tension required in order to keep the cable taut, the maximum tension is limited by the maximum torque that the servomotors can generate and the breaking force of the cables. The first step of the modelling consists in determining the unit vector associated with each cable (Eq. (1)) [14]. The wrench W of the forces and moments applied to the end-effector can then be defined (Eqs. (2) and (3)). ci ; where i = 1 to 4 ci    di where R = REuler wi = Rbi ∧ di T  W = w1 w2 w3 w4 di = −

(1) (2) (3)

Design of a Cable-Driven Parallel Robot

Finally, the matrix containing the cables tensions are determined as follows:   T = W+ −Fe − Wg + Null(W)λ  −1 where: W+ = WT WWT Fe , the external wrench applied on the end-effector Wg , the gravitational force of the end-effector Null(W), the null space of the matrix W λ, a scalar

163

(4)

As the end-effector fully supports the weight of the head (the robot works with gravity), there is no resistance from the subject to the robot’s movements: the external wrench is therefore equal to a null matrix (Fe = 06,1 ). Then, the combined weight of the head and the end-effector does not generate a moment around the vertical z axis of the end-effector because these two vectors are aligned:   (5) Wg = 0 0 −P = −mend −effector+head .g   T is thus obtained by adding a particular solution, W+ −Fe − Wg , to a homogeneous solution, Null(W)λ. The homogeneous solution is calculated from the determination of the optimal λ value [14].

5 Results and Discussions The different steps of the CDPR modelling are implemented on MATLAB software. The program input values correspond to the parameters of the robot and also to the initial position and orientation of the end-effector listed in Table 2. A spatial representation of the CDPR is given in Fig. 8. Table 2. Initialization values. Parameters

Values Parameters Values

Tmin [N]

0.1

ψ/x [°]

0

Tmax [N]

200

θ/y [°]

0

End-effector weight [kg] 6

ϕ/z [°]

0

xRe [mm]

yRe [mm]

447.5

497.5

The variations of the cable tensions for each generation of trajectories are given in Fig. 9. For a given orientation of the end-effector, the workspace is scanned in order to identify the space statistically accessible by the centre of the end-effector (Fig. 10). The statistically accessible space consists of all end-effector positions for which each of the cable tensions is greater or equal to Tmin . This space is greatly reduced with an end-effector orientation of only 1°. These results thus justify the importance to find an optimal solution of the CDPR.

164

A. Koszulinski et al.

Fig. 8. Position and orientation of the end-effector in relation to the frame.

(a)

(b)

(c) Fig. 9. The variation of the cable tensions for (a) the flexion-extension movement, (b) the lateral inclination movement and (c) the circular combined movement.

Design of a Cable-Driven Parallel Robot

165

Fig. 10. Space statistically accessible (represented in blue) by the end-effector for (a) an orientation of the end-effector of 0° and (b) an orientation of 5°.

6 Conclusion A novel device based on a cable driven parallel robot with four cables, intended to assist therapists in rehabilitation of the cervical spine joint, has been proposed in this paper. Its main advantage over existing robots is that it can cover the entire range of motion of the cervical spine. The trajectories of the head according to four different movements were first recorded using a motion capture system. Then, they were analysed to determine the working space required for each movement of the poly-joint. In addition, the design of the end-effector has been proposed with the aim of limiting the number of DoF necessary to perform the various tasks. This design was also intended to facilitate the modelling of the robot. The shape of the structure was chosen based on an evaluation sheet comprising several criteria such as size or rigidity. The kinetostatic model of the CDPR has been developed in this paper. The space statistically accessible by the robot as well as the evolution of the tension in each cable have also been studied. Future works will consist in studying the optimal design of the robot, by particularly using the minimal tensions distribution as well as the dexterity criterion over the whole workspace. Furthermore, the prototype will be manufactured to proceed the preliminary tests. Acknowledgments. This work was financially supported by the Pprime Institute, a CNRS research unit created in partnership with the University of Poitiers and ISAE-ENSMA and carried out within the CoBRA team.

References 1. Report of the French Association of Chiropractic. https://www.ifec.net/download/2-ifec-rap port-elaboration.pdf. Accessed 01 Oct 2020 2. Pathologies and treatments. https://sante.orthodz.com/2018/11/25/douleurs-au-cou-ou-cervic algie-origines-et-traitements/. Accessed 02 Oct 2020 3. Neck ache. https://www.pcnphysio.com/localiser-ma-douleur/douleur-au-cou/. Accessed 02 Oct 2020

166

A. Koszulinski et al.

4. Haghshenas-Jaryani, M., Patterson, R.M., Wijesundara, M.B., Bugnariu, N.: A pilot study on the design and validation of a hybrid exoskeleton robotic device for hand rehabilitation. J. Hand Ther. 33(2), 198–208 (2020) 5. Sargsyan, S., Arakelian, V., Briot, S. : Robotic rehabilitation devices of human extremities: design concepts and fonctional particularities. In: 11th Biennial Conference on Engineering Systems Design and Analysis (2012) 6. Proteor Post-surgery/rehabilitation. https://orthopedie.proteor.fr/famille,1410-post-chirur gie-reeducation.php. Accessed 01 Oct 2020 7. Demeure orthopaedic, Guide to orthopaedic apparatus, orthosis. https://demeureorthopedie. fr/images/GUIDE-PA-interactif.pdf. Accessed 02 Oct 2020 8. Colombia University. https://roar.me.columbia.edu/content/robotic-neck-brace. Accessed 20 Oct 2020 9. Prehab Exercises. https://prehabexercises.com/basic-assessments-and-movement-evalua tions-for-runners/prehab-exercises-amass-method-basic-assessments-for-running-cervicalspine-range-of-motion-head-and-neck-mobility/. Accessed 29 Sept 2020 10. Medico plus. https://medicoplus.com/traumatologia/aparato-locomotor. Accessed 29 Sept 2020 11. Kuo, C., Fanton, M., Wu, L., Camarillo, D.: Spinal constraint modulates head instantaneous center of rotation and dictates head angular motion. J. Biomech. 76, 220–228 (2018) 12. Facial shapes in aesthetic medicine. https://www.emotions.64g.ru/prof3/pro08en.htm. Accessed 15 Oct 2020 13. Boukraâ, Y.M.: Modélisation et commande d’un robot parallèle à câble pour la rééducation des membres inférieur (2019) 14. Peng, L., Yuanying, Q., Yu, S., Jiantao, C.: On the Minimum Cable Tensions for the CableBased Parallel Robots. J. Appl. Math. 2014, 8 (2014)

Design of a Cable-Driven Robot for Elbow and Wrist Rehabilitation Axel Fort1(B) , Octavie Somoza Salgado1 , Med Amine Laribi1 , Juan Sandoval1 , and Marco Ceccarelli2 1 Department of GMSC, Pprime Institute CNRS, ENSMA, University of Poitiers, UPR 3346,

Poitiers, France {axel.fort,octavie.somoza.salgado}@etu.univ-poitiers.fr, {med.amine.laribi,juan.sandoval}@univ-poitiers.fr 2 Laboratory of Robots Mechatronics, Department of Industrial Engineering, University of Rome Tor Vergata, 00133 Rome, Italy [email protected]

Abstract. This paper presents the design and an experimental characterisation of CADREW, a CAble Driven Robot for Elbow and Wrist rehabilitation. The device is an extended version of CADEL, a Cable-Driven Elbow assisting device, originally dedicated to elbow motion assistance. The new device design is conceived to be portable and user-oriented solution and its kinematics is formulated for functionality analysis. The proposed device allows motion assistance of the wrist as well as the elbow. The preliminary results of lab tests with a first prototype are discussed to characterize the feasibility and performance of the proposed design. Keywords: Medical robotics · Motion assistance · Cable driven devices · Design · Simulation

1 Introduction The use of robotic systems presents some advantages in terms of control and repeatability interesting in limb rehabilitation. The muscular recovery is linked to neuronal plasticity [1] so the goal of rehabilitation is to develop new neuronal interconnections through repetitive physical exercises of the injured limb. Patients then acquire new functions and offset the defect. The exercises for arm rehabilitation are based on its biomechanics [2] and assisted by a physiotherapist who helps to guide manually the limb during the movement. But this procedure is not perfect since physiotherapists cannot be able to perform this repetitive therapy effectively for a long period of time. A type of robots can help to do these rehabilitation exercises as an assistance device which patients can use by themselves even in domestic environments. Indeed, the use of cable robots for this application is an interesting solution, as reported in [3], in terms of adaptability and transport. They are lightweight, easy to use, and offer a larger working space as compared to conventional serial manipulators and rigid chain parallel structures [11]. They can assist patients during rehabilitation exercises for long periods of time and © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 167–175, 2021. https://doi.org/10.1007/978-3-030-75259-0_18

168

A. Fort et al.

with appropriate accuracy. Moreover, recording the data during the exercise can allow an objective measure of the progress and the result of the therapy. Among many examples of cable robots, we can cite the NeReBot (NEuro REhabilitation roBOT) [4], MEDARM [5] and Freebal [6] which are cable devices for the upper limb rehabilitation with different Degrees of Freedom (DoF). The Laboratory of Robot Mechatronics at the University of Rome Tor Vergata has developed a prototype of cable-driven robot, called CADEL [7]. This robot is designed to be transportable and usable at home. It is an adaptable and lightweight solution dedicated to the rehabilitation of the elbow through the execution of exercises involving the elbow joint. Its effectiveness was tested experimentally [12] and improvements are undergoing with new versions. The work presented in this paper consists in proposing a new device based on the design and requirements of the robot CADEL in [7] and [10] for the rehabilitation of the elbow and the wrist. This paper is organized as follow: Sect. 2 deals with the biomechanics description of the arm and the design requirements. Section 3 presents a description and a kinematic model of the proposed device. In Sect. 4, the numerical simulation of each arm joint motion is compared to the kinematic model. Section 5 presents the prototype with preliminary testing layout.

2 Upper Limb Motion and Requirements The movement of the upper limb depends on several joints including the elbow and wrist that allow the positioning of the hand in the space and thus the interaction of the patient with his environment.

Fig. 1. Biomechanics of the elbow and the wrist

Each of these joints has two DoF corresponding exclusively to rotations around different axes. Thanks to the use of a motion capture system, the amplitude of these rotations can be obtained. For this purpose, a zero position is determined, such that the forearm is perpendicular to the arm (α = 0◦ ; β = 0◦ ) and the hand is in the supination position (θ = 0◦ ; φ = 0◦ ) as shown in Fig. 1. According to previous studies [2], the flexion-extension motion (angle α) of the elbow reaches ±75° and the pronationsupination motion (angle β), ±40◦ . For the wrist according to [8], the flexion/extension motions (angle θ) assume an amplitude of ±85◦ and an amplitude of 25◦ for abduction and 45◦ for adduction (angle φ).

Design of a Cable-Driven Robot

169

As part of a physiotherapy, the above movements are reproduced and repeated to regain the mobility of the upper limb and allow patients to recover their daily activities without pain, as described in [9].

3 Cable Driven Device for Upper Limb Motion Assistance Based on the CADEL robot device in [7], the CADREW (CAble Driven Robot for Elbow and Wrist rehabilitation) device is also composed of rings placed on the different segments of the arm and it is equipped with servomotors allowing the arm to move. Several solutions have been considered to add the wrist rehabilitation device on the prototype for elbow rehabilitation. A first approach was the insertion of the servomotors allowing the movement of the wrist on the arm ring as shown on Fig. 2a, 2b and 2c. Indeed, a wider insertion of servomotors is easier. This solution has also the advantage of limiting the weight of the device on the forearm and hand and therefore the power provided by the servomotors is reduced. However, these solutions are not completely suitable since the lengths of the cables depend on the movement of the elbow, increasing the complexity of the device modelling as well as its design structure and biomechanics operation.

Fig. 2. Proposal for the insertion of servomotors on the arm ring (a) solution #2 (b) solution #3 (c) solution #4.

Another approach was to insert the servomotors on the forearm platform. The movement of the wrist does not depend on the elbow rotation in Fig. 3a and 3b. Therefore, it simplifies and limits the unwanted movements of the wrist during the execution of for elbow rehabilitation exercises.

Fig. 3. Proposal for the insertion of servomotors on the forearm and hand ring (a) solution #1 (b) solution #5

170

A. Fort et al.

As shown in Fig. 3b, we have chosen to distribute three servomotors allowing wrist movements, two on rings of forearm and one on hand for ergonomic reasons. In this solution, the movement of the forearm and the wrist is controlled by five cables with length Li. Each cable is wrapped around pulley, actuated by a servomotor Mi and attached to the mobile platforms at the point P as shown in Fig. 4a. A rolling system allows the motors on the arm to rotate on themself with an angle γ for staying in the same axis as the cable.

(a)

(b)

Fig. 4. (a) A kinematic scheme for CADREW (Cable-Driven Robot for Elbow and Wrist rehabilitation) with design and operation parameters; (b) Comparison of the cable length required for two types of servomotors mounting

From the kinematics of the device, the lengths Li of each cable are given by the norm of vectors M i P with P the anchor point A, B, C, D, E of the ith cable as: M i P = M i O1,2 + O1,2 P

(1)

According to wrist movements, the expression in R2(x2, y2, z2) is function of two rotation matrices: ⎛ ⎞ ⎛ ⎞ 10 0 Cϕ −Sϕ 0 R(θ , x) =⎝ 0 Cθ −Sθ ⎠ and R(, z) = ⎝ Sϕ Cϕ 0 ⎠ 0 Sθ Cθ 0 0 1 And for the elbow motion, the expression in R1(x1, y1, z1) is function of three rotation matrices: ⎛ ⎛ ⎞ ⎞ ⎛ ⎞ Cβ 0 Sβ 10 0 10 0 R(γ , x) = ⎝ 0 Cγ −Sγ ⎠; R(α, x) = ⎝ 0 Cα −Sα ⎠; R(β, y) = ⎝ 0 10 ⎠ −Sβ 0 Cβ 0 Sγ Cγ 0 Sα Cα Considering the movement of the forearm in the plane (y, z) and neglecting the rotation γ of servomotors, as shown on Fig. 4b, one can conclude that the length of the cable does not depend on the angles β and γ which can be considered equal to zero.

Design of a Cable-Driven Robot

We obtain the following equation:    L1 = L2 = r12 + r22 + h2f + h2a − 2 r1 r2 + hf ha Sα

171

(2)

For the motion of the wrist, the movements of adduction/abduction (angle φ) and of flexion/extension (angle θ) can be considered separately. We will neglect the movements of circumduction and admit that the movements are planar in each case. For the movement of flexion/extension, the lengths of cables can be computed as: 

2

  (3) + h2 + 2 h hh Cθ + r2 r3 L3 = L4 = r22 + r32 + h f

L5 =



h



−hh Cθ − LB Sθ − hf

2

f

+ (−hh Sθ + LB Cθ − r2 )2

(4)

Similarly, for the abduction/adduction motion of the wrist (angle φ), we obtain:  2  2 (5) L3 = r2 − r3 Cφ − hh Sφ + hf  + hh Cφ − r3 Sφ L4 =

 2  2 r3 Cφ − hh Sφ − r2 + hf  + r3 Sφ + hh Cφ

 2  2 hh Sφ + −hh Cφ − hf  + (LB − r2 )2 L5 =

(6) (7)

The cable motion requires motor action that produce cable tension. The tension in each cable must be always positive and can be computed using Jacobian relationship, which is a linear relationship between the cable tensions T and external wrench W e through the following relation: W e − JT T = 0

(8)

ui . R0 .O1,2 P i × ui R0 : the rotation matrix defining the platform’s orientation and ui the unit vectors along the cable segment given by ui = Li /Li 2 for (i = 1,..,5), ·2 is the Euclidean norm. Having two independent movements, we can define the rotation matrix in each task as RO = R(α, x) · R(β, y) for the elbow rehabilitation and RO = R(θ, x).R(φ, z) for the wrist rehabilitation. Thus, the operation is characterized by cable motions and tensions as calculated above.

Where T = [ti ]T and Jacobian matrix is defined by: JT =

4 Simulation and Performances Evaluation The CADREW operation can be characterized on Fig. 6 and 7 through numerical simulation under Meca3D software, where parameters are on Fig. 5.

172

A. Fort et al.

Fig. 5. CADREW parameters under Meca3D: (a) joints parameters, (b) CAD design

As shown on Fig. 6, curves have almost the same profile with a near factor probably due to the placement of the constraints of the different elements in SolidWorks due to the arm model we used. Also, our expectations on cable length variation depending on the angles θ and φ are respected. Indeed, the symmetrical cables work in an antagonistic way: L3 increases when L4 decreases during the abduction-adduction movement and L3 and L4 decreases when L5 increases during flexion-extension movement. According to Fig. 7, the cables placed symmetrically L1–L2 and L3–L4 assume the same tension. T5 is superior and has the role of compensating and driving the bending movement. Another interesting point, this analysis will guide us in the choice of servomotors in terms of power and size to carry the load of the system.

φ (°)

φ (°)

φ (°)

Fig. 6. Comparison between the computed theoretical equations (blue) and the simulations on SolidWorks-Meca3D (orange) for each movement.

Design of a Cable-Driven Robot

173

φ

Fig. 7. Computed cable tensions for the desired tasks (a) Flexion-extension of elbow for load of 3 kg (b) Flexion-extension of wrist for a load of 0,78 kg (c) Abduction-adduction of wrist for a load of 0,78 kg.

5 Lab Prototype and Tests This section is focused on the elbow part of CADREW, shown in Fig. 8a and 8b. This preliminary solution was tested to verify its functionality with the expected improvements in easiness of use and efficiency. These characteristics have been numerically evaluated during experimental tests with results reported in Fig. 10a and 10b. The prototype has been fixed to an aluminum structure simulating a human arm, as depicted in Fig. 8c. The arm ring is made of 170.7 mm diameters with a weight of 412 g, and the forearm ring is 140.7 mm diameter and 80g weight. All the 3D printed parts are made of PLA, combined with inflatable interface, with a rigid behavior because the exerted force are much below their elastic limit. A position control law has been used to execute a controlled flexion/extension trajectory, with a sampling time of 6ms. In order to verify the performance of the prototype, a desired trajectory has been executed for different loads: 0.5 kg, 1.0 kg, 1.5 kg and 2.5 kg. Figure 9 shows the angular trajectory imposed by the CADREW to the elbow of the test bench, having defined the zero position when the forearm is in the horizontal plane. Since the load of the forearm is distributed equitably between the two cables, the produced torque of the two motors is similar. Therefore, we only present in this paper the response of the motor placed at the right side. Fig. 10a and 10b shows the torque produced by the right-side motor for the different loads during tests in Fig. 9. Results show, as expected, the dependence between the produced torque and the load, but also an increase of torque during the bending movement of the elbow.

Fig. 8. A prototype of light CADEL (a) and (b), and test bench (c)

174

A. Fort et al.

Fig. 9. Test results with the light CADEL prototype for a flexion/extension exercise

Fig. 10. Test results with the CADREW prototype for a flexion/extension exercise: (a) right-side motor torque response for different loads vs time, (b) right-side motor torque vs elbow angle rotation.

6 Conclusion CADREW, a CAble Driven Robot for Elbow and Wrist rehabilitation, was presented in this paper as a portable, user-oriented and lightweight solution. Based on the CADEL device, several solutions were considered for insertion of the servomotors generating wrist motion. Simulation results and kinematic model were discussed to characterize and to show the feasibility of our device. A first prototype for elbow rehabilitation only was developed and tested on an aluminum pronation with different load ranges. Through lab rehabilitation exercises, the efficient of the first prototype for this application is shown. For future work, the printing of the ring of the hand and tests of the CADREW device in its entirety will be performed for its validation.

References 1. Cramer, S.C., Sur, M., Dobkin, B.H., O’Brien, C., et al.: Harnessing neuroplasticity for clinical applications. Brain 134(6), 1591–609 (2011) 2. Fornalski, S., Gupta, R., Lee, T.Q.: Anatomy and biomechanics of the elbow joint. Tech. Hand Up Extrem. Surg. 7(4), 168–78 (2003)

Design of a Cable-Driven Robot

175

3. Gonçalves, R., Alves, T., Carbone, G., Ceccarelli, M.: Cable-driven robots in physical rehabilitation: from theory to practice (2020) 4. Stefano, M., Patrizia, P., Mario, A., Ferlini, G., Rizzello, R., Rosati, G.: Robotic upper limb rehabilitation after acute stroke by NeReBot: evaluation of treatment costs. Biomed. Res. Int. 2014, 265634 (2014) 5. Cui, X., Chen, W., Xin, J.S., Agrawal, K.: Design of a 7-DOF cable-driven arm exoskeleton (CAREX-7) and a controller for dexterous motion training or assistance. IEEE/ASME Trans. Mech. 22(1), 1 (2016) 6. Stienen, A.H.A., et al.: Freebal: dedicated gravity compensation for the upper extremities. In: IEEE 10th International Conference on Rehabilitation Robotics, Noordwijk, pp. 804–808 (2007) 7. Ceccarelli, M., Ferrara, L., Petuya, V.: Design of a cable-driven device for elbow rehabilitation and exercise. In: Kecskeméthy, A., Geu Flores, F., Carrera, E., Elias, D. (eds.) Interdisciplinary Applications of Kinematics. Mechanisms and Machine Science, vol. 71. Springer, Cham (2019) 8. Fontaine, C., Bry, R., Laronde, P., Guerre, E., Aumar, A.: Anatomie descriptive, radiographique, topographique et fonctionnelle appliquée aux fractures de l’extrémité distale du radius. Hand Surg. Rehab. 35, S3–S14 (2016) 9. Vaida, C., Carbone, G., Major, K., Major, Z., Plitea, N., Pisla, D.: On human robot interaction modalities in the upper limb rehabilitation after stroke. Acta Tech. Napocensis Appl. Math. Mech. Eng. 60, 91–102 (2017) 10. Ceccarelli, M., Ferrara, L., Petuya, V.: Device for elbow rehabilitation, patent no. 102017000083887, Italy (2017) 11. Mao, Y., Agrawal, S.K.: Design of a cable-driven arm exoskeleton (CAREX) for neural rehabilitation. IEEE Trans. Robot. 28(4), 922–931 (2012) 12. Laribi, M.A., Ceccarelli, M.: Design and experimental characterization of a cable-driven elbow assisting device. ASME. J. Med. Dev. 15(1), 014503 (2021)

Controllable Passive Transtibial Prosthesis Design Ionel Cristian Vladu(B) , Florina Cristian Pan˘a(B) , Cristian Copilus, i, Iulian Petris, or, Ileana Vladu, and Nicu George Bîzdoac˘a University of Craiova, Craiova, Romania [email protected], [email protected]

Abstract. In medicine, a prosthesis or prosthetic implant is an artificial device that replaces a missing part of the body, which can be lost through trauma, disease, or illness present at birth. To help these amputated people, transtibial prostheses have been developed that have the role of increasing the motor abilities of the person in question. Among the different models of prostheses, the most frequently used are non-articulated prostheses, articulated prostheses, prostheses with energy re-turn/dynamic response, and microprocessor-controlled prostheses. The foot can vary the flexibility and elasticity of the complex structure to perform various challenging tasks, such as running, climbing, balancing, jumping, and lifting on the toes. To replace these functions, prostheses with different design and operating modes are used, depending on the activity performed. Thus, a person with social activity, or sports activity, requires the use of several prostheses with different modes of operation. The present paper proposes a transtibial prosthesis self-adaptable to the type of activity/movement. This is a passive prosthesis, controllable, with a universal joint (Hooke’s joint) embedded in the ankle, based on rheological fluids. The orientation and positioning of the mobile elements of the prosthesis are achieved by means of a system of elastic springs correlated with a rheological controllable universal joint. This paper presents the construction and operation of the prosthesis. At the end of the paper are presented the results of the simulation for loading the prosthesis foot and prosthesis joint element with load. Keywords: Transtibial prosthesis · Magnetorheological · Rheological stop-valve

1 Introduction In medicine, a prosthesis or prosthetic implant is an artificial device that inhabits a missing part of the body, which can be lost through trauma, disease, or illness present at birth. To help these people have been developed prostheses and orthoses to increase the motor abilities of the person in question (Fig. 1) [1]. Transtibial prostheses are the most requested. They can be classified into three categories: cosmetic prostheses, bodycontrolled prostheses, and externally operated prostheses. Externally operated prostheses can have passive joints and active joints. Of the various prosthesis models, the most commonly used are non-articulated prostheses, articulated prostheses, with energy return/dynamic response prostheses, and microprocessor-controlled prostheses. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 176–183, 2021. https://doi.org/10.1007/978-3-030-75259-0_19

Controllable Passive Transtibial Prosthesis - Design

177

To replace these functions, prostheses with different design and mode of operation are used, depending on the activity performed (stationary, slow walking, fast walking, running, climbing/descending stairs, etc.).

Fig. 1. Robotic prostheses for transtibial amputees

Thus, a person with active social activity, or sports activity, requires the use of several prostheses with different modes of operation. Figure 2 shows the main constructive types [2].

Fig. 2. Categories of prosthetic feet (a) SACH (solid ankle cushion heel), (b) Single axis, (c) Multi-axial, (d) Dynamic response, (e) Multiaxial dynamic response, (f) Externally powered

The present paper proposes a transtibial prosthesis self-adaptable to the type of activity/mode of movement. This is a passive type prosthesis controllable by means of a universal joint (Hooke’s joint) based on intelligent fluids, respectively magnetorheological fluids (MR). The proposed prosthesis consists of three systems that work together: the mechanical system, the sensory system, the control system. The mechanical system consists of a transtibial prosthesis that has a universal ankle joint, with rotational movement on two axes. The universal presented joint is a passive joint, whose rotational movement can be controlled by a rheological stop-valve. The joint also has a system of elastic springs which, in the absence of a load, reposition the foot in relation to the tibia. The sensory system provides information on the position and orientation of the foot and the prosthesis, as well as the distribution of body weight. This information is also taken from the valid foot. The control system determines the type of movement, respectively the step, based on the information provided by the sensory system. It also controls the position of the foot via the rheological stop-valve. The paper presents the construction and operation of the prosthesis. At the end of the paper are presented the results of the simulation for loading the prosthetic ankle joint with a load.

178

I. C. Vladu et al.

2 Prosthesis Design 2.1 General Presentation The sequence of movements of a single leg while walking is defined by the walking cycle. The gait cycle or step is divided into a stance phase (weight) and a swing phase (advancing the leg into space). Most of the cycle is dedicated to the process of supporting one leg (80%), while a smaller part is dedicated to supporting both legs (20%). Figure 3 [2] shows the phases of walking.

Fig. 3. The gait cycle or step

The ankle joint has 3 degrees of freedom in terms of movement, they are illustrated in Fig. 4. The mobile angular intervals of these movements are observed.

Fig. 4. Movements of ankle

As can be seen, the foot has a complex movement, adapted to the type of movement and the surface of the ground. Also, to the balance and dynamics of walking contributes the flexion of the fingers, the movement of the knee joint, hip, etc. Currently, articulated prostheses, passive or active, render only Dorsiflexion/Plantarflexion movement. Inversion/Eversion Movements control is also required for moving on rough terrain. Abduction/Adduction movement is required to change the direction of travel for brisk walking, or easy running. This movement cannot be implemented on a prosthesis yet because it is necessary to predict the change of direction. The

Controllable Passive Transtibial Prosthesis - Design

179

prosthesis shown (Fig. 5) allows the control of Dorsiflexion/Plantarflexion and Inversion/Eversion movements. From a constructive point of view, it consists of Socket and foot, joined by a universal ankle joint.

Soket

Inversion/

Dorsiflexion/

Ankle joint

Plantarflexion

Eversion

Foot

Fig. 5. Prosthesis construction

Currently, articulated prostheses, passive or active, perform only Dorsiflexion/Plantarflexion movements. Inversion/Eversion movements control is also required for moving on rough terrain. Abduction/Adduction movement is required to change the direction of travel for brisk walking, or easy running. This movement cannot be implemented on a prosthesis yet because it is necessary to predict the change of direction. The prosthesis shown (Fig. 5) allows the control of Dorsiflexion/Plantarflexion and Inversion/Eversion movements. From a constructive point of view, it consists of Socket and foot, joined by a universal ankle joint. Depending on the step of the step are performed, by Universal Ankle Joint (UAJ), Dorsiflexion/Plantarflexion and Inversion/Eversion movements. UAJ is a passive joint, the movement being created by the body weight at the contact of the prosthesis with the ground (during the stepping process). By controlling the dynamics of the rotational movements of the SAJ, we can control the dynamics of the prosthesis movement (respectively the foot in relation to the leg). 2.2 Foot Construction The foot consists of an elastic structure that replaces the cushioning when stepping (taken by the heel in the case of the foot) and also generates the force of pushing the toes when detaching the foot from the ground for its balancing movement forward. It is made of steel with high elasticity (Fig. 6). It is connected to the ankle joint by means of a rigid, removable part. This allows simply change of foot, for bigger elasticity or rigidity. 2.3 Universal Ankle Joint Universal Ankle Joint UAJ (like Hooke’s joint) (which replaces the ankle joint) is a passive joint, with hydraulically controlled movement through a magnetorheological

180

I. C. Vladu et al.

Fig. 6. Foot construction

stop-valve (MR) (Fig. 7). By controlling the flow of MR fluid through the stop-valve, we control the dynamics of the joint, respectively the dynamics of the foot. Thus, the prosthesis takes over the natural movements of the foot, for different travel regimes.

Ankle Joint (a)

(b)

Fig. 7. Universal ankle joint (a) prosthesis detail, (b) general view

UAJ (Fig. 8) allows two rotational movements (Dorsiflexion/Plantarflexion and Inversion/Eversion). MR stop-valve is not contained in the joint (it is located in the leg). UAJ is composed of two concentric spheres (7 and 15). Two spherical cap elements (6 and 17) rotate between the spheres around the axes of rotation 10 and 18. They are coupled to the elements 5 and 9 by means of the screws 16. The clamping elements (3 and 11) are in the middle of the elements. 5 and 9 and are connected to the prosthesis elements, respectively the leg and the foot. These two groups of elements (5, 6 and 9, 17) ensure the corresponding rotational movements Dorsiflexion/Plantarflexion and Inversion/Eversion. In the space between spheres 7 and 15 is MR fluid. This space is separated in two by the fixed element 14 (common to both spheres). We thus obtain two spaces, S1 and S2 with MR fluid, in which elements 5 and 9 rotate/move respectively. Element 5 divides space S1 into two spaces S11 and S12. When rotating element 6, because the MR fluid is incompressible, we must evacuate a quantity of fluid from S11 and add it to S12, respectively vice versa for the opposite movement. The MR fluid passes from S11 to S12 using the hole 8 route, the exhaust/intake tube 4 and 1, the MR stop valve, the exhaust/intake tube 2 and 4, the hole 18 (not marked in the figure). The same is true for item 17.

Controllable Passive Transtibial Prosthesis - Design

181

By controlling the flow rate of the MR fluid through the stop valve, we control the rotation dynamics of the UAJ, respectively the dynamics of the foot movement. 2.4 Magnetorheological Stop-Valve Magnetorheological Stop-Valve (MRSV) it consists of a flow channel passing through a magnetic field. Magnetorheological Fluid. It is composed of solid particles immersed in a fluid. When exposed to a magnetic field (transverse to the flow direction) the solid particles are arranged in chains of particles (along the field lines) that oppose the flow of the fluid, until it is blocked.

Fig. 8. Universal ankle joint, construction parts and section view

182

I. C. Vladu et al.

MRSV. It consists of a flow channel between two magnetic poles. There is a pressure difference between the ends of the flow channel (in the magnetic field). It depends on the type of MR fluid, the intensity of the magnetic field, the geometric parameters of the flow channel. By controlling the intensity of the magnetic field, we control the flow rate of the MR fluid.

2.5 Sensorial and Control System The sensory system provides information about the position of the legs and the stepping. Based on this information, the control system decides on the dynamics and position of the foot. These systems are not the subject of this paper.

3 FEM Analyses of Main Elements In the FEM analyses process, the prosthesis elements were subjected to stresses similar to those during walking. The person’s weight, used in the simulations, was 120 kg. Figure 9 and Fig. 10 presents some of the FEM analyses performed for sizing and optimizing the components of the prosthesis.

Fig. 9. Foot ground contact FEM analyses (a) in the phalanx area, (b) in the calcaneus area

Fig. 10. Universal ankle joint part FEM analyses (a) relax position, (b) maximum load

Controllable Passive Transtibial Prosthesis - Design

183

4 Conclusions The presented solution represents elements of originality both from a constructive point of view and of the control element. Similar prostheses, such as microprocessor-controlled prostheses with the dynamic response, are complex construction solutions with high volume and weight, and actuators. The prosthesis presented in the paper is a passive prosthesis, which allows the control of movement only using the action of body weight. Also, unlike passive prostheses, we have control of the movement dynamics performed by the magnetorheological stop valve. In order to be able to control the movements of the prosthesis, respectively its dynamics, the data provided by the sensory system attached to the prosthesis and the valid foot are processed. It consists of inertial sensors and pressure/weight sensors that determine the spatial movements of the legs, respectively the walking phase. The sensory system and the algorithm for determining the walking phase are not the subject of this paper. As future objectives, we proposed the optimization of the design of the mechanical structure, the realization of the sensory system and the algorithm for determining the step phase, and the optimization of the magnetorheological stop valve for the respective structure and requirements. Some of these goals are ongoing for implemented or tested. Acknowledgment. This work was supported by the European Social Fund within the Sectorial Operational Program Human Capital 2014–2020 and by the grant of the Romanian Ministry of Education and Research, CCCDI – UEFISCDI, project number PN-III-P2-2.1-PED-2019-0937, within PNCDI III.

References 1. Powered Lower-Limb Prostheses. https://research.vuse.vanderbilt.edu/cim/research_leg.html. Accessed 10 Jan 2021 2. Davis, A., Kelly, B., Spires, M.C.: Prosthetic Restoration and Rehabilitation of the Upper and Lower Extremity. DemosMedical (2013) 3. Zubieta, M., Eceolaza, S., Elejabarrieta, M.J., Bou-Ali, M.M.: Magnetorheological fluids: characterization and modeling of magnetization. Smart Mater. Struct. 18, 095019 (2009) 4. Stoian, V., Vladu, I.C.: Hyper-redundant robot control system in compliant motions. In: Proceedings of the 27th International Conference on Robotics in Alpe-Adria Danube Region. Mechanism and Machine Science. Advanced in Service and Industrial Robotics, vol. 67. Springer (2018) 5. Vladu, I., Strîmbeanu, D., Ivˇanescu, M., Vladu, I.C., Bîzdoaca, E.N.: Experimental results for magneto-rheological stop valve. In: 2012 16th International Conference on System Theory, Control and Computing, ICSTCC (2012)

A Study of Feasibility for a Novel Cable-Driven Upper-Limb Exoskeleton with Magnetic Balancing Jhon F. Rodríguez-León1 , Daniele Cafola2 , Brandon Suarez3 , Eduardo Castillo-Castañeda1 , and Giuseppe Carbone4(B) 1 Instituto Politécnico Nacional-CICATA Querétaro, 76090 Querétaro, Mexico 2 Biomechatronics Lab, IRCCS Neuromed, 86077 Pozzilli, IS, Italy 3 Universidad Politécnica de Querétaro, 76240 Querétaro, Mexico 4 DIMEG, University of Calabria, 87036 Rende, Italy

[email protected]

Abstract. This paper presents a novel concept for a cable-driven device for upperlimb exercising. The proposed device is based on a specific innovative architecture, which is driven by cables . Additionally, magnets are considered as a technical solution for partially compensating gravity and reducing energy consumption. The paper content includes a detailed description of the design process and simulations to validate the engineering feasibility and the effectiveness of the proposed design from power consumption viewpoint.

1 Introduction One aspect still requiring careful attention in modern robots is their energy efficiency. This can be conveniently achieved with optimal design procedures and a proper dynamic balancing strategy. This aspect is addressed in literature by several researchers. For example, papers [1, 2] as based on adding extra mass (counterweights) or other passive/active solutions with auxiliary links and/or springs for achieving energy saving. Exoskeleton type robots allow to mobilize high loads of objects with a minimum effort of both the upper and lower extremities. These machines made of rigid links of metal can deliver high forces to their wearer accurately and precisely and operate in parallel to the human skeleton [3]. Yet, this advantage has come at a cost: a significant inertia, which affects both the kinematics of human movement and the power requirements of the device [4], resulting in increased mechanical complexity and size. For example, Gaponov et al. proposed a fully portable exosuit for shoulder and elbow movements’ assistance, the exosuit was named Auxilio. The proposed exosuit is weights under 4 kg, which includes the frame, motors, control unit, and battery [5]. This weight needs to be carried by the wearer and results in additional power consumptions. This paper proposes to implement a novel device driven by cables to reduce the overall weight. Additionally, magnets are considered for partially compensating gravity and reducing energy consumption. Preliminary simulation results are reported to the engineering feasibility of the proposed design and to preliminarily validate the effectiveness of the proposed design concept. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 184–191, 2021. https://doi.org/10.1007/978-3-030-75259-0_20

A Study of Feasibility for a Novel Cable-Driven Upper-Limb Exoskeleton

185

2 Requirements for the Proposed Device 2.1 Desired Motions The proposed novel mechanism was created for the upper-limb exercising, the Fig. 1 shows the elbow flexion/extension motions, this motion implies bringing the forearm closer to the upper arm, reducing the angular value between the two limb segments, implicitly, and the extension is the motion of returning the forearm to its initial position and thus increasing the angle between the forearm and the upper arm. In this case, the device uses the cable driven the point attached q1 articulation to bend the elbow joint up to its required angular amplitudes.

Fig. 1. Human arm: elbow joint that interact during flexion and extension movements.

2.2 Conceptual Design of a Novel Cable-Driven Upper-Limb Exercising Device The design process towards proposed cable-magnet device starts with the design of a frame exoskeleton structure. This is designed to be comfortable for the user and lightweight. For that reason, the exoskeleton is made by lycra and few rigid elements. Figures 2 shows the main rigid elements of the proposed exoskeleton frame. Part A is the exoskeleton handles, these were designed for collocated the exoskeleton in the user’s back, working as a backpack. With this the user can wear the system easily. Part B points out the “x” behind the base. This “x” is formed by two ties which are made of lycra too. These ties were designed to bring support for all the base and to adjust with the user’s back, making the exoskeleton more resistant. Next to each side of the “x” there is part C. Part C is composed by two cylindrical covers, these covers were collocated over there to carry the cylinder container of the magnet system. Part D points out two extensions that protrude from the user’s back, which are the unique rigid part, both are made of plastic. These were designed to keep the cable upper than the user’s shoulder to avoid possible damages. Also, it brings support to the cable and work as a rail for it. For that reason, extensions have a middle hole which fits with the middle hole of the cylindrical containers. All the middle holes have the same diameter to avoid size changes with the magnet middle hole and the cable thickness.

186

J. F. Rodríguez-León et al.

Fig. 2. Exoskeleton parts.

3 Energy Storage and Gravity Compensation System The proposed exoskeleton includes magnetic-spring system, which is composed by three identical cylindrical magnets fixed in vertical position. Figure 3 shows the magnetic spring configuration, and the Table 1 shows the selected magnet proprieties.

Fig. 3. Magnet spring configuration.

According to Fig. 3, the magnets A and C are fixed to each side of the container, and the magnet B is floating between them. To calculate the force generated in the magnetic spring there have been developed several equations. First, there is the Coulomb’s law, which focuses on the magnet’s charges and the distance between them to obtain the magnetic force. [6–9] Also, there is the equation that uses low order polynomials to

A Study of Feasibility for a Novel Cable-Driven Upper-Limb Exoskeleton

187

Table 1. Magnet proprieties Material

NdFeB

Exterior diameter

3/4 (19 mm)

Interior diameter

1/4 (6.35 mm)

Thickness

1/8 (3.175 mm)

Tolerance

±0.1 mm

Mass

4.7 g

Grade

N35

Subjection force

3.3 kg

Magnetization type Axial

calculate it [10, 11] Another equation was developed in [12] which the authors define their formula as the sum of forces due to the four pairs of the surface. In this research we proposed an experimental characterization using two magnet (fixed and float) and a gram scale balance OHAUS model CS2000. D is the distance between magnets and the F repulsive force is calculated using the data of the balance and multiplying to gravity force 9.81 m/s2 . The Interaction displacement-force between magnets in function to the time is shows in the Fig. 4.

Fig. 4. Interaction displacement-force between magnets.

4 Integration of the Magnetic-Spring System Magnet B moves along the vertical axis when the arm moves too. Arm movement defines the magnet B position as in Fig. 5. For example, if the arm moves up then the magnet moves down, approaching to magnet C. Another important part is the cable that pass across the magnetic spring and around the user’s arm. This is fixed to the arm by two bracelets and is attached to the magnet B inside the magnetic spring, thus, if the arm moves then the magnet and the cable moves to the same direction. Achieving an effort reduction in the user while he moves his arm.

188

J. F. Rodríguez-León et al.

On [12] the authors used the following Eq. 1 to obtain the cable displacement on their system.    θ 2 2 (1) hf (θ ) = a + b cos φ + − 2b 2 In this case we used the same equation but with some variations to calculate the magnet and cable displacement on the present system.       θ 2 2  (2) − 2b Li =  a + b cos φ + 2 where Li is the cable displacement for i = {1, 2, 3} defines the number of displacements, a is the half of the arm width, b the distance between the fixed point (bracelet) and the elbow, φ is the value of arctan(a/b) and θ is the angle between the arm and the forearm.

Fig. 5. Magnetic-spring system in the novel cable-driven upper-limb exercising device.

5 Static Analysis So far, different categories of materials have been analyzed using the material performance indexes as well as Ashby charts. Some of the materials like Kevlar or Titanium alloys show excellent results with the material performance index. However, after looking at other properties of these materials such as cost or compressive strengths, Polymers, materials have been taken into consideration to allow the manufacturing using Rapid Prototyping technology. The most common polymers used for exoskeleton frame are PMMA, PE, PEEK, PTFE and PE. These polymers are shown in the Ashby chart, [13].

A Study of Feasibility for a Novel Cable-Driven Upper-Limb Exoskeleton

189

A technique frequently used to rehabilitate neurological patients such as the cerebrotonic stroke, to enhance the efficiency of arm movements during recovery, is compensating for gravity effects by providing brace-weight support (WS). Although it has been demonstrated that the use of WS arm in healthy subjects and stroke survivors leads to a general reduction in the degree of activation of upper limb muscles, as predicted, of different levels of WS in the movability and activity characteristics of upper limb muscles were not entirely in incident. Thus, to test the stiffness of the proposed structure the deadweight (26 N), [14], for each arm has been considered affecting all the structure by applying a pulling force. The Exoskeleton has been considered fixed as a bag to the subject fixing the due Degrees of freedom (DOF) to simulate the mentioned behavior. PEEK material has been chosen for the simulation and the manufacturing of the prototype as in Table 2. Table 2. Material properties Name

Polyetheretherketone (PEEK)

Model type

Linear Elastic Isotropic

Default failure criterion Unknown Tensile strength

9,5e + 07 N/m2

Compressive strength

1,25e + 08 N/m2

Elastic modulus

3,9e + 09 N/m2

Poisson’s ratio

0,4

Mass density

1.310 kg/m3

A Stress distribution results are shown in Fig. 6. In this simulation scenario, no stresses in the model exceed the material yield strength for all the components. The maximum computed stress value it is reached on the shoulder housing along the cables and at the junction between the cables. The stress reaches a maximum namely 1.796 N/m2 and minimum of 3.584 N/m2 . As expected, the stresses are distributed along the two part of the tubes containing the cables and is. As a further check, a Factor of Safety (FOS) distribution has been carried on. Since brittle materials with different tensile and compressive property are used, the Mohr-Coulomb stress has been used. Brittle materials do not have a certain output value, so the output strength for this criterion is not to be used to determine the limit stress. The stress criterion of Mohr-Coulomb was built on the principle of Mohr-Coulomb known as internal friction. Brittle materials do not have a certain output value, so the output strength for this criterion is not to be used to determine the limit stress. The theory predicts failure to occur when the combination of the maximum tensile principal stress σ1 and the minimum compressive principal stress σ3 exceeds their respective stress limits. Figure 7 show the FOS distribution where it can be seen that failure is not occurring in the simulated scenario showing the feasibility of the prototype.

190

J. F. Rodríguez-León et al.

Fig. 6. Stress distribution results.

Fig. 7. FOS distribution.

A characteristic curve of the magnetic-spring is obtained through the displacementforce relationship, which shows that the area where the greatest repulsion force is obtained is between 3 mm to 46 mm. The obtained result show that the configuration of the magnets must be modified to obtain a greater force in a greater displacement. In conclusion, the use of permanent magnets generates energy savings thanks to the force that is formed during the phenomenon of attraction and repulsion of the magnetic field and the action force that compresses them. Further studies will be made on how the proposed device can be integrated into specific grasp configuration planning such as proposed in [15]. Further research work will be devoted at addressing the peculiarities of cables into the proposed design by proper characterizations such as reported in [16–18]. A proper proof-of-concept demonstrator will be built in a near future.

6 Conclusions This paper outlines the design of a novel concept design for a cable-driven exoskeleton for upper-limb exercising. Simulations are mostly focused on the feasibility of a lightweight frame of 1850 g, which can be driven by cables. Additionally, magnets are proposed for

A Study of Feasibility for a Novel Cable-Driven Upper-Limb Exoskeleton

191

partially compensating gravity and reducing energy consumption. Preliminary simulation results show the engineering feasibility of the proposed design solution. Further investigations will be carried out towards a proper proof-of-concept demonstrator.

References 1. Arakelian, V.: Gravity compensation in robotics. Adv. Robot. 30(2), 79–96 (2016) 2. Gatti, G., Carbone, G.: Gravity compensation of robotic manipulators using non-linear spring configurations. Mech. Mach. Sci. 91, 310–317 (2021) 3. Xiloyannis, M., Chiaradia, D., Frisoli, A., Masia, L.: Physiological and kinematic effects of a soft exosuit on arm movements. J. NeuroEngineering Rehabil. 16(1), 1–15 (2019) 4. Stienen, A., Hekman, E., van der Helm, F.C.T., Van der Kooij, H.: Self-aligning exoskeleton axes through decoupling of joint rotations and translations. IEEE Trans. Robot. 25(3), 628–633 (2009) 5. Ganopov, I., Popov, D., Lee, S., Ryu, H.: Auxilio: a portable cable-driven exosuit for upper extremity assistance. Int. J. Control Autom. Syst. 15(1), 73–84 (2016) 6. Zhang, Q., Wang, Y., Kim, E.: Power generation from human body motion through magnet and coil arrays with magnetic spring. J. Appl. Phys. 115, 064908 (2014) 7. Zhang, Q., Wang, Y., Kim, E.S.: Electromagnetic energy harvester with flexible coils and magnetic spring for 1–10 Hz resonance. J. Microelectromech. Syst. 24(4), 1193–1206 (2015) 8. Saha, C.R., O’Donnell, T., Wang, N., Mccloskey, P.: Electromagnetic generator harvesting energy from human motion. Sens. Actuators A Phys. 147(1), 248–253 (2008) 9. Salauddin, M., Halim, M.A., Park, J.-Y.: A magnetic-spring-based, low-frequency-vibration energy harvester comprising a dual Halbach array. Smart Mater. Struct. 25, 095017 (2016) 10. Masoumi, M., Wang, Y.: Repulsive magnetic levitation-based ocean wave energy harvester with variable resonance: modeling, simulation and experiment. J. Sound Vib. 381, 192–205 (2016) 11. Mann, B., Sims, N.: Energy harvesting from the nonlinear oscillations of magnetic levitation. J. Sound Vib. 319, 515–530 (2009) 12. Xiloyannis, M., Cappello, L., Binh, K.D., Antuvan, C.W., Masia, L.: Preliminary design and control of a soft exosuit for assisting elbow movements and hand grasping in activities of daily living. J. Rehabil. Assist. Technol. Eng. (RATE) 557–561 (2016) 13. Fiorillo, L., D’Amico, C., Turkina, A.Y., Nicita, F., Amoroso, G., Risitano, G.: Endo and exoskeleton: new technologies on composite materials. Prosthesis. 2(1), 1–9 (2020) 14. Coscia, M., Cheung, V.C., Tropea, P., et al.: The effect of arm weight support on upper limb muscle synergies during reaching movements. J. NeuroEngineering Rehabil. 11, 22 (2014) 15. Yao, S., Ceccarelli, M., Carbone, G., Dong, Z.: Grasp configuration planning for a low-cost and easy-operation underactuated three-fingered robot hand. Mech. Mach. Theory 129, 51–69 (2018) 16. Hernández-Martínez, E.E., Ceccarelli, M., Carbone, G., López-Cajún, C.S., Jáuregui-Correa, J.C.: Characterization of a cable-based parallel mechanism for measurement purposes. Mech. Based Des. Struct. Mach. 38(1), 25–49 (2010) 17. Laribi, M.A., Carbone, G., Zeghloul, S.: On the optimal design of cable driven parallel robot with a prescribed workspace for upper limb rehabilitation tasks. J. Bionic Eng. 16(3), 503–513 (2019) 18. Gherman, B., Birlescu, I., Plitea, N., Carbone, G., Tarnita, D., Pisla, D.: On the singularityfree workspace of a parallel robot for lower-limb rehabilitation. Proc. Rom. Acad. Ser. A Math. Phys. Tech. Sci. Inf. Sci. 20(4), 383–391 (2019)

Robotic-Assisted Platform for Spinal Surgery: A Preliminary Study Térence Essomba1(B) , Med Amine Laribi2 , Juan Sandoval2 , Chieh-Tsai Wu3 , and Saïd Zeghloul2 1 Department of Mechanical Engineering, National Central University, No. 300, Zhongda Road,

Zhongli City, Taoyuan County 32001, Taiwan [email protected] 2 Department of GMSC, Pprime Institute CNRS, ENSMA, University of Poitiers, UPR 3346 Poitiers, France {med.amine.laribi,juan.sandoval,said.zeghloul}@univ-poitiers.fr 3 Department of Neurosurgery, Medical Augmented Reality Research Center, Chang Gung Memorial Hospital, Linkou Medical Center, Taoyuan City, Taiwan [email protected]

Abstract. This paper presents a preliminary study for the design of a roboticassisted platform dedicated to minimally invasive spinal surgery (MISS) with two possible human-robot collaboration modes: co-manipulation and tele-operation mode. A motion capture system composed of 8 high resolution infrared cameras has been used to study the surgeon gestures during a classical surgery performed on cadaver. The obtained results are analyzed to define the main robot specifications, such as its workspace. Keywords: Robot-assisted surgery · Cervical surgery · Motion capture

1 Introduction In general, cervical surgery is recommended to relieve neck pain, numbness, tingling and weakness, restore nerve function and stop or prevent abnormal motion in neck. Cervical spine refers to the upper region the spine located between the skull and the shoulders. Cervical spine surgery may be indicated for a variety of spinal neck problems. Further, this surgery may be performed for degenerative disorders, trauma or instability. These conditions may produce pressure on the spinal cord or on the nerves coming from the spine. Spine surgery is traditionally performed as open surgery. This entails opening the operative site with a long incision so the surgeon can view and access the spinal anatomy. However, technology has advanced to the point where more spine conditions can be treated with minimally invasive techniques, called Minimally Invasive Spinal Surgery (MISS).

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 192–198, 2021. https://doi.org/10.1007/978-3-030-75259-0_21

Robotic-Assisted Platform for Spinal Surgery

193

Nowadays, one of used clinical techniques in cervical surgery is based on the use of mechanical stabilization. This technique involves placing screws in the pedicle, the strongest bone regions of the vertebrae. This is especially critical in the cervical region, where the target bone mass is smaller and the spinal cord, nerve roots and vertebral arteries are all at risk. Therefore the challenge for the surgeon is to avoid damage to surrounding vital anatomical structures. The screws need to be placed with accurate gestures. A robotic system enabling the surgeon to precisely place implants into the vertebrae should enhance safety and may potentially improve surgical results. Currently, robotic spine surgery is in its beginning and most of its benefits come from the use of robots in a shared-control model to assist with the placement of pedicle screws. Further, robot-assisted spine surgery has been shown to increase the accuracy of pedicle screw placement and decrease radiation exposure to surgeons. The accuracy and precision afforded by the robots in spinal surgery potentially allow for even less tissue destructive and more meticulous Minimally Invasive Spinal Surgery (MISS). In the past decade, robotic surgical systems have been adopted in various surgical specialties to replace or complement traditional laparoscopic/endoscopic techniques [1–6]. The term, “robot,” implies a machine capable of carrying out a complex series of actions automatically. A true spinal surgery robot has yet to be created. Rather, “robots”—machines designed to interact and assist humans—could be a good candidate in an attempt to make spine surgery safer and more efficient. Robotics in spinal surgery has evolved relatively fast over the past decade. In 2004, the Mazor Spine Assist device was the first FDA (the US Food and Drug Administration) approved robot used to guide the placement of pedicle screws. Newer robotic devices including the Mazor X [7] (Medtronic and Mazor Robotics, Memphis, TN, USA), ExcelsiusGPS [8] (Globus Medical, Inc., Audubon, PA, USA), and ROSA [9] (Medtech Surgical, Inc., New York, NY, USA) have since been developed and FDA cleared for use in the spine. The utility of robotic technology in spine surgery is currently under investigation and has yet to achieve widespread adoption. Multiple studies have reported early experiences with robot-assisted pedicle screw placement and the outcomes are promising [10, 11]. Nonetheless, the role of robotics in a routine spinal practice remains somewhat unclear. Robots serve simply as guides to aid in pedicle screw insertion. Thus, robotics can provide precise guidance at any area of the spine that has been appropriately imaged and registered. However, their applications in spine surgery will continue to grow and evolve. In general, two human-robot collaboration modes can be proposed to perform the surgical task: comanipulation and tele-operation mode, as shown in Fig. 1. In the comanipulation mode (Fig. 1-(a)), the surgeon and robot simultaneously manipulate the surgical tool such that the surgeon gestures are guided by the robot according to the desired task. In the tele-operation mode (Fig. 1-(b)), the surgeon uses a haptic interface to control the robot’s motion from a distal site. Force feedback provided by the interface allows surgeon to modulate his movements whereas guidance features can also be activated in the robot, e.g. virtual fixtures.

194

T. Essomba et al.

Fig. 1. Two human-robot collaboration modes for MISS: a) comanipulation, b) tele-operation mode.

Regardless of the collaboration mode employed to establish the robotic assistance, it is essential to study the surgeon gestures, in order to obtain data inputs for the definition of the robot’s workspace, its optimal configuration, base placement and so on. In this paper we present a preliminary study of the surgeon gestures during a MISS. The experiments presented in this paper have been performed by a surgeon in cadavers under real conditions. Moreover, a motion capture system, i.e. Qualisys, has been used to record the surgeon gestures. This paper is organized as follows. Section 2 presents the experimental setup used for the study of the surgeon gestures. Section 3 details and discuss the preliminary results obtained during the experimental tests. Finally, a conclusion about the presented study is proposed.

2 Experimental Setup This section is dedicated to describe the experimental setup used to study the surgeon gestures during a classical cervical surgery. The performed task is the drilling process to prepare the screws’ placement in the pedicle for mechanical stabilization. The experiments were performed on cadaver by a neurosurgeon. The surgeon gestures and more specifically the drilling tool motions are recorded using a motion capture system (MoCap). Reflective markers are installed on the drilling tool as shown on the Fig. 2. Other markers are placed on the cervical spine as well as the head in order to detect all possible relative movements. Five markers on the drill, Oi i ∈ {1, 2, 3, 4, 5}, allow to compute the orientation of this latter, i.e. Euler angles, to the reference frame. Markers on the cervical spine and the head are used to emphasize possible relative displacements leading from drilling effort.

Robotic-Assisted Platform for Spinal Surgery

195

Fig. 2. Experimental setup with markers on drilling tool and cadaver.

The MoCap system is composed of a set of 8 high resolution cameras with 1MP resolution each one, i.e. Qualisys system. The choice of marker sets as well as segment references definition are based a specific geometry in order to facilitate the reference frames definitions. The drilling task is repeated several times under similar conditions to that found on operating room.

3 Preliminary Results A total of seven trajectories have been recorded. The specific placement of the reflective markers on the surgical drill has allowed the orientation calculation of its longitudinal axis. The evolutions of the Euler angles ψ and θ, as reported on Fig. 2, are computed during all trajectories and displayed on the Fig. 3-(a) and (b). Further detailed data are provided in Table 1. Based on the evolution observed, it can be noticed that the angular motion on each individual trajectory appear somewhat limited on every angle. The maximum amplitude measured are 8.3° on ψ and 10.1° on θ. On the other hand, these trajectories seem quite different from one another in terms of angular position. Although their tilt angle θ has a total amplitude of 12.2°, that amplitude goes to 91.8° for the angle ψ. These observations suggest that the motion for cervical drilling is mostly longitudinal. The longitudinal axis of the surgical drill must be first adjusted in terms of angular position using a large amplitude on ψ angle and limited on θ. During the actual drilling phase, very little angular adjustment shall be performed on the two angles with similar amplitude while the drill is being inserted through the cervical bone. The above requirement can help defining an appropriate mechanical architecture. Based on these conclusions, it can be suggested to propose spherical mechanism that generates a Remote Center of Motion. A longitudinal linear motion shall be added for the insertion of the drill. The angular motion of the mechanism could be possibly decoupled since the required amplitude is very different for the two angles.

196

T. Essomba et al.

Fig. 3. Evolution of the Euler angles ψ (a) and θ (b) of the surgical drill longitudinal axis during the recorded trajectories.

Robotic-Assisted Platform for Spinal Surgery

197

Table 1. Angular motion amplitude on the cervical drilling trajectories. Psi [°] Trajectory Max

Theta [°] Min

Amplitude Max Min Amplitude

1

175.1 171.7

3.4

60.2 53.4

6.8

2

155

151.2

3.8

58.4 53.8

4.6

3

124.8 116.5

8.3

60.3 50.2 10.1

4

119.9 117.5

2.4

51.7 48.1

3.6

5

91.2

87.9

3.3

57.7 52.4

5.3

6

90.8

88.4

2.4

53.7 50.1

3.6

87.1

83.3

3.8

52.8 51

1.8

175.1 171.7

8.3

60.3 53.8 10.1

2.4

51.7 48.1

7 Max Min

87.1

Amplitude

91.8

83.3

1.8

12.2

4 Conclusion This paper has presented a preliminary study useful for the design of a robotic-assisted platform dedicated to minimally invasive spinal surgery (MISS). Two possible humanrobot collaboration modes have been suggested, i.e. co-manipulation and tele-operation mode. As a preliminary study, the surgeon gestures have been recorded using a motion capture system composed of 8 high resolution infrared cameras. All spinal surgery, leading to seven trajectories, were performed on cadavers in a classic way. These experimentations allowed the analysis and characterization of the surgeon’s gestures as well as interaction performed throughout the real execution of the surgery tasks. The obtained data will be used as input first in the kinematic architecture definition of a drill holder and second the study optimal placement of robot in the co-manipulation mode or tele-operation mode. Acknowledgements. The present work has been supported by a collaboration project between Taiwan’s France’s CNRS (International Research Project-RACeS). This work was also sponsored by the French government research program Investissements d’avenir through the Robotex Equipment of Excellence (ANR-10-EQPX-44).

References 1. Zorn, K.C., Gofrit, O.N., Orvieto, M.A., et al.: Da Vinci robot error and failure rates: single institution experience on a single three-arm robot unit of more than 700 consecutive robotassisted laparoscopic radical prostatectomies. J. Endourol. 21, 1341–1344 (2007) 2. Yaxley, J.W., Coughlin, G.D., Chambers, S.K., et al.: Robotassisted laparoscopic prostatectomy versus open radical retropubic prostatectomy: early outcomes from a randomised controlled phase 3 study. Lancet 388, 1057–1066 (2016)

198

T. Essomba et al.

3. Wren, S.M., Curet, M.J.: Single-port robotic cholecystectomy: results from a first human use clinical study of the new da Vinci single-site surgical platform. Arch. Surg. 146, 1122–1127 (2011) 4. Tasci, A.I., Tufek, I., Gumus, E., et al.: Oncologic results, functional outcomes, and complication rates of roboticassisted radical prostatectomy: multicenter experience in Turkey including 1,499 patients. World J. Urol. 33, 1095–10295 (2015) 5. Soto, E., Lo, Y., Friedman, K., et al.: Total laparoscopic hysterectomy versus da Vinci robotic hysterectomy: is using the robot beneficial? J Gynecol Oncol 22, 253–259 (2011) 6. Kang, C.M., Kim, D.H., Lee, W.J., et al.: Conventional laparoscopic and robot-assisted spleenpreserving pancreatectomy: does da Vinci have clinical advantages? Surg. Endosc. 25, 2004– 2009 (2011) 7. Hu, X., Ohnmeiss, D.D., Lieberman, I.H.: Robotic-assisted pedicle screw placement: lessons learned from the first 102 patients. Eur. Spine J. 22, 661–666 (2013) 8. Theodore, N., Ahmed, A., Karim, B.: The history of robotics in spine surgery. Spine (Phila Pa 1976) 43(7), 23 (2018) 9. Lefranc, M., Peltier, J.: Evaluation of the ROSA™ Spine robot for minimally invasive surgical procedures. Expert Rev. Med. Devices 13, 899–906 (2016) 10. Keric, N., Doenitz, C., Haj, A., et al.: Evaluation of robotguided minimally invasive implantation of 2067 pedicle screws. Neurosurg. Focus 42, E11 (2017) 11. Lonjon, N., Chan-Seng, E., Costalat, V., et al.: Robot-assisted spine surgery: feasibility study through a prospective casematched analysis. Eur. Spine J. 25, 947–955 (2016)

Using a Robot Calibration Approach Toward Fitting a Human Arm Model Valerio Cornagliotto(B) , Elisa Digo, and Stefano Pastorelli Department of Mechanical and Aerospace Engineering, Politecnico di Torino, Turin, Italy [email protected]

Abstract. In the context of Industry 4.0, the human-robot interaction (HRI) can be improved by tracking the human arm in the workspace shared with the robot. This goal takes advantage of a customized human arm modeling and it should be conveniently achieved with a limited number of sensors and a reduced computational time. In this paper, considering the analogy between human and robotic arms, a new method for the identification of a custom-made human arm model was inspired by a robot calibration process. The Denavit-Hartenberg (DH) parameters of the arm model were estimated recording a suitable number of hand poses. Hence, a robotic arm was exploited to test the new method. To simplify the fitting procedure of a reliable robot model, the minimum number of the necessary end-effector (EE) poses was investigated. Through an optoelectronic system, the EE pose trajectory of a UR3 robot was recorded. The optimization of the DH parameters was repeatedly run decreasing the downsampling frequency of the acquired data and then the trajectory error was evaluated. A new reference dataset of robot configurations was acquired permutating the joints degrees of freedom among values of 0, +90, or −90°. Hence, the method to fit the model considering few EE poses was tested on six robot configurations randomly selected from the dataset. Overall, trajectory errors highlighted the applicability of this method in the context of HRI. Keywords: Optimization algorithm · Robotic model · Human-robot interaction · Industry 4.0 · Denavit-Hartenberg parameters · Optoelectronic system

1 Introduction Nowadays, the industrial world is involved in an important transition process called Industry 4.0 assigning a key role to collaborative robotics. Indeed, the contemporary presence of humans and robots in the same workspace enables to exploit both the flexibility and the perception of operators and the precision and the adaptability of robotic systems. Complementary skills of the human-robot team enhance the possibility of reaching a common goal with a safe, successful, and efficient execution [1, 2]. However, the human-robot interaction (HRI) could be improved with adaptive control algorithms identifying human actions, timings, and paths [3]. Accordingly, the motion of operators can be tracked with different technologies such as vision systems [4, 5] and/or wearable inertial sensors [6, 7]. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 199–207, 2021. https://doi.org/10.1007/978-3-030-75259-0_22

200

V. Cornagliotto et al.

The identification of the human body, and especially of human upper limbs, inside the workspace shared with the robot is based on the previous definition of a model. Some literature works have modelled the whole human body or a portion of it by using optoelectronic systems with markers on anatomical landmarks [8–11] or sensors of position and orientation [12]. All these studies have first tracked the human movement by positioning several markers or sensors along the whole kinematic chain. Then, they have proposed and refined a human model by minimizing differences between measured and modelled positions and/or orientations of links. Their approach usually requires long times to prepare subjects before the experiment and to compute data during the postprocessing phase. Both these aspects are not adequate for an industrial context of HRI, which requires easy wearable and non-invasive instruments and an almost real-time elaboration of data. Considering the analogy between human and robotic arms, the fitting of a human arm model could be inspired by a robot calibration process. In detail, a model of the robotic kinematic chain could be fitted by minimizing differences between measured and modelled poses of the end-effector (EE). Robot poses are usually acquired through external instruments such as laser trackers [13] or vision systems [14–16]. Some literature works have proposed procedures of robot calibration involving successive rotations around joint axes [17] or multiple configurations inside the workspace [18]. According to all these considerations, the principal aim of this study was to analyze how a modeling process of a kinematic chain is influenced by the input number of poses. Exploiting the custom-made model fitted on an individual, the configuration of the human arm kinematic chain can be identified only knowing the position and the orientation of the hand. The presented preliminary analysis was conducted on a robotic arm taking advantage that the joints degrees of freedom (DOFs) were accurately measured, but with a view to applying the same methodology on a human arm. Indeed, an optoelectronic system typical of human motion analysis was used to track the pose of the robot EE. Starting from a pre-planned trajectory, the model of the robot was fitted progressively reducing the number of considered poses. Subsequently, new robot models were fitted from different sets of six elementary joints configurations, with the attempt to simulate simple postures of the human arm and to overcome the limitations of unknown DOFs.

2 Materials and Methods The instrumentation adopted for the experimental study included a collaborative robot and an optoelectronic motion capture system (Fig. 1). The robot was the UR3 collaborative 6 DOFs robotic arm (Universal Robots, Denmark). Its nominal standard Denavit-Hartenberg (DH) parameters were considered (Table 1). A gripper tool was mounted on the EE of the robot, which was controlled through the dedicated software UR Polyscope. An Ethernet connection was established between the robot and a PC to record data. Acquisitions were made through the software Matlab® (MathWorks, USA) at 125 Hz. The optoelectronic system was composed of a V120: Trio tracking bar (OptiTrack, USA) and six passive reflective markers with a diameter of 6 mm. The bar was selfcontained, pre-calibrated, and equipped with three cameras able to detect infrared light

Using a Robot Calibration Approach Toward Fitting Model

Ztool

201

Ytool Zworld Yworld Xtool

s X world s

Fig. 1. Experimental setting with the UR3 robot and passive reflective markers. World and tool reference frames are depicted.

Table 1. Nominal DH parameters for each link of the UR3 robot. DH

1

2

3

4

5

6

α [deg] 90

0

0

90

−90

0

a [m]

0

−0.24365 −0.21325 0

0

0

d [m]

0.1519 0

θ [deg] 0

0

0

0.11235 0.08535 0.0819

0

0

0

0

and it was positioned in front of the robot. Three markers were placed on the table near the base of the robot to define a world reference frame. Another three markers were fixed on the robot tool to define a tool reference frame (Fig. 1). Recordings were made through the software Motive (OptiTrack, USA) at 120 Hz. A robot trajectory was planned in order to involve a large movement of all joints simultaneously, orienting the tool to avoid problems of markers occlusion (Fig. 2). During the trajectory execution, DOFs of the robot joints and 3D coordinates of markers were acquired independently. In the post-processing, recorded data were temporally synchronized and resampled at 60 Hz. Subsequently, starting from markers coordinates, two right-handed frames were defined on the table (world) and the tool (tool), respectively. Then, since the position of markers on the table with respect to the robot base was known, an orthonormal matrix describing the relative pose of the robot base with respect to the world reference frame Aˆ world base was defined. Analogously, the relative pose of the reference frame created from markers on the tool with respect to the robot EE reference frame Aˆ EE tool was determined. An algorithm for the robot calibration was implemented to fit the actual DH parameters (Fig. 3). The pose Aˆ world tool was directly calculated from acquired data. The pose world was obtained from the forward kinematics of the DH robot model as a function Tˆ tool of recorded joints DOFs q¯ .

202

V. Cornagliotto et al.

Fig. 2. Representative nominal model of the robot with the acquired trajectory.

Fig. 3. Flowchart of the method adopted to fit the actual DH parameters through a downsampled number of frames and results validation.

ˆ world From the relative pose Uˆ between Aˆ world tool and Ttool , position (p) and angular (θ ) errors were estimated. Accordingly, a cost function was defined as the sum of the two errors properly adjusted through weighting coefficients empirically set. Then, the cost function was minimized to find the best fitting DH parameters as follows in Eq. (1): n   kp pi (DH , q¯ ) + kθ θi (DH , q¯ ) arg minDH (1) i=1

where pi and θi represent the position error and the angular twist error, respectively, estimated for each frame i = [1, 2, …n]; kp and kθ represent the weighting coefficients of the position and angular errors, respectively (kp = 103 , kθ = 180/π ).

Using a Robot Calibration Approach Toward Fitting Model

203

In order to evaluate the minimum number of poses to fit a reliable model of the robot, the implemented algorithm was executed selecting   a progressively reducing number n of frames from the trajectory. The tool poses Aˆ D and the robot DOFs (¯qD ) were extracted from the downsampling and they were processed by the optimization algorithm. For each model obtained from a different number of n, the complete trajectory was simulated and compared with the experimental one. Hence, mean and standard deviation values of position and angular errors were calculated and compared among models. Subsequently, acquisitions were performed to create a reference database of several static elementary configurations of the robot. Sets of joints DOFs were imposed by permutation of values 0, +90, or −90°, avoiding self-collision among links. Then, based on previous results, the implemented fitting algorithm was applied to sets of six poses randomly selected from the database. The algorithm was run several times by changing the sets of configurations with a constant initial guess equal to nominal DH parameters approximated to 10−1 m. Moreover, for each obtained model, the complete trajectory was simulated, and position and angular errors were calculated.

3 Results Figure 4 shows mean and standard deviation values of position and angular errors estimated between experimental and simulation data within the complete trajectory. Each error corresponds to a different robot model fitted considering a progressively decreasing number of n frames (from 600 to 4). It is inferred that no less than 6 frames are necessary to deduce a still reliable model. Therefore, sets of six elementary configurations were gathered from the reference database.

Fig. 4. Position [m] and angular [deg] errors (mean ± σ) evaluated simulating the complete trajectory with robot models fitted considering a decreasing number of frames.

Figure 5 refers to a sample set of six static poses randomly selected inside the database. It describes trends of position and angular errors estimated between experimental and simulation data within the complete trajectory. Mean and standard deviation values are depicted with dotted lines defining a filled band (mean ± σ).

204

V. Cornagliotto et al.

Fig. 5. Position [m] and angular [deg] errors (trend, mean ± σ) evaluated simulating the complete trajectory with the robot model fitted considering the first set of six elementary poses.

Figure 6 shows mean and standard deviation values of position and angular errors corresponding to seven robot models fitted from different sets of poses randomly selected. Previous results in Fig. 5 are related to poses set #1.

Fig. 6. Position [m] and angular [deg] errors (mean ± σ) evaluated simulating the complete trajectory with robot models fitted considering different sets of six elementary poses.

4 Discussions This study aimed to analyze how an optimization process employed to obtain a reliable model of a kinematic chain is influenced by the input number of poses. An optoelectronic system with six markers was adopted to track the EE motion of a UR3 collaborative robot. Several DH models of the robot were defined through an optimization algorithm considering a progressively decreasing number of frames picked from a pre-planned trajectory. Position and angular errors between experimental and simulation data were evaluated. Subsequently, based on previous results, several sets of six static elementary configurations of the robot were used to fit DH robot models. Finally, position error and angular twist error were evaluated on a generic trajectory.

Using a Robot Calibration Approach Toward Fitting Model

205

Figure 4 shows error values consistent with the precision of the adopted motion capture system. In detail, the mean value of position errors is less than 1 mm considering a number of frames ≥6. On the contrary, the position error estimated at 4 frames shows a significant increase (around 3 mm). Considering the mean value of angular errors, it progressively increases by reducing the number of frames (from around 0.5° to 1°). As Fig. 5 shows, the DH robot model fitted through six static poses can depict the trajectory with slight position and angular errors. Moreover, Fig. 6 demonstrates that the six poses necessary to create a reliable DH robot model could be randomly chosen restricting mean errors in the range 0.5–2.5 mm and 0.4–0.7°. Overall, the evaluation of errors for a decreasing number of frames suggests that six poses are sufficient to guarantee a good compromise between two fundamental aspects when the modeling procedure is implemented for human arm motion tracking in an industrial context: the reduction of elementary postures dataset and the good reliability of the fitted model. Since DOFs of a human arm are not known a priori as the ones of a robot, it would be necessary to define six elementary postures for which DOFs could be easily imposed according to biomechanics standards, such as the neutral posture and permutations of shoulder/elbow/wrist flexion/extension, abduction/adduction and medial/lateral rotations [19]. Applying the presented identification methodology to the human arm, it stands to reason that the kinematic model will be in agreement with biomechanical literature works [8–12].

5 Conclusions Overall, DOFs of the human arm cannot be accurately measured without specific instrumentation nor positioning a large number of markers along the whole kinematic chain. However, these conditions are not suitable for the industrial context of collaborative robotics. The present study demonstrates that the acquisition of six poses of the end effector of a robot arm are sufficient to obtain a reliable model of the kinematic chain, based on DH parameters. Considering the analogy between robotic and human arms, a human limb model can be defined through a process similar to the one applied in robot calibration. In detail, the model of the human arm could be fitted starting from six elementary postures which might be reproduced by properly constraining the upper limb. Hence, in future works the most effective method to set the DOFs of the human arm in elementary reference postures will be investigated. Once the subject-specific model will be defined and fitted, it could be exploited to reconstruct the real-time spatial configuration of the upper limb. This could be achieved by solving the inverse kinematics of the hand posture, also taking into account the biomechanical constraints on the range of motion of the human arm joints.

206

V. Cornagliotto et al.

References 1. Ajoudani, A., Zanchettin, A.M., Ivaldi, S., Albu-Schäffer, A., Kosuge, K., Khatib, O.: Progress and prospects of the human–robot collaboration. Auton. Robots 42, 957–975 (2018). https:// doi.org/10.1007/s10514-017-9677-2 2. Bauer, A., Wollherr, D., Buss, M.: Human–robot collaboration: a survey. Int. J. Humanoid Robot. 5, 47–66 (2008) 3. Lasota, P.A., Fong, T., Shah, J.A.: A survey of methods for safe human-robot interaction. Found. Trends Robot. 5, 261–349 (2017). https://doi.org/10.1561/2300000052 4. Melchiorre, M., Scimmi, L.S., Mauro, S., Pastorelli, S.P.: Vision-based control architecture for human–robot hand-over applications. Asian J. Control, 105–117 (2020). https://doi.org/ 10.1002/asjc.2480 5. Scimmi, L.S., Melchiorre, M., Mauro, S., Pastorelli, S.P.: Implementing a vision-based collision avoidance algorithm on a UR3 robot. In: 2019 23rd International Conference Mechatronics Technology ICMT 2019 (2019). https://doi.org/10.1109/icmect.2019.8932105 6. Digo, E., Antonelli, M., Pastorelli, S., Gastaldi, L.: Upper limbs motion tracking for collaborative robotic applications. In: International Conference on Human Interaction and Emerging Technologies (2020) 7. Digo, E., Antonelli, M., Cornagliotto, V., Pastorelli, S., Gastaldi, L.: Collection and analysis of human upper limbs motion features for collaborative robotic applications. Robotics (2020). https://doi.org/10.3390/robotics9020033 8. Laitenberger, M., Raison, M., Périé, D., Begon, M.: Refinement of the upper limb joint kinematics and dynamics using a subject-specific closed-loop forearm model. Multibody Syst. Dyn. 33, 413–438 (2015). https://doi.org/10.1007/s11044-014-9421-z 9. Reinbolt, J.A., Schutte, J.F., Fregly, B.J., Koh, B. Il, Haftka, R.T., George, A.D., Mitchell, K.H.: Determination of patient-specific multi-joint kinematic models through two-level optimization. J. Biomech. 38, 621–626 (2005). https://doi.org/10.1016/j.jbiomech.2004. 03.031 10. Ayusawa, K., Ikegami, Y., Nakamura, Y.: Simultaneous global inverse kinematics and geometric parameter identification of human skeletal model from motion capture data. Mech. Mach. Theory 74, 274–284 (2014). https://doi.org/10.1016/j.mechmachtheory.2013.12.015 11. Begon, M., Wieber, P.B., Yeadon, M.R.: Kinematics estimation of straddled movements on high bar from a limited number of skin markers using a chain model. J. Biomech. 41, 581–586 (2008). https://doi.org/10.1016/j.jbiomech.2007.10.005 12. Prokopenko, R.A., Frolov, A.A., Biryukova, E.V., Roby-Brami, A.: Assessment of the accuracy of a human arm model with seven degrees of freedom. J. Biomech. 34, 177–185 (2001). https://doi.org/10.1016/S0021-9290(00)00179-2 13. Nubiola, A., Bonev, I.A.: Absolute calibration of an ABB IRB 1600 robot using a laser tracker. Robot. Comput. Integr. Manuf. 29, 236–245 (2013). https://doi.org/10.1016/j.rcim. 2012.06.004 14. Zhang, X., Song, Y., Yang, Y., Pan, H.: Stereo vision based autonomous robot calibration. Rob. Auton. Syst. 93, 43–51 (2017). https://doi.org/10.1016/j.robot.2017.04.001 15. Meng, Y., Zhuang, H.: Autonomous robot calibration using vision technology. Robot. Comput. Integr. Manuf. 23, 436–446 (2007). https://doi.org/10.1016/j.rcim.2006.05.002 16. Motta, J.M.S.T., De Carvalho, G.C., McMaster, R.S.: Robot calibration using a 3D visionbased measurement system with a single camera. Robot. Comput. Integr. Manuf. 17, 487–497 (2001). https://doi.org/10.1016/S0736-5845(01)00024-2 17. Zhang, T., Song, Y., Wu, H., Wang, Q.: A novel method to identify DH parameters of the rigid serial-link robot based on a geometry model. Ind. Rob. (2020). https://doi.org/10.1108/ IR-05-2020-0103

Using a Robot Calibration Approach Toward Fitting Model

207

18. Gao, G., Sun, G., Na, J., Guo, Y., Wu, X.: Structural parameter identification for 6 DOF industrial robots. Mech. Syst. Signal Process. 113, 145–155 (2018). https://doi.org/10.1016/ j.ymssp.2017.08.011 19. Gates, D.H., Walters, L.S., Cowley, J., Wilken, J.M., Resnik, L.: Range of motion requirements for upper-limb activities of daily living. Am. J. Occup. Ther. 70 (2016)

Feasibility Study of a Passive Pneumatic Exoskeleton for Upper Limbs Based on a McKibben Artificial Muscle Stefania Magnetti Gisolo1 , Giovanni Gerardo Muscolo2 , Maria Paterna1 , Carlo De Benedictis1(B) , and Carlo Ferraresi1 1 Department of Mechanical and Aerospace Engineering, Politecnico di Torino, Turin, Italy

[email protected] 2 Department of Computer Science, University of Verona, Verona, Italy

Abstract. Exoskeletons are wearable structures or systems designed to enhance human movement and to improve the wearer’s strength or agility, providing auxiliary support aimed at reducing efforts on muscles and joints of the human body. The aim of this work is to discuss on the feasibility of a new passive upper limb exoskeleton, based on the use of pneumatic artificial muscles, and characterized by extreme lightness, cheapness, and ease of use. A broad overview of the state of the art on current exoskeletons is introduced. Then the concept of the new device is presented, and different transmission architectures between pneumatic muscle and limb are discussed. The study demonstrates the potential effectiveness of such a device for supporting an operator in heavy work condition. Keywords: Passive exoskeleton · Pneumatic artificial muscles · Deformable actuators · Wearable systems · Mechanisms

1 Introduction The exoskeleton technology originally focused on military and rehabilitation applications. In the last years, attention is paid to the application of these systems into manufacturing industry. The exoskeletons can be actuated actively or passively [1], depending on the kind of support to be provided. Active exoskeletons require an external power source in order to supply and control the actuation system, which could be pneumatic, hydraulic or electric. Due to their power consumption and battery life, they have reduced load capacity and poor power-to-weight ratio. On the other hand, passive exoskeletons cannot actively assist or control human motion but they use mechanical elements (like springs or dampers), which can store and release energy when required by the human body, without significantly increasing the inertia of the whole system. Even though passive exoskeletons are potentially less effective, they are lighter and simpler because no control is present, providing a high power-to-weight ratio. Most of the devices available in the literature and on the market use electric motors or pneumatic muscles as actuators, and the actuation unit is usually external, or located on the user’s back. To reduce the size and inertia of such systems, cable-based actuation © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 208–217, 2021. https://doi.org/10.1007/978-3-030-75259-0_23

Feasibility Study of a Passive Pneumatic Exoskeleton

209

is often used as an alternative to separate electric DC motor for each revolute joint. Cable-driven exoskeletons can also be designed to implement speed reductions with low friction forces and zero backlash [2–4]. The main advantage of these systems is their capability of allowing a natural motion of the shoulder complex by adding passives Degrees of Freedom (DoF). In other solutions [5, 6], pulleys are used to transmit the motion to the actuation system, although a high number of idle pulleys can be often required to widen the workspace, increasing the bulkiness of the device and the occurrence of friction related issues. Complex architectures as 7 DoF upper limb exoskeletons require particular attention to avoid singularities that can occur in some configurations. Moreover, misalignments between the human arm joint axes and the exoskeleton joint axes can lead to discomfort and pain and should be taken into account during design [7–10]. Thanks to their intrinsic deformability and similarity to human muscles, pneumatic artificial muscles (PAMs) [2] have been employed in systems interacting with human beings, providing safety, compliant behavior, and higher power-to-weight ratio than electric motors [11]. The main drawbacks of this technology are the reduced stroke, the high nonlinear behavior which requires sophisticated control, and the unidirectional tension force. Mainly to improve the stroke of these deformable actuators, as well as to overcome the limitations in terms of the direction of the force exerted, several architectures have been developed for the transmission of the motion between the PAM and the end effector [12–14]. Given their passive characteristics when inflated with pressurized air, PAMs can also be employed as passive elements within the structure of an exoskeleton to provide a support for the user, similarly to spring-based systems that use elastic components to balance the effects of gravity, reducing weight and volume with respect to active solutions [15, 16]. For industrial applications, the aim is typically to assist workers in load handling, repetitive movements, overhead manipulation of objects and relieving them from excessive physical fatigue in manual tasks, always guaranteeing the user’s movements and providing a relatively high shoulder flexion angle. Due to their advantages, passive devices are mostly employed. In particular, they often include passive spring-based mechanisms and leverages to provide the assistive torque at the shoulder [17–21]. The aim of this work is to present a novel passive upper limb exoskeleton meant to assist the user keeping his arms in elevated position, based on a McKibben PAM, with a lightweight structure and simple architecture. The design of the device is presented and its feasibility is discussed by means of simulations, in particular regarding the different solutions adopted to convey the tension force generated by the PAM to the end effector placed on the arm of the user.

2 Materials and Methods 2.1 Original Design of the Exoskeleton and PAM Modeling During overhead handling tasks, the torque created by the weight of the arms should be balanced by the contraction of the human shoulder complex muscles. If the user is required to keep the arm lifted for a long time, a compensating device can be necessary to reduce the human shoulder effort, and to hold that position. The torque due to gravity to

210

S. Magnetti Gisolo et al.

be compensated can be calculated from the inertial body parameters [22], and it assumes a maximum value when the elevation angle θ 1 of the upper arm (mainly in flexion) with respect to vertical position is 90° and the forearm elevation angle θ 2 is 0° (see Fig. 1).

Fig. 1. Torque due to gravity with respect to the elevation angle of the upper arm θ 1 and the forearm flexion angle θ 2 .

To verify the feasibility of the use of McKibben muscles in the exoskeleton, it was decided to consider the heaviest working condition, and to study the behavior of the system when only the elevation angle θ 1 varies, while maintaining the angle θ 2 of flexion of the forearm equal to zero. The torque shown in Fig. 1 (at the top), plus an eventual contribution due to external loads exerted on the arm of the operator, should be partially or totally provided by the exoskeleton, depending on the application and on the physical characteristics of the user. The forearm was considered extended to simulate a typically laborious position, however similar considerations can be deduced for other configurations. A simple representation of the device studied in this work is presented in Fig. 2. In the original design (Fig. 2, left), a transmission based on cable and pulley is used to transmit the PAM tension force to the bracelet (i.e., the end effector) located on the user’s upper arm. Alternatively, the fixed pulley could be replaced by a fixed passing point for the cable. With respect to the scheme shown in Fig. 2 (left), the geometric parameters a, c0 , α and β are constant, so the segment c can be calculated:  c = c0 cos β (1) By neglecting the fixed pulley radius, b is the distance of the attachment point of the cable on the arm’s support with respect to the pulley, and it is only function of the elevation angle θ 1 . The angle γ is calculated as:  2   b + c2 − a 2 , b = a2 + c2 + 2ac cos(θ1 + β − α) (2) γ = cos−1 2bc

Feasibility Study of a Passive Pneumatic Exoskeleton

211

By calculating the semi-perimeter S and the area A of the triangle (a, b, c), the lever arm r of the tension force produced by the pneumatic muscle is:    r = 2A b, A = S(S − a)(S − b)(S − c), S = (a + b + c) 2 (3)

Fig. 2. Different designs of the exoskeleton: original design (left), cam-based design (right).

Among the several solutions available, a commercial McKibben artificial muscle (DMSP-20-200N, Festo, Germany, nominal length l0 = 0.21 m) has been considered for this application. A static characteristics of the force exerted by the pneumatic muscle, function of the percentage contraction k and of the relative supply pressure p, has been produced, using the exponential function to approximate the modified Hill’s muscle model, through MATLAB Curve Fitting Toolbox (The MathWorks, USA) [23], see Fig. 3: Fmu = (a1 p + a2 )ea3 k + a4 kp + a5 p + a6

(4)

Fig. 3. Static characteristics of the McKibben muscle (DMSP-20-200N, Festo, Germany) at different supply pressures.

212

S. Magnetti Gisolo et al.

When the arm is in its initial position at θ 1 = 90°, the initial contraction of k 1 = 5% is chosen. Therefore, the initial length L 1 is:   k1 L1 = l0 1 − = 0.1995 m (5) 100 The muscle is pressurized at the desired pressure value, so it contracts lifting the arm until the torque generated by the muscle reaches the static equilibrium with the moment due to gravity. For each elevation angle θ 1 of the arm, the shortening b of the wire, which is considered non-extendible, is equal to the contraction of the pneumatic muscle:   b(θ1 ) = b 90◦ − b(θ1 ) (6) Finally, the actual length of the pneumatic muscle L and its percentage of contraction k can be calculated: L(θ1 ) = l0 − b(θ1 ) 

L(θ1 ) k(θ1 ) = 100 1 − l0

(7)  (8)

By selecting different air pressure p, the tension force F mu can be derived from Eq. (4) for each elevation angle θ 1 and used to estimate the torque M mu exerted by the pneumatic muscle about the shoulder: Mmu = Fmu r

(9)

The matching between the device torque M mu and the gravitational load required (Fig. 1) is relevant for the correct functionality of the exoskeleton. 2.2 Cam-Based Design of the Transmission System With respect to the solution presented in Fig. 2 (left), an additional cam-based architecture has been studied. The adoption of a cam profile for transmitting the force of the actuator to the limb of the operator is aimed at approximating as much as possible the torque generated at the shoulder by the actuator to that created by the gravitational load, so as to extend as much as possible the range of the elevation angle at which the operation is to be performed. The cam profile and position should be selected in order to not disturb the user’s view and to avoid increasing the bulkiness of the system with respect to the original design. To this outcome, the cam profile should not exceed the eye position, with respect to the shoulder joint (SJ), that can be estimated from the literature [22].

Feasibility Study of a Passive Pneumatic Exoskeleton

213

The proposed architecture is based on a cam profile formed on a rigid shoulder pad centered on the ideal center of the SJ (see Fig. 2, right). The same pneumatic muscle shown in the original design is assumed. The cam design and a scheme of its implementation in the final device have been performed mainly by graphical approach, as presented in Fig. 4. In the conceiving of the exoskeleton, it was decided to design it hypothesizing a working range of the elevation angle θ 1 between 90° and 135°, assuming assistance for operations in an elevated position. This implies that to move outside this range (particularly in positions below 90°) the operator would have to exert a significant effort at the shoulder level. A non-working position below 90° could be reached by reducing the pressure in the actuator. For a defined working range of elevation angle and a nominal supply pressure (p = 3 bar), the initial lever arm r i of the muscle force has been deducted from the equilibrium of the gravitational load. To determine the muscle force at θ 1 = 90°, an initial percentage contraction k 1 of 5% has been selected. Then, the desired lever arm r f of the muscle force at the maximum elevation angle has been chosen, ranging between 40 mm and 70 mm. This range was selected to limit the resulting cam radius, since a large cam profile could be difficult to implement in the final design. The limit values for r f have been selected by iterative tuning, in order to get force and contraction values that could be achieved by the PAM used, given the static characteristics shown in Fig. 3. The cam profile has been graphically obtained by circular arc interpolation, in particular by ensuring the tangency between the cable and the profile itself, for the initial (θ 1 = 90°) and final (θ 1 = 135°) configurations. Intermediate positions were extrapolated from the CAD design and imported into MATLAB to calculate the muscle force lever arm r for each configuration between θ 1 = 90° and θ 1 = 135°. Similarly, the free cable length and the resulting percentage contraction k of the PAM were calculated as presented in Eq. (7). Finally, from Eqs. (4) and (9), the muscle tension and torque were respectively evaluated.

Fig. 4. Cam-based architecture for r f = 40 mm configuration: graphical approach to evaluate cam profile (left), implementation of the cam in the exoskeleton (right).

214

S. Magnetti Gisolo et al.

3 Results and Discussion The functionality of the exoskeleton depends on the matching between the torque generated by the device and the external load exerted on the arm of the operator. For this reason, the supporting torque provided by the actuator at the SJ was computed for both configurations shown in the previous section and compared to the gravitational load. The configurations tested are briefly resumed in the following: • Configuration A: original design, cable and pulley transmissions system. • Configuration B: cam-based transmission system, with the cam centered at the SJ. The original configuration (A) was tested in a “no load” condition, subjected only to the weight of the arm, and in a loaded condition, with a 1 kg load in the hand. The results of these simulations are presented in Fig. 5. With respect to Fig. 2 (left), the exoskeleton structure was based on the following geometrical parameters and condition: a = 0.15 m, c0 = 0.2 m, α = 80°, β = 14°, θ 2 = 0°. The loaded condition represents a typical situation in which the user is holding a tool with lifted arms. In this condition, the initial contraction is assumed to be 5% as for the unloaded condition, but it results in a lower equilibrium position as shown in Fig. 5, since the supporting and gravitational torques match at lower elevation angles θ 1 . However, even in the “no load” condition, the system could not fulfill one of the most critical specifications of this design, that is an adequate matching between the gravitational torque and the one provided by the device. The difference between the torque profiles shown in Fig. 5 and the torque due to gravity requires an additional contribution from the operator to move the arm within the range of elevation angles selected.

Fig. 5. Torque due to the weight of the arm, in unloaded and loaded condition, and torques exerted by the exoskeleton (configuration A) at different supply pressures.

The performance of the exoskeleton in the original configuration A was then compared to the behavior of the system with the cam-based design (configuration B). The

Feasibility Study of a Passive Pneumatic Exoskeleton

215

same geometrical parameters used for configuration A were selected (c0 = 0.2 m, β = 14°, θ 2 = 0°). Simulations have been made for final lever arm r f ranging from 40 to 70 mm. The best results have been obtained for r f equal to 40 and 50 mm and are presented in Fig. 6. The limited mismatch between the torque profiles can be quite easily provided by the user, whose activity can be effectively supported by the device in a very extended working range. Appropriate regulation of the supply pressure can adapt the device action to the operating conditions. Between the presented solutions, configuration B is the most promising also regarding the reduction of the bulkiness of the system, with the cam, centered at the SJ, being easily integrated within a shoulder pad, without the need for a supporting bar (as the one shown in Fig. 2, on the left, with length a).

Fig. 6. Torque due to the weight of the arm and torques exerted by the exoskeleton (configuration B) at different supply pressures. Each plot refers to a different value of final lever arm r f .

4 Conclusion This study demonstrates the feasibility of a passive exoskeleton for the upper body based on the use of a McKibben pneumatic artificial muscle. The exoskeleton may effectively support an operator that must perform a task with upper limbs in an elevated position. The performance of the exoskeleton strictly depends on the system of transmission of the actuator force to the operator’s limb. A transmission based on the sliding of a tendon cable on a cam-shaped shoulder strap is particularly convenient, since it allows for extending the compensatory effect of the actuator over an extended working range of the shoulder joint. However, the effectiveness of the result requires an accurate design of the cam profile. The structure of the exoskeleton may be extremely simple and lightweight. The device may be very easily adapted to the anthropometric characteristic of the operator and to the kind of work that must be performed. The proposed architecture has been designed to support the upper limb lifted in the sagittal plane. However, limited abduction/adduction and internal/external rotation

216

S. Magnetti Gisolo et al.

angles can be allowed by appropriate design of the transmission system, in particular regarding the connection between the bracelet and the fixed frame integral with the trunk. For instance, a 2 DoF joint centered at the SJ could be implemented to allow such movements.

References 1. Gopura, R., Kiguchi, K.: Mechanical designs of active upper-limb exoskeleton robots: state-of-the art and design difficulties. In: IEEE International Conference on Rehabilitation Robotics, ICORR 2009, pp. 178–187 (2009) 2. Sanjuan, J.D., et al.: Cable driven exoskeleton for upper-limb rehabilitation: a design review. Robot. Auton. Syst. 126, 103445 (2020) 3. Nef, T., et al.: ARMin - exoskeleton for arm therapy in stroke patients. In: 2007 IEEE 10th Inernational Conference on Rehabilitation Robotics, pp. 68–74 (2007) 4. Cappello, L., et al.: A series elastic composite actuator for soft arm exosuits. In: 2015 IEEE International Conference on Rehabilitations Robotics, pp. 61–66 (2015) 5. Perry, J.C., Rosen, J., Burns, S.: Upper-limb powered exoskeleton design. IEEE/ASME Trans. Mechatron. 12(4), 408–417 (2007) 6. Gopura, R., Kiguchi, K., Yi, Y.: SUEFUL-7: a 7DOF upper-limb exoskeleton robot with muscle-model-oriented EMG-based control. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1126–1131 (2009) 7. Lo, H.S., Xie, S.Q.: Exoskeleton robots for upper-limb rehabilitation: state of the art and future prospects. Med. Eng. Phys. 34, 261–268 (2012) 8. Cui, X., et al.: Design of a 7-DOF cable driven arm exoskeleton (CAREX-7) and a controller for dexterous motion training or assistance. IEEE/ASME Trans. Mechatron. 22(1), 161–172 (2017) 9. Park, H.S., Ren, Y., Zhang, L.Q.: IntelliArm: an exoskeleton for diagnosis and treatment of patients with neurological impairments. In: 2008 2nd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics, pp. 109–114 (2008) 10. Dehez, B., Sapin, J.: ShouldeRO, an alignment-free two-DOF rehabilitation robot for the shoulder complex. In: 2011 IEEE International Conference on Rehabilitations Robotics, pp. 1–8 (2011) 11. He, J., et al.: RUPERT: a device for robotic upper extremity repetitive therapy. In: 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, pp. 6844–6847 (2005) 12. Sasaki, D., Norisugu, T., Takaiwa, M.: Development of active support splint driven by pneumatic soft actuator (ASSIST). In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation, pp. 520–525 (2005) 13. Klein, J., et al.: Biomimetic orthosis for the neurorehabilitation of the elbow and shoulder (BONES). In: 2008 2nd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics, pp. 535–541 (2008) 14. Tsagarakis, N.G., Caldwell, D.G.: Development and control of a ‘soft-actuated’ exoskeleton for use in physiotherapy and training. Auton. Robots 15, 21–33 (2003) 15. Rahman, T., et al.: Passive exoskeletons for assisting limb movement. J. Rehabil. Res. Dev. 43, 583–590 (2006) 16. Rahman, T., et al.: A simple technique to passively gravity-balance articulated mechanisms. ASME Trans. Mech. Des. 117(4), 655–658 (1995) 17. Yin, P., et al.: Effects of a passive upper extremity exoskeleton for overhead tasks. J. Electromyogr. Kinesiol. 55, 102478 (2020)

Feasibility Study of a Passive Pneumatic Exoskeleton

217

18. Huysamen, K., et al.: Evaluation of a passive exoskeleton for static upper limb activities. Appl. Ergon. 70, 148–155 (2018) 19. HIEIBER: Hyetone Industrial EXO Intelligent Boost Exoskeleton Robot (HIEIBER). Guangzhou Hyetone Mechanical and Electrical Equipment Co., Ltd, The China (2019). www. hyetone.com 20. Maurice, P., et al.: Objective and subjective effects of a passive exoskeleton in overhead work. IEEE Trans. Neural Syst. Rehabil. Eng. 28(1), 152–164 (2019) 21. Zhu, Y., et al.: Automatic load-adapting passive upper limb exoskeleton. Adv. Mech. Eng. 9, 1–8 (2017) 22. De Leva, P.: Adjustments to Zatsiorsky-Seluyanov’s segment inertia parameters. J. Biomech. 29(9), 1223–1230 (1996) 23. Pitel, J., Tothova, M.: Modeling of pneumatic muscle actuator using Hill’s model with different approximations of static characteristics of artificial muscle. In: MATEC Web of Conferences, vol. 76, 02015 (2016)

Control

Adapting Behaviour of Socially Interactive Robot Based on Text Sentiment Sarwar Hussain Paplu(B) , Mohammed Nazrul Islam Arif, and Karsten Berns Department of Computer Science, RRLAB, Technische Universit¨ at Kaiserslautern, 67663 Kaiserslautern, Germany {paplu,m arif19,berns}@cs.uni-kl.de https://agrosy.informatik.uni-kl.de/en/

Abstract. The assessment of emotion is a complex process that encompasses information from various channels or sources, e.g., verbal, paralinguistic, non-verbal or even textual cues. Understanding the emotion of an interaction partner and exhibiting proper bodily reactions play a crucial role in establishing an engaging interaction. The evaluation of emotion, in general, is significant as far as emotionally intelligent human-robot interaction is concerned. In this paper, a text-based interaction system has been introduced to establish affective communication. A chatbot system has been implemented on a humanoid robot named ROBIN. The sentiment of the user input texts has been analyzed with MITIE machine learning toolkit. The robot considers emotional states of the users’ text utterances and reacts with appropriate speech, gesture, and facial expressions. This triggers real-time affective behavior from the robot’s perspective. The proposed system has been tested on several interaction scenarios to validate the approach from a human-centered perspective. Keywords: Human-robot interaction

1

· Text sentiment · Chatbot

Introduction

Text conveys not only some specific meaning but also bears attitudinal information, e.g., emotional states. It is important to understand what is being communicated and in what manner speech is directed. Over the years, text-based interaction introduced a new dimension in establishing interaction between humans and virtual agents or robots. There are several modalities of Human-Robot Interaction (HRI) available in relevant research domain, with verbal and non-verbal modalities of interaction being the prominent ones [1]. Integration of communicative intents with the help of text-based sentiment analysis can improve the emotion assessment approaches that use only non-verbal cues, e.g., recognition of gestures, body postures, and facial expressions [2]. In other words, non-verbal cues are not sufficient to understand communicative intents. c The Author(s), under exclusive license to Springer Nature Switzerland AG 2021  S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 221–228, 2021. https://doi.org/10.1007/978-3-030-75259-0_24

222

S. H. Paplu et al.

Chatbot platforms ensure constant speech feedback from virtual assistants or real-world robots. Keeping in line with this, such modalities have been adapted in humanoid robots [3], with less exploration in the real-time analysis of text emotion and the subsequent display of appropriate robot behaviors (e.g. speech, facial expressions, gestures or postures). With the world moving towards the automation and robotics, humanoid robots have great prospect in the technology-driven future world to help people and to make life much more effortless. In the future, we can imagine a humanoid robot helping us in our household chores, daily activities, healthcare etc. [4]. However, these systems can be improved taking extensive emotion assessment into consideration. Simple question-answering systems [5] often fail to interact in an affective manner as these systems mostly focus on the delivery of a reply rather than assessing sentiment of the queries. Moreover, social assistive robots performing redundant activities on a daily basis can be more effective if emotional attributes are incorporated. Humans tend to show change in behavior based on an interaction partner’s speech emotion or the sentiment inherent in texts [6]. Therefore, change in robot behavior based on an interlocutor’s textual emotional states can trigger more natural interaction. The approach introduced in this paper takes this aspect of Human-Human Interaction into consideration and establishes interaction based on real-time analysis of textual emotion. This triggers more realistic and humanlike behaviour from a robot. The robot behaviour, in this context, refers to the combination of speech derived from the chatbot output and a good number of pre-implemented gestures and facial expressions [7]. Our previous research on the humanoid robot ROBIN [8] focused more on vision-based interaction. However, this work introduces an approach to establish real-time affective textual communication with the interactive robot. The rest of the paper is structured as follows: Sect. 2 summarizes the relevant state-of-the-art, Sect. 3 explains the related technologies used for our implementation: chatbot system, sentiment analyzer. The details of the experiments and human-centered evaluation results are provided in Sect. 4.

2

State-of-the-Art

Alan Turing’s question of whether machines can think has spawned numerous, passionate debates in the context of artificial intelligence. The first attempt to process natural language by a computer program called Eliza [9] was conducted between 1964 and 1966. There have also been several attempts to make a conversation between humans and robots. In [10], a robotic system named “CleverNAO” can interact with people in English. It is a combination of a chatbot application called “Cleverbot” and a NAO robot equipped with speech synthesis. A dialogue manager was developed for a Humanoid robot named ERICA [3] that converses with one or more human interlocutors. She perceives the environment and users through microphones, video cameras, depth and motion sensors. They integrate dialogue manager with a statement-response component, generating dialog about focused words detected in the user’s utterance. However, there is a

Adapting Robot Behaviour Based on Text Sentiment

223

proactive initiator which generates dialog based on events detected by ERICA. The robot continuously monitors its environment in search of cues about the intent of the user. In [11], an approach has been applied to augment a conversational robot interface to facilitate the generation of topically relevant replies and emotionally appropriate expressions during real-time interactions with users. The humanoid robot interface is implemented to detect the topic and sentiment of the user’s text-transcribed speech. It consists of a topic-detection subsystem that generates a topically relevant reply and a sentiment-analysis subsystem that obtains an overall sentiment score of input text. This study only deals with small documents (around 20 words) consisting of text-transcribed conversational responses but it shows that automatic topic detection and real-time sentiment analysis of the user text-input made the interaction more engaging. Although several related research works have been conducted to utilize emotions in HRI, it is still a challenging task when it comes to analyzing emotions and topics on-the-fly as far as textual communication is concerned. In this paper, we endeavour to extract text sentiments and display actions or reactions from a humanoid robot to bring the robot close to an emotionally intelligent entity.

3

Implementation

3.1

Textual Communication

Chatbots are often considered to be an efficient tool establishing interaction between a human and a virtual agent or a robot. Our chatbot program preprocesses the user input, removing all the punctuation and extra spaces. Several conversational datasets e.g. simple dialogs1 , chatbot-nlp2 have been integrated to create a knowledge base containing all possible chatbot outputs. In order to match the exact user inputs, the concept of “fuzzy string search” has been used. Upon pre-processing of the text inputs, a list of keywords are generated. The Levenshtein distance has been used to measure the similarity between the user prompt and the keywords of the knowledge base. A response is picked up from a pool of possible responses from the knowledge base. The selection process is determined by a randomizer function. In this work, we have opted for a keyword-matching chatbot in which a keyword is just a phrase or even a word that the chatbot might recognize from the user’s input which, in turn, triggers a response. Even if the input keyword does not match with knowledge base, the chatbot provides us with some reasonable responses, e.g. “I do not understand what you exactly mean. Could you please explain in other words?”, “I have no idea what you are talking about” etc. This ensures a robust flow of interaction. The chatbot can also handle redundancy by simply storing the previous response and matching with the present input.

1 2

https://www.kaggle.com/grafstor/simple-dialogs-for-chatbot. https://www.kaggle.com/hassanamin/chatbot-nlp.

224

3.2

S. H. Paplu et al.

Sentiment Analysis

Sentiment Analysis is the computational study of peoples opinion, attitude, and emotion towards an entity [12]. Social media apps and monitoring companies relay on sentiment analysis and machine learning to assist them in gaining insights of user behavior. During textual communication in the context of HRI, the quality and the sense of a conversation can be identified with the analysis of sentiment. MIT Information Extraction (MITIE)3 toolkit has been incorporated with our chatbot as a sentiment analyzer. The library uses state-of-the-art statistical Machine Learning (ML) approaches. It is built on top of a high-performance machine-learning library named DLIB [13]. MITIE offers several pre-trained ML models trained with a variety of linguistic resources for English, Spanish, and German. It uses the Bag-of-Words(BoWs) technique for feature selection, which treats a document or a sentence as a group of words. The other functionalities of BoWs include the removal of stop-words and stemming steps (setting the stem or root of a word i.e. flies→fly) to select the features. MITIE performs the tokenization of the text input before the selection of features and uses NLTK’s [14] default Part-of-Speech (POS) tagger for POS tagging. The toolkit leveraged Support Vector Machines (SVMs) approach for the classification tasks. The main principle of SVMs lies in the determination of linear separators in the search space which can best separate the different classes. A diverse set of emotion words and sentences have been leveraged from various sources4 to train a sentiment model containing labels of six basic emotions: happiness, sadness, surprise, anger, fear and disgust. The trained model have been tested against the chatbot inputs to get the emotional state of the robot. For example, the prompt “You look terrible” triggers “sad” emotional state of the robot. 3.3

Workflow

Figure 1 illustrates the overall workflow of the implementation architecture. An interface that takes chatbot input has been implemented. This input triggers a response based on the knowledge base of the chatbot Sect. 3.1. The generated response and the user input are fed into sentiment analyzer module simultaneously. The important features of the texts have been extracted utilizing Bag of Words (BoW) [15] technique. This triggers a sentiment score and an emotional state, e.g., happiness, sadness, surprise etc. of an input text. In the end, we get a response generated by the chatbot, sentiment score along with emotional state of the input text. The proposed system can be applied to a diverse set of scenarios, provided that the knowledge base of the chatbot is adapted accordingly. This ensures the robustness and flexibility of our system. An existing XML based dialog system [7] has been utilized to handle the real-time communication process and the display of robot’s behaviour. The output of the chatbot along with the name of the robot’s gestures and facial expressions is transferred to the 3 4

https://github.com/mit-nlp/MITIE. https://goo.gl/B64Bex.

Adapting Robot Behaviour Based on Text Sentiment

225

dialog system on-the-fly. As a result, the robot performs relevant actions based the content of the textual messages.

Fig. 1. Block diagram showing the workflow of the proposed system

4

Experimentation and Evaluation

A humanoid interactive robot named ROBIN has been used for the experimentation. The robot is equipped with 34 degrees of freedom, enabling the display of various gestures and postures. Inspired by the work of Paul Ekman [16], the robot’s face comprises of 40 Action Units (AUs): the building-blocks to create facial expressions e.g., happiness, sadness, anger, surprise etc. Moreover, ROBIN’s perception system is based on a ASUS Xtion RGB-D Kinect sensor [8], which recognizes gestures, facial expressions and activity of an interaction partner. The robot has a built-in text-to-speech synthesizer for German and English language with male and female voices. ROBIN’s perception and control system run on a robotic framework called Finroc [17], written in C++. The system proposed in this paper has been implemented on this framework. A total of 21 university students were invited to interact with the robot via our chatbot interface. Specific interaction scenarios have been created to observe how the robot behaves for a specific text input. The participants were asked to interact with ROBIN considering various scenarios. There is a “University info” scenario in which the interlocutors ask the robot questions related to our university. The questions may range from administrative queries or statements to getting locations of the important buildings of the university. A “Favourite Game: Football” scenario has also been implemented in which the robot responds to queries related to football. The robot can also handle “Small Talk” scenarios in which the participants engage into few “conversation starter” at the beginning of the interaction. The proposed system can be applied to any specific scenario, provided that the knowledge base is adapted accordingly. Each participant interacted in all the scenarios. The subjects were asked to pay special attention on how the robot behaves after each text query or statement. We provided each of these interlocutors with a questionnaire comprising of questions

226

S. H. Paplu et al.

e.g., (i) were the responses appropriate based on the user query or the statement? (ii) whether the emotional expression was consistent with the robot’s response. Figure 2 illustrates that a large number of participants (76%) considered accuracy of emotional expression to be “satisfactory” and 67% of them rated response appropriateness as “satisfactory”. However, in some situations our emotion analyzer misclassified the emotional state of input text, resulting in inappropriate emotional behaviour of the robot. In addition, 19% of the subjects expressed their dissatisfaction in terms of the responses generated by the chatbot. In most cases, ROBIN was successful at responding users’ queries or statements.

Fig. 2. Questionnaire-based evaluation results of the experiments conducted

A sample conversation between an interaction partner and the robot is as follows: Human: Hi ROBIN: How are you? Human: I am fine ROBIN: Great Human: What are you doing? ROBIN: Why talking about me, let’s talk about you instead Human: Tell me something ROBIN: Continue, I’m listening Human: What makes you irritated? ROBIN: Segmentation faults make me unhappy (emotional state: anger) Human: Today is a gloomy day! ROBIN: Gloomy weather makes me sad (emotional state: sad) . . Human input: Bye ROBIN output: Ok! Goodbye (emotional state: happy) It is noticeable that the text inputs at the beginning of the conversation (“small talk”) triggered “happy” emotional state out of the robot. At one stage, the robot switches its emotional behaviour from “happiness” to “anger” as the preceding text input of the human “What makes you irritated?” is classified as “angry” by

Adapting Robot Behaviour Based on Text Sentiment

227

MITIE sentiment analyser. The emotional state of these user inputs have been classified with MITIE (Sect. 3.2). We also get a corresponding sentiment confidence score, ranging from 0 to 1, out of the user input. Figure 3 depicts a sample user interface of the proposed system that establishes affective communication with the robot, with robot’s emotional reaction shown above the interface.

Fig. 3. ROBIN showing an emotional state (“anger”) based on the text input

5

Conclusion

This paper introduces a dialogue-based communication system, triggering responses from a robot with relevant emotional behaviour. The evaluation of the emotional state of a robot based on text sentiment is crucial to ensure situational behaviour. The experimental results validate that the proposed system is capable of establishing real-time affective human-robot interaction. The most challenging part of this work lies in the assessment of accurate emotional state from the textual conversation as high-level linguistic cues, e.g., sarcasm, irony etc. are often misinterpreted by the sentiment analyzer. This work can be extended by integrating a suitable speech recognition system with the robot so that the interlocutors can establish face-to-face verbal communication. The visual cues of ROBIN’s perception system can be incorporated with the proposed system to improve the recognition of human emotions from the robot’s perspective.

228

S. H. Paplu et al.

References 1. Thomaz, A., Hoffman, G., Cakmak, M.: Computational human-robot interaction. Found. Trends Robot. 4(2–3), 105–223 (2016) 2. Al-Darraji, S.: Perception of Nonverbal Cues for Human-Robot Interaction. Verlag Dr. Hut (2016) 3. Milhorat, P., Lala, D., Inoue, K., Zhao, T., Ishida, M., Takanashi, K., Nakamura, S., Kawahara, T.: A conversational dialogue manager for the humanoid robot ERICA. In: Advanced Social Interaction with Agents, pp. 119–131. Springer (2019) 4. Fridin, M.: Storytelling by a kindergarten social assistive robot: a tool for constructive learning in preschool education. Comput. Educ. 70, 53–64 (2014) 5. Brill, E., Dumais, S., Banko, M.: An analysis of the AskMSR question-answering system. In: Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002), pp. 257–264 (2002) 6. Muench, F., Baumel, A.: More than a text message: dismantling digital triggers to curate behavior change in patient-centered health interventions. J. Med. Internet Res. 19(5), e147 (2017) 7. Paplu, S.H., Mishra, C., Berns, K.: Pseudo-randomization in automating robot behaviour during human-robot interaction. In: 2020 Joint IEEE 10th International Conference on Development and Learning and Epigenetic Robotics (ICDLEpiRob), pp. 1–6. IEEE (2020) 8. Zafar, Z., Paplu, S.H., Berns, K.: Automatic assessment of human personality traits: a step towards intelligent human-robot interaction. In: 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids), pp. 1–9. IEEE (2018) 9. Weizenbaum, J.: ELIZA–a computer program for the study of natural language communication between man and machine. Commun. ACM 9(1), 36–45 (1966) 10. Serrano, J., Gonzalez, F., Zalewski, J.: CleverNAO: the intelligent conversational humanoid robot. In: 2015 IEEE 8th International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), vol. 2, pp. 887–892. IEEE (2015) 11. Russell, E.: Real-time topic and sentiment analysis in human-robot conversation (2015) 12. Medhat, W., Hassan, A., Korashy, H.: Sentiment analysis algorithms and applications: a survey. Ain Shams Eng. J. 5(4), 1093–1113 (2014) 13. King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755– 1758 (2009) 14. Bird, S., Klein, E., Loper, E.: Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit. O’Reilly Media Inc., Sebastopol (2009) 15. Zhang, Y., Jin, R., Zhou, Z.H.: Understanding bag-of-words model: a statistical framework. Int. J. Mach. Learn. Cybern. 1(1–4), 43–52 (2010) 16. Ekman, P.: Basic emotions. In: Handbook of Cognition and Emotion, vol. 98, no. 45–60, p. 16 (1999) 17. Reichardt, M., F¨ ohst, T., Berns, K.: Introducing FINROC: a convenient real-time framework for robotics based on a systematic design approach. Technical report, Robotics Research Lab, Department of Computer Science, University of Kaiserslautern, Kaiserslautern, Germany (2012)

Modular Real-Time System for Upper-Body Motion Imitation on Humanoid Robot Talos Kristina Savevska(B) , Mihael Simoniˇc, and Aleˇs Ude Humanoid and Cognitive Robotics Lab, Department of Automatics, Biocybernetics and Robotics, Joˇzef Stefan Institute, Jamova 39, 1000 Ljubljana, Slovenia [email protected]

Abstract. In this paper, real-time motion transfer from a human demonstrator to an advanced humanoid robot Talos is presented. The objective of the motion transfer is to reproduce the demonstrated motion as close as possible with the real robot. Using simple motion transfer while considering physical constraints and safety issues of the robot, a successful motion imitation approach is proposed. By using a low-cost RGB-D camera, human motions are being captured and transferred to the robot. Our main focus was to enable real-time motion imitation and implement a safety module to prevent the robot from executing fast, non-safe movements. While the stability of the robot has not yet been considered, with the proposed approach Talos can safely imitate upper-body human motion in real-time without significant delays. The system was experimentally evaluated in simulation and on the real robot. Keywords: Motion imitation Humanoid robots

1

· Human-like motion generation ·

Introduction

Most of the currently deployed robotic systems can only perform specific tasks for which they have been programmed. While this allows for high precision and repeatability, such robots usually operate in well-structured environments designed in such a way that the work can be automated and there are only limited external disturbances. The future generation of home and service robots, including humanoid robots, is expected to perform tasks in more natural and highly dynamic environments made for humans. Programming a complex system such as a humanoid robot usually requires expert knowledge. Due to many degrees of freedom and issues with the stability of a free-standing humanoid robot, it is very challenging to program a humanoid robot to perform even a simple task. As we cannot rely on skilled operators in home environments, it is important to develop intuitive and effective solutions for transferring human knowledge to a humanoid robot. Learning from demonstration (LfD) allows endusers without in-depth programming skills to teach a robot [1]. One of the basic c The Author(s), under exclusive license to Springer Nature Switzerland AG 2021  S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 229–239, 2021. https://doi.org/10.1007/978-3-030-75259-0_25

230

K. Savevska et al.

prerequisites for LfD is the ability to capture movements performed by a human demonstrator and mapping them to the robot joints [1]. To capture human motion, methods like visual observation, teleoperation, kinesthetic teaching, and various solutions based on wearable sensors have been proposed [4]. There are multiple ways to capture human motion by visual observation. They include marker-based or optical tracking and also direct human motion extraction from digital images [17]. While marker-based systems and systems relying on wearable sensors provide more accurate detection of human motions, they require the attachment of devices to each segment on the human body, which limits their applicability outside of the laboratory environment [10]. Hence many recent approaches rely on RGB-D cameras and avoid using markers. It was shown that LfD based on real-time human motion imitation is an efficient and user-friendly way to learn new skills on humanoid robots [5]. Based on the captured movements from the demonstrator, the goal of motion imitation is to generate appropriate robot commands that reproduce the demonstrated motion [16]. However, it can be challenging to generate feasible motions for the robot and ensure stable execution due to the differences in the human and humanoid robot kinematics and weight distribution [8,19]. In [15] upperbody imitation was treated as an optimization problem constrained by captured human motions and physical constraints of the robot. The effectiveness of the proposed approach was demonstrated on the humanoid robot HRP-2. In [11] imitation was achieved using a geometric approach with additional safety scaling in the joint limits on the Darwin-OP humanoid robot. In [18] motion imitation was achieved both on a small-size humanoid robot HOAP-3 and a full-size humanoid CB-i [3] using reinforcement learning to optimize the trade-off between unconstrained movement imitation and the humanoid robot’s balance. Multiple motion imitation approaches were demonstrated on a small-size humanoid robot NAO, such as solving inverse kinematics [12] and mapping kinematic chains between a human and a robot [19] to compute joint angles or calculate appropriate control values using particle filter [9]. The aforementioned approaches provided simple lower body controllers for maintaining NAO’s stability. Lower body control of NAO while imitating human motion has been presented in [2]. Stable motion imitation based on prioritized task control on a small robot COMAN was realized in [7]. The scope of this paper is to provide a reproduction module for human motion captured with a Microsoft Kinect v2 sensor in order to generate real-time, safe robot motions for the humanoid robot Talos. The paper is structured as follows. In Sect. 2 we introduce the robotic platform used in this paper. The process of end-to-end motion imitation is described in detail in Sect. 3. In Sect. 4 we present the experimental results gathered in simulation and on the real humanoid robot. Section 5 concludes the paper and proposes the next steps for future work.

2

Humanoid Robot Talos

This paper tackles human motion imitation on the humanoid robot Talos designed by Pal Robotics. Talos is a human-size robot with 32 degrees of freedom,

Motion Imitation on Humanoid Robot Talos

231

of which two are in the head, two in the torso, seven in each of both arms, one in each of both grippers, and six in each of both legs. All joints are rotational. They are equipped with powerful motors to carry out fast movements and with torque sensors (except in the wrist and head joint), which enables the implementation of advanced control strategies for safe human robot-collaboration [14]. However, the robot does not have safety brakes in the joints, which constraints the maximum joint speed around the joint limits. Considering this safety issue we are presenting a safety module around joint limits while maintaining the speed as close as possible to the demonstrated movements. 2.1

Software Architecture

To accomplish imitation of the captured human movement, we use a modular system architecture based on Robotic Operating System (ROS) [13]. We separate the imitation process into three main modules as shown in Fig. 1. The Body Tracking Module estimates the position and orientation of human joints. This data is provided to the Motion Imitation Module responsible for calculating reference joint position values for the robot. Motion Control Module is composed of multiple submodules for controlling each limb of the robot where calculated joint angles are transferred to robot joint motors. Modular architecture eases the transfer of the imitation process evaluation from simulation to the real robot. On the tracking level, modularity enables the application of different tracking software. Decomposition into multiple independent control modules enables interchangeability as well, with the possibility to easier modify, change, and replace each of the modules (e.g. different robot controllers).

Kinect Depth Sensor

Depth frames

Skeleton data

Motion Imitation Module Human-Robot Joint Mapping

Joint angles

Motion Control Module

Motor commands

Gazebo Real robot

Head controller Right arm controller

Body Tracking Module

Safety scaling

Left arm controller Torso controller

Fig. 1. Modular system architecture. The Body Tracking Module continuously provides human body posture (skeleton) estimates to the motion imitation module, which calculates the robot joint angles needed to reproduce the human posture and sends them to the Motion Control Module. The Motion Control Module consists of low-level controllers that communicate with the simulator or the real robot.

232

2.2

K. Savevska et al.

Body Tracking

In our system, we utilize Kinect v2 time-of-flight RGB-D camera, which can capture an RGB-D stream at 30 frames per second (fps) with 70◦ horizontal and 60◦ vertical field of view. To extract human motion, we use API provided by Nuitrack Body Tracking SDK.1 Nuitrack is a cross-platform, sensor-agnostic middleware capable of realtime full-body skeletal tracking. Up to 19 joints that closely correspond with human body parts can be detected as shown in Fig. 2. Each of the joints includes the position in real world coordinates as well as position in normalized projective coordinates and orientation.

Fig. 2. Screenshot of a skeleton as recognized by Nuitrack. The skeleton is represented as a set of 19 joints. By connecting the adjacent joints, a simplified model of the human skeleton is obtained.

3

Imitation Process

The process of motion imitation consists of joint calculation based on the data from the body tracker and joint evaluation through safety check functions. Our main focus was on the safety module and processing the measured human joint angles before sending them to the control module. 3.1

Joint Calculation

Based on the orientations of individual skeleton joints extracted from the body tracker, we calculate the upper body joint configuration for the Talos robot (4 joints for each arm, 2 for the torso and 2 for the head). Skeleton joints orientations are given with quaternions rather than geometrically more intuitive elemental rotations. In order to calculate joint space configuration for the robot independent rotations are required. To this end, we use 1

https://github.com/3DiVi/nuitrack-sdk.

Motion Imitation on Humanoid Robot Talos

233

conversion from quaternion to Euler angles. Given a quaternion q = v +u, v ∈ R, u ∈ R3 , we calculate ⎤ ⎡ ⎤ ⎡ atan2(2(vux + uy uz ), 1 − 2(u2x + u2y )) ϕq ⎦, ⎣ ϑq ⎦ = ⎣ asin(2(vuy − uz ux )) (1) 2 2 ψq atan2(2(vuz + ux uy ), 1 − 2(uy + uz )) where ϕq , ϑq , ψq are the Euler angles known as yaw, pitch and roll. Besides Euler angles, we use conversion to axis-angle representation as well. Rotation given by the quaternion q can be expressed as a rotation around axis aq by angle αq aq =

u , |u|

αq = 2atan2(|u|, v),

(2)

where |u| is the Euclidean norm of the vector part of the quaternion. To calculate joint configuration for the torso of the robot, we first calculate the Euler angles from quaternion of the torso joint, qtorso , from skeleton using Eq. (1). The first angle in the robot’s torso configuration refers to the rotation around the z axis of the torso (deflection) and the second angle is the rotation around the x axis (inclination). Therefore these angles are calculated as T

θtorso = [ϕqtorso , −ϑqtorso ] .

(3)

Similarly, joint space coordinates for the head are calculated using Eq. (1), where the Euler angles are calculated using the orientation of the neck joint, qneck , of the tracked skeleton as: θhead = [−ϑqneck , ϕqneck ]T .

(4)

For calculating the configuration for the robot’s arms, we use orientations of the shoulder and elbow joints from the skeleton to control the first 4 joints of the robot. The first three angles of the robot’s arm correspond to yaw, roll and pitch of the shoulder joint, respectively, calculated from qshldr using Eq. (1). Angle between the upper arm and forearm is calculated from qelbow using Eq. (2). Additional offsets have been added to match the range of the robot’s joint angles. Note that, we did not implement motion imitation for Talos wrists as Nuitrack does not provide orientations of the wrists therefore θ5−7 are not changing. The left arm’s joint configuration can be determined as T  θlarm = ϕqlshldr , ψqlshldr + π2 , −ϑqlshrld , −αqlelbow + π4 , θ5 , θ6 , θ7 .

(5)

Accordingly, configuration for the right arm is calculated using right shoulder and right elbow quaternions. Finally, for a smooth continuous motion, a low-pass second order Butterworth filter is applied to the calculated angles. Hereinafter we refere to these angles as the human angles.

234

K. Savevska et al.

3.2

Safety Processing

The second stage of the motion imitation process is verifying the imitation angles and applying safety checks before sending them to the controller. To achieve this we limit joint velocities by saturation and by additional constraints near the joint limits. The first step is to apply a smoothing filter to the measured human motion data. This module runs at a higher frequency (120 Hz) than the frequency of the body tracker (30 Hz) to achieve smooth movement on the robot. The joint angles for robot control at 120 Hz are computed as follows θprocessed (t) = αθprocessed (t − Δt) + (1 − α)θhuman (t)

(6)

where θhuman are the estimated angles of human body motion obtained at 30 Hz frequency and θprocessed are the angles processed at the multilevel safety check at 120 Hz. They are used as the desired joint angles for the robot control. 0 ≤ α ≤ 1 is the smoothing parameter that ensures the smooth transition from the θprocessed (t − Δt) to θhuman (t) and Δt is the robot servo rate, i.e. Δt = 1/120. Using Eq. (6) we apply simple exponential smoothing [6] that guarantees smooth convergence of the high frequency processed angles towards desired human joint angles. Reducing Large Steps: The first level of the safety module prevents the robot from moving too fast. Firstly, the joint angle step between θprocessed (t) computed in (6) and the current robot angles θrobot (t) is calculated: Δθ(t) = θprocessed (t) − θrobot (t)

(7)

When Δθ(t) exceeds a threshold, in our experiments 80% of the maximum allowed joint angle change, the movement is scaled with a step executed with 80% percent of the maximum speed: θprocessed (t) = θrobot (t) + min(Δθ(t), 0.8 Δθmax ).

(8)

Joint Limits: When approaching the physical joint limits θmin or θmax of the robot, the movement has to gradually slow down and stop completely before the joint limit is reached as the robot does not have brakes. Velocity is limited by calculating a speed scale factor w based on the actual joint position θ. To ensure smooth stopping, limiting is implemented using a double-sided logistic function in the areas close to the limits. This function is defined as ⎧ ⎪ 0, for θ ≤ θmin , and θ ≥ θmax , ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ 1 s ⎪ ⎪ ⎨ 1+e−k(θ−θ0 ) , for θmin < θ < θmin + 2 θrange , (9) w(θ) = ⎪ ⎪ 1 s ⎪ , for θmax − 2 θrange < θ < θmax , ⎪ ⎪ 1+ek(θ−θ0 ) ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩1, + sθ ≤θ≤θ − sθ , for θ min

2 range

max

2 range

235

Speed scale factor w

Motion Imitation on Humanoid Robot Talos

1

θmin θmin + 2s θrange

θmax − 2s θrange θmax

Joint position θ

Fig. 3. Speed scaling given actual joint position

k=

10 s 2 θrange

(10)

where θ0 is the angle value of the sigmoid’s midpoint, k calculated in (10), modifies the steepness of the curve, and s is related to the percentage of the joint’s full range θrange = θmax − θmin at which the safety scaling is applied, see Fig. 3. In our experiments, s was set to 0.2. Given the safety speed scaling factor w we then calculate safe joint configuration as: (11) θprocessed (t) = θrobot (t) + w(θprocessed (t))Δθ(t).

4

Experimental Evaluation

The presented method of motion imitation was executed first in simulation using Gazebo simulation environment. Imitation of the captured motion from the human was achieved in a simulated environment in nearly real time, without significant delays while reproducing normal speed motion, see Fig. 4a. In Fig. 4b, longer delays are visible when reproducing a fast elbow movement due to the safety evaluation of the captured movements, yet the direction of the demonstrated movement is kept the same. We analyzed the delays between the demonstrated and reproduced movements using cross-correlation between the two signals

236

K. Savevska et al.

Fig. 4. Plots of reproducing an elbow motion with a) normal speed, b) fast movement, c) movement beyond the joint limit (dashed horizontal line). The calculated joint position for imitation is shown in blue, the processed position after safety scaling in orange and the measured robot’s elbow position in green.

τdelay = argmax(corr(θhuman (t), θrobot (t))).

(12)

t∈R

Using Eq. (12), the delay of 0.004 s was estimated for a normal speed movement, while longer delays of up to 0.25 s were calculated for the faster movement shown in Fig. 4b. The purpose of the proposed safety mechanism is also visible when the robot is approaching joint limits, where demonstrated movement beyond the physical limit of the joint is not imitated, see Fig. 4c. Simulation results along with the body tracking view are shown in Fig. 5. Experimental results on the real robot, shown in Fig. 6, also show that realtime, smooth motion imitation can be achieved without visible delays. Even though the stability control was not yet integrated, the robot could imitate the human motion while remaining in a stable position.

Fig. 5. Motion imitation in simulation in different poses. Body tracking result is shown on the left and tracked pose in Gazebo on the right.

Fig. 6. Motion imitation on Talos.

Motion Imitation on Humanoid Robot Talos

237

Fig. 7. Motion imitation module can be used to teach simple tasks. From left to right, demonstration, body tracker, motion imitation in Gazebo and motion execution on the real robot are shown.

The operation of the procedure was also demonstrated in a basic learning from demonstration task. The demonstrator first shows the motion of placing the bottle on a table and at the same time monitors the motion imitation in the Gazebo environment. The robot learns the movement and repeats it to place the bottle on the table (Fig. 7). Full video showing the real-time motion imitation in the simulation and on the real robot is available at https://youtu.be/-Hy8MNnF6Pk.

5

Conclusions

In this paper, we proposed a real-time upper body motion imitation on the humanoid robot Talos. The experiments show that with the proposed method we can achieve real-time imitation with no visible delays. Successful implementation of a safety mechanism prevents unpredictable fast movements that may cause the robot to fall or self-collide. At this stage of development, the main focus was on the implementation of the safety module and achieving real-time imitation movement. We demonstrated that real-time imitation can be done even on an advanced, human-size robot with a ROS-based control system. The presented modular architecture of the system enables future extensions by integrating with functionalities of different ROS packages, such as different body trackers, collision avoidance mechanisms, stability controllers, and learning pipelines. In future work, we intend to address issues such as stability and selfcollision. By addressing these issues, we will gain the ability to realize also lower body motion imitation. Acknowledgments. This work has received funding from program group Automation, robotics, and biocybernetics (P2-0076). K.S. is a holder of Ad Futura scholarship (287th Public Call). The authors would like to thank Rok Pahiˇc, Zvezdan Lonˇcarevi´c, Bojan Nemec and Andrej Gams for their valuable suggestions in the software development process.

238

K. Savevska et al.

References 1. Billard, A., Calinon, S., Dillmann, R., Schaal, S.: Robot programming by demonstration, pp. 1371–1394. Springer, Heidelberg (2008) 2. Chen, J., Wang, G., Hu, X., Shen, J.: Lower-body control of humanoid robot NAO via Kinect. Multimed. Tools Appl. 77(9), 10883–10898 (2018) 3. Cheng, G., Hyon, S., Morimoto, J., Ude, A., Colvin, G., Scroggin, W., Jacobsen, S.C.: CB: a humanoid research platform for exploring neuroscience. In: IEEE-RAS International Conference on Humanoid Robots, pp. 182–187 (2006) 4. Dillmann, R.: Teaching and learning of robot tasks via observation of human performance. Robot. Auton. Syst. 47(2), 109–116 (2004) 5. Elbasiony, R., Gomaa, W.: Humanoids skill learning based on real-time human motion imitation using Kinect. Intell. Serv. Robot. 11(2), 149–169 (2018) 6. Fried, R., George, A.C.: Exponential and holt-winters smoothing. In: Lovric, M. (ed.) International Encyclopedia of Statistical Science, pp. 488–490. Springer, Heidelberg (2011) 7. Gams, A., Van den Kieboom, J., Dzeladini, F., Ude, A., Ijspeert, A.J.: Real-time full body motion imitation on the COMAN humanoid robot. Robotica 33, 1049– 1061 (2015) 8. Koenemann, J., Burget, F., Bennewitz, M.: Real-time imitation of human wholebody motions by humanoids. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 2806–2812 (2014) 9. Kondo, Y., Yamamoto, S., Takahashi, Y.: Real-time posture imitation of biped humanoid robot based on particle filter with simple joint control for standing stabilization. In: Joint International Conference on Soft Computing and Intelligent Systems (SCIS) and International Symposium on Advanced Intelligent Systems (ISIS), pp. 130–135 (2016) 10. M¨ undermann, L., Corazza, S., Andriacchi, T.P.: The evolution of methods for the capture of human movement leading to markerless motion capture for biomechanical applications. J. NeuroEngineering Rehabil. 3(1) (2006). https:// jneuroengrehab.biomedcentral.com/about. Article no. 6 11. Nguyen, V.V., Lee, J.: Full-body imitation of human motions with kinect and heterogeneous kinematic structure of humanoid robot. In: IEEE/SICE International Symposium on System Integration (SII), pp. 93–98 (2012) 12. Ou, Y., Hu, J., Wang, Z., Fu, Y., Wu, X., Li, X.: A real-time human imitation system using Kinect. Int. J. Soc. Robot. 7(5), 587–600 (2015) 13. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: ROS: an open-source Robot Operating System. In: ICRA Workshop on Open Source Software, Kobe, Japan (2009) 14. Stasse, O., Flayols, T., Budhiraja, R., Giraud-Esclasse, K., Carpentier, J., Mirabel, J., Del Prete, A., Sou`eres, P., Mansard, N., Lamiraux, F., et al.: TALOS: a new humanoid research platform targeted for industrial applications. In: IEEE-RAS International Conference on Humanoid Robotics (Humanoids), pp. 689–695 (2017) 15. Suleiman, W., Yoshida, E., Kanehiro, F., Laumond, J.P., Monin, A.: On human motion imitation by humanoid robot. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 2697–2704 (2008) 16. Ude, A., Atkeson, C., Riley, M.: Programming full-body movements for humanoid robots by observation. Robot. Auton. Syst. 47, 93–108 (2004) 17. Vakanski, A., Janabi-Sharifi, F.: Robot Learning by Visual Observation. Wiley, Hoboken (2017)

Motion Imitation on Humanoid Robot Talos

239

18. Vuga, R., Ogrinc, M., Gams, A., Petriˇc, T., Sugimoto, N., Ude, A., Morimoto, J.: Motion capture and reinforcement learning of dynamically stable humanoid movement primitives. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 5284–5290 (2013) 19. Zhang, L., Cheng, Z., Gan, Y., Zhu, G., Shen, P., Song, J.: Fast human whole body motion imitation algorithm for humanoid robots. In: IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1430–1435 (2016)

Cloud-Enabled Bi-manual Collaborative Robot with Enhanced Versatility for Customized Production Aleksandar Rodi´c1(B) , Jovan Šumarac1 , Ilija Stevanovi´c1 , and Miloš Jovanovi´c2 1 Institute Mihajlo Pupin, University of Belgrade, Belgrade, Serbia

[email protected] 2 School of Computing, Belgrade, Serbia

Abstract. The paper presents a new prototype of a bi-manual, collaborative service robot for industrial purpose designed for use in the so – called customized production within the Industry 4.0 initiative. The robot is a significant technological means of supporting Small and Medium-sized Enterprises (SMEs) engaged in the manufacturing of products intended for the market. The bi-manual collaborative robot has the features of enhanced operational versatility and flexibility within technological production, thanks to a specific mechanical construction with variable robot geometry, cloud-supported control structure, advanced algorithms of artificial intelligence, inherently embedded in the perception and control system. A feature that makes a special contribution to the technological capabilities of robots refers to the embodiment of the human skill of manipulating objects and planning the accomplishing of technological tasks based on know-how and experience. The prototype of the robot is a mobile, portable system, adaptable to different technological conditions of customized production, suitable and safe for co-working with humans. The developed prototype uses all the designed advantages of its functional modules. The possible imperfections of the 4DOF robot torso mechanism are compensated by the variable geometry of the structure as well as by the versatility of the sensorial system and robot controllability. Since the prototype device is in the phase of mechanical integration, an appropriate complementary avatar has been also designed to demonstrate the capabilities of this intelligent system in virtual reality. Keywords: Industry 4.0 · Cloud robots · Customized production · Bi-manual collaborative robot · Industrial humanoid

1 Introduction Humans perform various manipulative tasks thanks to their ability to adapt their body to the work space i.e. physical environment. That means that a person can perform work tasks by taking different body positions: sitting, standing, bent, squatting, etc. Conventional industrial robots have limited possibilities of structural adaptation in the workspace due to their specific mechanical design. Therefore, the workspace must be adapted to © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 240–249, 2021. https://doi.org/10.1007/978-3-030-75259-0_26

Cloud-Enabled Bi-manual Collaborative Robot

241

them in such a way that the robots can perform their function unhindered. In this paper, a mobile, self-reconfigurable robotic torso is designed that allows bending/straightening of the mechanism and its height adjustment from the squat to a fully extended position, as illustrated in Fig. 1. As the manipulative system, two lightweight robot arms (universal robots UR-5) were added and integrated into the mechanical structure of the robot torso. The operation range and the payload capacity of the robotic arms correspond to the capabilities of the average developed human. In a word, our goal in the paper was to create a variant of an industrial humanoid, i.e. service robot that would as much as possible mimic human manipulative abilities and enable enhanced operation versatility and higher level of collaborative skills needed for co-working with humans. Designed robotic system was developed for the customized manufacturing and collaborative work with humans or robots. The paper presents a self-configurable, modular, bi-manual, industrial service robot for collaborative tasks in manufacturing. The robot is mobile and modular. It consists of a mobile platform (motorized cart), robot trunk (torso) of variable geometry and two light weight robot arms – Universal Robots UR-5. The kinematic scheme of the robot is synthesized in such a way that the system has a maximum reachability in the task space that represents a significantly enlarged working area. Comparing the robot with a man, considered manipulative robotic system may take the operating positions that correspond to the position of workers in a crouch, half bent or in the extended body posture. The system is devoted to enhance the technological concept Industry 4.0 contributing to the customized manufacturing characteristic for the Small and Medium Enterprises (SMEs). The paper is organized in 5 sections starting with introduction. Section 2 presents state-of-the-art in this field as well as the progress beyond. The Sect. 3 presents an innovative mechanical design of the industrial service robot with two lightweight robot arms, designed for safe and reliable collaborative work and quick reconfiguration of the technological batch within manufacturing line. The Sect. 4 deals with the control architecture consisting of three functional levels: strategic, tactical and operation. In Sect. 5 some simulation results using robot avatar are presented. With the Sect. 5, the paper is concluded and further research and development plans on these topics will be listed as well.

2 Progress Beyond State-of-the-Arts Modern requirements of industrial manufacturing require a significantly higher level of flexibility and autonomy of robotic systems than the current ones, as well as a qualitatively higher level of robot intelligence and the possibility of human-robot or robotrobot communication. This includes the use of collaborative robotic systems of the new generation characterized by biologically inspired humanoid structure (with body segmentation), higher levels of intelligence and expertise as well as the use of intuitive multi-modal interface (use of speech, gestural symbolism, messaging, etc.) for mutual more efficient communication. At the moment of the world pandemic caused by the Covid-19 virus, there is an increasing need to construct robotic systems that should replace human auxiliary work in technological operations in which the participation of two or more workers is necessary, such as the assembly of bulky items or processing with tools that require multiple auxiliary operations at the same time. The ultimate

242

A. Rodi´c et al.

goal to be achieved in this paper by developing a collaborative humanoid robot for industrial purposes is greater flexibility of production as well as substitution of human labor in technological tasks where interactive work of two or more workers is necessary [1]. Industrial production, sustainable even in the conditions of large pandemics, can be achieved by minimizing degree of mutual contacts between human workers and by substitution them by collaborative robots. In [2], it was introduced the design of a wheeled-legged mobile manipulation platform capable of executing demanding manipulation tasks, and demonstrating significant physical resilience while possessing a body size (height/width) and weight compatible to that of a human. The achieved performance is the result of combining a number of design and implementation principles related to the actuation system, the integration of body structure and actuation, and the wheeled-legged mobility concept. The humanoid robot Rollin’ Justin [3] is a platform for research in service robotics. Rollin’ Justin’s compliant light-weight arms and four-finger hands enable broad research into sensitive, ambidextrous manipulation. The mobile base allows autonomous operation over a long range. Motion detection sensors and stereo cameras enable 3D reconstruction of the robot’s environment. Its multiple actuated degrees of freedom allow Rollin’ Justin to pursue several goals at the same time while complying with a task hierarchy. By designing Herb 2.0 [4] researchers interested in collaborative manipulation with humans, focusing on robot-human hand-overs. Many of the potential tasks for personal robots, such as fetching objects for the elderly or individuals with motor-impairment, were identified to involve hand-over interactions. Different aspects of robot-human hand-overs have been studied within robotics, including motion control and planning, grasp planning, social interaction and grip forces during hand-over. The key objective of designing the Herb 2.0 bi-manual robot platform has been a commitment to working in a home environment and understanding the nuances of how humans structure their environments. The goal in this paper was to develop a cloud-enabled industrial human-size service robot with enhanced versatility that would be applicable in various tasks adapted to the customized manufacturing in the smaller, market-oriented series. A step forward, compared to the classic industrial robots, it makes the first-generation of intelligent industrial service robots that possess significant level of AI (including perception, context understanding, task planning, decision making, etc.). This assumes the robot possesses necessary prior knowledge about the predefined technological tasks that robot will handle (e.g. move, palletize, assemble, etc.) or that will accomplish (e.g. welding, polishing, drilling, etc.). Within the robot controller, information on objects (e.g. CAD model, photos, color, texture, material of structure, etc.) as well as description of technological operation are prepared and saved in advance. That assumes knowledge about the sequence of tasks, i.e. phases of the technology operations. The development of the industrial humanoid presented in this paper was preceded by extensive research and experimental work in the field of development of mobile platforms, light robotic arms, anthropomorphic hands and cloud control systems in the Robotics Lab. of the Mihajlo Pupin Institute [5–9].

Cloud-Enabled Bi-manual Collaborative Robot

243

3 Mechanical Designs The mechanism of a bi-manual collaborative service robot for customized production consists of several functional modules (Figs. 1 and 2): (i) a motorized carts representing a base and at the same time a mobile platform for transporting and carrying on a dualarm manipulation system, (ii) robot torso where two industrial lightweight robotic arms are mounted, (iii) two universal robots UR-5, (iv) robot end-effectors realized as twofingered or three-fingered robotic grippers that allow different techniques of gripping objects, (v) three-axis force and torque sensor placed at the wrist of robot hand (gripper), (vi) a wrist camera. The mechanical structure of the robot’s torso is designed so that the robot can change its kinematic chain configuration from a fully extended position to a fully contracted one (bent position). The appearance of the robot torso is shown in Fig. 1 in the upright position (standing) as well as in the bent position (squat). In this way, the torso of the robot can change its geometry as needed. This allows the robot to perform various service tasks at different height levels, at the floor, on the table or above on the shelf. The variability of the robot geometry gives it increased application flexibility because the mechanism is more adaptable to the environment.

Fig. 1. Mobile bi-manual industrial service robot with a variable geometry. Robot illustration in a fully extended (standing) position – left side, and in a fully contracted (squatting) position – right side.

The robot cart (Fig. 1) is mounted on two motorized and two auxiliary wheels. Trolley is controlled by a joystick, by use of a differential change of wheel speed. Total weight of the robot with the cart is 217 kg (141 torso and 76 trolley). The maximum speed of trolley, which ensures a safe transport, is 1 m/s. In the transport position, robot must assume a completely bent position as illustrated in Fig. 1, right side. By placing the service robot on the mobile platform, a faster reconfiguration of the technological line can be achieved, as well as better utilization of the working space in the workshop. The trolley is battery operated or by cable using an AC/DC power adapter.

244

A. Rodi´c et al.

The robot torso at its upper end, on the left and right side, has two flanges to which two light industrial robotic arms UR-5 are connected. The payload capacity of one arm is 5 kg and the operation range corresponds to one average human arm. The height of the robot corresponds to the average height of a human (1786 mm). Since the robot and human are approximately of the same height and arm’s reach, the physical interaction of the technical and biological system is facilitated because they are at the same level and similar manipulative capabilities.

Fig. 2. Drawing of the mobile bi-manual industrial service robot of human-size dimensions. Explanation of the particular positions in the drawing: 1) chest, 2) waist, 3) upper leg segment, 4) lower leg, 5) base (motorized wheelchair), 6), 7) shaft sleeves, 8) leg bracket, 9) robot arm UR-5, 10) flat washer, 11) screw, 17) electrical and control cabinet.

4 Intelligent Control Architecture Controlling a bi-manual collaborative robot prototype is a complex task because the system consists of four independent subsystems (motorized trolley, robot torso, left and right robotic arm with grips) that are kinematic and dynamically coupled to each other (Fig. 3). The entire control system has a distributed hierarchical structure. It is subordinated to the higher levels of control as shown in the block diagram presented in Fig. 4.

Cloud-Enabled Bi-manual Collaborative Robot

245

Robotic perception includes data acquisition from the system and its environment and sensor-data fusion of heterogeneous information, collected from different sensors (encoders, tachogenerators, proximity sensors, temperature sensors, tactile sensors, force and torque sensors, noise sensors, etc.) and machine vision. In the background of the collected heterogeneous information, there are system state variable classifiers that serve for data interpretation and context (physical environment) understanding.

Fig. 3. Laboratory prototype of the robot torso (left side). Robot avatar presented in Virtual Reality (right side).

Recognizing objects and understanding environment are important link in the chain of a reliable system control. Versatile perception enables planning of manipulative tasks of industrial service robot and system optimization. Decision-making and task planning at the higher level of system control enable robot to smoothly and accurately perform assigned technological operations within the technological manufacturing line. Since the bi-manual manipulation robot has a distributed hierarchical control structure (Fig. 4), with subsystems each of them performing a particular task, the task distribution control-block enables robot subsystems to operate synergistically and to accomplish common tasks assigned to the robotic system. Before the robot begins to accomplish the planned technological operation, it is necessary to transfer the appropriate knowledge (know-how) on how to perform that operation in the most efficient way. In a word, it is necessary to transfer to the robot the “skill” of how to perform the task. The skill is reflected in acquiring the techniques of achieving fine, complementary movements (very small linear movements up to 1 mm or angular displacements for just 1–2°) by the robot’s end-effector or additional tactile perception that registers very small oscillations in an imposed value of force or moment on the robot hand (gripper or tool). Possession of the skill leads the robotic arm to perform the operation according to the control algorithm which guarantees achievement of necessary accuracy of manipulation.

246

A. Rodi´c et al.

Fig. 4. High-level system description of the cloud-enabled intelligent control of the mobile bimanual service robot with variable geometry.

The cloud control structure, shown in Figs. 4, enables technological task to be assigned to the robot remotely from the project office (CAD design bureau), as well as to transfer the necessary knowledge about the technology process to the robot. The presented high-level cloud architecture is realized on three functional levels: physical, network and application. At the physical level, robot performs operation i.e. technological task, the network level is used for communication between agents and data transfer from the central server as well as backward. At the application level, detailed information on application are created, stored in database and distributed up to the physical level. From the physical level the actual information about the manufacturing and technology line are sent back for analysis and processing.

5 Use Case In order to demonstrate the manipulative capabilities of the bi-manual service robot in industrial customized tasks and to verify the methodology of synchronized control of two robotic arms described in the previous sections, one characteristic example of the assembly task was chosen. For that purpose, a cube measuring 75 × 75 mm was introduced, which consists of two complementary parts – model (shape or mold) and corresponding mask. The parts of the cube have asymmetrical geometry (Fig. 5a) and the gap between the parts is 1 mm in order to avoid jams when joining. The model and mask of the cube are designed as a 3D puzzle that needs to be assembled so that a regular cube is formed when they are assembled (Fig. 5a). In order to accomplish assembling of the parts, robot must grasp both of them by grippers and bring parts accurately relative to each other so that they fold into a correct cube (Fig. 5a). The phases of a bi-manual

Cloud-Enabled Bi-manual Collaborative Robot

247

robot assembling process are shown in Fig. 6. The corresponding joint angles of the left-side and the right-side robot arm are presented in Fig. 5b.

Fig. 5. a) The object subjected to assembling operation – a 3D cube consists of two parts. b) Diagrams of robot joint angles – the left-side and right-side robot arm joint trajectories.

In the simulation example, the care should be taken with the applied mounting force that should be kept at the level of 10 N to prevent the cube of mechanical damaging. The task of parts assembling is performed in several phases as presented in Fig. 6. Initially, robot recognizes the parts of the cube on the table in front (Fig. 6a). At the high level of control robot determine the strategy of grasping parts and do path planning. Grasping of the cube parts are performing by use of the both robot arms and by using visual feedback to bring parts (the model and its’ mask) closer to each other (Fig. 6d). Then, applying a combined position/speed and force/torque control algorithms robot brings the parts into a close contact (Fig. 6e). Finally, it is necessary to accomplish fine mutual displacements (piece of millimeters) of one side in relation to the other in order to fit the wedges with corresponding holes. In this phase, impedance control is applied to the robot on the right-side robot hand and admittance control on the left-side arm. The process of assembling parts model (the red part) and corresponding mask (the blue one) is graphically illustrated in Fig. 6e while the assembled object is illustrated in Fig. 5a. When the tolerances between the parts are very narrow, it takes a lot of manipulative skill to assemble the parts without damage. Then an experience and a manipulative skill are needed, which are reflected in the fine, micro movements of the robot grippers and tactile sensations that serve both to guide parts in this process. At this stage of assembly, the visual feedback has no effect at all on the process as the connecting parts are already in touch. Then, it the robot applies control algorithms that use force and complementary speed feedback to ensure smooth assembling.

248

A. Rodi´c et al.

Fig. 6. Snapshots of dual-arm robot in characteristic positions corresponding to the phases of technology operation observed: a) initial robot position, b) approaching to the parts, c) grasping, d) mutual positioning of the parts, e) assembling, f) object releasing after assembling.

6 Conclusion Enhanced versatility of the collaborative bi-manual robot and extended flexibility of the customized industrial production are achieved through application of: (i) adaptive mechanical structure of robot torso with variable geometry, (ii) application of artificial intelligence algorithms that allow imitation of human manipulative skills. (iii) cloudenabled control architecture of bimanual service robot that uses facilities of the distributed computer-sensor-communication network. One such architecture supports the hierarchically distributed control of the bi-manual service robot presented in this paper developed for industrial manufacturing within the Industry 4.0 initiative. Aknowledgements. Development of the robot prototype is the result of the bilateral research and development project entitled “Development and experimental verification of performance of mobile dual-arm robot for cooperation with human” from the “Science and Technology Development Program - Funding of Joint Research and Development Projects of the R. Serbia and the P.R. China”, 2017–2019.

Cloud-Enabled Bi-manual Collaborative Robot

249

References 1. Rodi´c, A., Stevanovi´c, I., Jovanovi´c, M.: Smart cyber-physical system to enhance flexibility of production and improve collaborative robot capabilities–mechanical design and control concept. In: Aspragathos, N.A., Koustoumpardis, P.N., Moulianitis, V.C. (eds.) Proceedings of the 27th International Conference on Robotics in Alpe-Adria Danube Region (RAAD 2018). Advances in Service and Industrial Robotics, Part of the Mechanisms and Machine Science book series, Mechan. Machine Science, vol. 67, pp. 627–639 (2018) 2. Kashiri, N., et al.: CENTAURO: a hybrid locomotion and high power resilient manipulation platform. IEEE Robot. Autom. Lett. 4(2), 1595–1602 (2019). https://doi.org/10.1109/LRA. 2019.2896758 3. Fuchs, M., et al.: Rollin’ Justin - Design considerations and realization of a mobile platform for a humanoid upper body. In: 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan, pp. 4131–4137 (2009). https://doi.org/10.1109/robot.2009.5152464 4. Srinivasa, S.S., et al.: Herb 2.0: lessons learned from developing a mobile manipulator for the home. Proc. IEEE 100(8), 2410–2428 (2012). https://doi.org/10.1109/JPROC.2012.2200561 5. Rodi´c, A., Miloradovi´c, B., Popi´c, S., Spasojevi´c, S., Karan, B.: Development of modular compliant anthropomorphic robot hand. In: Pisla, D., Bleuler, H., Rodic, A., Vaida, C., Pisla, A. (eds.) New Trends in Medical and Service Robots. Theory and Integrated Applications, Series: Mechanisms and Machine Science, Springer Publishing House, 2014, vol. 16, VIII, 238, p. 167, 30 September 2013. ISBN 978-3-319-01591-0 6. Rodi´c, A., Miloradovi´c, B., Popi´c, S., Urukalo, Ð.: On developing lightweight robot-arm of anthropomorphic characteristics. In: Bleuler, H., Pisla, D., Rodic, A., Bouri, M., Mondada, F. (eds.) New Trends in Medical and Service Robots. Book 3, Series: Mechanisms and Machine Science, Springer Publishing House, vol. 38 (2015), ISBN: 978-3-319-23831-9, Book ID: 332595 _1_En 7. Rodi´c, A., Jovanovi´c, M., Popi´c, S., Mester, G.: Scalable experimental platform for research, development and testing of networked robotic systems in information structured environments. In: Proceedings of the IEEE SSCI2011, Symposium Series on Computional Intelligence, Workshop on Robotic Intelligence in Information Structured Space, Paris, France, pp. 136–143 (2011) 8. Rodi´c, A., Stojkovi´c, I.: Building of open structure wheel-based mobile robotic platform. In: Prof. Dr. Habib, M.K., Prof. Dr. Davim, J.P. (eds.) Interdisciplinary Mechatronics: Engineering Science and Research Development, ISTE-Willey, London, UK, 624 pages, pp. 385–421 (2013). ISBN: 978-18-4821-418-7 9. Rodi´c, A., Jovanovi´c, M., Vujovi´c, M., Urukalo, D.J.: Application-driven cloud-based control of smart multi-robot store scenario. In: Rodi´c, A., Borangiu, Th. (eds.) Advances in Robot Design and Intelligent Control, Proceedings of the 25th International Conference on Robotics in Alpe-Adria Danube Region, Series: Advances in Intelligent Systems and Computing, vol. 540, pp. 347–360 (2016)

Multi-objective Trajectory Tracking Optimization for Robots with Elastic Joints Werner Ainhauser1,2(B) , Johannes Gerstmayr1 , and Andrea Giusti2 1

University of Innsbruck, Technikerstraße 13, 6020 Innsbruck, Austria [email protected], [email protected] 2 Fraunhofer Italia Research, 39100 Bolzano, Italy [email protected]

Abstract. We consider the trajectory tracking optimization for an elastic joint robot with noisy joint-velocity readings and actuation torque limits. We propose an approach targeting the minimization of the trajectory tracking error, of the velocity noise amplification on the control commands, and of the trajectory execution time. We integrate the use of a feedforward/feedback trajectory tracking control scheme with efficient and accurate simulations of the controlled system dynamics. These are used within a genetic algorithm optimization scheme for optimizing the control gains and trajectory execution time. We verify the effectiveness of our proposed approach using simulations. Keywords: Multi-objective control optimization · Elastic joints Parallel robotics · Genetic algorithm · Trajectory tracking

1

·

Introduction

Robots with elastic joints enjoy increasing popularity in recent decades. Flexible structural elements can increase resiliency in case of shocks, provide a safer interaction with humans or the environment, and can store and release potential energy thus enabling highly dynamic movements. In addition to bio-inspired robotic solutions with soft materials, compliance can be introduced through elastic joints [17]. However, flexible joints come along with an increasingly complex control problem since, for each elastic coupling, both motor and link-side dynamics must be considered. For such a control problem, a variety of solutions already exist e.g., [2,6,11]. Previous work that consider the optimization of the tracking control performance with focus on fuzzy logic [12,18], neural networks [1,16] or genetic algorithms [14,15] are presented. With respect to this previous work, we focus on control optimization to increase the trajectory tracking performance of flexible joint robots accounting for joint velocity measurement noise and force/torque limits. We propose a multi-objective trajectory tracking optimization approach c The Author(s), under exclusive license to Springer Nature Switzerland AG 2021  S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 250–258, 2021. https://doi.org/10.1007/978-3-030-75259-0_27

Multi-objective Trajectory Tracking Optimization

251

using genetic algorithms considering three objectives: positional accuracy, trajectory execution time, and sensor noise amplification. We use genetic algorithms combined with realistic multibody simulation powered by EXUDYN [10] to find the Pareto-optimal front, based on which the gains of a feedforward/feedback tracking controller can be selected systematically. We present simulation results of a five-bar planar parallel robot with five revolute joints which verify the effectiveness of our approach. The paper is structured as follows. In Sect. 2 we describe the problem in detail. Our proposed approach is described in Sect. 3, followed by the presentation of simulation results in Sect. 4. Conclusions are drawn in Sect. 5.

2

Problem Statement

We consider serial and parallel robots having N joints, from which M joints are actuated and compliant. Non-redundant actuation is assumed for parallel robots, while full actuation by compliant joints is assumed for serial robots. By considering Spong’s assumptions [20], the dynamic model for control purpose of such systems can be written as: ˙ + De qa − τe = τext , M (q)¨ qa + h(q, q) ¨ ˙ + τe = u, Jm θ + Dm θ˙ + f (θ) Ke (θ − qa ) = τe .

(1) (2) (3)

where q ∈ RN is the link-side joint position vector, qa ∈ RM the link-side joint position vector of actuated joints only, θ ∈ RM the motor-side joint position vector, M (q) ∈ RM ×M the inertia matrix of the rigid links. The vector of elastic torques reads τe ∈ RM which is based on diagonal joint stiffness matrix Ke ∈ RM ×M , and τext ∈ RM is the external applied torque vector mapped onto the actuated joint space. Further, Jm ∈ RM ×M is the motor inertia diagonal matrix, Dm ∈ RM ×M is a diagonal motor damping matrix, u ∈ RM is the vector ˙ ∈ RM represents motor-side friction. The diagonal of motor torques and f (θ) M ×M describe the stiffness and the decoupled damping matrices Ke and De ∈ R of the elastic joints. Finally, h(q, q) ˙ ∈ RM is a generalized term given by h(q, q) ˙ = C(q, q) ˙ q˙a + Dl q˙a + g(q), where C(q, q) ˙ q˙a ∈ RM denotes the vector of Coriolis and centrifugal terms, Dl ∈ M ×M contains viscous link-side damping coefficients and the vector g(q) ∈ RM R considers gravitational terms. On one hand we assume availability of noise-free joint position measurements for both the link and motor side, which is realistic when considering modern high resolution encoders. On the other hand, we consider noisy velocity readings and actuation torque limits. For the considered systems and assumptions, we face the problem of optimizing the tracking control for sufficiently smooth desired trajectories qd ∈ RN while considering the following objectives: i) minimization of the

252

W. Ainhauser et al.

trajectory tracking error Ep , ii) minimization of the time to perform the trajectory Et , and iii) minimization of sensor noise amplification on the control commands En . For Ep we use the integral square error in the discrete time domain:  2 Ep = (qd − q) . (4) t

As a metric of the sensor noise amplification En we consider the 2-norm of the error propagation of Eq. (6). Since we only consider noise at the velocity measurement, En is given by:    ∂u    . En =  (5) ∂ θ˙ 2

3

Proposed Approach

The basic idea of our proposed approach consists in using i) a feedback control law for trajectory tracking of elastic joint robots providing a straightforward velocity noise amplification relation along with smooth trajectory planning (Subsect. 3.1), ii) an accurate yet computationally efficient modeling tool for simulation of the trajectory execution of flexible multibody systems (Subsect. 3.2), within iii) a genetic algorithm for performing the optimization of the controller gains and trajectory execution time (Subsect. 3.3). 3.1

Motion Control and Planning

Many control schemes have been proposed for the tracking control problem of robots with elastic joints as e.g., inverse-dynamics [20], passivity-based [5], or a combination of these also using feedforward terms [6,11]. For reducing velocity noise amplification effects, we favour the use of feedforward terms as in [6] for approximately cancelling couplings of the system dynamics with the following control scheme,  (6) u = ud + Kp eθ + Ki eθ + Kd e˙ θ , where the vector eθ = θd − θ represents the deviation between desired motor position θd ∈ RM and the actual motor position θ ∈ RM , and Kp , Ki and Kd ∈ RM ×M are diagonal matrices containing the proportional, integral or respectively differential controller gains. Further, ud ∈ RM is the feedforward term which can be interpreted as the desired motor torque trajectory, obtained by rearrangement of the dynamic model from Eqs. (1)–(3), ud = Jm θ¨d + Dm θ˙d + f (θ˙d ) + Md q¨d + hd + De q˙a,d ,

(7)

with the desired link-side joint position trajectory qa,d ∈ RM . For simplicity M (qd ) is denoted as Md and h(qd , q˙d ) is denoted as hd with the desired linkside joint position trajectory qd . The desired motor position trajectory θd and

Multi-objective Trajectory Tracking Optimization

253

its derivatives θ˙d ,θ¨d are obtained by transferring the desired trajectory from the link-side joint space to the motor-side joint space – again through rearrangement of the dynamic model from Eqs. (1)–(3) θd = qa,d + Ke−1 (Md q¨a,d + hd + De q˙a,d ),   ... ˙ d + De q¨a,d , ˙ d q¨a,d + h θ˙d = q˙a,d + Ke−1 Md q a,d + M   ¨ d + De ... ˙ d ... ¨ d q¨a,d + h q a,d + M q a,d . θ¨d = q¨a,d + Ke−1 Md q¨ ¨a,d + 2M

(8)

Now, by first inserting the feedforward term from Eq. (7) in the control law (6) for the system in (2), the closed-loop error dynamics can be written as:  ¨θ + (Kd + Dm ) e˙ θ + Kp eθ + Ki eθ = ψ. (9) Jm e The vector ψ is a perturbation term which arises from mismatches between the desired and executed trajectories, model inaccuracies and external disturbance. By applying the Laplace transform to the j-th coordinate of Eq. (9) and rearranging the resulting equation in the Laplace domain leads to the transfer function from the perturbation ψj to the error eθ,j , Gj (s) =

eθ,j s = 3 , 2 ψj s Jm,j + s (kd,j + Dm,j ) + skp,j + ki,j

(10)

which is stable if the denominator of the transfer function is a Hurwitz polynomial. The latter condition is considered during the optimization to quickly discard unacceptable values of the control gains. In order to provide a continuous trajectory of the desired motor torque from Eq. (7), the desired joint space trajectory qd must be at least four times continuously differentiable. A sufficiently smooth trajectory through desired via-points can be planned based on B-spline interpolation by assuming that the first four derivatives – velocity, acceleration, jerk and snap – of qd are zero at the beginning and at the end of the trajectory. This results in eight boundary conditions for the spline interpolation, which requires the desired trajectory to be a ninth-degree polynomial. 3.2

Multibody Dynamics Simulation

The proposed optimization scheme relies on accurate and efficient computation of the closed-loop system dynamics. A conventional constrained (redundant) multibody system dynamics approach [19] is used to solve the dynamics of the parallel robot with compliant joints and control. An open-source code for constrained flexible multibody systems named EXUDYN has been developed by one of the co-authors and is available on GitHub [10]. Due to the redundant formalism, additional sub-components (inertia, actuators, springs-dampers, constraints) or additional physical effects (friction, flexibility of the bodies) can be included in a simpler way allowing to adapt the model to the real system.

254

W. Ainhauser et al.

The model for the closed loop kinematics is based on planar rigid bodies [19], having two translational and one rotational coordinates. Each of the revolute joints is modeled with two algebraic constraints for the relative displacement. The motor inertia is modeled by 1D (rotational) masses, which are attached to the arms via spring-damper elements. The control law is applied using Python user functions, allowing for arbitrary coupling to sensors measuring the arm rotation. In total, the model has 14 displacement/rotation coordinates and ten constraint equations, which are solved by an implicit trapezoidal rule (similar to Newmark method), using a constraint reduction to velocity level. For more details see the description in EXUDYN [10]. 3.3

Control Optimization Using Genetic Algorithms

Based on the parallelism with the theory of evolution, descriptive terms from biology are used in genetic algorithms [13]. In this context, an individual represents a solution of the optimization problem. Many individuals form a population or a generation, which basically represents one iteration of the genetic algorithm. For a given trajectory we compute the Pareto-optimal front using the non-dominated sorting genetic algorithm NSGA-II [8] making use of the open-source Python module DEAP [9] for parallel evolutionary computation. We apply simulated binary crossover [7]. Further, bounded polynomial mutation is used as in the original implementation of NSGA-II. The mutation probability is implemented adaptively as in [3]. Our optimization problem includes four design variables: the control gains kp , ki and kd and the time ttraj , in which the trajectory is performed. Beyond the tracking error objective Ep as in (4), we use the trajectory duration directly as an optimization objective thus Et = ttraj . If the same control gains are applied to all actuators, from (5) and (6), with Kd = kd · I, it follows that the noise amplification metric simplifies to En = kd . We constrain the design space by two conditions. Before simulation, the stability of the control gains is checked by means of Eq. (10). In addition, the simulation is aborted if the planned motor torque trajectory exceeds the maximum available torque.

4

Simulation Results

In our simulation test-bed, we consider a parallel planar robot, composed of 5 revolute joints connected by rigid links as illustrated in Fig. 1. The robot is actuated by two motors at the joints A and E that are connected via flexible couplings. The passive joints B, C, and D are considered to be rigid. The robot is modeled according to Eqs. (1)–(3) with the mechanical parameters listed in Table 2 and simulated using [10]. The noise of the joint velocity estimates is considered based on joint position measurements as in [4]. Additional simulation parameters are listed in Table 1. The trajectory to be optimized is planned based on 9 via points in our simulation. This results in a rectangular-shaped desired trajectory as shown in Fig. 3. For the proposed trajectory a Pareto-optimal front is

Multi-objective Trajectory Tracking Optimization

255

Table 1. Control design variables and simulation parameters.

Fig. 1. Parallel 5R-mechanism

Param. Description

Value

kp

Proportional control gain

[0,200]

ki

Integral control gain

[0, 50]

kd

Differential control gain

[0, 50]

ttraj

Time to perform trajectory [0.5, 2] s

ts

Control sampling time

μ

Mean value of velocity noise 0

1 ms

σ

Stand. dev. of velocity noise 6 · 10−3

m s

Table 2. Mechanical parameters used in the simulation; Matrix I2 represents the identity matrix in R2 . Parameter Description

Value

la

0.31 m

Upper arm length

lf

Forearm length

0.36 m

d

Distance between motor axes

0.21 m

ma

Mass of upper arm link

0.186 kg

mf

Mass of forearm link

0.208 kg

mtc

Tool carrier mass (neglected)

0 kg

Ja

Arm link inertia with respect to the center of mass

5.997 · 10−3 kgm2

Jf

Forearm link inertia with respect to the center of 9.019 · 10−3 kgm2 mass

Jm

Motor inertia matrix

1.464 · 10−2 I2 kgm2

Kt

Stiffness matrix of the elastic coupling

400 Nm/rad

Dt

Damping matrix of the elastic coupling

2 · 10−4 I2

Dm

Damping matrix of the motors

4 · 10−4 I2

τmax

Maximum available motor torque

±25 Nm

computed as described in Subsect. 3.3. In particular, we set a crossover probability of pc = 0.7. We select an initial population size of 100 individuals with a maximum of 250 generations. The design variables kp , ki , kd and ttraj are distributed uniformly at the initial population with the particular ranges listed in Table 1. The results of the optimization are presented in Fig. 2. The two-dimensional representations of the Pareto front illustrate the relationships between the optimization objectives. According to intuition, a longer trajectory duration provides a higher position accuracy. Also, increased noise amplification reduces the position error since more aggressive differential control action result in these cases.

256

W. Ainhauser et al.

Fig. 2. Pareto front of the proposed control optimization. The color bar refers to the position error.

Fig. 3. End effector trajectories for a fast and a slow Pareto-optimal solution (with joint A at the origin [0, 0]).

As shown in Fig. 2, the control gains can now be optimally configured for a particular scenario. As an example, two possible Pareto-optimal end-effector trajectories are shown in Fig. 3. The dotted line represents the solution with the highest positional accuracy in the entire design space. In contrast, the solution with the dashed line was selected under the condition of a maximum trajectory duration of 0.65 s, while the position error is supposed to be minimal. By ignoring noise amplification in the selection of these example trajectories (the dotted solution) an aggressive differential control action results providing higher position accuracy as well as higher noise amplification on the control commands.

5

Conclusion

An approach to deploy elastic joint robots with trajectory tracking performance that can be optimized for given scenarios is presented. We consider not only position accuracy but also trajectory duration and sensor noise amplification. Therefore, we formulate a multi-objective optimization problem and simulate

Multi-objective Trajectory Tracking Optimization

257

the Pareto-optimal front using genetic algorithms. The Pareto front shows possibilities and limits of the design space and can be used as a reference for a systematic selection of control gains. Future work will include the experimental validation of the proposed approach.

References 1. Al-Khayyt, S.Z.S.: Tuning PID controller by neural network for robot manipulator trajectory tracking. Al-Khwarizmi Eng. J. 9(1), 19–28 (2013) 2. Albu-Sch¨ affer, A., Ott, C., Hirzinger, G.: A unified passivity-based control framework for position, torque and impedance control of flexible joint robots. Int. J. Robot. Res. 26(1), 23–39 (2007) 3. B¨ ack, T., Sch¨ utz, M.: Intelligent mutation rate control in canonical genetic algorithms. In: International Symposium on Methodologies for Intelligent Systems, pp. 158–167. Springer (1996) 4. Belanger, P.R.: Estimation of angular velocity and acceleration from shaft encoder measurements. In: Proceedings of ICRA 1992, pp. 585–586. IEEE Computer Society (1992) 5. Brogliato, B., Ortega, R., Lozano, R.: Global tracking controllers for flexible-joint manipulators: a comparative study. Automatica 31(7), 941–956 (1995) 6. De Luca, A.: Feedforward/feedback laws for the control of flexible robots. In: Proceedings of ICRA 2000, vol. 1, pp. 233–240. IEEE (2000) 7. Deb, K., Agrawal, R.B., et al.: Simulated binary crossover for continuous search space. Complex Syst. 9(2), 115–148 (1995) 8. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002) 9. Fortin, F.A., De Rainville, F.M., Gardner, M.A., Parizeau, M., Gagn´e, C.: DEAP: evolutionary algorithms made easy. J. Mach. Learn. Res. 13, 2171–2175 (2012) 10. Gerstmayr, J.: EXUDYN: flexible multibody dynamics simulation with Python and C++. https://github.com/jgerstmayr. Accessed 01 Mar 2021 11. Giusti, A., Malzahn, J., Tsagarakis, N.G., Althoff, M.: On the combined inversedynamics/passivity-based control of elastic-joint robots. IEEE Trans. Robot. 34(6), 1461–1471 (2018) 12. Kazemian, H.B.: The SOF-PID controller for the control of a MIMO robot arm. IEEE Trans. Fuzzy Syst. 10(4), 523–532 (2002) 13. Kumar, M., Husain, M., Upreti, N., Gupta, D.: Genetic algorithm: review and application. Available at SSRN 3529843 (2010) 14. Nguyen, V., Morris, A.S.: Genetic algorithm tuned fuzzy logic controller for a robot arm with two-link flexibility and two-joint elasticity. J. Intell. Robot. Syst. 49(1), 3–18 (2007) 15. Niu, G., Qu, C.: Global asymptotic nonlinear PID control with a new generalized saturation function. IEEE Access 8, 210513–210531 (2020) 16. Perez, J., Perez, J.P., Soto, R., Flores, A., Rodriguez, F., Meza, J.L.: Trajectory tracking error using PID control law for two-link robot manipulator via adaptive neural networks. Procedia Technol. 3, 139–146 (2012) 17. Pratt, G.A., Williamson, M.M.: Series elastic actuators. In: Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots, vol. 1, pp. 399–406. IEEE (1995)

258

W. Ainhauser et al.

18. Salas, F., Llama, M., Santibanez, V.: A stable self-organizing fuzzy PD control for robot manipulators. Int. J. Innov. Comput. Inf. Control 9(5), 2709–2717 (2013) 19. Shabana, A.A.: Dynamics of Multibody Systems. Cambridge Univ. Press, Cambridge (2013) 20. Spong, M.W.: Modeling and control of elastic joint robots. J. Dyn. Syst. Meas. Control 109(4), 310–318 (1987)

Stabilization of Mobile Weapon Aiming Karol Dobrovodský(B) and Pavel Andris Institute of Informatics, Slovak Academy of Sciences, Dúbravská cesta 9, 842 37 Bratislava, Slovakia {utrrdobr,utrrandr}@savba.sk

Abstract. The stabilization of geo coordinates of the mobile weapon aiming is considered. In the horizontal position of the chassis, servo coordinates of the optical axis coincide with its geo coordinates. Nevertheless, in the case of the non-horizontal position of the chassis, servo and geo coordinates split. A digital compass combined with an inclinometer is used for measurement of the initial orientation of the chassis in respect of the North and the horizontal plane. Three optical gyroscopes are used for measurement of the chassis incremental change of orientation. The goal of the proposed stabilization is to maintain geo coordinates of the optical axis generated by the superior fire control system while moving in an uneven terrain. Keywords: Mobile weapon · Digital compass · Inclinometer · 3D gyroscope · Coordinates stabilization

1 Introduction Current fire control systems have the separate azimuth angle stabilization and the separate elevation angle stabilization. Such 2D stabilization helps to maintain the observed object in the field of view of the camera but fails to compensate for the lateral tilt of the vehicle chassis. Modern weapon systems are equipped with a control system, whose task is to support the operator during all combat operations. The visual subsystem of day and night vision detects and recognizes objects that the operator searches for, [1]. The problem with finding objects while driving a vehicle in the terrain is to keep the recognized object in the field of view of the camera. To this end, it is necessary to stabilize the direction of the optical axis with respect to the horizontal plane by compensating for changes in the vehicle chassis orientation. Stabilizing the geo coordinates of the aiming not only facilitates the process of finding a target but is even an essential part of aiming when shooting while driving. While tracking an object, the control system measures its distance, its motion parameters, and predicts the hit point, [2]. It automatically performs ballistic aiming at the predicted hit point before firing, [3]. Stabilization allows the operator to search for targets and automatically aim even when the vehicle is leaning and changing direction to avoid obstacles. The aim of the proposed 3D automatic stabilization is to maintain the generated orientation of the optical axis of the weapon with respect to the horizontal plane even in case of the lateral tilt of the vehicle chassis. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 259–265, 2021. https://doi.org/10.1007/978-3-030-75259-0_28

260

K. Dobrovodský and P. Andris

Two degrees of freedom are sufficient to achieve the desired direction of the optical axis when tracking the target or to achieve the desired ballistic aiming. The desired direction can thus be controlled by two rotational axes of the azimuth and elevation servo systems. The third degree of freedom is the twist around the optical axis. The geo twist angle does not affect the direction of the optical axis, but the value will be calculated for parallax compensation in the superior fire control system. These are the reasons why it is necessary to differentiate between the azimuth and the elevation angle relative to the vehicle chassis platform, and the azimuth and the elevation angle relative to the geographical North and the horizontal plane. The vehicle chassis orientation in 3D space has three degrees of freedom. We will use a digital compass with tilt measurement relative to the horizontal plane to measure the initial orientation of the vehicle chassis. Subsequently, we use a triple of optical gyroscopes, rigidly connected so that their axes form a dextrorotatory vector base, to measure incremental changes in the orientation of the vehicle chassis.

2 Vector Bases The vector base EARTH: e1 , e2 , e3 fixed to the Earth is rotating around e3 . The vector base GROUND: g1 , g2 , g3 fixed to the Earth is rotating around e3 together with the Earth. Mutual orientation (GROUND, EARTH) is constant with e1 = g1 . The vectors g1 , g2 define the horizontal plane at the battlefield rotating with the Earth. The vector base CHASSIS: c1 , c2 , c3 is fixed to the chassis of the vehicle collinearly with the mobile platform. The unit vectors c1 , c2 are transversely and longitudinally collinear with the chassis. The vector base OPTICS: o1 , o2 , o3 at the end of the kinematic chain of the mobile weapon is rotated around c3 (servo azimuth angle) and then around o1 (servo elevation angle). Mutual orientation (OPTICS, CHASSIS) is controlled by two servo systems: azimuth around c3 and elevation around o1 . The unit vector o2 is representing the orientation of the optical axis. The subject of the measurement is a matrix of coordinates of the chassis orientation. Translatory motion does not enter the issue. Precisely defined relations between vector bases are consistently algebraic, not geometric. Vector bases do not have the origin of a coordinate system. This means that a graphical representation of vectors would only be a choice of their location, and the image would complicate the situation more than explain it.

3 Initial Orientation The initial orientation of the vector base CHASSIS: c1, c2, c3 with respect to the vector base GROUND: g1, g2, g3 is measured by the device located in the gyro module connected to the chassis. The device contains a digital compass in combination with an inclinometer. Let us denote N the value read from the compass and X, Y the values read from the inclinometer expressed in angular units. Based on the values N, X, Y we define nonnegative angles f1 , f2 and coordinates (u1 , u2 , 0) of the unit vector u f1 = −N

(1)

Stabilization of Mobile Weapon Aiming

f2 = |X| + |Y| (u1 , u2 , 0) = (Y, −X, 0)/sqrt (X2 + Y2 ).

261

(2)

The angle f1 around g3 denotes the geo azimuth to the North, the angle f2 denotes the amount of tilt relative to the horizontal plane, and the vector u denotes the dextrorotatory tilt axis of the chassis in the horizontal plane. Let’s denote C1 = cos(f1 ), S1 = sin(f1 ), C2 = cos(f2 ), S2 = sin(f2 ), and define the rotation matrices, [4]: ⎡ ⎤ C1 , −S1 , 0 Z(f1 ) = ⎣ S1 , C1 , 0 ⎦ (3) 0, 0, 1 ⎤ ⎡ S2 u2 (1 − C2 )u21 + C2 , (1 − C2 )u1 u2 , (4) R(u, f2 ) = ⎣ (1 − C2 )u1 u2 , (1 − C2 )u22 + C2 , −S2 u1 ⎦ −S2 u2 , S2 u1 , C2 The resulting rotation matrix of the vector base CHASSIS relative to the vector base GROUND is the product of the rotation matrix Z(f1 ) and the rotation matrix R(u, f2 ). Let’s denote the resulting orientation symbolically as (CHASSIS, GROUND) = Z(f1 ) R(u, f2 )

(5)

The initial orientation (CHASSIS, GROUND) is measured when the chassis has been stationary for at least some time so that the tilt measurement relative to the horizontal plane is not disturbed by acceleration.

4 Incremental Change of Orientation The incremental change in chassis orientation is measured by the three gyroscopes. The gyroscope module contains three optical gyroscopes firmly connected to form a dextrorotatory vector base. In an incremental measurement mode, the registers of these gyroscopes provide three rotations about the respective axes over a period of time since the last measurement. Let us denote X, Y, Z values read from registers of individual gyroscopes. We define the non-negative angle f and the coordinates of the unit vector v:    f = sqrt X2 + Y2 + Z2 (6) (v1 , v2 , v3 ) = (X, Y, Z) f. The non-negative angle f is the magnitude and the vector v is the axis of the dextrorotatory rotation of the vector base CHASSIS: c1 , c2 , c3 with respect to its previous orientation. Let’s denote C = cos(f), S = sin(f) and define the rotation matrix, [4]: ⎡ ⎤ (1 − C)v21 + C, (1 − C)v1 v2 − Sv3 , (1 − C)v1 v3 + Sv2 R(v, f) = ⎣ (1 − C)v1 v2 + Sv3 , (1 − C)v22 + C, (7) (1 − C)v2 v3 − Sv1 ⎦ 2 (1 − C)v1 v3 − Sv2 , (1 − C)v2 v3 + Sv1 , (1 − C)v3 + C The rotation matrix R(v, f) defines the change of orientation since the last measurement in the incremental mode.

262

K. Dobrovodský and P. Andris

5 Discrete Orientation Changes For discrete orientations, we can symbolically write: (CHASSISi+1 , GROUNDi+1 ) = (CHASSISi+1 , CHASSISi )(CHASSISi , GROUNDi )(GROUNDi , GROUNDi+1 ), (8) where i = 0, 1, …, n − 1. The rotation matrix of the initial orientation of the vehicle chassis with respect to the North and in respect of the horizontal plane is: [CHASSIS0 , GROUND0 ] = Z(f1 ) R(u, f2 )

(9)

The vehicle chassis orientation is subject to changes due to off-road movement and Earth rotation. Orientation changes are measured discreetly in steps of 40 ms by the three mutually fixed gyroscopes. The respective rotation matrices measured in incremental measurement mode represent the change of orientation since the last measurement:

(10) CHASSISi+1 , CHASSISi = R(v, f) Orientation changes of the vector base GROUND will not be measured but calculated as a result of the Earth’s rotation. The Earth’s angular velocity vector is virtually constant and its orientation at the battlefield is known. We can write for the vector w of the Earth’s angular rate of rotation w = |w|e3 ,

(11)

where e3 is a unit vector in the direction of the Earth’s axis of rotation and the magnitude |w| = 15 [deg/hod]. Considering the relatively small value, let’s point out that the corresponding value of the magnitude of the angle of rotation in 40 ms is 0.00016667°. The vector base GROUND: g1 , g2 , g3 on the battlefield with north latitude 48° is rotating together with the Earth about the axis e3 while e1 = g1 . The constant orientation of GROUND in respect of EARTH is given by an angle of magnitude f0 around e1 = g1 . We can write for coordinates wg and magnitude f0 wg = we X(f0 ) = (0, S, C)|w|,

f0 = |90−latitude|

(12)

where S = sin(f0 ), C = cos(f0 ), and X(f0 ) is the constant rotation matrix [GROUND, EARTH]. The incremental change of the vector base GROUND: g1 , g2 , g3 with respect to its previous orientation is

 GROUNDi+1 , GROUNDi = R wg /|w|, 0.00016667 . (13) Finally, the discrete change of the chassis orientation with respect to the North and the horizontal plane in steps of 40 ms is

CHASSIS = i+1 , GROUNDi+1   (14) R−1 wg |w|, 0.00016667 [CHASSISi , GROUNDi ] R(v, f).

Stabilization of Mobile Weapon Aiming

263

6 Geo Coordinates Stabilization The established vector bases GROUND : g1 , g2 , g3 ,

CHASSIS : c1 , c2 , c3 ,

OPTICS : o1 , o2 , o3 , (15)

form a kinematic chain of rotation matrices of the mobile weapon: [OPTICS, GROUND] = [CHASSIS, GROUND] [OPTICS, CHASSIS]

(16)

Let’s define the geo coordinates as three angles: pan, tilt, twist, by the equation [OPTICS, GROUND] = Z(pan) X(tilt) Y(twist)

(17)

In general, pan-tilt-twist angles can be extracted from any rotation matrix, regardless of how many rotations were used to generate it, [5]. Pan is geo azimuth, tilt is geo elevation, and the twist is geo twist of the optical axis o2 . The rotation matrix [OPTICS, CHASSIS] is given by angles of the two servo systems: azimuth around c3 and elevation around o1 . Let’s denote f3 the azimuth angle and f4 the elevation angle. The corresponding rotation matrix is [OPTICS, CHASSIS] = Z(f3 ) X(f4 )

(18)

Discrete measurement of the [CHASSISi , GROUNDi ] rotation matrices in each 40 ms step with respect to the North and the horizontal plane allows us to join (16), (17), (18) into the equation Z(pan) X(tilt) Y(twist) = [CHASSISi , GROUNDi ] Z(f3 ) X(f4 )

(19)

For i = 0, [CHASSISi , GROUNDi ] in (19) is from (9). For all i > 0, we have [CHASSISi , GROUNDi ] in (19) from (14). The Eq. (19) will be solved either with respect to unknown pan, tilt, and twist angles or with respect to unknown f3 , f4 , and twist angles. Stabilization of geo coordinates of the fire control system is a subsystem that can be turned on or off. When the operator activates the stabilization, the superior steering system begins to wait for the conditions for measuring the initial inclination of the chassis relative to the horizontal plane to be met. The initial inclination of the chassis must be constant for at least half a minute. Subsequently, the initial values of the geo coordinates are calculated from the Eq. (20). Z(pan) X(tilt) Y(twist) = R, R = [CHASSIS0 , GROUND0 ] Z(f3 ) X(f4 )

(20)

The solution of goniometric Eq. (20) with rotation matrix R = [rij ] is pan = atan2(r22 ; −r12 ) tilt = atan2(sqrt(sumsq(r12 ; r22 )); r32 ) twist = atan2(r33 ; −r31 )

(21)

264

K. Dobrovodský and P. Andris

For all i > 0, during the stabilization of geo coordinates pan, tilt, the servo coordinates f3 , f4 are calculated from the Eq. (22). Z(f3 ) X(f4 ) Y(−twist) = S, S = [CHASSISi , GROUNDi ]−1 Z(pan) X(tilt)

(22)

Analogously, the solution of goniometric Eq. (22) with rotation matrix S = [sij ] is f3 = atan2(s22 ; −s12 ) f4 = atan2(sqrt(sumsq(s12 ; s22 )); s32 ) twist = −atan2(s33 ; −s31 )

(23)

Geo coordinate twist is not stabilized, but passively calculated. Remember that twisting of the optical axis o2 doesn’t change the direction of o2 . The value of twist angle is calculated for the parallax compensation in the superior fire control system.

Fig. 1. Operator stand of the superior fire control system in the laboratory.

When the operator deactivates the stabilization, the superior steering system begins to ignore data from the inclinometer, the digital compass, and the 3D gyroscope. Subsequently, the new values of the geo coordinates are calculated from the Eq. (20) with [CHASSIS0 , GROUND0 ] = I (identity matrix). Thus geo coordinates pan, tilt become to be equal to servo coordinates f3 , f4 . The twist geo coordinate becomes to be zero. The same remains to be valid until the stabilization remains deactivated. At this point, it should be emphasized that 3D stabilization is a new subsystem of the superior fire control system. 3D stabilization of the defined geo coordinates pan, tilt

Stabilization of Mobile Weapon Aiming

265

does not mean keeping them at constant values, but at values generated by the superior fire control system, regardless of 3D changes in the orientation of the chassis during driving in uneven terrain (Fig. 1).

7 Conclusion The fire control system [3] with 2D stabilization has been verified at the shooting range. Nevertheless, an advanced original method of 3D stabilization has been implemented and algorithmically verified in the laboratory. The digital compass in combination with an inclinometer is used to measure the initial orientation of the vehicle chassis with respect to the geographical North and the horizontal plane. Three optical gyroscopes are used to measure the development of the initial chassis orientation. Moreover, an original Earth rotation compensation method has been introduced in order not to measure the initial orientation too frequently. Acknowledgment. We thank the support of the Scientific Grant Agency of the Ministry of Education of the Slovak Republic and the Slovak Academy of Sciences, project number 2/0155/19.

References 1. Dobrovodsky, K., Andris, P.: Real time sub image localization for tracking. In: Proceedings of the 27th Conference on Robotics in Alpe-Adria-Danube Region Mechanisms and Machine Science, RAAD 2018, , vol. 67, pp. 588–596. Springer Switzerland (2019) 2. Dobrovodsky, K., Andris, P.: Real time recognition and tracking of moving objects. Comput. Inform. 33(6), 1213–1236 (2014) 3. Dobrovodsky, K., Andris, P.: Aiming procedure for the tracking system. In: Advances in Robot Design and Intelligent Control: Proceedings of the 24th International Conference on Robotics in Alpe-Adria-Danube Region (RAAD) Advances in Intelligent Systems and Computing, vol. 371, pp. 369–378. Springer Switzerland (2016) 4. Dai, J.S.: Euler-Rodrigues formula variations, quaternion conjugation and intrinsic connections. Mech. Mach. Theory 92, 144–152 (2015) 5. Wohlhart, K.: Decomposition of a finite rotation into three consecutive rotations about given axes. In: Proceedings of the 6th International Conference on Theory of Machines and Mechanisms, IFToMM, Liberec (1992)

Using the Digital Twin Concept to Design and Validate a Robotized Bin Picking Process Silviu R˘aileanu(B) , Florin Anton, Theodor Borangiu, and Silvia Anton Department of Automation and Applied Informatics, University Politehnica of Bucharest, Bucharest, Romania {silviu.raileanu,florin.anton,theodor.borangiu, silvia.anton}@cimr.pub.ro

Abstract. The paper describes the design and implementing solution for the digital twin of a robotized 3D vision-based bin picking process. The authors describe the design of the gripper and the robot-vision control for an efficient and safe bin picking process, which is validated by simulation. The shape, dimension, and orientation of the end effector are computed to allow the robot to handle parts safely, avoiding collisions with both the bin walls and neighbouring parts. A strategy is also proposed to handle parts in layered mode. Keywords: Bin picking · Digital Twin · Industrial robot · Collision avoidance

1 Introduction Process automation is gaining more terrain in industrial applications but is limited by certain technological aspects which makes automation harder. Some of these aspects are: i) handling delicate objects (e.g.: vegetables in the food supply chain) and ii) dealing with unstructured environments, e.g., extracting known parts randomly located in a bin, an operation usually performed at the beginning of a part processing sequence. Bins storing bulk parts are frequently encountered in value chains both in logistics and manufacturing. It is usually the human operator who performs bin picking, due to the complexity of the part detection problem, grasping technique, and dexterity needed to extract parts without collisions that might affect the structure of the stacked parts. Over the past decade there has been done a lot in computer vision to enable robots to safely handle parts randomly located in a 3D environment, resulting thus in a wide variety of solutions like: SICK [1], Cognex [2], PickIT [3], a.o. This process is found in literature under the keyword bin picking [4] and is operated by a system composed of: i) a 3D vision solution able to scan the environment (stereo vision or laser scanner), locate parts and generate robot data for safe handling operations using information about the end effector geometry and operation (e.g., impactive or astrictive), and ii) a vertical articulated robot equipped with a robust and dexterous end effector that can approach a recognized part under any inclination, as computed by the vision solution (Fig. 1). From a theoretical point of view this process is solved (there are standardized © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 266–274, 2021. https://doi.org/10.1007/978-3-030-75259-0_29

Using the Digital Twin Concept to Design

267

vision applications as well as different types of robot arms able to execute the processes described above), but when it comes to implementation some choices must be made especially for the end effector design and establishing the robot motion strategy. These choices are influenced by customer constraints such as: possible bin placements in the work cell and distances to the robot and processing machine imposed by safety rules, access of the AGVs transporting bins to the workstation, emptying the entire bin, short cycle times for bin picking + machine tending.

Fig. 1. Robotized bin picking system using 3D vision

Since this problem has many variables it is best to use a simulation to model, validate and compute the operation cycle time for the set of robot motions. The evaluation of the cycle time is important to see if the investment in the automation solution is less than the cost of employing human personnel doing the same job. In this context we propose to use a model-driven Digital Twin (DT) [5] to model the solution and validate it through offline programming. Thus, the article is structured as follows: Sect. 2 presents the physical process to be automated and its digital model; Sect. 3 describes the robot and vision strategies used to pick parts while avoiding collisions with the bin walls and with other parts; Sect. 4 presents the realization of the model-driven DT based on findings derived from the validation of the proposed structure; Sect. 5 presents the physical implementation of the bin picking process as designed in Sects. 3 and 4. The article ends with a set of conclusions and future work.

2 Application Description The process to be automated is a part feeding system for a CNC machining centre operating at high productivity. The cycle time is 80 s, and two parts are being processed simultaneously. Because both the bin picking and the load/unload operations are very complex it was decided to split the process in two robotized subprocesses inter-connected by a buffer. Thus the desired cycle time of the bin picking process will be 40 s for a single part. In this time interval the following sequential actions must be performed: 1) bin scanning through laser swipe; 2) part identification and localization along with the computation of the grasping transformation; 3) data transfer and part handling using the robot. The equipment used to perform this task consists 6-d.o.f. industrial robot ABB IRB 6620 [6] chosen for its large working envelope within which the bottom level of the bin can be reached, and a Scanning Ruler from SICK using the PLB5.7 software for

268

S. R˘aileanu et al.

image processing which was chosen by the customer. By analysing the vision solution, the following operation times have been recorded: 10 s for the laser swipe and 3 s for the computation of a grasping transformation. The shape and the dimensions of the generic metallic part being manipulated are given in Fig. 2.

Fig. 2. The shape and dimensions of the handled part for the given the robot flange

The main issue here is that because the robot flange is very large compared with the handled part, the parts located near the bin walls cannot be picked with a symmetric gripper. Also, because the robot must descend into the bin, a rotational grasping method should be used to always orient the flange towards the bin’s centre to avoid the collision with the walls. Therefore an asymmetric end effector with a thin magnet at its end tip was proposed as shown in Fig. 3. The maximum inclination of parts in the bin was taken into account for the gripper’s design to allow handling parts rotated over X and/or Y with a maximum angle of 30° (this is the maximum inclination accepted by the vision application to recognize the parts).

Fig. 3. Gripper design data (left), maximum inclination (right) and rotational grasping (bottom)

Using the Digital Twin Concept to Design

269

The parts are disposed in the bin on 10 layers of 63 pieces, in total about 630 parts (variations may occur due to supplier variability). All these parts must be manipulated. Using the available data concerning the possible placement and orientation of parts in the bin we concluded that it is possible to safely handle all parts with an asymmetric gripper (Fig. 3, left) with an offset of 400 mm on X and 255 mm on Z. In the following sections we will demonstrate how we have validated this proposal.

3 Collision Free Handling Strategy To extract parts from the bin, a strategy for collision avoidance must be developed. The SICK solution is used to recognize and localize the objects inside the bin, but it can also recognize the bin limits if the bin is visible. In the case presented, the parts were placed in the bin and isolated from the environment with a plastic foil covering completely the bin’s interior. To allow the robot picking the parts, an operator places the bin always in the same fixed position and then either cuts the foil around the edges of the bin or folds it downward over the bin’s walls. In the first case the bin edges can be easily recognized, but during parts handling, when the objects’ z level decreases, the foil tends bending towards the interior of the bin and covering again the objects, blocking in this way their recognition. In the second case the operator fixes the foil on the top edges of the bin, so the foil will not cover anymore the parts but will cover the bin’s edges preventing the SICK vision software to recognize the bin’s limits which makes impossible detecting and preventing the collisions between the robot end effector and the bin’s walls. The second bin preparation procedure was accepted even if the bin is not recognized, because in this case at least the objects can be recognized and handled without the need to reposition the foil during the bin unloading process. The SICK artificial vision solution should not only detect, recognize, and locate parts, but also should be able to detect collisions, unfortunately this feature is less well working when the edges of the bin are not permanently visible to be detected. Regarding the risk of collisions two situations can be distinguished: the first situation is when the robot end effector can collide with the objects inside the bin; the second one is when the robot can collide with the edges of the bin. The first situation can occur since the SICK vision software is detecting a set of objects and the best recognized objects are the ones laying in the central area of the bin, so the artificial vision system is sending to the robot the coordinates of such best recognized objects. In this way the robot will “dig a hole” inside the bin and finally will collide with the objects on the edges of the hole. This problem was solved by specifying an adjustable bottom of the bin, so the system is recognizing only the objects which are higher than the virtual bottom of the bin; when there are no more objects recognized at the “current bottom” level, the bin’s virtual bottom is decreased, and the process resumes until the virtual bottom of the bin overlaps with the bin’s real bottom. The second situation is when the robot can collide with the edges of the bin. In this case two situations were identified: 1) the robot arm will collide with the bin or 2) the end effector will collide with the bin. The first situation can be solved by defining a disk as a part of the end effector (Fig. 3 bottom), placed at the contact point of the end effector and the robot flange mounting point. Thus, the end effector is covering enough

270

S. R˘aileanu et al.

space to avoid collisions between the robot arm and the bin if the edges of the bin are not covered and the vision system will detect collisions. The last type of collisions between the end effector and the bin was solved by using a function which computes an alternate position for object grasping which is based on the grasping point suggested by the vision system but with different orientation in order to reposition the robot in such way that the flange will be positioned always close to the centre of the bin (this solves also the first case if the edges of the bin are covered). The function relies on the previously trained positions of the bin’s corners (C1 to C4) and the target point TP (the point which is used to access the part, see Fig. 4).

C1

Robot end effector C4 Center

C2 C3

Fig. 4. Orientation of the end effector

Each point C1 to C4 is trained using the same distance on the Z axis of the robot base frame, the points being learned in the counter clockwise order. The centre point of the surface is defined as: CPx = (C1x + C2x + C3x + C4x )/4   CPy = C1y + C2y + C3y + C4y /4 The other components of the CP are the same as for C1 to C4. The flange centre point (on the grasping position) is defined as: FP = RelTool(TP, toolx , 0, −toolz ) where RelTool is a transform relative to the tool which makes three translations from TP on X,Y and Z. The value for Y is 0 because the tool centre point is only translated on X and Z. The alternative grasping point is defined by: TPf = RelTool(TP, 0, 0, 0, Rz ) where Rz is a rotation on the Z axis and has the following form: Rz = 180 − θf       θf = arctan FPy − TPy , (FPx − TPx ) + arctan CPy − TPy , (CPx − TPx ) θf being the angle (FP, TP, CP). In Fig. 5 we can see a test where the first picture presents the point suggested by the vision system and the second picture presents the final computed grasping point.

Using the Digital Twin Concept to Design

271

4 RobotStudio Implementation Traditionally a digital twin consists of “objects in the real world dynamically connected to their respective real-world twin, sometimes in real-time, to collect and organize data from the corresponding real-world objects” [7]. From the point of view of modelling reality there are also two types of DT: i) DT with a physical counterpart or data-driven DT and ii) DT without a physical counterpart or model-driven DT which is a digital simulation used for modelling, design (analysis and forecasting). In the second case there is no IoT and no setup. Its usage case comprises different processes and views outcomes prior to developing infrastructure. Commonly used for PLM, the term ‘product’ was extended to the terms ‘resource’ and ‘system’ used in manufacturing processes, and there exist now aggregated DT for Smart Manufacturing [8].

Fig. 5. Testing the grasping

To validate the proposed relative locations of the robot, bin and 3D scanner, the gripper form and dimensions, the part handling strategy, and to compute the average part handling cycle time a model-driven digital twin had been developed. This DT was implemented with the RobotStudio default application used to program ABB robots [9]. This application has a feature enabling offline programming using a virtual controller. The generated trajectories can be visualized in a 3D graphical interface which is also able to detect collisions and simulate gravitational laws (e.g.: parts that fall according to gravity and physically interact). This feature is labelled by ABB Digital Twin and is mainly used to develop model-driven DTs, but there are functionalities which can link digital objects to the physical world. In our case we developed the standard model for a bin picking application and configured it with the parameters computed in the previous sections (the position of the bin relative to the robot, gripper dimension and form, and trajectory), see Fig. 6. The simulation is done in two successive steps: i) first the generation of random positioned and oriented parts in the bin and ii) then the handling of the previous parts using an active component to locate the fallen parts. The random generation and positioning of parts is done using a specific feature of RobotStudio, namely dynamic behaviour of components which enables the simulation to control the motion of the parts according to gravity. Thus, 630 parts are generated in a matrix above the bin and when simulation starts, they fall into the bin according to gravity. This part is very CPU- intensive and

272

S. R˘aileanu et al.

after its completion, in order not to consume time again the state is saved and from this point the second simulation (bin picking) starts. This bin picking simulation is done with collision detection enabled between the following two sets: robot & end effector (Set1) and bin & random generated parts (Set2). Upon completion no collision was detected, and the average cycle time was approximatively 13 s (Fig. 6). This cycle time comprises only the movement of the robot. For a real application, this time will increase with the end-effector operation time, in the case of a magnet approximatively 1 s for one grip/release operation. The connection between the Rapid code (ABB programming language which is used to generate the trajectories) and the digital twin is realized through a set of Active Components [9] as depicted in Fig. 7, simulating thus the operation of a real vision system. The robot requests a localisation of a part (starting from the top) through a digital signal (req_scan) which executes the Closest Object block. The output of this block is a reference to an object whose position is computed using a Position sensor and is afterwards forwarded to the trajectory code using a Rapid Variable block which is executed with a different signal (req_set). The completion of the simulated vision flow sequence is signalled using a digital input (scan_complete).

Fig. 6. Robot trajectory and cycle time computed based on Digital Twin approach

Fig. 7. Flow of active components used to simulate 3D vision operation

Using the Digital Twin Concept to Design

273

5 Conclusions and Future Work The paper presented the methodology used for the design and validation of an end effector used for bin picking, based on the digital twin concept. As a partial conclusion for the digital simulation, the designed end effector (Fig. 9) was validated and the cycle time was estimated at around 20 s. The cycle time (Fig. 8) is composed of the time spent on trajectory (~13 s), the time actuating the end effector (~2 s) and approximately 5 s laser operation time (which cannot be done completely in parallel with robot movement). Concerning future research, we aim at improving the cycle time by handling multiple parts using a single scan.

Fig. 8. Composition of the workstation cycle composed of robot movement and laser scanning

Fig. 9. Test realization of the end effector (left) and final implementation (right)

References 1. SICK, PLB 5.7 Robot Guidance Systems, Operating instructions. https://www.sick.com/ag/ en/system-solutions/robot-guidance-systems/plb/c/g259663. Accessed Jan 2020 2. Cognex, 3D-A5000 Series Area Scan 3D Camera. https://www.cognex.com/products/machinevision/3d-machine-vision-systems/3d-a5000-series-area-scan, Accessed in January 2021 3. Pickit. https://www.pickit3d.com/. Accessed Jan 2021 4. Spenrath, F., Pott, A.: Gripping point determination for bin picking using heuristic search. Procedia CIRP 62, 606–611 (2017). https://doi.org/10.1016/j.procir.2016.06.015

274

S. R˘aileanu et al.

5. R˘aileanu, S., Borangiu, T., Iv˘anescu, N., Morariu, O., Anton, F.: Integrating the Digital Twin of a Shop Floor Conveyor in the Manufacturing Control System, Service Oriented Holonic and Multi-agent Manufacturing Systems for Industry of the Future, Springer book series Studies in Computational Intelligence, vol. 853, Ch. 10, pp. 134-145 (2019) 6. ABB IRB 6620. https://new.abb.com/products/robotics/industrial-robots/irb-6620. Accessed Jan 2021 7. Gonzalez C., Robotics Blog: Digital Twins for Robot Installations, 27 October 2020. https:// www.asme.org/topics-resources/content/robotics-blog-digital-twins-for-robot-installations 8. Qia, Q., Taoa, F., Zhaob, D.: Digital Twin service towards smart manufacturing. Procedia CIRP 72, 237–242 (2018). https://doi.org/10.1016/j.procir.2018.03.103 9. ABB, Operating manual, RobotStudio 2019.5, Document ID: 3HAC032104-001, Revision: AB. Accessed Jan 2021

Preliminary Study of Self-adaptive Constrained Output Iterative Learning Control for Robotic Manipulators Kaloyan Yovchev(B) and Lyubomira Miteva Faculty of Mathematics and Informatics, Sofia University, Sofia, Bulgaria [email protected], [email protected]

Abstract. The Iterative Learning Control (ILC) can be applied within the control of industrial robots with uncertainties in their mathematical model for accomplishing precise trajectory tracking. The method calculates the tracking error and corrects the output control signals in accordance with a predefined learning operator. This research will focus on the influence of the ILC learning operator over the performance of the ILC. The proposed in this preliminary study algorithm for Self-adaptive Constrained Output ILC (SCOILC) will try to analyze how the learning operator can be optimized for the given task and afterwards to use this analysis in order to enhance the ILC performance when the task is slightly changed. The algorithm extends the convergent Constrained Output ILC (COILC) method. SCOILC adds an initial step of regression analysis of the executions of the COILC for selected trajectory with different learning operators. This allows the ILC procedure to self-adapt to the specific type of trajectories needed for execution of a specific domain of tasks. The conducted experiments led to a conclusion that this initial adaptation process reduces the overall convergence time of the ILC when it is applied to a modification of the trajectory which was used during the adaptation step. Keywords: Regression analysis · Iterative learning control · Robotic manipulators · Computer simulation

1 Introduction The Iterative Learning Control (ILC) is a nondual type of adaptive control suitable for systems that work in repetitive manner. The ILC method was introduced by Uchiyama in 1978 [1] and later developed by Arimoto [2, 3]. The industrial robotic arm manipulators are systems that are required to perform repetitively a given task and with high tracking precision. The following publication will focus only on the industrial type of robotic arm manipulators with an open kinematic chain. The ILC can be applied for accomplishing precise trajectory tracking to such robots for which the parameters of their mathematical model cannot be fully estimated. The ILC method will start with an initial guess for the needed output control signals which are computed on the basis of the estimated mathematical model parameters. Afterwards, the ILC will need a number of iterations. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 275–283, 2021. https://doi.org/10.1007/978-3-030-75259-0_30

276

K. Yovchev and L. Miteva

After each iteration the method calculates the tracking error and corrects the output control signals in accordance with a predefined learning operator. The iterative process will be stopped when a predefined desired accuracy threshold is achieved. This results in a natural iterative self-learning process. In order for the ILC to be applied to a physical robot it must be generating a convergent process. The convergence of the ILC method nowadays is well studied in the literature. The convergence for nonlinear systems (such as the robotic manipulators) was proven by Heizinger et al. in 1989 [4]. The mechanical characteristics of the robotic manipulators and their hard constrained state space required an additional research of the reported by Longman, R.W. and Huang, Y. problem with growth of transient error within the first few iterations [5]. This problem interferes with the direct application of the ILC to robotic manipulators with constrained state space. Straightforward and computationally effective solution to the transient growth problem is the Bounder-Error Algorithm (BEA), proposed by Delchev in 2013 [6]. This algorithm is proven to generate a convergent ILC process for nonlinear systems. BEA introduces a special stop condition which enforces limits over the maximum possible tracking error (the deviation from the desired trajectory) of any ILC iteration. This effectively eliminates the growth of the transient error. Later, the stop condition was additionally researched and altered so that it is applicable to state space constrained robotic manipulators but with a reduction of their workspace [7]. Finally, the described in [8] Constrained Output ILC (COILC) method allowed the ILC to be used in the control of robotic manipulators without enforcing any additional limits over their workspace. Nowadays, the ILC researchers are trying to improve the performance of the ILC methods by using various machine learning techniques. The relationship between ILC and machine learning techniques, such as reinforcement learning is investigated in [9, 10]. The research [11] proposes an adaptive ILC scheme where the adaptation is supervised by reinforcement learning in order to achieve smooth and safe handling of fragile objects. Other researchers incorporate neural networks techniques [12, 13] or fuzzy neural network techniques [14, 15] within the ILC control in order to reduce the uncertainty of the model used for the design of the controller. This research will focus on the influence of the ILC learning operator over the performance of the ILC. In the typical use case, the industrial robotic manipulators which are welding, or painting are executing trajectory movements. High precision of those movements is required. However, even if the robots will be welding or painting different details it can be assumed that those details have similarities. For example, car doors (or car hoods, etc.) can be painted or welded with almost similar trajectory movements. The movements will differ at their end positions due to the different sizes of the details. If the COILC is directly applied, it will have to learn the control signals for every single detail. The proposed in this preliminary study self-adaptive COILC will try to analyze how the learning operator can be optimized for the given task and afterward use this analysis in order to enhance the ILC performance when the task is slightly changed. In other words, if the task is painting car doors the ILC will adapt its learning operator so that it reduces the needed time for convergence when learning the trajectory for painting a specific car door. Afterwards the adapted learning operator will be used when the robot has to learn to paint another door.

Preliminary Study of Self-adaptive Constrained Output

277

This paper is organized as follows. Section 2 formulates the problem of applying ILC to execution of similar trajectory movements. Section 3 gives an analysis of the performance of the ILC with different learning operators and state space constraints and proposes an algorithm for self-adaptation of the ILC to the given set of tasks. Section 4 validates the proposed algorithm through a computer simulation.

2 Problem Formulation For this research we will consider the SEIKO Instruments SCARA-type robot TT-3000 with the kinematics of this robot as shown in Fig. 1. The joint lengths are l1 = 25 cm and l2 = 22 cm. This robot has well-known dynamical equations of motions and parameter estimations as described in research [16]. Those will be used for developing a computer simulation similar to one used and described in [17] where the focus was on the motion tracking in state space coordinates. This paper will focus on trajectory tracking of specific trajectories planned in workspace coordinates. The use of a planar SCARA-type robot will allow us easier representation of the results.

Fig. 1. Kinematics of SEIKO instruments TT-3000 robot.

The standard ILC control scheme is shown in Fig. 2, where P represents the robot arm; L is the learning controller; M is the memory of the control system; l = 1, . . . , N is the current iteration number and N is the total number of iterations; ul is the feedforward term of the control law for iteration l; ql is the actual output trajectory; and qd is the desired output trajectory. The offline computed feed-forward term ul+1 is supposed to decrease the tracking error of the robot’s motion on the next iteration. The learning update law of the standard ILC scheme can be synthetized as follows of the considered ILC scheme requires the specification of feed-forward and feedback controllers, respectively. The feed-forward controller is based on the update control law that improves the feed-forward control term:    (1) ul+1 = ul + L ql q¨ d − q¨ l   where L ql (t) , l = 0, 1, . . . , N is a learning operator; u0 (t) ≡ 0 is the initial feedforward control input; t ∈ [0, T ] denotes time, where [0, T ] is the robot tracking time interval. ˆ The robustness and convergence of the ILC update law (1) if L(q) ≡ A(q) is proven ˆ in [18], where A is the corresponding estimate of the inertia matrix A. The estimated

278

K. Yovchev and L. Miteva

Fig. 2. Iterative learning control scheme.

inertia matrix takes into account how the characteristics of each joint affect the robot ˆ of A will improve the convergence rate. The robot used for motion. Better estimation A this research have the following joint constraints Qimin ≤ qi ≤ Qimax , where 1 ≤ i, j ≤ 2 as described in Table 1. Table 1. TT-3000 hard constraints for the computer simulation. Joint constraint [rad] Q1min

Q1max Q2min

−2.50 2.50

Q2max

−1.57 1.57

The control of the selected TT-3000 robot will have to minimize the tracking error of any attainable desired trajectory qd by using the described in [8] COILC procedure: a. Set the initial iteration number l = 0 and begin the iterative procedure. b. Starting from the initial position ql (0) the system is tracking the desired trajectory under the control ul (t) until the first moment Sl for which there exists j : 1 ≤ j ≤ 2 and either qjl (Sl ) = Qjmin or qjl (Sl ) = Qjmax or the end position ql (T ) is reached. When t = Sl , Sl ∈ (0, T ], the tracking process stops. c. After the current tracking performance has finished, the learning controller updates the feed-forward control term according to the following learning update law: ul+1 (t, Sl ) = ul (t, Sl−1 ) + u∗l (t, Sl ), u0 (t, S−1 ) ≡ u0(t),    L ql (t), t q¨ d (t) − q¨ l (t) , t ∈ [0, Sl ], Sl ∈ (0, T ]; u∗l (t, Sl ) = 0, ∀t ∈ (Sl , T ]

(2)

d. If the current maximum trajectory tracking error δql (t)∞ is less than or equal to an acceptable tracking accuracy, then exit from the learning procedure, else set l = l +1 and go to step (b).

Preliminary Study of Self-adaptive Constrained Output

279

Two similar to each other trajectories (trajectory A and B) are shown in Fig. 3 with red and black line. The blue dots in Fig. 3 represent the workspace of the robot. When this algorithm is applied to the trajectories shown in Fig. 3 the robot needs to execute 10 iterations with a total time of 92.7 s for the trajectory A and 11 iterations with a total time of 102.6 s for trajectory B in order to converge with a maximum trajectory tracking error δql (t)∞ < 0.01. The next section analyses how the selection of the learning operator affects the convergence time of trajectory A and proposes an algorithm for self-adaptation of the learning operator.

Fig. 3. The blue dots represent the workspace of the robot. The red and the black lines represent two similar desired trajectories A and B planned in the workspace.

3 Self-adaptive Constrained Output ILC As already stated, the estimated inertia matrix takes into account how the characteristics of each joint affect the robot motion. This leads to a conclusion that the learning operator cannot be randomly chosen if we want a fast convergence rate. Furthermore, randomly chosen learning operators can result in a non-convergent ILC process. For this study we will assume that the inertia matrix Aˆ is estimated correctly. We will introduce an additional parameter of the COILC - the scalar learning gain s by which the matrix s ˆ will be multiplied and the learning operator will become L(q) ≡ sA(q). This will alter the u∗l (t, Sl ) from the COILC update law (2) as follows: u∗l (t, Sl )

 =

   ˆ ql (t) q¨ d (t) − q¨ l (t) , t ∈ [0, Sl ], Sl ∈ (0, T ]; sA 0, ∀t ∈ (Sl , T ]

(3)

Next, the gain s from (3) will take values between 0.2 and 1.4 and the COILC will be executed over the same trajectory (trajectory A from Sect. 2) and the separate executions will be compared. The execution times are plotted in Fig. 4a. By regression analysis we can find the optimal value for the gain s for execution of trajectory A. From Fig. 4b it can be concluded that the dataset is best fitted with a quadratic equation.

280

K. Yovchev and L. Miteva

Fig. 4. a. COILC convergence for different values of the learning gain s; b. Regression analysis of the COILC executions of trajectory A with different values of the learning gain s.

So, the COILC can be modified by adding an additional scalar learning gain s by which the learning operator is multiplied. This learning operator gain s can be found by regression analysis of the different values of s when a training trajectory tracking is executed. This training trajectory must represent as best as possible the set of trajectories which the robot will be executing during its further work time. After the gain s is selected then it is used in each subsequent ILC executions. The process of finding the gain s is the step of self-adaptation of the used in the COILC process learning operator to the corresponding domain of executed tasks (trajectory movements). The COILC constrains the output trajectory at each iteration l, so that the state space constraints cannot be violated during the ILC procedure and the adaptation step can be safely executed. The described self-adaptation step together with the COILC can be summarized as Self-adaptive COILC (SCOILC) algorithm. The performance of the proposed SCOILC is evaluated through numerical experiments in the following section.

4 Simulation Results For the numerical experiments of the SCOILC algorithm a computer simulation is used as described in Sect. 2. The gain s is found over trajectory A. For this we had to run ILC procedure 7 times. The regression analysis resulted in the following quadratic equation: y = 412.75x2 − 633.99x + 312.54. The found scalar learning operator gain s is equal to 0.76. For the next experiment the SCOILC with the found gain s was used for tracking of the trajectory B. The convergence comparison is shown in Fig. 5. The SCOILC needed only 7 iterations with total time of 65.6 s for convergence. This is an improvement of 37 s and about 36% faster than the COILC method for the specific task. For the next experiment the joint 2 limits parameters of the COILC were selected close to the desired trajectory of the joint 2, so that Q2min = 0.79 rad and Q2max = 1.47 rad. The COILC was executed with performance results shown in Fig. 5. With those selection of parameters, the first iterations were aborted earlier as seen in Fig. 6 which led to a convergence for 6 iterations and 50.6 s. This is a major improvement of

Preliminary Study of Self-adaptive Constrained Output

281

Fig. 5. Convergence comparison for trajectory B: a. maximum tracking error; b. iteration times.

the convergence time of the COILC of 52 s or about 49.4% faster convergence. This leads to a conclusion that joint limits of the COILC can also be adapted to the actual task of the robot in order for the execution time to be improved.

Fig. 6. Comparison of tracked trajectory of joint 2 for: a. the robot joint constraints; b. the joints constraints selected accordingly to the desired trajectory.

5 Conclusion This paper proposes a novel and computationally efficient algorithm for performance enhancement of the ILC methods when they are applied within the control of robotic manipulators. The SCOILC algorithm combines the convergent COILC method with regression analysis of the executions of the ILC procedure with different learning operators for selected trajectory. The different learning operators are generated by using the estimated inertia matrix of the robotic manipulator. The regression analysis is done offline, and the algorithm does not require additional specific hardware. Also, does not add additional computational cost during the execution of the ILC iterations or during the learning procedure. This allows the ILC process to adapt to the specific type of trajectory needed for execution of a specific domain of tasks. The conducted experiments

282

K. Yovchev and L. Miteva

show that this initial adaptation process reduces the convergence time of the ILC when it is applied to a modification of the trajectory movement (task) which is used during the adaptation. This algorithm improves the overall performance of ILC for industrial robotic manipulators which have to execute a set of similar trajectories when there exist inaccuracies in the estimated parameters of their mathematical model by reducing the overall ILC execution time. Acknowledgement. The research presented in this paper was supported by the Bulgarian National Science Fund under the contract # KP-06-M47/5 – 27.11.2020.

References 1. Uchiyama, M.: Formulation of high-speed motion pattern of a mechanical arm by trial. Trans. SICE (Soc. Instrum. Control Eng.) 14(6), 706–712 (1978) 2. Arimoto, S., Kawamura, S., Miyazaki, F.: Iterative learning control for robot systems. In: Proceedings of IECON, Tokyo, Japan (1984) 3. Arimoto, S., Kawamura, S., Miyazaki, F.: Bettering operation of robots by learning. J. Robot. Syst. 1(2), 123–140 (1984) 4. Heizinger, D., Fenwick, B., Paden, B., Miyazaki, F.: Robust learning control. In: Proceedings of the 28th Conference on Decision and Control, Tampa, FL (1989) 5. Longman, R., Huang, Y.: The phenomenon of apparent convergence followed by divergence in learning and repetitive control. Intell. Autom. Soft Comput. 8(2), 107–128 (2002) 6. Delchev, K.: Iterative learning control for nonlinear systems: a bounded-error algorithm. Asian J. Control 15(3), 1–8 (2013) 7. Yovchev, K., Delchev, K., Krastev, E.: State space constrained iterative learning control for robotic manipulators. Asian J. Control 20(3), 1145–1150 (2018) 8. Yovchev, K., Delchev, K., Krastev, E.: Constrained output iterative learning control. Arch. Control Sci. 30(1), 157–176 (2020) 9. Zhang, Y., Chu, B., Shu, Z.: A preliminary study on the relationship between iterative learning control and reinforcement learning. IFAC-PapersOnLine 52(29), 314–319 (2019) 10. Kuo, W.-L., Cheng, M., Lin, H.: Reinforcement Q-learning and ILC with self-tuning learning rate for contour following accuracy improvement of biaxial motion stage (2017) 11. Nemec, B., Simoniˇc, M., Likar, N., Ude, A.: Enhancing the performance of adaptive iterative learning control with reinforcement learning. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, pp. 2192–2199 (2017). https:// doi.org/10.1109/IROS.2017.8206038 12. Patan, K., Patan, M., Kowalów, D.: Neural networks in design of iterative learning control for nonlinear systems. IFAC-PapersOnLine 50(1), 13402–13407 (2017) 13. Xu, P., Xu, Q.: Neural network-based AILC for nonlinear discrete-time system with iteration varying initial error and reference trajectory. In: Proceedings of the 2018 5th International Conference on Information Science and Control Engineering (ICISCE), Zhengzhou, China, pp. 854–858, July 2018 14. Wang, Y., Chien, C., Chi, R.: A fuzzy neural network based adaptive terminal iterative learning control for nonaffine nonlinear discrete-time systems. In: Proceedings of the 2014 International Conference on Fuzzy Theory and Its Applications (iFUZZY 2014), pp. 163–167, Kaohsiung, China, November 2014 15. Wang, Y.-C., Chien, C.-J., Lee, D.-T.: A fuzzy neural network direct adaptive iterative learning controller for robotic systems. IFAC Proc. Volumes 41(2), 6519–6524 (2008)

Preliminary Study of Self-adaptive Constrained Output

283

16. Wakui, S., Mita, T.: A parameter identification method of horizontal robot arms. Adv. Robot. 4(4), 337–352 (1990) 17. Yovchev, K., Miteva, L., Delchev, K., Krastev, E.: Iterative learning control of hard constrained robotic manipulators. In: Zeghloul, S., Laribi, M., Sandoval Arevalo, J. (eds.) Advances in Service and Industrial Robotics. RAAD 2020. Mechanisms and Machine Science, vol. 84. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-48989-2_52 18. Delchev, K.: Simulation-based design of monotonically convergent iterative learning control for nonlinear systems. Arch. Control Sci. 22(4), 371–384 (2012)

Author Index

A Ainhauser, Werner, 250 Andris, Pavel, 259 Anton, Florin, 266 Anton, Silvia, 266 Arif, Mohammed Nazrul Islam, 221 B Berns, Karsten, 127, 221 Bîzdoac˘a, Nicu George, 176 Borangiu, Theodor, 266 Botta, Andrea, 39 Brecelj, Tilen, 135 C Cafola, Daniele, 184 Carbonari, Luca, 39 Carbone, Giuseppe, 13, 31, 184 Castillo-Castañeda, Eduardo, 184 Cavallone, Paride, 39 Ceccarelli, Marco, 3, 167 Chikurtev, Denis, 145 Copilus, i, Cristian, 176 Cornagliotto, Valerio, 199 D Dadiotis, Ioannis D., 47 De Benedictis, Carlo, 208 De Martin, Andrea, 69 Denarda, Alessandro, 13 Digo, Elisa, 199 Dobrovodský, Karol, 259 E Essomba, Térence, 192

F Ferraresi, Carlo, 208 Fort, Axel, 167 G Gadringer, Stefan, 78 Gams, Andrej, 119 Gastaldi, Laura, 86 Gattringer, Hubert, 78 Gazvoda, Samo, 104 Gerstmayr, Johannes, 250 Ghiss, Moncef, 95 Giusti, Andrea, 250 Guida, Roberto, 69 H Havlik, Stefan, 23 Henrich, Dominik, 61 Hernández-Cerero, R., 3 Hricko, Jaroslav, 23 J Jovanovi´c, Miloš, 240 K Keen, Hannan Ejaz, 127 Koszulinski, Alizée, 157 Koustoumpardis, Panagiotis N., 47 L Laribi, Med Amine, 95, 157, 167, 192 Leal-Naranjo, J. A., 3 Lonˇcarevi´c, Zvezdan, 119

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 S. Zeghloul et al. (Eds.): RAAD 2021, MMS 102, pp. 285–286, 2021. https://doi.org/10.1007/978-3-030-75259-0

286 M Magnetti Gisolo, Stefania, 208 Malyshev, Dmitry, 31 Manuello Bertetto, Andrea, 13 Mauro, Stefano, 69 Mehdi, Syed Atif, 127 Miteva, Lyubomira, 275 Mueller, Andreas, 78 Muscolo, Giovanni Gerardo, 208 N Nozdracheva, Anna, 31 O Ollivier, Léa, 157 P Pan˘a, Florina Cristian, 176 Panero, Elisa, 86 Paplu, Sarwar Hussain, 221 Pastorelli, Stefano, 69, 86, 199 Paterna, Maria, 208 Petriˇc, Tadej, 135 Petris, or, Iulian, 176 Pisla, Doina, 13 Q Quaglia, Giuseppe, 39 R R˘aileanu, Silviu, 266 Raviola, Andrea, 69 Rodi´c, Aleksandar, 240 Rodríguez-León, Jhon F., 184 Rybak, Larisa, 31

Author Index S Sakellariou, John S., 47 Salgado, Octavie Somoza, 167 Sandoval, Juan, 95, 157, 167, 192 Sauer, Lukas, 61 Savevska, Kristina, 229 Segagliari, Margherita, 86 Semenenko, Tatiana, 31 Simoniˇc, Mihael, 229 Sorli, Massimo, 69 Stantinat, Eliot, 157 Stevanovi´c, Ilija, 240 Suarez, Brandon, 184 Šumarac, Jovan, 240 T Tagliavini, Luigi, 39 Torres-SanMiguel, C. R., 3 Trabelsi, Amir, 95 U Ude, Aleš, 229 V Vladu, Ileana, 176 Vladu, Ionel Cristian, 176 W Wu, Chieh-Tsai, 192 Y Yovchev, Kaloyan, 145, 275 Z Zaheer, Muhammad Hassan, 127 Zeghloul, Saïd, 192 Žlajpah, Leon, 104