Frontiers in Robotics and Electromechanics 9811976848, 9789811976841

Table of contents :
Preface
Contents
Editors and Contributors
Part I Modeling and Control of Manipulators, Multi-link Robots
1 Simulation of Foot Movement During Walking Based on the Study of Different Step Parameters
1.1 Introduction
1.2 Kinematic Equations of the Key Points of the Foot
1.3 Construction of the Trajectory of the Key Points of the Foot
1.4 Smoothing and Polynomialization of the Trajectory
1.5 Comparison of the Experimental Trajectory with the Mathematically Processed Trajectory
1.6 Solving the Inverse Kinematics Problem
1.7 Lower Limb Motion Modeling with MATLAB
1.8 Conclusion
References
2 Mathematical Modeling of the Biomechanical Rehabilitation System of Foot Exoskeleton in Frontal and Sagittal Planes
2.1 Introduction
2.2 The Structural Diagram of the Ankle Rehabilitation System
2.2.1 The Kinematic Diagram
2.3 Angle of Rotation Ranges of the Foot
2.4 Mathematical Model of the Biomechanical System
2.4.1 Mathematical Model of the System in the Sagittal Plane
2.5 Experimental Results of Foot Motion in the Sagittal Plane
2.5.1 Investigation of the Mathematical Model in the Frontal Plane
2.5.2 Experimental Results of Foot Movement in the Frontal Plane
2.6 Conclusion
References
3 Simulation of Controlled Motion of the Actuator of Robotic Systems in the Presence of Coupling Forces
3.1 Mathematical Model of Coupling Forces
3.2 Calculation Scheme of the Interaction of the Device with the Environment and Nodes
3.3 Mathematical Expressions for Modeling
3.4 Block Diagram of the Device Control System
3.5 Description of the Mathematical Model of the Automatic Control System
3.6 Conclusions
References
4 Digital Control by Robot Manipulator with Improved Rigidity
4.1 Introduction
4.2 Operational Unit Positioning Control System Statics
4.3 Operational Unit Positioning Control System Dynamics
4.4 Digital Control by Manipulator Positioning
4.5 Conclusion
References
5 Performance Evaluation of Multigrid Brute-Force Solutions of Inverse Kinematics Problem for the Robotis OP2 Humanoid Hand
5.1 Introduction
5.2 System Setup
5.3 Algorithms Description
5.3.1 Brute-Force Algorithm
5.3.2 Matrix-Based Brute-Force Algorithm
5.4 Testing
5.5 Experimental Results Analysis
5.6 Discussion
5.7 Conclusions
References
6 Neural Network Approach for Solving Inverse Kinematics Problem of Modular Reconfigurable Systems
6.1 Introduction
6.2 Related Works
6.3 Material and Methods
6.3.1 Algorithm of ANNs Learning for Solving IK Problem of Modular RSs
6.3.2 Dataset Generation
6.4 Experiments of Modeling an Autonomous Robotic System
6.5 Conclusion
References
7 Algorithm of Trajectories Synthesis for Modular Wheeled Inspection Robot
7.1 Introduction
7.2 Mathematical Model
7.3 Parametric Analysis of Robot Kinematics
7.4 Features of the Robot Dynamics
7.5 Conclusion
References
Part II Interaction and Control of Robot Group
8 Study of Algorithms for Coordinating a Group of Autonomous Robots in a Formation
8.1 Introduction
8.2 Mathematical Model of the UAV and the Formulation of the Problem
8.3 Formation Algorithms
8.4 Results of the Study of Formation Algorithms
8.5 Conclusion
References
9 Intelligent System for Countering Groups of Robots Based on Reinforcement Learning Technologies
9.1 Introduction
9.2 Problem Description
9.3 Algorithms
9.4 Neural Network Architecture
9.5 Preliminary Processing
9.6 Reward Function
9.7 Training
9.8 Conclusion
References
10 LIRS-MazeGen: An Easy-to-Use Blender Extension for Modeling Maze-Like Environments for Gazebo Simulator
10.1 Introduction
10.2 Related Work
10.3 Maze Generation Module
10.3.1 Graphical User Interface
10.3.2 Experimental Considerations
10.3.3 3D-Maze Modeling
10.4 Maze Environment Validation
10.5 Conclusions
References
11 Modeling of Joint Motion Planning of Group of Mobile Robots and Unmanned Aerial Vehicle
11.1 Introduction
11.2 Problem Statement
11.3 Review of Methods for Planning the Movement of Mobile Robots in a Group
11.4 Applying Tangential Avoidance to Calculate the Paths of Mobile Robots in a Group
11.5 Experiments
11.6 Formation and Landing
11.7 Determining How to Build a Communication Field to Transmit Data from a UAV to a Group of Ground Robots
11.8 Conclusion
References
12 Simulation of Controllable Motion of a Flying Robot Under the Action of Aerodynamic Force of a Bioinspired Flapping Wing
12.1 Introduction
12.2 Flapping Wing Kinematics
12.3 Model of Force in the Interaction of a Wing with Air
12.4 Aircraft Motion Control
12.5 Conclusion
References
13 Approaches to Optimizing Individual Maneuvers of Unmanned Aerial Vehicle
13.1 Introduction
13.2 Mathematical Model of the Aircraft and Problem Formulation
13.3 Modification of Direct Control Method
13.4 General Methodological Foundations of UAV Formation Control
13.5 Results of the Study
13.6 Conclusion
References
14 Neural Network Technologies in the Tasks of Estimating and Forecasting the Resource of Power Supply Systems in Robotic Complexes
14.1 Introduction and Problem Statement
14.2 Related Works
14.3 Investigation of the Influence of Temperature and Electric Field Strength on the Breakdown of Polymer Materials of the Thermofluctuation Theory
14.3.1 Problem Statement and Its Solution
14.3.2 Calculation of the Time to Breakdown Based on the Thermofluctuation Theory
14.3.3 Experimental Investigation
14.4 Investigation of Magnetic Induction in an Inhomogeneous Dielectric
14.4.1 Case of Homogeneous Insulation
14.4.2 Case of Inhomogeneous Insulation
14.5 Conclusion
References
15 Instantaneous Common-Mode Voltage Reduction of Three-Phase Multilevel Voltage Source Inverter Under Quarter-Wave-Symmetric Space Vector PWM with Full Set of Vectors
15.1 Introduction
15.2 Zero-Common-Mode-Voltage Space Vectors Applying
15.3 Common-Mode Voltage Reduction Under Applying of Full Set of Space Vectors
15.4 Computer Simulation
15.5 Conclusion
References
16 System of Decentralized Control of a Group of Mobile Robotic Means Interacting with Charging Stations
16.1 Introduction
16.2 Synthesis of a Method for Decentralized Control of the Process of Functioning of a Generalized Doubly Connected Technical System
16.2.1 Problem Statement
16.2.2 Control System for a Distributed Generalized Network, Including Mobile Robotic Vehicles and Docking Stations
16.2.3 Method of Decentralized Interaction of Mobile and Stationary Objects of the DGN
16.3 Decentralized Control Method for a DGN Containing a Group of UAVs and a Station for Recharging Them
16.4 Simplified Model of UAV Energy Consumption
16.5 Required Energy Characteristics of a Recharging Station Based on a Wind Power Plant
16.6 Simulation
16.7 Conclusion
References
17 Method for Optimizing the Trajectory of a Group of Mobile Robots in a Field of Repeller Sources Using the Method of Characteristic Probabilistic Functions
17.1 Introduction
17.2 The Problem Statement
17.3 Methods for Constructing a Program Trajectory of a Ground-Based Robotic Platform Under the Influence of Disturbance Sources
17.3.1 Method for Optimizing Local Sections of the Trajectory Based on the Characteristic Probabilistic Function
17.3.2 Method for Reducing Oscillations and Trajectory Length When Planning Motion in a Region with Sources
17.4 Correction of Individual Trajectories of Ground-Based Robotic Platforms Forming a Group Under the Influence of Disturbance Sources
17.5 Additional Correction of Trajectories Taking into Account Obstacles
17.6 Simulation Results
17.7 Conclusion
References
Part III Heterogeneous Robots in Monitoring and Service Tasks
18 Control of Robotic Mobile Platform for Monitoring Water Bodies
18.1 Introduction
18.2 Robotic Mobile Platform Scheme
18.3 Mathematical Model of RMP
18.4 RMP Control System
18.5 RMP Motion Planning Algorithm
18.6 Synthesis of the Optimal Regulator
18.7 Results of Simulation
18.8 Conclusions
References
19 Control System of Small-Unmanned Aerial Vehicle for Monitoring Sea Vessels on Coastal Territory of Ecuador
19.1 Introduction
19.2 Structure of the System for Monitoring Sea Vessels in Coastal Areas
19.3 SUAV Motion Control
19.4 SUAV Control Algorithms
19.5 Planning of SUAV Flight Trajectory
19.6 Vision System
19.7 Conclusions
References
20 Development of an Algorithm for Coverage Path Planning for Survey of the Territory Using UAVs
20.1 Introduction
20.2 Requirements for the Algorithm for Constructing a Covering Trajectory
20.3 Development of an Algorithm for Constructing a Covering Path for a Polygon
20.4 Result of Operation of the Algorithm
20.5 Conclusion
References
21 Computer Vision System of Robot Control for Monitoring Objects in Radioactive Areas
21.1 Introduction
21.2 Scheme and Principle of the Mobile Robot Operation
21.3 Mathematical Model and Principle of the Control System Operation
21.4 Formation of the Data About a Studied Object
21.5 Mathematical Modeling of the Control System in the Object Scanning Mode
21.6 Conclusion
References
22 Method for Searching Deployment Zones of Ground Seismic Sensors by a Heterogeneous Group of UAVs in an Environment with a Complex Topography
22.1 Introduction
22.2 Related Works
22.3 Method of Searching Deployment Zones for Ground Seismic Sensors by Means of a Heterogeneous Group of UAVs
22.4 Results
22.5 Conclusion
References
23 Method of Autonomous Survey of Power Lines Using a Multi-rotor UAV
23.1 Introduction
23.2 Description of the Developed Method of Autonomous Survey of PTL
23.3 Description of the Developed Algorithms for UAV Movement According to Given Patterns of Trajectories
23.4 Experimental Results
23.5 Conclusion
References
24 Walking Robots for Agricultural Monitoring
24.1 Introduction
24.2 Application Scenario
24.3 Types of Walking Robots and Their Gaits
24.4 Work Requirements
24.5 Industrial Analogues
24.6 Conclusion
References
25 Computational Approach to Optimal Control in Applied Robotics
25.1 Introduction
25.2 Problem Statement of the Synthesized Optimal Control
25.3 Methods Overview
25.4 Computational Example
25.5 Conclusions
References
26 Highly Maneuverable Small-Sized Wheeled Mobile Robotic Construction Platform
26.1 Introduction
26.2 Proposed Structure and Description of the MRCP
26.3 Mathematical Model of MRCP Movement with Two Independent Driving Wheels
26.4 MRCP Movement Control
26.5 Planning of MRCP Trajectory Movement
26.6 Conclusions
References
27 Control System for Robotic Towing Platform for Moving Aircraft
27.1 Introduction
27.2 Control Methods for Wheeled Robots
27.3 Design of a Robotic Aircraft Tug
27.4 External Environment Perception System
27.5 Description of the Positioning Control System Using an Optical Sensor Based on an Optronic Matrix
27.6 Description of the Work Area
27.7 Construction of a Piecewise Linear Trajectory
27.8 Determination of Positioning Relative to the Line with the Help of an Optronic Matrix
27.9 Synthesis of Multi-channel Controller for Robot Motion
27.10 Example of Simulation of RMB Motion Along a Contrasting Line
27.10.1 Simulation Results
27.11 Conclusion
References
28 Simple Task Allocation Algorithm in a Collaborative Robotic System
28.1 Introduction
28.2 Problem Statement and Algorithmic Solution
28.3 MATLAB Simulation
28.4 Conclusions and Future Work
References

Citation preview

Smart Innovation, Systems and Technologies 329

Andrey Ronzhin Viacheslav Pshikhopov   Editors

Frontiers in Robotics and Electromechanics 123

Smart Innovation, Systems and Technologies Volume 329

Series Editors Robert J. Howlett, Bournemouth University and KES International, Shoreham-by-Sea, UK Lakhmi C. Jain, KES International, Shoreham-by-Sea, UK

The Smart Innovation, Systems and Technologies book series encompasses the topics of knowledge, intelligence, innovation and sustainability. The aim of the series is to make available a platform for the publication of books on all aspects of single and multi-disciplinary research on these themes in order to make the latest results available in a readily-accessible form. Volumes on interdisciplinary research combining two or more of these areas is particularly sought. The series covers systems and paradigms that employ knowledge and intelligence in a broad sense. Its scope is systems having embedded knowledge and intelligence, which may be applied to the solution of world problems in industry, the environment and the community. It also focusses on the knowledge-transfer methodologies and innovation strategies employed to make this happen effectively. The combination of intelligent systems tools and a broad range of applications introduces a need for a synergy of disciplines from science, technology, business and the humanities. The series will include conference proceedings, edited collections, monographs, handbooks, reference books, and other relevant types of book in areas of science and technology where smart systems and technologies can offer innovative solutions. High quality content is an essential feature for all book proposals accepted for the series. It is expected that editors of all accepted volumes will ensure that contributions are subjected to an appropriate level of reviewing process and adhere to KES quality principles. Indexed by SCOPUS, EI Compendex, INSPEC, WTI Frankfurt eG, zbMATH, Japanese Science and Technology Agency (JST), SCImago, DBLP. All books published in the series are submitted for consideration in Web of Science.

Andrey Ronzhin · Viacheslav Pshikhopov Editors

Frontiers in Robotics and Electromechanics

Editors Andrey Ronzhin St. Petersburg Federal Research Center of the Russian Academy of Sciences St. Petersburg, Russia

Viacheslav Pshikhopov Research Institute of Robotics and Control Systems Southern Federal University Rostov-on-Don, Russia

ISSN 2190-3018 ISSN 2190-3026 (electronic) Smart Innovation, Systems and Technologies ISBN 978-981-19-7684-1 ISBN 978-981-19-7685-8 (eBook) https://doi.org/10.1007/978-981-19-7685-8 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Preface

Digital transformation of society and declining cost of robots open up new areas of application of robotic technology. A retrospective analysis of the development of robotics allows us to assess the future development of the industry and highlight the most promising research areas for the development of intelligent control systems and electromechanics of heterogeneous robots. The book presents solutions and notes of the problems of individual robotic devices, as well as heterogeneous groups of robots when performing technological tasks that require information, physical or energy interaction with people, the environment and other robots, including service-charging platforms. The book examines the models, algorithms and software, hardware control of ground, water and underwater robots, unmanned aerial vehicles, as well as their built-in and attached equipment, including video cameras, sensors, repeaters, manipulators, grippers, and other actuators. Examples of bioinspired insectomorphic and four-legged robots, unmanned aerial vehicles with flapping wings, which allow them to move in an aggressive environment with moving obstacles and changing weather conditions, are considered. Multilink systems for manipulating objects and exoskeletons for enhancing the physical capabilities of a person and rehabilitation of the patient’s musculoskeletal system have been analyzed. Due to the observance of the correct angles of the patient’s hip and foot, as well as their precise positioning in the exoskeleton, rehabilitation is much faster. To implement manipulators, various kinematic schemes are used, including those with a pyramidal structure, and their control systems are tuned through algorithms for learning an artificial neural network, reinforcement learning, an adaptive forecasting strategy, and a semi-Markov model. Decomposition of tasks in collaborative robotic systems is discussed. For complex and lengthy composite operations in the control system, it is possible to take into account the requirements for providing time to the rest of the participating people (depending on their current productivity and taking into account the drop in the efficiency of completing tasks on time) and periodic maintenance of collaborative robots. v

vi

Preface

Approaches to the calculation of trajectories of movement of robots in twodimensional and three-dimensional environments with obstacles are considered. When constructing the trajectory of unmanned aerial vehicles of an aircraft type, the location of take-off and landing points is taken into account, which ensures a decrease in the number of maneuvers while driving. Simulation of labyrinthine environments is performed in the Gazebo simulator and other computer platforms. The robotic solutions of authors from Azerbaijan, Japan, Mexico, Russia, and Taiwan, considered in the book, are implemented for practical tasks of monitoring power lines, water resources, agricultural resources, hazardous radioactive territories, the movement of ships on the coastal territory, airfield transport, the formation of dynamic communication systems, and construction production. St. Petersburg, Russia Rostov-on-Don, Russia

Andrey Ronzhin Viacheslav Pshikhopov

Contents

Part I 1

2

3

Modeling and Control of Manipulators, Multi-link Robots

Simulation of Foot Movement During Walking Based on the Study of Different Step Parameters . . . . . . . . . . . . . . . . . . . . . . . Andrey Malchikov, Alexander Pechurin, and Andrey Jatsun

3

Mathematical Modeling of the Biomechanical Rehabilitation System of Foot Exoskeleton in Frontal and Sagittal Planes . . . . . . . . Andrey Knyazev, Andrey Jatsun, and Andrey Fedorov

19

Simulation of Controlled Motion of the Actuator of Robotic Systems in the Presence of Coupling Forces . . . . . . . . . . . . . . . . . . . . . . Evgeny Subbotin and Andrey Jatsun

33

4

Digital Control by Robot Manipulator with Improved Rigidity . . . . Eugene Larkin, Aleksey Bogomolov, and Aleksandr Privalov

5

Performance Evaluation of Multigrid Brute-Force Solutions of Inverse Kinematics Problem for the Robotis OP2 Humanoid Hand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Linar Zagidullin, Tatyana Tsoy, Kuo-Hsien Hsia, Edgar A. Martínez-García, and Evgeni Magid

61

Neural Network Approach for Solving Inverse Kinematics Problem of Modular Reconfigurable Systems . . . . . . . . . . . . . . . . . . . . Aleksei Erashov, Anton Saveliev, and Dmitriy Blinov

77

Algorithm of Trajectories Synthesis for Modular Wheeled Inspection Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ildar Nasibullayev and Oleg Darintsev

93

6

7

45

vii

viii

Contents

Part II

Interaction and Control of Robot Group

8

Study of Algorithms for Coordinating a Group of Autonomous Robots in a Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Viacheslav Pshikhopov, Mikhail Medvedev, and Boris Gurenko

9

Intelligent System for Countering Groups of Robots Based on Reinforcement Learning Technologies . . . . . . . . . . . . . . . . . . . . . . . . 135 Vladimir Parkhomenko, Tatiana Gayda, and Mikhail Medvedev

10 LIRS-MazeGen: An Easy-to-Use Blender Extension for Modeling Maze-Like Environments for Gazebo Simulator . . . . . 147 Anzhelika Iskhakova, Bulat Abbyasov, Timophey Mironchuk, Tatyana Tsoy, Mikhail Svinin, and Evgeni Magid 11 Modeling of Joint Motion Planning of Group of Mobile Robots and Unmanned Aerial Vehicle . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 Aleksander Shirokov, Alexander Salomatin, Rinat Galin, and Vasiliy Zorin 12 Simulation of Controllable Motion of a Flying Robot Under the Action of Aerodynamic Force of a Bioinspired Flapping Wing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 Sergey Efimov, Oksana Emelyanova, and Sergey Jatsun 13 Approaches to Optimizing Individual Maneuvers of Unmanned Aerial Vehicle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Oleg Korsun, Alexander Stulovskii, Sergey Kuleshov, and Alexandra Zaytseva 14 Neural Network Technologies in the Tasks of Estimating and Forecasting the Resource of Power Supply Systems in Robotic Complexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 Nikolay Poluyanovich, Dmitriy Burkov, and Marina Dubyago 15 Instantaneous Common-Mode Voltage Reduction of Three-Phase Multilevel Voltage Source Inverter Under Quarter-Wave-Symmetric Space Vector PWM with Full Set of Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 Nikolay Lopatkin 16 System of Decentralized Control of a Group of Mobile Robotic Means Interacting with Charging Stations . . . . . . . . . . . . . . . . . . . . . . . 235 Vladimir Kostyukov and Viacheslav Pshikhopov 17 Method for Optimizing the Trajectory of a Group of Mobile Robots in a Field of Repeller Sources Using the Method of Characteristic Probabilistic Functions . . . . . . . . . . . . . . . . . . . . . . . . 265 Vladimir Kostyukov and Viacheslav Pshikhopov

Contents

ix

Part III Heterogeneous Robots in Monitoring and Service Tasks 18 Control of Robotic Mobile Platform for Monitoring Water Bodies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 Sergey Efimov, Dzhamil Safarov, and Sergey Jatsun 19 Control System of Small-Unmanned Aerial Vehicle for Monitoring Sea Vessels on Coastal Territory of Ecuador . . . . . . . 295 Andres Santiago Martinez Leon, Luis Miguel Mosquera Morocho, and Oksana Emelyanova 20 Development of an Algorithm for Coverage Path Planning for Survey of the Territory Using UAVs . . . . . . . . . . . . . . . . . . . . . . . . . . 315 Valeria Lebedeva and Igor Lebedev 21 Computer Vision System of Robot Control for Monitoring Objects in Radioactive Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 Andrei Malchikov and Petr Bezmen 22 Method for Searching Deployment Zones of Ground Seismic Sensors by a Heterogeneous Group of UAVs in an Environment with a Complex Topography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343 Roman Iakovlev, Valeria Lebedeva, Ivan Egorov, Vitaly Bryksin, and Andrey Ronzhin 23 Method of Autonomous Survey of Power Lines Using a Multi-rotor UAV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 Anton Saveliev and Igor Lebedev 24 Walking Robots for Agricultural Monitoring . . . . . . . . . . . . . . . . . . . . 377 Dmitry Dobrynin and Yulia Zhiteneva 25 Computational Approach to Optimal Control in Applied Robotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387 Elizaveta Shmalko 26 Highly Maneuverable Small-Sized Wheeled Mobile Robotic Construction Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 Ekaterina Saveleva and Evgeny Politov 27 Control System for Robotic Towing Platform for Moving Aircraft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419 Dmitry Afonin, Alexander Pechurin, and Sergey Jatsun 28 Simple Task Allocation Algorithm in a Collaborative Robotic System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433 Rinat Galin, Roman Meshcheryakov, and Mark Mamchenko

Editors and Contributors

About the Editors Prof. Andrey Ronzhin is Director of St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS). He is a specialist in the field of human– machine interaction research, artificial intelligence and robotics. Under his leadership, the technological foundations for the application of artificial intelligence and machine learning methods in the digital means of natural language analysis and control of interoperable socio-cyber-physical and robotic systems were developed. Mathematical and algorithmic models, methods and prototypes of proactive automation tools for information, physical and energy interaction of heterogeneous robotic and cyber-physical complexes have been developed. He is Deputy Editor-in-Chief of the journal Informatics and Automation, Member of the Editorial Board of the journals Analysis and Data Processing Systems, Speech Technologies. He is Member of the Scientific Boards of Departments of the Russian Academy of Sciences “Scientific Foundations of Information Technologies and Automation,” “Robotics and Mechatronics,” “Theory and Control Processes,” “Mechanical Engineering.” He is Member of Scientific Board of Robotics and Mechatronics of the Russian Academy of Sciences, the Academy of Navigation and Motion Control. He is Co-chair of International Conference on Interactive Collaborative Robotics (ICR) and Chair of Program Committee of International Conference on Agriculture Digitalization and Organic Production (ADOP). He is Co-editor of 11 books and Co-author of over 300 refereed journal papers, 4 manuals, 2 monographs. Prof. Viacheslav Pshikhopov is Director of Research Institute of Robotics and Control Systems, Southern Federal University. He is Director of Coordinating Research and Development Center for Control Systems of Southern Federal University and Founder and Leader of Youth Design Agency “Robotics and intellectual systems.” He is Member of the Academy of Engineering Sciences, named after A. M. Prokhorov. His research interests cover control systems of mobile robots and manipulators; adaptive and optimal control of moving objects; group control

xi

xii

Editors and Contributors

of mobile robots, bio-inspired control of robots. Under his leadership, the control theory of complex dynamical systems in uncertain environment was developed. The bio-inspired method of unstable modes for control of mobile robots and the group of mobile robots was developed, and the problem of analytical design of optimal control performance for manipulators and mobile robots was solved. The developments of his scientific school have been implemented in dozens of samples of mobile robots, including aeronautical platforms, wheeled and tracked robots and unmanned boats. He is one of the organizers and coordinators of the Interdepartmental Coordinating Scientific and Technical Council for Information Management and Processing Systems. He is Member of the Editorial Board of the journals Facta Universitatis, Mechatronics, Automation, Control, Informatics and Automation. He is Co-author of over 320 refereed journal papers, 20 patents, 11 monographs.

Contributors Bulat Abbyasov Kazan Federal University, Kazan, Russia Dmitry Afonin Southwest State University, Kursk, Russia Petr Bezmen Southwest State University (SWSU), Kursk, Russia Dmitriy Blinov St. Petersburg State University of Aerospace Instrumentation, St. Petersburg, Russia Aleksey Bogomolov St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, St. Petersburg, Russia Vitaly Bryksin Immanuel Kant Baltic Federal University (IKBFU), Kaliningrad, Russia Dmitriy Burkov Southern Federal University (SFEDU), Rostov-on-Don, Russia Oleg Darintsev Mavlyutov Institute of Mechanics, UFRC RAS, Ufa, Russia; Ufa State Aviation Technical University, Ufa, Russia Dmitry Dobrynin Federal Research Center “Computer Science and Control” of the Russian Academy of Sciences, Moscow, Russia Marina Dubyago Southern Federal University (SFEDU), Rostov-on-Don, Russia Sergey Efimov Southwest State University, Kursk, Russia; Voronezh State Technical University, Voronezh, Russia Ivan Egorov LLC R-Sensors, Moscow Region, Russia Oksana Emelyanova Department of Mechanics, Mechatronics and Robotics, Southwest State University (SWSU), Kursk, Russia

Editors and Contributors

xiii

Aleksei Erashov St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, St. Petersburg, Russia Andrey Fedorov Southwest State University, Kursk, Russia Rinat Galin V.A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences, Moscow, Russia Tatiana Gayda Southern Federal University, Rostov-on-Don, Russia Boris Gurenko Southern Federal University, Rostov-on-Don, Russia Kuo-Hsien Hsia Graduate School Engineering Science and Technology, National Yunlin University of Science and Technology, Douliou, Yunlin, Taiwan Roman Iakovlev St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, St. Petersburg, Russia Anzhelika Iskhakova Kazan Federal University, Kazan, Russia Andrey Jatsun Southwest State University, Kursk, Russia Sergey Jatsun Southwest State University, Kursk, Russia Andrey Knyazev Southwest State University, Kursk, Russia Oleg Korsun State Research Institute of Aviation Systems, Moscow, Russia Vladimir Kostyukov Southern Federal University, Rostov-on-Don, Russia Sergey Kuleshov St. Petersburg Federal Research Center of the Russian Academy of Sciences, St. Petersburg, Russia Eugene Larkin Tula State University, Tula, Russia Igor Lebedev St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, St. Petersburg, Russia Valeria Lebedeva St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, St. Petersburg, Russia Nikolay Lopatkin Shukshin Altai State University for Humanities and Pedagogy, Biysk, Altai Region, Russia Evgeni Magid Laboratory of Intelligent Robotic Systems, Intelligent Robotics Department, Kazan Federal University, Kazan, Russia Andrei Malchikov Southwest State University (SWSU), Kursk, Russia Andrey Malchikov Southwest State University, Kursk, Russia

xiv

Editors and Contributors

Mark Mamchenko V.A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences, Moscow, Russia Andres Santiago Martinez Leon Department of Mechanics, Mechatronics and Robotics, Southwest State University (SWSU), Kursk, Russia Edgar A. Martínez-García Institute of Engineering and Technology, Department of Industrial Engineering and Manufacturing, Autonomous University of Ciudad Juarez, Cd Juárez, Mexico Mikhail Medvedev Southern Federal University, Rostov-on-Don, Russia Roman Meshcheryakov V.A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences, Moscow, Russia Timophey Mironchuk Kazan Federal University, Kazan, Russia Luis Miguel Mosquera Morocho Department of Mechanics, Mechatronics and Robotics, Southwest State University (SWSU), Kursk, Russia Ildar Nasibullayev Mavlyutov Institute of Mechanics, UFRC RAS, Ufa, Russia Vladimir Parkhomenko Southern Federal University, Rostov-on-Don, Russia Alexander Pechurin Southwest State University, Kursk, Russia Evgeny Politov Southwest State University, Kursk, Russia Nikolay Poluyanovich Southern Federal University (SFEDU), Rostov-on-Don, Russia Aleksandr Privalov Tula State Lev Tolstoy Pedagogical University, Tula, Russia Viacheslav Pshikhopov Southern Federal University, Rostov-on-Don, Russia Andrey Ronzhin St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, St. Petersburg, Russia Dzhamil Safarov Azerbaijan Technological University, Ganja, Azerbaijan Alexander Salomatin V.A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences, Moscow, Russia Ekaterina Saveleva Southwest State University, Kursk, Russia Anton Saveliev St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, St. Petersburg, Russia Aleksander Shirokov V.A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences, Moscow, Russia; National Research University “MPEI”, Moscow, Russia

Editors and Contributors

xv

Elizaveta Shmalko Federal Research Center “Computer Science and Control” of the Russian Academy of Sciences, Moscow, Russia Alexander Stulovskii State Research Institute of Aviation Systems, Moscow, Russia Evgeny Subbotin Southwest State University, Kursk, Russia Mikhail Svinin College of Information Science and Engineering, Ritsumeikan University, Kusatsu, Shiga, Japan Tatyana Tsoy Laboratory of Intelligent Robotic Systems, Intelligent Robotics Department, Kazan Federal University, Kazan, Russia Linar Zagidullin Laboratory of Intelligent Robotic Systems, Intelligent Robotics Department, Kazan Federal University, Kazan, Russia Alexandra Zaytseva St. Petersburg Federal Research Center of the Russian Academy of Sciences, St. Petersburg, Russia Yulia Zhiteneva State University of Humanities and Technology, Moscow Region, Orekhovo-Zuevo, Russia Vasiliy Zorin V.A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences, Moscow, Russia

Part I

Modeling and Control of Manipulators, Multi-link Robots

Chapter 1

Simulation of Foot Movement During Walking Based on the Study of Different Step Parameters Andrey Malchikov , Alexander Pechurin , and Andrey Jatsun

Abstract In this work, data on the movement of the human foot are experimentally obtained, by solving the inverse problem of kinematics, the laws of change in the angles of rotation of the lower limbs in time are obtained. For the construction of rehabilitation exercises, the parameters of the movements of the legs of a healthy person were used, which in the future will allow using the patterns obtained to recreate the patient’s stepping movements in an exoskeleton—a robotic electromechanical system that allows reproducing the movements of the lower extremities. The use of simplified laws of foot movement leads to the formation of an incorrect gait of a person and increases the rehabilitation period. This complicates the development of a criterion for assessing the fidelity of reproduction. It is necessary to correctly select the points of the foot for measurements, since the real trajectory changes from step to step. Therefore, a method is considered for approximating the trajectory of the foot movement, taking into account the anthropometric parameters of the patient, based on video analysis of the gait, taking into account uncertainties, and statistical processing of the results.

1.1 Introduction According to the WHO statistics, 80% of the population suffers from various musculoskeletal system diseases, and most of them are able-bodied people aged 30 to 50 years [1]. This development leads to an increase in the number of people with lower limb injuries, whose independent movement is difficult or impossible. Recently, treatment of patients with OA injuries has been provided using various rehabilitation mechatronic systems [2–6]. The use of such modern systems contributes to a significant reduction in the patient’s recovery time after injuries, which increases the efficiency of rehabilitation. Of the various variations of devices for OA restoration, the most accurate gait simulator is the lower limb exoskeleton—a bipedal robot that A. Malchikov · A. Pechurin (B) · A. Jatsun Southwest State University, St. 50 Years of October, 94, Kursk 305040, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_1

3

4

A. Malchikov et al.

provides the implementation of exercises to restore individual parts of the lower limbs or simulate human gait in real-life conditions [7]. In the process of developing and creating exoskeletons, modern methods of computer modeling are used to study the characteristics of patient and exoskeleton lower limb movement, which makes it possible to process various movement patterns and optimize the parameters of the HMS control system and design personalized rehabilitation systems for patient rehabilitation based on them. Exoskeleton operation in the mode set by a rehabilitation therapist is ensured by precise determination of the laws of motion of the feet and center of mass, taking into account anthropometric parameters of the person. This data makes it possible to calculate optimal control actions and create exoskeleton movement patterns for people with various degrees of musculoskeletal disorders.

1.2 Kinematic Equations of the Key Points of the Foot Let us assume that the foot [8] (Fig. 1.1) consists of phalanx 1, tarsus 2, and tibia 3, which are solid bodies. Therefore, the distances between the points A2 A1 , A1 S R , A2 H R , and A1 H R are constant values. At the point A2 , there is a two-coordinate hinge that allows the rotation of the link A2 A1 H R relative to the tibia. At the point A1 , there is a cylindrical joint that rotates the link A1 S R with respect to the basic link of the foot A2 A1 H R . Let us assume that the links of the foot in the sagittal plane move plane-parallel. To set the motion of the key points belonging to the links of the foot, it is necessary to determine the trajectory of motion of point A2 , the geometric characteristics of the

Fig. 1.1 Diagram of the location of key points on the human foot: a on the human foot, b geometric diagram, where: A2 is ankle joint; A1 is metatarsal joint; S R is extreme point of thumb; HR is extreme point of tarsus; 1 is toe link; 2 is tarsal link; 3 is tibia link; 4 is given trajectory; φ12 is rotation angle of toe link relative to tarsal link; φ23 is rotation angle of tarsal link relative to tibia link

1 Simulation of Foot Movement During Walking Based on the Study …

5

links, and the relative φ12 (t), φ23 (t) and absolute angles φ10 (t), φ20 (t) of the foot rota T tion. To do this, we will introduce the radius vector r (0) A2 = X A2 (t), Y A2 , Z A2 (t) , whose projections are functions of time obtained by video analysis that determine the position of the GSS at any given time, the speed of the point r˙ A2 is, respectively, equal T   T (0) (1) (1) (1) = X , Y , Z , to r˙ A2 = X˙ A2 (t), Y A2 , Z˙ A2 (t) . The vectors: r (1) A1 S R A1 S R A1 S R A1 S R T T   (2) (2) (2) (2) (2) (2) r (2) , r (2) determine the A2 H R = X A2 H R , Y A2 H R , Z A2 H R A2 A1 = X A2 A1 , Y A2 A1 , Z A2 A1 positions of the key points of the foot, the numerical values are filled based on the corresponding anthropometric parameters. Equations determining the position of the metatarsal joint (1.1): (0) (2) r (0) A1 = r A2 + T23 · r A2 A1 ,

(1.1)

where ⎡

⎤ cos(φ23 (t)) 0 − sin(φ23 (t)) T23 = ⎣ − sin(φ23 (t)) · sin(φ23 x(t)) cos(φ23 x(t)) − cos(φ23 (t)) · sin(φ23 x(t)) ⎦, cos(φ23 x(t)) · sin(φ23 (t)) sin(φ23 x(t)) cos(φ23 (t)) · cos(φ23 x(t)) where φ23 x(t) is an angle of rotation of the foot in the plane Z O OY O , equal to zero in this study. Equations determining the position of the heel (1.2): (0) (2) r (0) H R = r A2 + T23 · r A2 H R .

(1.2)

Equations determining the position of the right big toe (1.3): (0) (2) (1) r (0) S R = r A2 + T23 · r A2 H R +12 ·r A1 S R ,

(1.3)

where ⎤ ⎡ cos(φ12 (t) + φ23 (t)) 0 − sin(φ12 (t) + φ23 (t)) ⎥ ⎢ T12 = ⎣ − sin(φ12 (t) + φ23 (t)) · sin(φ23 x(t)) cos(φ23 x(t)) − cos(φ12 (t) + φ23 (t)) · sin(φ23 x(t)) ⎦, cos(φ12 (t) + φ23 x(t)) · sin(φ23 (t)) sin(φ23 x(t)) cos(φ12 (t) + φ23 (t)) · cos(φ23 x(t))

Next, let’s determine the velocities of the corresponding points, for point A1 (1.4 (0) (0) r˙ A1 = r˙ A2 +˙23 · r (2) A2 A1 ,

(1.4)

where T˙23 = dTdt23 Equations speeds of points H R and S R (1.5), (1.6): (0) (0) r˙ HR = r˙ A2 + T23 · r (2) A2 H R ;

(1.5)

6

A. Malchikov et al. (0) (0) (1) r˙ SR = r˙ A2 +˙23 · r (2) A2 A1 +˙12 · r A1 S R ,

where T˙12 =

(1.6)

dT12 . dt

1.3 Construction of the Trajectory of the Key Points of the Foot Construction of trajectories of movement of key points of a foot of the person, occurs by means of a video file of experiment [9], and also the program of the analysis and modeling of video fragments is “tracker”, intended for manual or automatic tracing of object with imposing on an initial video stream. The program “tracker” allows to receive data on position of object, its velocity and acceleration that allows to construct kinematic and dynamic models of point masses or systems of two bodies [10] (Fig. 1.2). In this work, the main varying step parameters are: step length this ls m and step tempo this ps Hz, determined with the help of a metronome. In Figs. 1.3 and 1.4, the

Fig. 1.2 Building trajectories of movement of key points of the human foot using a video analysis program: 1 is the main area of the program, 2 is graphs of changes in the trajectory of movement, 3 is table of numerical values of the trajectory, 4 is tools for constructing the trajectory, and 5 is points that make up the trajectory of the key points of the foot

1 Simulation of Foot Movement During Walking Based on the Study …

7



Fig. 1.3 Variation plots X A2 (t), Z A2 (t) and Z A2 X A2 : 1, 7, 13 at ls = 0.4 m and ps = 1 Hz, 2, 8, 14 at ls = 0.4 m and ps = 1.5 Hz, 3, 9, 15 at ls = 0.8 m and ps = 1 Hz, 4, 10, 16 at ls = 0.8 m and ps = 1.5 Hz, 5, 11, 17 at ls = 1.15 m and ps = 1 Hz, 6, 12, 18 at ls = 1.15 m and ps = 1.5 Hz

movement trajectories of point A2 belonging to the ankle joint and the foot rotation angles at different step parameters are shown. In Fig. 1.5, the combined graphs of foot motion and rotation in the second half of the step cycle are shown [11]. At point 1, the foot is fixed stationary on the supporting surface, at (a) the dip and transfer phases occur, which can be functionally represented as a roll over the forefoot rotating around the metatarsal joint. At point 2, the transfer period begins and the entire foot detaches from the surface. Point (b) is characterized by the acceleration and advancement phases, during which the foot makes a movement and a pivot in the air. The final phase of the carry period is braking at point 3 and the heel makes contact with the supporting surface. Section (c) is characterized by a partial support period including contact and loading phases. In this section, the foot rolls from heel to full foot, the position of the foot at point 4 is identical to the position of the foot at point 1. After processing the human step by the video analysis program, the obtained data arrays are transferred to MATLAB workspace. Then, using numerical differentiation (1.7) derivatives are found: f =

f i+t − f i . t

(1.7)

8

A. Malchikov et al.

Fig. 1.4 Variation plots φ20 (t), φ12 (t) and φ10 (t): 1, 7, 13 at ls = 0.4 m and ps = 1 Hz, 2, 8, 14 at ls = 0.4 m and ps = 1.5 Hz, 3, 9, 15 at ls = 0.8 m and ps = 1 Hz, 4, 10, 16 at ls = 0.8 m and ps = 1.5 Hz, 5, 11, 17 at ls = 1.15 m and ps = 1 Hz, 6, 12, 18 at ls = 1.15 m and ps = 1.5 Hz

Fig. 1.5 Diagrams of changes: the position of point A2 , the absolute angle of rotation of the metatarsophalangeal foot φ20 , the angle of rotation of the toes φ12

1 Simulation of Foot Movement During Walking Based on the Study …

9

1.4 Smoothing and Polynomialization of the Trajectory To reduce measurement error, to “smooth” [12] values obtained during the experiment, the “smooth” function of MATLAB software package is used, which processed data of vector-column y using a sliding average filter [13]. An example of function usage is yy = smooth(y). Finding the law of motion from the numerical data obtained in the analysis of human foot motion is done by approximating the function [14] “polyfit” software package MATLAB, which returns the coefficients of the polynomial of the n-th degree, which are suitable for describing the approximated data [15]. The final laws of change of the foot coordinates will have the form (1.8) [16]: ⎡ A 2 = ⎣ p X A2 =

nx  i=0

ai · t i , p Z A2 =

nz 

⎤ b j · t j ⎦,

(1.8)

j=0

where nx and nz are the order of the polynomial, t is the step time, the coefficients ai and bj are found using the “polyfit” function described above. As a result of numerical processing of the experimental data, the trajectory of ankle joint movement was obtained, shown in Fig. 1.6.

Fig. 1.6 Graphs of changes in the experimentally obtained and processed trajectory of motion of point A2 and its derivatives

10

A. Malchikov et al.

1.5 Comparison of the Experimental Trajectory with the Mathematically Processed Trajectory The trajectories of the key points of the foot can be reconstructed using the trajectory of point A2 , as well as the values of absolute and relative foot angles reconstructed from the experimental trajectory or with the help of sensors. Figures 1.7 and 1.8 show the results of comparing the trajectory plots obtained during the experiment and calculated by formulas (1.1)–(1.6). To estimate the trajectory errors, Fig. 1.9 shows graphs of the difference between the trajectories obtained during the experiment and those calculated using formulas (1.1)–(1.6) of the key points of the foot. The error was calculated according to the law (1.9): X HR = X HR − p X HR .

(1.9)

The graphs shown in Fig. 1.9 show that the maximum value of the error exceeds 0.015 m. Such values of deviations are caused by an error in the visual arrangement of points by the operator of the “tracker” program, which led to inconstant values of lengths between the key points of the foot (Fig. 1.10). However, it should be noted that the laws of movement of the key points of the foot obtained using formulas (1.1)–(1.6) are suitable for the initial levels of rehabilitation, such as walking in an exoskeleton with additional support or in a rehabilitation complex with back fixation. With the use of high-speed cameras and

Fig. 1.7 Comparison of the measured and calculated trajectories of movement of points: 1, 2, 4, 6 measured point trajectories this pA2 , pH R , pA1 , pS R ; 3, 5, 7 is calculated point trajectories is A2 , H R , A1 , S R

1 Simulation of Foot Movement During Walking Based on the Study …

11

Fig. 1.8 Plots of coordinates of characteristic points of the foot and their derivatives, obtained during the calculation

Fig. 1.9 Graphs of the key point trajectory errors

markers suitable for automatic processing, the approach presented is capable of demonstrating greater accuracy.

1.6 Solving the Inverse Kinematics Problem In the presented study, the human lower extremities are considered as a multi-link mechanism with rotational joints, kinematically calculated as a manipulator with variable start and end points (Fig. 1.11).

12

A. Malchikov et al.

Fig. 1.10 Graphs of length changes between the key points of the foot

Fig. 1.11 Schematic of the human lower extremity model when supported a on the left leg, b on the right leg, c on the left leg in the frontal plane

1 Simulation of Foot Movement During Walking Based on the Study …

13

The principle of changing start and end points is as follows: as soon as the free leg has finished moving along a given trajectory, it becomes the supporting leg. If the left leg is considered to be the supporting leg, then S L is taken as the starting point, and S R is taken as the ending point. As soon as point A2 completes the movement along the given trajectory, the right leg becomes the supporting leg, and S R and S L are considered the starting and ending points. Finding the vectors of generalized coordinates (1.10a, 1.10b) when supporting the left and right leg, respectively, for a given foot trajectory, is done by solving the inverse kinematics problem [17, 18]. q = [φ90 , φ89 , φ78x , φ78 , φ67 , φ56 , φ45 , φ34 , φ23x , φ23 , φ12 ]T ;

(1.10a)

q = [φ10 , φ21 , φ32x , φ32 , φ43 , φ54 , φ65 , φ76 , φ87x , φ87 , φ98 ]T ,

(1.10b)

where: φ10 = φ90 + φ89 + φ78 + φ67 + φ56 + φ45 + φ34 + φ23 + φ12 ; φ21 = −φ12 ; φ32 = −φ23 ; φ43 = −φ34 ; φ54 = −φ45 ; φ65 = −φ56 ; φ76 = −φ67 ; φ87 = −φ78 ; φ98 = −φ89 . To obtain motion patterns [19] of the lower limbs, it is also necessary to know the law of motion of the operator’s center of mass. Since the trajectory of the center of mass cannot be plotted using markers as for the foot, the position of the center of mass is found theoretically in this work, and the extreme points of stability of the center of mass are used instead of the law of motion [20] (Fig. 1.12). However, the system cannot move without a horizontal displacement of the center of mass, so as an assumption, let’s set the law of change of X C . Thus, the law of change of coordinates of point C will have the form (1.11): ⎧ 5  ⎪ ⎪ ⎪ ck · t k , D6 ∧ D2 ≥ X C ≥ K 6 ∧ K 2 ⎨ XC = i=0

⎪ ⎪ ⎪ ⎩

K 6 ∧ D6 ≥ YC ≥ K 2 ∧ D2

.

(1.11)

maxC min

ZC

C

Using xd as the desired configuration of the mechanism, extending the forward kinematics function f (q) in a Taylor series with respect to the configuration qd (1.12), where xd = f (qd ) [21]: q d = q + J + · (xd − f (q)).

(1.12)

The data received by and returned by the inverse kinematics problem solving module are shown in Fig. 1.13.

14

A. Malchikov et al.

Fig. 1.12 Schematic of the two-pointed phase of motion, with the stability zone of the system: C is center of mass of the system, K 6 , D6 , K 2, D2 are extreme points of the stability zone, L islength of the foot, W is width of the foot

Fig. 1.13 Diagram of the module for solving the inverse kinematics problem

Let us introduce a vector-function −F(q), which determines the values corresponding to the projections of the center of mass, the foot and the position of the back, depending on the generalized coordinates q (1.13). F(q) = [r C (q), r A2 (q), φ90 , φ89 , φ50 , φ20 , φ12 ]T .

(1.13)

1 Simulation of Foot Movement During Walking Based on the Study …

15

Let us also introduce a vector function (1.14):  F (t) = [r C (t), r A2 (t), φ90 (t), φ89 (t), φ50 (t), φ20 (t), φ12 (t)]T .

(1.14)

Let us denote the pseudonverse Jacobi matrix (1.13):  J+ =

∂F ∂q

+ (1.15)

Then, in discrete form, we rewrite the expression in the form (1.16): q k+1 = q k + JF+  F (t),

(1.16)

where  F (t) is the increment of the function  F (t) at the time step t. The resulting relation allows us to find the vector of generalized coordinates q at the k + 1st time step by the known value of q at the k-th step.

1.7 Lower Limb Motion Modeling with MATLAB Using MATLAB program, we wrote a program of mathematical model of the exoskeleton step with the specified laws of motion of the foot and center of mass, as well as the ability to hold in the vertical position of the back support link. Figures 1.14, 1.15 and 1.16 show a graphical representation of the simulation results. Fig. 1.14 Motion animation frame of the mathematical model of the lower extremities: 1 is a geometric model of the mechanism, 2 is an assumed trajectory of the center of masses, 3 is a polynomialized experimental trajectory

16

A. Malchikov et al.

Fig. 1.15 Angle and derivatives graphs: φ89 , φ78 , φ67 , φ56

Fig. 1.16 Angle and derivatives graphs: φ45 , φ34 , φ23 , φ12

As can be seen from Fig. 1.14, the mechanism is able to perform motion according to the given laws, including holding the “back” link in a vertical position without loss of stability. The result of solving the inverse kinematics problem is the values of kinematic characteristics of the rotational joints between the links of the lower limbs (Figs. 1.15 and 1.16).

1 Simulation of Foot Movement During Walking Based on the Study …

17

The kinematic characteristics of the mechanism obtained as a result of modeling are sufficient to create from them entry-level movement patterns used in rehabilitation systems.

1.8 Conclusion A method for constructing and processing the trajectory of human foot movement from a video fragment of the experiment was developed and a mathematical model of human step based on the movement of a two-linked foot was created. The direct problem of kinematics, the inverse problem of speed kinematics were solved and mathematical modeling of the gait of the mechanism replacing the lower limbs of a person was carried out. Mathematical modeling in MATLAB allowed to obtain data on the kinematic characteristics of human gait, necessary to create patterns of movement of the rehabilitation device. The simulation results demonstrate that the proposed methods of building a trajectory of movement and modeling the human gait based on the multi-linked foot allow us to repeat the human gait with satisfactory accuracy, taking into account anthropometric parameters of the foot and obtain data for creating rehabilitation patterns of lower limb movement. Acknowledgements The article was prepared with the support of the Russian Scientific Foundation project 22-21-00464 “Development of models and control algorithms for biotechnical walking systems”.

References 1. Dynamics of the spread of OA diseases in Russia and in the world. Spine and Joint Health. https://spinet.ru/public/dinamika_rasprostraneniy_oda.php, 14 Jun 2022 2. Vorobev, A.A., Petrukhin, A.V., Zasypkina, O.A., Krivonozhkina, P.S.: Exoskeleton as a new tool for habilitation and rehabilitation of people with disabilities (review). Modern Technol. Med. 7(2), 51–62 (2015). (In Russ.) 3. Shakti, D., Mathew, L., Kumar, N., Kataria, C.: Effectiveness of robo-assisted lower limb rehabilitation for spastic patients: a systematic review. Biosens. Bioelectron. 117, 403–415 (2018) 4. Zhang, X., Yue, Z., Wang, J.: Robotics in lower-limb rehabilitation after stroke. Behav. Neurol. (2017) 5. Cafolla, D., Russo, M., Carbone, G.: CUBE, a cable-driven device for limb rehabilitation. J. Bionic Eng. 16(3), 492–502 (2019) 6. Shi, D., Zhang, W., Zhang, W., Ding, X.: A review on lower limb rehabilitation exoskeleton robots. Chin. J. Mech. Eng. 32(1), 1–11 (2019) 7. Yatsun, S.F., Yatsun, A.S., Korenevsky, N.A.: Experience in designing rehabilitation exoskeletons. Med. Eng. 3, 48–51 (2017) 8. Charles, J.P., Grant, B., D’Août, K., Bates, K.T.: Foot anatomy, walking energetics, and the evolution of human bipedalism. J. Hum. Evol. 156, 103014 (2021)

18

A. Malchikov et al.

9. Hannink, J., Ollenschläger, M., Kluge, F., Roth, N., Klucken, J., Eskofier, B.M.: Benchmarking foot trajectory estimation methods for mobile gait analysis. Sensors 17(9), 1940 (2017) 10. Tracker Video Analysis and Modeling Tool. https://physlets.org/tracker/, 14 Jun 2022 11. Skvortsov, D.V.: Clinical analysis of movements. In: Gait Analysis (1996) (In Russ.) 12. Mikheev, S.E.: On function smoothing. In: Proceedings of the Karelian Scientific Center of the Russian Academy of Sciences (2014) (In Russ.) 13. Smooth MathWorks. https://www.mathworks.com/help/curvefit/smooth.html, 14 Jun 2022 14. Polyfit MathWorks. https://www.mathworks.com/help/matlab/ref/polyfit.html, 14 Jun 2022 15. Vershik, A.M., Malozemov, V.N., Pevnyi, A.B.: The best piecewise polynomial approximation. Sib. 16(5), 925–938 (1975). (In Russ.) 16. Richter, C., Bry, A., Roy, N.: Polynomial trajectory planning for aggressive quadrotor flight in dense indoor environments. Robot. Res., 649–666 (2016) 17. Koltygin, D.S., Sedelnikov, I.A., Petukhov, N.V.: Analytical and numerical methods for solving the inverse kinematics problem for the robot Delta. Bull. Irkutsk State Tech. Univ. 21(5), 124 (2017) 18. Shmalko, E., Rumyantsev, Y., Baynazarov, R., Yamshanov, K.: Identification of neural network model of robot to solve the optimal control problem. Inf. Autom. 20(6), 1254–1278 (2021). https://doi.org/10.15622/ia.20.6.3 19. Yatsun, S.V., Al Manji, H.H.M., Postolny, A.A., Yatsun, A.S.: Modeling of gait patterns of patients with musculoskeletal system damage using an exoskeleton. Izvestia Southwestern State Univ. 23(6), 176–188 (2020) (In Russ.) 20. Kapustin, A.V., Loskutov, Yu.V., Kudriavtsev, I.A., Belogusev, V.N.: Ways to maintain a stable position of a rehabilitative medical exoskeleton while walking. Bull. Volga Region State Technol. Uni. Ser.: Mater. Des. Technol. 3, 44–54 (2018) 21. Spong, M.W., Hutchinson, S., Vidyasagar, M.: Robot Modeling and Control, vol. 3, pp. 75–118. Wiley, New York (2006)

Chapter 2

Mathematical Modeling of the Biomechanical Rehabilitation System of Foot Exoskeleton in Frontal and Sagittal Planes Andrey Knyazev , Andrey Jatsun , and Andrey Fedorov Abstract Ankle injuries are the most common injuries to the lower extremities. Despite the close attention of traumatologists, both abroad and in our country, the problem of treating injuries of the ankle joint remains unresolved. This is due to the difficulties of treatment, the frequent occurrence of complications and not always positive results, which were noted in more than one third of patients. Therefore, the relevance of the apparatus for mechanotherapy of the ankle joint is high. It is necessary for the rehabilitation complex to be able to change the position of a person’s foot, thereby performing the trajectories of movement chosen by the rehabilitation therapist, which are necessary for the restoration of the joint. Due to the observance of the correct angles of inclination of the patient’s foot, as well as its precise positioning, rehabilitation should take place faster and better than therapy without a specialized apparatus. Such a device can be in demand both among professional athletes who are prone to injuries of this type, and in trauma departments of hospitals. The purpose of this study is to create a rehabilitation device that provides programmable spatial movement of the foot around a virtual hinge.

2.1 Introduction Injuries of the ankle joint are the most common among injuries of the lower extremities. Despite the close attention of traumatologists, both in our country and abroad, the problem of ankle injuries treatment remains unresolved to the end [1–8]. Therefore, the relevance of ankle mechanotherapy apparatus is high. It is essential that the rehabilitation unit is able to change the position of the person’s foot, thereby performing the trajectories of movement chosen by the rehabilitation therapist, which are necessary for the restoration of the joint. Due to the correct angle of the patient’s foot, as well as its precise positioning, rehabilitation will be faster and better than therapy without a special device. Such a device may A. Knyazev (B) · A. Jatsun · A. Fedorov Southwest State University, St. 50 Years of October, 94, 305040 Kursk, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_2

19

20

A. Knyazev et al.

be in demand both by professional athletes exposed to this type of injury and in the trauma departments of hospitals [9–14]. There are a large number of devices for ankle rehabilitation: ORMED FLEX– 02 (Russia) [15], ARTROMOT SP3 (Germany) [16], Kinetec Breva Ankle CPM (Russia) [17]. The disadvantage of the above devices is the relative displacement of the ankle joint during operation. This can negatively affect the process of rehabilitation with the help of hardware [18–21]. The aim of the work is to improve the efficiency of the rehabilitation process by developing and creating an ankle rehabilitation system (RAS) equipped with a virtual joint that provides precise positioning of the rotation axes relative to the ankle joint. To improve the rehabilitation process, it is essential that the rotation axes of the joints intersect with the center of the ankle joint [22–24]. Platform positioning accuracy for devices with rotational motion actuators is comparable to devices equipped with translational motion actuators. However, when using translational motion actuators as an actuator, it is possible to move the platform linearly along the vertical axis, which can improve the quality of rehabilitation.

2.2 The Structural Diagram of the Ankle Rehabilitation System The kinematic capabilities of the ankle rehabilitation system (RAS) are focused on turning the controlled platform in the frontal and sagittal planes. RAS (Fig. 2.1) consists of a platform connected to the device frame by linear electric actuators by means of articulated joints of different classes. The actuators are connected to the framework by means of hinges A1 , B1 , C 1 . Actuators are connected to the platform by means of joints A, B, C. The coordinate system 0XYZ is fixed. The coordinate system 01 X 1 Y 1 Z 1 is related to the center of the ankle joint, and the coordinate system 02 X 2 Y 2 Z 2 shows the deviation of the ankle joint from the initial position by angles ψ, ϕ.

2.2.1 The Kinematic Diagram The kinematic diagram of the RAS is shown in Fig. 2.1. By changing the lengths of the actuators, it is possible to change the position of the platform, respectively, and the patient’s foot. Joints A, B, C, A1 , B1 are two-coordinate, joint C 1 , in turn, is one-coordinate, able to change its position in the sagittal plane. This system thus forms a parallel manipulator. In order for the system to work properly, the axes of the joints (A, B, C) must intersect the center of the ankle joint at point O2 , in three planes. Point O2 must stay

2 Mathematical Modeling of the Biomechanical Rehabilitation System …

21

Fig. 2.1 Kinematic diagram of the device. The figure shows the following designations: 1 is patient’s foot; 2 is ankle joint (virtual joint); 3 is patient’s tibia; 4 is controlled platform; 5 is linear actuator 1 (l1 ); 6 is linear actuator 2 (l2 ); 7 is linear actuator 3 (l3 ); and 8 is platform in the base position

stationary and, respectively, all movements of the platform must be around this point. Thus, point O2 will be a virtual joint.

2.3 Angle of Rotation Ranges of the Foot The ankle joint is formed by the tibia and talus bones. The articular surfaces of the tibia bones and their ankles encompass the talus block like a fork. In this joint, flexion (movement toward the plantar surface of the foot) and extension (movement toward the dorsal surface of the foot) are possible around the transverse axis passing through the talus block. The subtalar joint is responsible for pronation (rotation of the foot with outward rotation of the sole) and supination (rotation of the foot with inward rotation of the sole) of the foot [4]. The human foot can perform movements in three planes, but the rotation around the vertical axis is performed by rotating the tibia, so when working with a machine with the shin fixed, you can get new injuries. The ranges of foot rotation angles for a healthy person are shown in Fig. 2.2. Plantar flexion—45°; dorsiflexion—25°; inversion—40°; pronation—30°. The ranges of the patient’s foot rotation angles are determined by physiological features, type and nature of injury. We will take the platform rotation angles as ψ and ϕ, thus, −27, 3◦ ≤ ψ ≤ 44, 6◦ − 27, 3◦ ≤ ψ ≤ 44, 6◦ , −30◦ ≤ ϕ ≤ 40◦ .

22

A. Knyazev et al.

Fig. 2.2 Ranges of rotation angles of the healthy ankle

2.4 Mathematical Model of the Biomechanical System During the operation of the device, there are reaction forces between the patient’s foot and the platform. To account for these forces, it is necessary to introduce a model that will describe the interactions between the foot and the platform.

2.4.1 Mathematical Model of the System in the Sagittal Plane Foot acts on the platform as a distributed load, let’s replace it with two concentrated forces. Let’s assume that the foot is in contact with the platform by the heel and metatarsal bones. Together they form a bone system, which is connected to the plantar side of the foot by muscular tissues. Figure 2.3 shows a diagram of the system. The Z0X coordinate system is global, and the Z 1 01 X 1 coordinate system is local. The rotation angle of the platform is ψ, the rotation angle of the bone system is ψ 1 . Points A and D are at the contact points of the bone system and the platform, points B and C are at the joints of the soft tissues with the bone system. Let us write down the equation of moments for the patient’s foot in the sagittal plane (2.1): Jbs ψ¨ 1 = M R + M(G bs ) − Mr ,

(2.1)

2 Mathematical Modeling of the Biomechanical Rehabilitation System …

23

Fig. 2.3 Schematic of the system in the sagittal plane

where Jbs is moment of inertia of the bone system; M R is the moment created by the support reaction forces; M(G bs ) is the moment created by the gravity of the bone system; Mr is the moment of resistance of the leg muscles. The moment created by the support reaction forces will be as follows (2.2)–(2.4): M R = −M R1 + M R2 ;

(2.2)

M R1 = R1 ∗l1 ;

(2.3)

M R2 = R2 ∗l2 .

(2.4)

Let’s represent the soft tissues as a Kelvin-Voigt model, with viscous and elastic components arranged in parallel. On this basis, we will find the forces R1 , R2 by the formulas (2.5), (2.6):   R1 = c(ψ − ψ1 ) + μ ψ˙ − ψ˙ 1 ;

(2.5)

  R2 = c(ψ1 − ψ) + μ ψ˙ 1 − ψ˙ ,

(2.6)

where c is reduced muscle stiffness coefficient; μ is reduced muscle viscosity coef ficient; (ψ − ψ1 ) is muscle deformity; ψ˙ − ψ˙ 1 is the velocity of muscular tissue deformation. Thus, the equations for finding the deformation, the rate of deformation of the metatarsal and heel bones will look as follows (2.7)–(2.10):

24

A. Knyazev et al.

z 1B = B O2 − AO2 (sin(ψ1 − ψ));

(2.7)

   z˙ 1B = B O2 − AO2 ψ˙ 1 − ψ˙ cos(ψ1 − ψ) ;

(2.8)

z 1D = D O2 − C O2 (sin(ψ − ψ1 ));

(2.9)

   z˙ 1D = D O2 − C O2 ψ˙ − ψ˙ 1 cos(ψ − ψ1 ) .

(2.10)

Let’s write the equation: Jbs ψ¨ 1 = −l1 (c(B O2 − AO2 ) sin(ψ1 − ψ) + μ(B O2 − AO2 )   ψ˙ 1 − ψ˙ cos(ψ1 − ψ)) + l2 (c(D O2 − C O2 ) sin(ψ − ψ1 )   + μ(D O2 − C O2 ) ψ˙ − ψ˙ 1 cos(ψ − ψ1 ) + MG − Mr . This expression is a nonlinear inhomogeneous differential equation. To linearize such an equation, the following transformations must be performed. For small values of the angle difference (ψ1 − ψ), the sinus of the difference of these angles will be equal to this difference, and the cosine of this difference may be taken as 1. In the expression (ψ − ψ1 ), the sinus of this difference is taken as the difference of these angles with a negative sign, and the cosine may also be taken as 1. Thus, after the above-described transformations, the following equation was obtained (2.11): Jbs ψ¨ 1 + (l1 μB O2 − l1 μ AO2 + l2 μD O2 − l2 μC O2 )ψ˙ 1 + (l1 cB O2 − l1 c AO2 +l2 cD O2 + l2 cC O2 )ψ1 = (l1 μB O2 − l1 μAO2 + l2 μD O2 − l2 μC O2 )ψ˙ +(l1 cB O2 − l1 c AO2 − l2 cD O2 − l2 cC O2 )ψ + MG − Mr (2.11) Investigation of foot motion relative to ankle joint with a given platform motion in the sagittal plane. Let us investigate the motion of the foot relative to the ankle joint using the mathematical model (2.11). A computational experiment method was used. The input data  are  the rotation angle of the platform (ψ) and the angular velocity of the platform ψ˙ . In the right part of Eq. (2.11), there is a variable ψ, it has a polynomial character and is written as follows, function graphs are shown in Figs. 2.4 and 2.5: ψ(t) =

n=5 

ai t i ;

i=0

˙ ψ(t) =

n=5  i=0

ai it i−1 .

2 Mathematical Modeling of the Biomechanical Rehabilitation System …

25

The output data are the rotation angle of the bone system (ψ1 ) and the angular velocity of the bone system (ψ˙ 1 ). Thus, the solution of Eq. (2.11) after appropriate transformations is shown below. ψ1 (t) = C1 e−1.1t + C2 e1.3t − 18.72t 2 − 6.06t 3 + 46.53t 4 − 26.1t 5 − 11.22 ψ˙ 1 (t) = −1.1C1 e−1.1t + 1.3C2 e1.3t − 37.44t − 18.18t 2 + 186.12t 3 − 130.5t 4 − 11.22

Fig. 2.4 Plots of the rotation angle of the platform and bone system on time in the sagittal plane. Plot of the rotation angle error

Fig. 2.5 Plots of the angular velocity of the platform and bone system on time in the sagittal plane. Plot of the angular velocity error

26 Table 2.1 Initial conditions integration for the model in the sagittal plane

A. Knyazev et al. t =0 ψ1 = 0

t =0 ψ˙ 1 = 0

Based on the resulting formulas, let us find the integration constants. The initial conditions are shown in Table 2.1. Thus, the integration constants: C1 = 10.62; C2 = 1.58.

2.5 Experimental Results of Foot Motion in the Sagittal Plane Figure 2.4 shows the dependence of the platform rotation angle on time, as well as the dependence of the bone system rotation angle on time and the graph of changes in the rotation angle error. In Fig. 2.5, you can see time dependence of angular velocity of the platform, as well as time dependence of angular velocity of the bone system and graph of angular velocity error variation. So, given Eqs. (2.5), (2.6), we got dependences of support reaction forces R1 , R2 from time. Dependencies are shown in Fig. 2.6.

2.5.1 Investigation of the Mathematical Model in the Frontal Plane Figure 2.7 shows a diagram of the system in the frontal plane.

Fig. 2.6 Dependences of support reaction forces R1 , R2 on time

2 Mathematical Modeling of the Biomechanical Rehabilitation System …

27

Fig. 2.7 Diagram of the system in the frontal plane

The coordinate system Z0Y is global and the coordinate system Z 1 01 Y 1 is local. The angle of rotation of the platform is ϕ, the angle of rotation of the bone system is ϕ 1 . Points F and H are at the points of contact between the bone system and the platform; points E and G are at the joint between the soft tissue and the bone system. Let us write down the equations of moments for the patient’s foot in the frontal plane: Jbs ϕ¨1 = M R + M(G bs ) − Mr .

(2.12)

The moment created by the support reaction forces will be as follows: M R = −M R3 + M R4 ; M R3 = R3 ∗l3 ; M R4 = R2 ∗l4 . To find the support reaction torque, we represent the muscle tissues as a KelvinVoigt model, with viscous and elastic components placed in parallel. On this basis, we find the support reaction forces using the following equations: R3 = c(ϕ − ϕ1 ) + μ(ϕ˙ − ϕ˙1 );

(2.13)

R4 = c(ϕ1 − ϕ) + μ(ϕ˙1 − ϕ). ˙

(2.14)

Thus, the equations for finding the deformation, the rate of deformation of the bone system will look like this:

28

A. Knyazev et al.

z 1E = E O2 − F O2 (sin(ϕ1 − ϕ)); z˙ 1E = E O2 − F O2 ((ϕ˙1 − ϕ) ˙ cos(ϕ1 − ϕ)); z 1G = H O2 − G O2 (sin(ϕ − ϕ1 )); z˙ 1G = H O2 − G O2 ((ϕ˙ − ϕ˙1 ) cos(ϕ − ϕ1 )). Let’s write the Eq. (2.12): Jbs ϕ¨1 = −l3 (c(E O2 − F O2 ) sin(ϕ1 − ϕ) + μ(E O2 − F O2 ) ˙ cos(ϕ1 − ϕ)) + l4 (c(H O2 − G O2 ) sin(ϕ − ϕ1 ) (ϕ˙1 − ϕ) + μ(H O2 − D O2 )(ϕ˙ − ϕ˙1 ) cos(ϕ − ϕ1 ) + MG − Mr . For small values of the angle difference (ϕ1 − ϕ), the sinus of the difference of these angles will be equal to this difference, and the cosine of this difference can be taken as 1. In the expression (ϕ − ϕ1 ), the sinus of this difference is taken as the difference of these angles with a negative sign, and the cosine can also be taken as 1. Thus, after the above transformations, the following equation was obtained. Jbs ϕ¨1 + (l3 μE O2 − l3 μF O2 + l4 μH O2 − l4 μG O2 )ϕ˙1 + (l3 cE O2 − l3 cF O2 − l4 cH O2 + l4 cG O2 )ϕ1 = (l3 μE O2 − l3 μF O2 + l4 μH O2 − l4 μG O2 )ϕ˙ (2.15) + (−l3 cE O2 − l3 cF O2 − l4 cH O2 + l4 cG O2 )ϕ + MG − Mr . Investigation of motion of the foot relative to the ankle joint for a given platform motion in the sagittal plane. Let us investigate the motion of the foot relative to the ankle joint using the mathematical model (2.15). A computational experiment method was used. The inputs are the rotation angle of the platform (ϕ) and the angular velocity of the platform (ϕ). ˙ In the right part of the equation there is a variable ϕ. It has a polynomial character and is written as follows, the graphs of functions are shown in Figs. 2.8 and 2.9: ϕ(t) =

n=5 

ai t i ;

i=0

ϕ(t) ˙ =

n=5 

ai it i−1 .

i=0

The output data are the rotation angle of the bone system (ϕ1 ) and the angular velocity of the bone system (ϕ˙1 ). Thus, after appropriate transformations, the laws of rotation angle and rotation velocity of the bone system are described below.

2 Mathematical Modeling of the Biomechanical Rehabilitation System …

29

Fig. 2.8 Plots of the rotation angle of the platform and bone system on time in the frontal plane. Graph of the rotation angle error

Fig. 2.9 Plots of the angular velocity of the platform and bone system on time in the frontal plane. Diagram of angular velocity error

ϕ1 (t) = C1 e−79.16t + C2 e2.98t + 28.8t 2 − 87.6t 3 + 73.8t 4 − 18t 5 − 4.18; ϕ˙1 (t) = −79.16C1 e−79.16t + 2.98C2 e2.98t + 57.6t − 262.8t 2 + 295.2t 3 − 90t 4 − 4.18.

Based on the resulting formulas, let us find the integration constants. The initial conditions are shown in Table 2.2. Thus, the integration constants: C1 = 4.078; C2 = 0.102.

30 Table 2.2 Initial conditions integration for the model in the frontal plane

A. Knyazev et al. t =0

t =0

ϕ1 = 0

ϕ˙1 = 0

Fig. 2.10 Dependences of support reaction forces R3 , R4 on time

2.5.2 Experimental Results of Foot Movement in the Frontal Plane Figure 2.8 shows the dependence of the platform angle of rotation on time, as well as the dependence of the angle of rotation of the bone system on time and a graph of the change in the error of the angle of rotation in the frontal plane. In Fig. 2.9, you can see the dependence of angular velocity of the platform on time, as well as the dependence of angular velocity of the bone system on time and a graph of change in the error of angular velocity in the frontal plane. Thus, based on Eqs. (2.13) and (2.14), we obtained dependencies of support reaction forces R3 , R4 on time. Dependencies are shown in Fig. 2.10.

2.6 Conclusion This work has described the physiological features of the ankle joint. The limiting angles of rotation of the joint, the location and dimensions of the bone system, the location of the ankle joint, and the stiffness and toughness of the muscles have been considered. These data were used to synthesize a mathematical model. Next, a scheme and description of the device, which is a parallel manipulator, the main idea of which,

2 Mathematical Modeling of the Biomechanical Rehabilitation System …

31

is to implement the concept of a virtual joint was presented. A mathematical model of the biomechanical system for the frontal and sagittal planes was also compiled and investigated. On the basis of the linearized model, we can get the values of support reaction forces arising from the deformation of the ankle joint muscular system. As a result of the work, we obtained the functions of changes in the foot rotation angles and angular velocities, the support reaction forces, on the basis of which we plotted the angle and angular velocity versus time, the angle and angular velocity error versus time, and the support reaction forces plots. Acknowledgements The article was prepared with the support of the Strategic Project “Priority2030. Creation of robotic tools to expand human functionality”.

References 1. Wang, C., Fang, Y., Guo, S., Zhou, C.: Design and kinematic analysis of redundantly actuated parallel mechanismsfor ankle rehabilitation. Robotica 33(2), 366–384 (2015) 2. Zeng, D., Wu, H., Zhao, X., Lu, W., Luo, X.: A new type of ankle-foot rehabilitation robotbased on muscle motor characteristics. IEEE Trans. Autom. Sci. Eng. 8, 189–195 (2020) 3. Antonellis, P., Galle, S., Clercq, D.D., Malcolm, P.: Altering gait variability with an ankle exoskeleton 13(10) (2018) 4. Alvarez-Perez, M.G., Garcia-Murillo, M.A., Cervantes-Sánchez, J.J.: Robot-assisted ankle rehabilitation: a review, disability and rehabilitation. Disabil. Rehabil. Assist. Technol. 15(4), 394–408 (2020) 5. Zhang, L., Li, J., Dong, M, Fang, B., Cui, Y., Zuo, S., Zhang, K.: Design and workspace analysis of a parallel ankle rehabilitation robot (PARR). J. Healthcare Eng. 10 (2019) 6. Jamwal, P.K., Xie, S., Aw, K.C.: Kinematic design optimization of a parallel ankle rehabilitation robot using modified genetic algorithm. Robot. Auton. Syst. 57(10), 1018–1027 (2009) 7. Vallés, M., Cazalilla, J., Valera, Á., Mata, V., Page, Á., Díaz-Rodríguez, M.: A 3-PRS parallel manipulator for ankle rehabilitation: towards a low-cost robotic rehabilitation. Robotica 35(10), 1939–1957 (2017) 8. Zhang, M., McDaid, A., Veale, A.J., Peng, Y., Xie, S.Q.: Adaptive robot with trajectory tracking control of a parallel ankle rehabilitation joint-space force distribution 7, 85812–85820 (2019) 9. Jamwal, P., Hussain, S., Xie, S.: Restage design analysis and multicriteria optimization of a parallel ankle rehabilitation robot using genetic algorithm. IEEE Trans. Autom. Sci. Eng. 12(4), 1433–1446 (2014) 10. Khalid, Y.M., Gouwanda, D., Parasuraman, S.: A review on the mechanical design elements of ankle rehabilitation robot. Proc. Inst. Mech. Eng. [H] 229(6), 452–463 (2015) 11. Zeng, X., Zhu, G., Zhang, M., Xie, S.Q.: Reviewing clinical effectiveness of active training strategies of platform-Based ankle rehabilitation robots. J. Healthcare Eng. 1–12 (2018) 12. Wang, C., Fang, Y., Guo, S.: Multi-objective optimization of a parallel ankle rehabilitation robot using modified differential evolution algorithm. Chinese J. Mech. Eng. 28(4), 702–715 (2015) 13. Hassan, M., Khajepour, A.: Optimization of actuator forces in cable-based parallel manipulators using convex analysis. IEEE Trans. Rob. 24, 736–740 (2008) 14. Farjadian, A.B., Nabian, M., Hartman, A., Corsino, J., Mavroidis, C., Holden, M. K.: Position versus force control: using the 2-DOF robotic ankle trainer to assess ankle’s motor control. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 1186–1189 (2014)

32

A. Knyazev et al.

15. ORMED FLEX 02 ankle apparatus. https://www.ormed.ru/katalog/passivnaya-reabilitatsiya/ ormed-flex-02-dlya-golenostopnogo-sustava/ 08 May 2022 16. ARTROMOT SP3 trainer for continuous passive development of the ankle joint. https://rea med.su/catalog/product/artromot-sp3/ 08 May 2022 17. Rehabilitation simulator for passive development of the ankle joint. https://www.beka.ru/ru/ katalog/domashnyaya-reabilitatsiya/kinetec-breva-ankle-cpm/ 08 May 2022 18. Anatomy of the foot. https://www.sportmedicine.ru/foot_anatom.php/ 08 May 2022 19. Jatsun, S.M., Jatsun, A.S., Korenevskiy, N.: Experience in the development of rehabilitation exoskeletons. Biomed. Eng. 51(3) (2017) 20. Zhang, M., Davies, T.C., Xie, S.: Effectiveness of robot-assisted therapy on ankle rehabilitation—a systematic review. J. Neuroeng. Rehabil. 10(1), 1–16 (2013) 21. Li, J., Fan, W., Dong, M.: Research on control strategies for ankle rehabilitation using parallel mechanism. Cognitive Comput. Syst. 2(3), 105–111 (2020) 22. Jatsun, S.M., Jatsun, A.S., Rukavitsyn, A.N., Politov, E.N.: New approaches to rehabilitation of the ankle joint using a mechanotherapeutic apparatus. Biomed. Eng. 52, 37–41 (2018) 23. Nakayama, A., Ruelas, D., Savage, J., Bribiesca, E.: Teleoperated service robot with an immersive mixed reality interface. Inf. Automat. 20(6), 1187–1223 (2021). https://doi.org/10.15622/ ia.20.6.1 24. Jatsun, S.M., Jatsun, A.S., Rukavitsyn, A.N.: Designing a mechanotherapy device for rehabilitation of lower extremities of humans. Biomed. Eng. 50, 128–133 (2016)

Chapter 3

Simulation of Controlled Motion of the Actuator of Robotic Systems in the Presence of Coupling Forces Evgeny Subbotin

and Andrey Jatsun

Abstract The article is devoted to mathematical modeling of robotic devices in the presence of dry friction with coupling. By dry friction with adhesion, we will understand such drag forces that change the value of the force modulus depending on the velocity. And, the value of the drag force at zero velocity is many times greater than the value of the drag force at non-zero velocity. The appearance of such forces is associated with the action of various external factors, such as corrosion, icing, and adhesion. The need to control the motion of the actuator in the presence of such forces, which are many times greater than the frictional forces acting on the actuator in motion, requires the creation of special means to ensure a given motion of the actuator. The article considers features of RTS control in the presence of coupling forces, as well as issues of synthesis of parameters of electric drive control system in the presence of coupling forces, and also gives an example of application of the developed theory for control of a robotic device for correct switching a 35–110 kV disconnector.

3.1 Mathematical Model of Coupling Forces At the moment, the friction effect has been considered in many scientific papers. There are several mathematical descriptions of the friction process itself, which quite accurately describe the complex phenomena occurring in reality [1–7]. One of the most common is the Amonton-Coulomb dry friction model. It is often used for mathematical modeling of the phenomenon of movement of mechanisms and further development of automatic control system, taking into account friction forces, thanks to which it is possible to reduce the static error and achieve more accurate control [8–12]. However, very often, in addition to the friction force directly present during motion, there may be a friction force during rest. Such frictional forces occur due to E. Subbotin · A. Jatsun (B) Southwest State University, 50 let Oktyabrya, 94, Kursk, Russia e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_3

33

34

E. Subbotin and A. Jatsun

external factors such as corrosion, icing, etc., and can be much greater than frictional forces: FFr1  FFr2 . The ratio of the friction force of rest to the friction force of motion is much greater than 1: FFr1  1. FFr2 Experiments have shown that this ratio can be in the range of 87–100. This in turn creates the need to separately consider these forces in mathematical modeling and automatic control system design. At the beginning of the movement of the mechanism due to the clutch forces, it is necessary to apply much more torque than during the movement itself. Hence, special motion control algorithms that control the position of the actuator are needed to overcome them [13–15]. Figure 3.1 shows a graph of dry friction showing the change in forces with respect to velocity. At the very beginning of the motion there is a rest friction force FFr1 , in overcoming which only the friction force of motion FFr2 will act when the speed is further increased. Figure 3.2 shows the same graph, but with the addition of the viscous friction component. In this case the component FFr1 remains unchanged, and FFr2 begins to grow with increasing velocity. Fig. 3.1 Diagram of friction forces change by velocity

Fig. 3.2 Diagram of friction forces with a viscous component on the velocity

3 Simulation of Controlled Motion of the Actuator of Robotic Systems …

35

Fig. 3.3 Picture of the connection: a connection of mechanism elements, b connection of the part to the surface, c connection on the working body

During the clutch moment, the resting friction force acts on the actuating elements of the mechanism. At the beginning of the movement dry friction acts, the model of which is presented below: ⎧ ˙ ⎨ N f sign(ϕ), FFr2 = F0 , , ⎩ N f sign(F0 ), where N is the normal support reaction, f is the friction coefficient, F 0 is the resultant of all forces except friction forces, ϕ˙ is the speed. The connection that occurs between the working parts of the mechanism can be formed for a number of reasons, for example, heavy icing inherent in the Arctic latitudes. In addition, such connections can occur during various technical operations, such as lifting any parts from the surface, moving them, as well as directly between the links of the working body of the device (Fig. 3.3). The theory of damage accumulation has been applied to describe the failure process of the connection that occurs between the actuating links of the mechanism. It is used to describe fatigue failures that occur as a result of alternating loads and lead to the formation and development of cracks. It is based on the kinetic equation of the form: dψ = f (ψ), dt where ψ is the continuity of the body, varying from 1 to 0, where at 1 there is no damage and at 0 the body is destroyed. To simulate damage accumulation, a simulation model is used that takes into account all the influences applied to the joint, such as the impulse generated by the

36

E. Subbotin and A. Jatsun

Fig. 3.4 Simulation model structure

striker, the frequency of the striker, and the constant torque generated by the motor. The structure of the model is shown in Fig. 3.4. The result of all influences is the continuity index of the body, which shows the amount of damage and the moment in which it occurred. The transition function of this model will look as follows: ψ=

C1 C2 C0 + + , Q ω Mdr

where Q is striker inertia, ω is beating frequency, M dr is constant torque of electric drive, C 0 , C 1 , C 2 are coefficients selected experimentally. The inertia of the striker varies depending on its speed up to the moment of impact with the working surface: Q = m x, ˙ where m is the mass of the striker, x˙ is the striker velocity.

3.2 Calculation Scheme of the Interaction of the Device with the Environment and Nodes In this paper, a mathematical model of robotic system used in conditions of corrosion, icing, and other factors was created. A mathematical description of these phenomena is given. For this purpose, an upgraded model of dry friction of Amontont-Coulomb was applied. The general diagram of the rotational transmission link is presented in Fig. 3.5. The device has two modes: static mode and dynamic mode. In the static mode, the device overcomes the moment of resistance created by the clutch forces resulting from the disconnector operating conditions due to the joint operation of the rotary motion actuator and the striking striker. In this mode, there is no movement, but the striker module mounted on the actuator creates the impact force necessary to overcome the initial resistance. After breaking the connection and overcoming the maximum torque, the main action mode begins, in which the system starts moving at the nominal torque M c1 . The striker itself operates on the basis of an electromagnet whose core is suspended on a spring, which provides the oscillating action.

3 Simulation of Controlled Motion of the Actuator of Robotic Systems …

37

Fig. 3.5 Schematic diagram of the rotation transfer link of the robotic device: 1, 2 are elements of the working body of the device, 3 is synchronizing link, and 4 is module of the striker

3.3 Mathematical Expressions for Modeling Two modes of operation are applicable to the system: 1. Static mode of the mechanism ϕ˙1 = 0. At the beginning of the static mode, the clutch force is destroyed. This is achieved by the impact of the striker module. In this position, there is no motion, so the sum of all moments acting in the system will be zero. The equation of moments: 

Mi = 0.

i=1

In this case, the main generalized drag force FFr1 and the impact force FS will act on the system. As a result, the equation of moments will look like this: FFr1 l − FS l1 − Mdr = 0. Here, F S is the thrust force with which the core of the striker strikes to break the connection. FS =

dWm = (B H/2)S, dx

where W m is magnetic energy, B is induction of electromagnet, H is magnetic field strength, and S is cross-sectional area. The hammer strikes periodically according to a given law:

38

E. Subbotin and A. Jatsun

m x¨ + u x˙ + cx = FS , where m is the mass of the striker, u is the viscosity factor, c is the spring stiffness factor. Let us transform this equation by dividing all parts by the component m: x¨ + 2n x˙ + k 2 x = f S , where k 2 = c/m is the square of the natural frequency. The moment the striker comes into contact with the tool element, it is subjected to an impact effect. 2. Mechanism dynamics mode ϕ˙1 = 0. The equation of moments: J ϕ¨ =

3 

Mi ,

(3.1)

i=1

where J is the equivalent moment of  inertia of the system, ϕ—is ¨ the angular 3 Mi —is the sum of system moments. acceleration of the disconnector blades, i=1 Let us indicate the moments of the system in Eq. (3.2): J ϕ¨ = Mred − Mc ,

(3.2)

where M red is the drive output torque, M c is the drag torque when the system moves. Let’s express the drive torque M red through the current I: Mred = Mdr n = C M I n,

(3.3)

where M red is the torque produced by the electric motor, n—is the gear ratio of all drive gears, C M —torque constant, I—current supplied to the motor windings. The law of the total current of the electric motor winding: L

dI + I R + Cω ωdr = U, dt

(3.4)

where L is the inductance of the electric motor, R is the active resistance of the electric motor, Cω —speed constant of the electric motor, ωdr —angular speed of the electric motor, and U—system voltage. Equations (3.2), (3.3) and (3.4) represent a system describing the motion of the device:

3 Simulation of Controlled Motion of the Actuator of Robotic Systems …

39

Fig. 3.6 Block diagram of the device control system

J ϕ¨ = Mred − Mc ; Mred = C M I n; dI L + I R + Cω ωdr = U. dt

(3.5)

3.4 Block Diagram of the Device Control System Figure 3.6 shows a schematic diagram of the automatic control system of the device. The computational module of the control system works according to the algorithm shown in Fig. 3.7. According to it, initially the main motor starts to work, which creates a constant torque. If there is no movement after the start of operation, the striker module, which is an electromagnetic drive, starts to operate. It creates an impulse action to break the connections that have arisen. If there is speed, the striker module stops working and the main motor continues to operate.

3.5 Description of the Mathematical Model of the Automatic Control System We investigate the control of the main drive of the device. The system works according to a system of equations: ⎧ J ϕ¨ = Mred − Mc ; ⎪ ⎪ ⎨ Mred = C M I n; ⎪ L dI + I R + Cω ωdr = U ; ⎪ ⎩ dt U = U (ϕ).

(3.6)

40

E. Subbotin and A. Jatsun

Fig. 3.7 Algorithm of device operation

The control in the main action mode is based on the angle ϕ ∗ , which is set by law and can range from 90° to 190°: ϕ∗ =

5 

ai t i ,

(3.7)

i=1

where ai are coefficients of the polynomial, t is time, and i is the ordinal number of the polynomial elements. Use the 5th order polynomial to set the change in the angle over time. It will look like the graph shown in Fig. 3.8. The system is regulated in the P block and is controlled by a proportionaldifferential regulator:

3 Simulation of Controlled Motion of the Actuator of Robotic Systems …

41

Fig. 3.8 Diagram of the desired angle change over time for the main action mode

Fig. 3.9 Diagram of the change in the value ψ

 (ϕ ∗ − ϕ) , U = k p ϕ ∗ − ϕ + kd t

(3.8)

where U—setting voltage, k p —the proportional coefficient, ϕ ∗ —desired angle, ϕ— angle actual, kd —the differential coefficient, t—time. Research results and conclusions. Figure 3.9 shows a graph of the change in the value of ψ. Consider the simulation results for a given finite angle of 90°. Figure 3.10 shows a graph of the change in the angle of the mechanism after overcoming the initial resistance.

3.6 Conclusions As a result of mathematical modeling, we developed a mathematical model of the device, taking into account the friction and drag forces, as well as the properties of the

42

E. Subbotin and A. Jatsun

Fig. 3.10 Diagram of changes in the angle of the mechanism

electric motor, revealed the need to introduce a regulator smoothing its work in the system. It is also necessary to set the desired change in the angle by some law, limiting the stroke of the mechanism to a certain value required by the operator. Scheme of the considered device for switching the contact group is shown in Fig. 3.11. The results of theoretical research have made it possible to create a robotic module for a 35–110 kV high-voltage disconnector. The transfer of rotary motion to the disconnector is carried out by means of the rotation transmission mechanism, the scheme of which is shown in Fig. 3.11. A distinctive feature of this device is that the rotation drive, which provides rotation of the executive body at a given angle additionally contains a special shock hammer, which allows to break the connection resulting from adhesion or other factors and eliminates the movement of the executive elements of the device.

3 Simulation of Controlled Motion of the Actuator of Robotic Systems …

43

Fig. 3.11 Schematic diagram of the device, where 1—motor, 2—gearbox, 3—cable transmission, 4—manual drive, 5—rotation transmission mechanism, 6—executive element of the robotic device consisting of two parts, 7—beater module, and 8—connection

Acknowledgements The article was prepared with the support of the Strategic Project “Priority2030. Creation of robotic tools to expand the functionality of a person”.

References 1. Phuong, T.H., Belov, M.P., Van Thuy, D.: Adaptive model predictive control for nonlinear elastic electrical transmission servo drives. In: 2019 IEEE NW Russia Young Researchers in Electrical and Electronic Engineering Conference (ElConRusNW), pp. 704–708 (2019) 2. Szabat, K., Serkies, P., Nalepa, R., Cychowski, M.: Predictive position control of elastic dualmass drives under torque and speed constraints. In: 14th International Power Electronics and Motion Control Conference EPE-PEMC 2010, pp. 79–83 (2010) 3. Safarov, D.I., Jatsun, S.F.: Vibrating technological processes of wastewater treatment (2004) 4. Kireenkov, A.A., Semendyaev, S.V., Filatov, V.F.: Experimental study of coupled twodimensional models of sliding and twisting friction. Izvestia RAN MTT 6, 192–202 (2010) 5. Kireenkov, A.A.: Generalized two-dimensional model of sliding and twisting friction. Dokl. Phys. 431(4), 482–486 (2010) 6. Pennestrì, E., Rossi, V., Salvini, P., Valentini, P.P.: Review and comparison of dry friction force models. Nonlinear Dyn. 83(4), 1785–1801 (2016) 7. Al-Bender, F.: Fundamentals of friction modeling. In: ASPE Spring Topical Meeting on Control of Precision Systems (2010) 8. Borello, L., DallaVedova, M.D.L.: Dry friction discontinuous computational algorithms. Int. J. Eng. Innov. Technol. 3(8), 1–8 (2014)

44

E. Subbotin and A. Jatsun

9. Shmalko, E., Rumyantsev, Y., Baynazarov, R., Yamshanov, K.: Identification of neural network model of robot to solve the optimal control problem. Inf. Autom. 20(6), 1254–1278 (2021). https://doi.org/10.15622/ia.20.6.3 10. Do, T.N., Tjahjowidodo, T., Lau, M.W.S., Phee, S.J.: Nonlinear modeling and parameter identification of dynamic friction model in tendon sheath for flexible endoscopic systems. In: 10th International Conference on Informatics in Control, Automation and Robotics, pp. 5–10 (2013) 11. Van Geffen, V.: A study of friction models and friction compensation (2009) 12. Liu, H., Song, X., Nanayakkara, T.: Friction estimation based object surface classification for intelligent manipulation. In: IEEE ICRA 2011 Workshop on Autonomous Grasping (2011) 13. Marques, F., Flores, P., Lankarani, H.M.: On the frictional contacts in multibody system dynamics. In: ECCOMAS Thematic Conference on Multibody Dynamics (2015) 14. Qi, Z., Xu, Y., Luo, X., Yao, S.: Recursive formulations for multibody systems with frictional joints based on the interaction between bodies. Multibody Syst. Dyn. 24, 133–166 (2010) 15. Kudra, G., Awrejcewicz, J.: Approximate modelling of resulting dry friction forces and rolling resistance for elliptic contact shape. Eur. J. Mech. A Solids 42, 358–375 (2013)

Chapter 4

Digital Control by Robot Manipulator with Improved Rigidity Eugene Larkin , Aleksey Bogomolov , and Aleksandr Privalov

Abstract Technological robot, as part of digital production enterprise is considered. It is shown, that exactness and performance of operation execution are the main characteristics, which determine product quality in digitized industry. Manipulator design with pyramidal structure, in which mechanism is actuated with linear drives, placed in parallel, is described. Mathematical model of exactness evaluation, when there is dry friction at actuators, is worked out. It is shown that mechanics parallelism leads to cross links between grasp coordinates control channels. Flowchart of manipulator under investigation, as the object under control, is given. The analytical descriptions of object under control and control algorithm, underlying Von Neumann type controller, are fulfilled with description of delays, linked with features of controller operation, namely, operator-by-operator interpretation of algorithm, unfolding in physical time. For estimation of time delays at interface between controller and manipulator semi-Markov model of Von Neumann computer operation is worked out. The method of estimation lags in contours, based on transformation the ergodic semi-Markov model into non-ergodic one, and computation expectations and dispersions of time interval from-point-to-point wandering through semi-Markov process, is proposed.

E. Larkin Tula State University, Lenin Av. 92, 300012 Tula, Russia A. Bogomolov St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, 39, 14th Line, 199178 St. Petersburg, Russia A. Privalov (B) Tula State Lev Tolstoy Pedagogical University, 125, Building 4, Prospekt Lenina, 300026 Tula, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_4

45

46

E. Larkin et al.

4.1 Introduction Widespread use of robots in digitized technologies is rather modern concept of industry development [1, 2]. In order to digitizing process is been accompanied with improvement the quality of products it is necessary to increase the requirements for execution of operations by robots, involved into technologies [3]. The difficulty lay in the fact, that source of labor objects, processing machine and robot itself are installed on different bases, and there is known awkwardness of measurement space position of robot operational unit and exactly send it to labor object, or to processing machine. Difficulties increase significantly when sequencing manipulators are used. In this case error, generated by the previous link is multiplied on the error of next link that significantly reduces the accuracy of operation execution as a whole [4]. To improve operational unit positioning one should use manipulator as rigid as possible. From this point of view flat triangle, and three dimensional tetrahedron based on four triangles have highest possible rigidity and may be recommended for manipulator design [5]. The increased rigidity of pyramid manipulator under investigation is explained by such kinematics scheme, which has a minimum number of joints and mechanical gears. Another source of manipulator inaccuracy is operation of digital control system, based on Von Neumann controllers which interpret control algorithm sequentially operator-by-operator, unfolding in real physical time [6, 7]. Due to that, computers, born time delays which affects control system performance, hence reduces manipulator exactness during transition process. Methods of managing by Pyramid manipulator with use Von Neumann controllers are not widespread, that confirms necessity and relevancy of investigation in the area.

4.2 Operational Unit Positioning Control System Statics Robotic arms in general, and pyramidal manipulator in particular, belong to a class of technology equipment, in which position of operational unit awkwardly to measure directly, but it may be intermediately estimated by state of its parts, f.e. actuators. Taking into account this fact, common structure of digital control system is shown in Fig. 4.1. The digital control system should include programmable block of actuators desired state l A , l B , l C calculation, according desired Cartesian coordinates x D , y D , z D of operational unit, which may be set by a higher level control system, or by a human operator (in Fig. 4.1a is not shown). Digital control system under investigation physically fulfills desired states l A , l B , l C , and generates real states of actuators lA , lB , l C , which on physical level form real space position of operational unit x D , yD , zD . So, on the exactness of operational unit positioning both kinematics, and dynamics of digital control system has an impact.

4 Digital Control by Robot Manipulator with Improved Rigidity Fig. 4.1 A common structure of DCS a, and the kinematics diagram of the pyramid manipulator b

⎯lA ⎯xD Inverse ⎯l ⎯yD B kinematics ⎯lC ⎯zD task

47 lA

Digital control system kinematics task

lB lC

xD Direct yD kinematics zD task

a

D

z

lB

zD

lA

y

ϑA

b

B

yD O -b

ϑC

lC

MB g

ϑB

MA g a

A С

xD

x

MC g b

The pyramid manipulator kinematics diagram is shown in Fig. 4.1b. Manipulator includes three linear actuators, AD, BD, CD, each placed at its own two-degree hinge A, B, C, on the rigid immovable base ABC. Actuator’s opposite ends converge in the hinge D, physically located on the pyramid top, at the actuator AD. With the base ABC Cartesian coordinate system xOyz is linked, initial points O of which coincide with the hinge A of the actuator AD. Axis Ox of Cartesian system passes through center of symmetry of the isosceles triangle ABC, in which AB = AC, toward the side BC. Axis Oz is the vertical one, and axis Oy complements xOyz till the right coordinate system. At the actuator AD point D manipulator operating unit, having mass M, is rigidly installed, and its coordinates x D , yD , zD manipulator control system must provide during operation. To control coordinates x D , y D , z D one should control linear actuators AD, BD, CD lengths, l A , l B , l C , correspondingly [8, 9]. So, required lengths of linear actuators to obtain coordinates x D , yD , zD are equal to: ⎧ 2 2 2 2 ⎪ ; ⎨l A = x D + yD + zD ( )2 2 2 (4.1) l B = (x D − a) + y D − b + z 2D ; ⎪ ( )2 ⎩ 2 2 2 l C = (x D − a) + y D + b + z D , where a is the distance from the origin point O till the line BC, parallel to the axes Oy; b is the distances from axis Ox till points B and C.

48

E. Larkin et al.

According the inverse kinematics problem (4.1), which in the system is solved algorithmically, the direct kinematics task is solved at the physical level and implies establishing of coordinates x D , yD , zD according values d A , d B , d C achieved: ⎧ ( 2 ) 1 2 + 2l 2A − l 2B − lC2 ; ⎪ ⎨ x D = 4a (2a + 2b ) 1 2 l − l 2B ; yD = / 4b C ⎪ ))2 ( 1 ( 2 ))2 (1( ⎩ z D = l 2A − 4a 2a 2 + 2b2 + 2l 2A − l 2B − lC2 − 4b lC − l 2B .

(4.2)

The static exactness of coordinate x D , yD , zD establishing depends on exactness of parameters a, b, l A , l B , l C maintaining, and may be estimated with use sensitivity functions apparatus [10]. Common formula for determining of functions φ(a, b, l A , l B , lC ), where φ ∈ {x D , y D , z D }, shifting from pre-determined value, when parameters a, b, l A , l B , lC deviate from their nominal values, is as follows: εφ =

5 Σ

ζn εn ,

(4.3)

n=1

where εφ is function deviation; ζ1 = | | ) ( ∂φ | ∂φ | · l , ζ = | A A 4 min max ∂l A ∂l B | l A =l Anom

|

| | · anom , ζ2 = ∂φ · bnom , ζ3 = ∂b |b=b nom | ) ( nom ) ( | · l Bmin Bmax , ζ5 = ∂l∂φC | · lCmin Cmax are

∂φ | ∂a |a=a

l B =l Bnom

sensitivity functions of dimensionality [m]; ε1 = l B −l Bnom l Bmin Bmax

lC −lCnom lCmin Cmax

a−anom , anom

lC =lCnom nom ε2 = a−a , anom

ε3 =

l A −l Anom l Amin Amax

,

, ε5 = are dimensionless deviations of parameters from their ε4 = nominal (nom) values; lCmin Bmin Amin are minimal values of linear actuators lengths; lCmax Bmax Amax are maximal values of l A , l B , l C . A sensitivity functions expanded form have been summarized in Table 4.1. In Table 4.1, formulae parameters a, b, l A , l B , l C are of nominal values, so the table describes main feature of parallel mechanical systems exactness of every parameter influence on the exactness of all functions. To assess the influence of parameters on values x D , yD , zD , one should find the maxima of functions ζn in their domains, that may be done by any of known maximum search methods.

4.3 Operational Unit Positioning Control System Dynamics Dynamics of pyramid manipulator is determined by linear actuator performance, mass of the operational unit, moments of inertia of linear actuators about hinges A, B, C, dissipative forces in the form of viscous friction in the hinges and linear actuators, and hard nonlinearities at the junction of the linear drives and manipulator itself. Every linear drive may be considered as a rod, which rotates in its own hinge in two planes under force actions, generated by two other drives. Rod itself operate in tension, not bending, hinged rod operates for rotation. So forces, applied to the

4 Digital Control by Robot Manipulator with Improved Rigidity

49

Table 4.1 Expanded form of sensitivity functions Parameter

ζn for functions φ ∈ {x D , y D , z D } xD

a

2a 2 −2b2 −2l 2A +l 2B +lC2 4a

yD

zD

0

−ax D ∂aD / l 2A −x 2D −y 2D

l 2 −l 2 − C4b B

[ ] ∂x ∂y −b x D ∂bD +y D ∂bD / l 2A −x 2D −y 2D

∂x

b

b2 a

lA

l 2A a

0

] [ ∂y ∂x l A l A −x D ∂l D −y D ∂l D A A / l 2A −x 2D −y 2D

lB

l2 − 2aB

l2 − 2bB

] [ ∂x ∂y l B l B −x D ∂l D −y D ∂l D B B / l 2A −x 2D −y 2D

lC

lC2 − 2a

lC2 2b

] [ ∂y ∂x lC lC −x D ∂l D −y D ∂l D C C / l 2A −x 2D −y 2D

drive, may be considered as the force, directed along linear actuator rod, and the force, directed perpendicular to linear actuator, as a stick, at that both forces are located in the plane, formed by the drive under investigation, and the acting drive. As an example, in Fig. 4.2 the plane, formed by centerlines of actuators A and B is shown [11]. In Fig. 4.2, lengths of linear actuators A, B are nominated as lA and l B correspondingly; angles of same rods rotation in the plane ABD are nominated as ψ B AD , ψ AB D ; the angle between centerlines of actuators A and B is nominated as ψ AD B ; vector force, acting on the actuator A from the actuator B is nominated as RBA ; longitudinal and transverse components of vector force RBA are nominated as RBA,l , RBA,t correspondingly; longitudinal and transverse components of vector force RBA are nominated as RAB,l , RAB,t correspondingly. Angles between actuators centerlines and the bases of triangles are defined by the sine theorem: Fig. 4.2 Operation of actuators pair in the ABD plane

RBA

RAB,t

RBA,t

lB,w lA,w lA

RBA,l

D ψADB

RAB

RAB,l

ψABD

lB A

ψBAD

lAB

B

50

E. Larkin et al.

lA lB l AB = = for the plane AB D; sin ψ AB D sin ψ B AD sin ψ AD B lA lC l AC = = for the plane AC D; sin ψ AC D sin ψC AD sin ψ ADC lB lC l BC = = for the plane BC D, sin ψ BC D sin ψC B D sin ψ B DC

(4.4)

√ where l AB = l AC = a 2 + b2 l BC = 2b. According to Newton’s third law: ⎧

R B A,l cos ψ B AD + R B A,t sin ψ B AD = −R AB,l cos ψ AB D − R AB,t sin ψ AB D ; R B A,l sin ψ B AD + R B A,t cos ψ B AD = −R AB,l sin ψ AB D + R AB,t cos ψ AB D

for the plane ABC; ⎧

RC A,l cos ψC AD + RC A,t sin ψC AD = −R AC,l cos ψ AC D − R AC,t sin ψ AC D ; RC A,l sin ψC AD + RC A,t cos ψC AD = −R AC,l sin ψ AC D + R AC,t cos ψ AC D (4.5)

for the plane ACD; ⎧

RC B,l cos ψC B D + RC B,t sin ψC B D = −R BC,l cos ψ BC D − R BC,t sin ψ BC D ; , RC B,l sin ψC B D + RC B,t cos ψC B D = −R BC,l sin ψ BC D + R BC,t cos ψ BC D

for the plane BCD. Weights of moving parts of linear actuators W A = (0, 0, M A g), W B = (0, 0, M B g), and W C = (0, 0, M C g), where M A g, M B g, M C g are masses of moving parts; g is acceleration of gravity (Fig. 4.1), are applied to points, situated from origin at the distances lA,w , l B,w , l C,w , along corresponding actuator. Weight vectors are directed opposite to coordinate Oz, and in vertical plane unfold to longitudinal and transversal components as follows: ⎧

W A,l = M A g sin ϑ A ; W A,t = M A g cos ϑ A ;

⎧

W B,l = M B g sin ϑ B ; W B,t = M B g cos ϑ B ;

⎧

WC,l = MC g sin ϑC ; WC,t = MC g cos ϑC ,

(4.6)

where ϑ A , ϑ B , ϑC are elevation angles of corresponding actuators. Longitudinal components create loads to liner actuators stocks, while transversal components rotate actuators, as a rod, at the proper hinges. Due to rotations are considered in the ABD, ACD, BCD planes, transversal component W A,t should be unfolded on the component, operating in the planes ABD, ACD; W B,t —on the component, operating in the planes ABD, BCD; W C,t —on the component, operating in the planes ACD, BCD. Defined vector-components lie on the intersection of corresponding planes and planes, orthogonal to corresponding centerlines at points lA,w ,

4 Digital Control by Robot Manipulator with Improved Rigidity

51

l B,w , l C,w . Orthogonal planes are described by following equations: ⎧ ⎨ x cos αx, A + y cos α y, A + y cos αz,A − l A,w = 0; x cos αx,B + y cos α y,B + y cos αz,B − l B,w = 0; ⎩ x cos αx,C + y cos α y,C + y cos αz,C − lC,w = 0,

(4.7)

where cos αx, A (B,C) , cos α y, A (B,C) , cos αR,A(B,C) are the direction cosines of centerlines A(B, C), respectfully; cos αx, A = xl AD ; cos α y, A = yl AD ; cos αz, A = zl AD ; cos αx,B = xl BD ; cos α y,B = y Dl B−b ; cos αz,B = zl BD ; cos αx,C = xlCD ; cos α y,C = y DlC+b ; cos αz,C = zlCD . Directions of transverse weight forces subcomponent are defined by linear equation systems: Definition of W A,t subcomponents, situated at planes ABD and ACD: ⎧ ⎪ x cos αx, A + y cos α y, A + y cos αz,A − l A,w = 0; ⎪ ⎪ | | ⎪ ⎪ ⎨ || x y z 1 || | 0 0 0 1| | | ⎪ ⎪ | a −b 0 1 | = 0; ⎪ ⎪ | | ⎪ ⎩|x y z 1| D D D ⎧ ⎪ x cos αx, A + y cos α y, A + y cos αz,A − l A,w = 0; ⎪ ⎪| | ⎪ ⎪ | ⎨ | x y z 1 || | 0 0 0 1| | | ⎪ ⎪ | a b 0 1 | = 0; ⎪ ⎪ | | ⎪ ⎩|x y z 1| D

D

D

definition of W B,t subcomponents, situated at planes ABD and BCD: ⎧ ⎪ x cos αx,B + y cos α y,B + y cos αz,B − l B,w = 0; ⎪ ⎪ | | ⎪ ⎪ ⎨ || x y z 1 || | 0 0 0 1| | = 0; ⎪ || ⎪ ⎪ a −b 0 1 || ⎪ | ⎪ ⎩|x y z 1| D D D ⎧ ⎪ x cos αx,B + y cos α y,B + y cos αz,B − l B,w = 0; ⎪ ⎪ | | ⎪ ⎪ ⎨ || x y z 1 || | a b 0 1| | | ⎪ ⎪ | a −b 0 1 | = 0; ⎪ ⎪ | | ⎪ ⎩|x y z 1| D

D

D

definition of W C,t subcomponents, situated at planes ACD and BCD:

52

E. Larkin et al.

⎧ ⎪ ⎪ |x cos αx,C + y| cos α y,C + y cos αz,C − lC,w = 0; ⎪ ⎪ ⎪ ⎨ || x y z 1 || | 0 0 0 1| | | = 0; ⎪ ⎪ ⎪ || a b 0 1 || ⎪ ⎪ ⎩|x y z 1| D D D ⎧ ⎪ x cos αx,C + y cos α y,C + y cos αz,C − lC,w = 0; ⎪ ⎪ | | ⎪ ⎪ ⎨ || x y z 1 || | a b 0 1| | = 0. ⎪ || ⎪ ⎪ a −b 0 1 || ⎪ | ⎪ ⎩|x y z 1| D D D

(4.8)

Solution of linear systems (4.8) yields direction cosines shown at Table 4.2. Values of angles between transversal vector-components W A,t , W B,t , W C,t and their vector-subcomponents are collected in Table 4.3. Table 4.2 Direction cosines Actuator

Plane

Component

Subcomponent

Direction cosines

A

ABD

W A,t

W A,t ( AB D)

r x, A,1 , r y, A,1 , r R,A,1

A

ACD

W A,t

W A,t ( AC D)

r x, A,2 , r y, A,2 , r R,A,2

B

ABD

W B,t

W B,t ( AB D)

r x,B,1 , r y,B,1 , r R,B,1

B

BCD

W B,t

W B,t (BC D)

r x,B,2 , r y,B,2 , r R,B,2

C

ACD

W C,t

W C,t ( AC D)

r x,C,1 , r y,C,1 , r R,C,1

C

BCD

W C,t

W C,t (BC D)

r x,C,2 , r y,C,2 , r R,C,2

Table 4.3 Angles between transversal vector-components and vector-subcomponents Actuator A

Angle ] [ α W A,t , W A,t (AB D)

Direction cosines arccos

r ·r +r ·r y. A +r z.A.1 ·r z.A /( x. A.1 x.A y.A.1 )( ) 2 2 2 2 2 2 +r +r r x. A,1 y. A,1 z.A,1 r x. A +r y.A +r z. A

A

] [ α W A,t , W A,t ( AC D)

arccos

r ·r +r ·r y. A +r z.A.2 ·r z. A /( x.A.2 x. A y. A.2 )( ) 2 2 2 2 2 2 r x. A,2 +r y.A,2 +r z. A,2 r x.A +r y.A +r z.A

B

] [ α W B,t , W B,t (AB D)

arccos

r ·r +r ·r y.B +r z.B.1 ·r z.B /( x.B.1 x.B y.B.1 )( ) 2 2 2 2 +r 2 +r 2 r x.B,1 +r y.B,1 +r z.B,1 r x.B y.B z.B

B

] [ α W B,t , W B,t (BC D)

arccos

r ·r +r ·r y.B +r z.B.2 ·r z.B /( x.B.2 x.B y.B.2 )( ) 2 2 2 2 2 2 +r +r r x. A,2 y. A,2 z. A,2 r x. A +r y.A +r z. A

C

] [ α W C,t , W C,t ( AC D)

arccos

r ·r +r ·r y.C +r z.C.1 ·r z.C /( x.C.1 x.C y.C.1 )( ) 2 2 2 2 +r 2 +r 2 +r y.C,1 +r z.C,1 r x.C,1 r x.C y.C z.C

C

] [ α W C,t , W C,t (BC D)

arccos

r ·r +r ·r y.C +r z.C.2 ·r z.C /( x.C.2 x.C y.C.2 )( ) 2 2 2 2 +r 2 +r 2 r x.C,2 +r y.C,2 +r z.C,2 r x.C y.C z.C

4 Digital Control by Robot Manipulator with Improved Rigidity

53

Table 4.4 Values of subcomponents Actuator

Subcomponent

Determined value

A

W A,t ( AB D)

W A,t sin α [ W A,t ,W A,t ( AC D)] sin{α [ W A,t ,W A,t ( AB D)]+α [ W A,t ,W A,t ( AC D)]}

A

W A,t ( AC D)

W A,t sin α [ W A,t ,W A,t ( AB D)] sin{α [ W A,t ,W A,t ( AB D)]+α [ W A,t ,W A,t (AC D)]}

B

W B,t ( AB D)

W B,t sin α [ W B,t ,W B,t (BC D)] sin{α [ W B,t ,W B,t ( AB D)]+α [ W B,t ,W B,t (BC D)]}

B

W B,t (BC D)

W B,t sin α [ W B,t ,W B,t (AB D)] sin{α [ W B,t ,W B,t ( AB D)]+α [ W B,t ,W B,t (BC D)]}

C

WC,t (AC D)

WC,t sin α [ W C,t ,W C,t (BC D)] sin{α [ W C,t ,W C,t ( AC D)]+α [ W C,t ,W C,t (BC D)]}

C

WC,t (BC D)

WC,t sin α [ W C,t ,W C,t ( AC D)] sin{α [ W C,t ,W C,t ( AC D)]+α [ W C,t ,W C,t (BC D)]}

) yD √ , sin ϑ A 2 2 , cos ϑ A 2 x D +y D ( x D +y D ) r are direction cosines of W vector-component; , r , r A,t x,B x,B x,B ( ) y D +b x D −a √ sin ϑ B √ = , sin ϑ , cos ϑ are B B (x D −a)2 +(y D +b)2 (x D −a)2 +(y D +b)2 ( ) vector-component; ) r x,C , r x,C , r x,C = direction cosines of W B,t ( ( ) In Table 4.3: r x, A , r x, A , r x, A =

( sin ϑ A √ x2D

y D −b x D −a , sin ϑC / , cos ϑC r x,C , r y,C , r z,C are (x D −a )2 +( y D −b)2 (x D −a )2 +( y D −b)2 direction cosines of W C,t vector-component. Values of subcomponents W A,t (ABD), W A,t (ACD), W B,t (ABD), W B,t (BCD), W C,t (ACD), are determined from vectors based parallelogram as it follows from Table 4.4. Formulae obtained for determining forces between actuators and weight forces, affected on actuators, permit to form three sets of differential equations, describing actuators operation, every of which includes force equation, describing longitudinal movement of actuator stock, and moment equations describing actuator double rotation at the hinge [12, 13]:

sin ϑC /

⎧ ¨ ˙ ⎪ ⎨ M A l A˙ + ηl l A = κu A − R B A,l − RC A,l − W A,l − FA ; d( J A ψ B AD ) + ηr ψ˙ B AD = l A R B A,l + l A,w W A,t ( AB D); dt ⎪ ⎩ d( J A ψ˙ C AD ) + ηr ψ˙ C AD = l A RC A,l + l A,w W A,t ( AC D); dt ⎧ ¨ ˙ ⎪ ⎨ M B l B˙ + ηl l B = κu B − R AB,l − RC B,l − W B,l − FB ; d( J B ψ AB D ) + ηr ψ˙ AB D = l B R AB,l + l B,w W B,t ( AB D); dt ⎪ ⎩ d( JB ψ˙ C B D ) + ηr ψ˙ C B D = l B RC B,l + l B,w W B,t (C B D); dt ⎧ ¨ ˙ ⎪ ⎨ MC lC˙ + ηl lC = κu C − R AC,l − R BC,l − WC,l − FC ; d( JC ψ AC D ) + ηr ψ˙ AC D = lC R AC,l + lC,w WC,t ( AC D); dt ⎪ ⎩ d( JC ψ˙ BC D ) + ηr ψ˙ BC D = lC R BC,l + lC,w WC,t (BC D), dt

(4.9)

54

E. Larkin et al.

where ηl is the coefficient of viscous friction during linear actuators stocks longitudinal movement; ηκ is the coefficient of viscous friction during linear actuators rotation in hinges; κ is the actuators transmission ratio; uA , uB , uC are control signals, which are formed on the output of digital control system according operational unit desired cartesian coordinates x D , y D , z D , the current manipulator state x D , yD , zD and managing principle, underlying into control algorithm; F A , F B , F C are dry friction forces, acted in A, B, C actuators, correspondingly; J A , J B , J C are moments of inertia of actuators. Systems (4.9) should be fulfilled by dry friction characteristics and actuator stock movement speed limitations as follows: FA = −κ F · FA,Σ · sgn ˙ l A ; FB = −κ F · FB,Σ · sgn ˙ l B ; FC = −κ F · FC,Σ · sgn ˙ lC ; (4.10) where κ F is the dry friction coefficient; FA,Σ , FB,Σ , FC,Σ are corresponding total forces, originated dry friction; l˙Amax , l˙Bmax , l˙Cmax are limit values of stock speeds. Expressions (4.10), close description of pyramid manipulator.

4.4 Digital Control by Manipulator Positioning In robotic complexes manipulators are object under control, so structure of digital control system by manipulator is shown in Fig. 4.3.

uc,A

NA(s)

uA

ФAA(s) ФBA(s)

l d (s) - Φ0,c(s)l0,c(s)

l0c,A

N0,A(s)

l0,A

1/s

lA

ФCA(s) Ф0,A(s) ФAB(s)

uc,B NB(s) l0c,B N0,B(s)

uB

l0,B

ФBB(s)

1/s

lB

ФCB(s) Ф0,B(s) ФAC(s)

uc,C

l0c,C

NC(s)

N0,C(s)

Digital controller

uC

l0K

ФBC(s)

1/s

lB

xD(lA,lB,,lC), yD(lA,lB,,lC), zD(lA,lB,,lC)

Fig. 4.3 Digital control system by pyramid manipulator

ФCC(s) Ф0,B(s) Manipulator, object under control

xD yD zD

4 Digital Control by Robot Manipulator with Improved Rigidity

55

Structure of manipulator as object under control does not include blocks, describing dry friction, because dry friction, as dissipative force may be included into other dissipative force, namely viscous friction. Besides, on the structure restrictions (11) are not reflected, Transfer functions ϕ A A (s), ϕ B A (s), . . . , ϕ BC (s), ϕCC (s) may be received through joint solution Eqs. (4.2)–(4.8) with respect to l A , l B , l C , linearization expressions obtained, and application to the linear system the Laplace transform [14]. Transfer functions mat be ordered to the matrix (4.12): ⎡

⎤ ϕ A A (s) ϕ AB (s) ϕ AC (s) Φ(s) = ⎣ ϕ B A (s) ϕ B B (s) ϕ BC (s) ⎦, ϕC A (s) ϕC B (s) ϕCC (s)

(4.12)

where s is complex Laplace variable. At practice actuators states l˙A , l˙B , l˙C are measured with sensors, output of which may be described as (4.13): l0, A (s) = ϕ0, A (s) · l0, A (s), l0,B (s) = ϕ0,B (s) · l0,B (s), l0,C (s) = ϕ0,C (s) · l0,C (s)

(4.13)

where ϕ0, A (s), ϕ0,B (s), ϕ0,C (s) are sensor’s transfer functions. For a convenience transfer functions also may be ordered into the matrix (4.14): ⎡

⎤ 0 0 ϕ0, A (s) Φ 0 (s) = ⎣ 0 0 ⎦. ϕ0,B (s) 0 0 ϕ0,C (s)

(4.14)

Digital controlled processes data l0, A (s), l0,B (s), l0,C (s) according algorithm, which semantics in the case of linear data processing may be described as (4.15): uc (s) = l d (s) − Φ 0,c (s) · l 0,c (s),

(4.15)

where uc (s) is the data-vector, components of which are transmitted to inputs of linear actuator; l d (s) is the vector of desired values of actuator’s lengths; l 0,c (s) is the vector of current values of actuator’s lengths; Φ 0,c (s) is the transfer function matrix from vector current values to vector of actuator inputs. On practice input of vector l 0,c (s) elements and output of vector uc (s) elements are executed in sequence, so in Von Neumann controller there are time intervals between transactions. Time intervals are ordered in delay transfer functions matrices (4.16), (4.17): ⎡

⎤ 0 0 exp(−τ A s) ⎦ N(s) = ⎣ 0 0 exp(−τ B s) 0 0 exp(−τC s)

(4.16)

56

E. Larkin et al.

) ( ⎤ 0 0 exp −τ0, A s ) ( ⎦, N 0 (s) = ⎣ 0 exp −τ0,B s ) ( 0 0 0 exp −τ0,C s ⎡

(4.17)

where τ A , τ B , τC are interface delays in transmitting signals uA , uB , uC from digital controller to actuator’s inputs; τ0, A , τ0,B , τ0,C are interface delays in transmitting signals l0, A , l0,B , l0,C from manipulator to digital controller. In common reaction of closed loop digital control system on the input vector l d (s) is as follows (4.18): [ ]−1 l(s) = N(s)Φ(s) · E + N(s)Φ(s)Φ 0,c (s)N 0 (s) · Φ 0,c (s) l d (s),

(4.18)

where E is the 3 × 3 unit diagonal matrix. Matrices, characterizing delays, are both in the numerator, and in denominator of (4.17). The matrix N(s), situated in the numerator, define so called data skew and common lag of external commands execution. Matrices N(s), N 0 (s), appearing in denominator, born feedback signal pure lag, therefore has significant impact on a performance and stability of controlled manipulator functioning. To estimate delays τ A , τ B , τC , τ0, A , τ0,B , τ0,C it is necessary to analyze control algorithm execution on the Von Neumann type controller. For simplicity let us assume, that algorithm has six transactions operators only (Fig. 4.4a, b). Operators of reading data l 0,c (s) have numbers from 1 to 3, operators of writing data uc (s) have numbers from 4 till 6. [ ] The model of algorithm is the semi-Markov process h(t) = h i, j (t) , where hi,j (t) is weighted time of sojourn the process at the state i with further jumping to the state j. In common case process has the structure due to which from any state it can switch to any other state [15–17]. Due to transactions periodicity semi-Markov process is the ergodic one and does not contain absorbing or semi-absorbing. Contribute to time intervals τ A , τ B , τC , τ0,A , τ0,B , τ0,C may both direct switching, and wandering through the semi-Markov process from one state till another. Weighted time of sojourn in the state i is determined as h i, j (t) = pi, j ξi, j (t), where 1 ≤ i, j ≤ 6; pi, j is the probability

1

2

3

1

2

3

4

5

6

4

5

6

a

b

Fig. 4.4 Common structure of control algorithm a and structure for delay estimation b

4 Digital Control by Robot Manipulator with Improved Rigidity

57

of switching from the state i to the state j; ξi, j (t) is pure time density of sojourn the process in the state j. For estimation of time delay, the semi-Markov matrix h(t) should be transformed (4.19) as follows (Fig. 4.4b): ] [ h(t) → h' (t) = ξi j (t) · pi' j , Let it is necessary to estimate delay between execution of i-th and j-th operator. So, i-th state one should do the starting state, and j-th state one should do the absorbing state. For this purpose, i-th column and j-th row should be zeroed and probabilities in all other rows and columns should be recalculated as follows (4.19): ' pik =

pik . 1 − pi j

(4.19)

After recalculation, semi-absorbing states, emerging after j-th column zeroing become non-absorbing, time density of wandering may be defined as follows (4.20): ↔

[

ξ i j (t) = V i · L

−1

] ∞ Σ { [ ' ]}λ L h (t) V j,

(4.20)

λ=1

where V i is the row vector of size 6, i-th element of which is equal to 1, and all other elements are equal to zero; V j is the column vector of size 6, j-th element of which is equal to 1, and all other elements are equal to zero. ↔

For time density ξ i j (t) according to “three sigma rule” [18, 19] may be obtained estimations of time delays, which influence on the system performance (4.21): / τi j = T˜i j + 3 D˜ i j ,

(4.21)

)2 ↔ ⎧∞ ↔ ⎧∞( where T˜i j = 0 ξ i j (t)tdt; D˜ i j = 0 t − T˜i j ξ i j (t)(t)dt. So, on the stage of pyramid manipulator design one can estimate control algorithm runtime, error, which born transient process under applied control algorithm and correct algorithm for achieving better results [20–22].

4.5 Conclusion As a result, the mathematical models of pyramid manipulator describing achievement by operational unit desired position, estimation of its exactness, pyramid manipulator as object under control, and digital control system as a whole are worked out. It is proved, that due to pyramid structure dynamic model has cross links, changing length of every actuator leads to necessity of changing lengths of other manipulators, if one want to achieve desired position of operational unit. Also, it is proved, that closing

58

E. Larkin et al.

feedback loops when digital control by manipulator, leads to decreasing control system performance, so at the control system design stage it is necessary to optimize control algorithm runtime for reaching quality characteristics of control process. Further investigations in the domain may be directed to working out methods of practical digital control by pyramid manipulator for achievement higher accuracy of operational unit positioning.

References 1. Malin, L.M., Säfsten, K., Winroth, M.: Manufacturing strategy formulation, leadership style and organizational culture in small and medium-sized enterprises. Int. J. Manuf. Technol. Manage. (IJMTM) 30(5), 306–325 (2016) 2. Roy, R., Stark, R., Tracht, S., Takata, S., Mori, M.: Continuous maintenance and the future— foundation and technological challenges. CIRP Ann. 65(2), 667–688 (2016) 3. Siciliano, B., Khatib, O., Kröger, T.: Springer Handbook of Robotics, vol. 200 (2008) 4. Gong, C., Yuan, J., Ni, J.: Nongeometric error identification and compensation for robotic system by inverse calibration. Int. J. Mach. Tools Manuf. 40(14), 2119–2137 (2000) 5. Larkin, E., Dolgov, A., Osetrov, A., Osetrov, C.: Industrial robot module, Patent No. 103086 (RF) IPC B 25 J 9/08. Tula State University Bull, vol. 9 (2010) 6. Landau, I.D., Zito, G.: Digital Control Systems, Design, Identification and Implementation, vol. 130 (2006) 7. Aström, J., Wittenmark, B.: Computer Controlled Systems: Theory and Design. Tsinghua University Press. Prentice Hall, p. 557 (2002) 8. Lambrechts, P., Boerlage, M., Steinbuch, M.: Trajectory planning and feedforward design for electromechanical motion systems. Control. Eng. Pract. 13(2), 145–157 (2005) 9. Morella, A., Tarokhb, M., Acosta, L.: Solving the forward kinematics problem in parallel robots using Support Vector Regression. Eng. Appl. Artif. Intell. 26(7), 1698–1706 (2013) 10. Zimenko, D., Polyakov, A., Efimov, D., Kremlev, A.: Feedback sensitivity functions analysis of finite-time stabilizing control system. Int. J. Robust Nonlinear Control 27(15), 2475–2491 (2016) 11. Dreizler, R.M., Luedde, C.S.: Theoretical Mechanics: Theoretical Physics, vol. 1 (2011) 12. Swevers, J., Verdonck, W., Joris, D.S.: Dynamic model identification for industrial robots. IEEE Control Syst. 27(5), 58–71 (2007) 13. Karnopp, D.C., Margolis, D.L., Rosenberg, R.C.: System Dynamics: Modeling, Simulation and Control of Mechatronic Systems. Willey, p. 636 (2012) 14. Schiff, J.L.: The Laplace Transform: Theory and Applications. Springer, USA, p. 233 (1991) 15. Bielecki, T.R., Jakubowski, J., Niew˛egłowski, M.: Conditional Markov chains: properties, construction and structured dependence. Stochastic Process. Appl. 127(4), 1125–1170 (2017) 16. Ching, W.K., Huang, X., Ng, M.K., Siu, T.K.: Markov Chains: Models, Algorithms and Applications. Internat. Ser. Oper. Res. Management Sci. 189, 241 (2013) 17. Janssen, J., Manca, R.: Applied Semi-Markov Processes. Springer Science & Business Media (2006) 18. Kobayashi, H., Marl, B.L., Turin, W.: Probability, Random Processes, and Statistical Analysis: Applications to Communications, Signal Processing, Queueing Theory and Mathematical Finance. Cambridge University Press (2011) 19. Pukelsheim, F.: The three sigma rule. Am. Statistician 48(2), 88–91 (1994)

4 Digital Control by Robot Manipulator with Improved Rigidity

59

20. Wu, M., He, Y., She, J.H., Liu, G.P.: Delay-dependent criteria for robust stability of time-varying delay systems. Automatica 40(8), 1435–1439 (2004) 21. Balabanov, A., Bezuglaya, A., Shushlyapin, E.: Underwater Robot Manipulator Control. Inform. Autom. 20(6), 1307–1332 (2021). https://doi.org/10.15622/ia.20.6.5 22. Zhang, X.M., Min, W.U., Yong, H.E.: Delay dependent robust control for linear systems with multiple time-varying delays and uncertainties. Control Decis. 19(5), 496–500 (2004)

Chapter 5

Performance Evaluation of Multigrid Brute-Force Solutions of Inverse Kinematics Problem for the Robotis OP2 Humanoid Hand Linar Zagidullin , Tatyana Tsoy , Kuo-Hsien Hsia , Edgar A. Martínez-García , and Evgeni Magid Abstract Humanoid robots target to remove human labor from multiple working environments including the ones that were initially constructed for a human. Robot limbs operation requires solving an inverse kinematics problem, and a standard solution could involve algebraic, geometric, or numerical approaches. This paper presents two brute-force off-line approaches for a Robotis OP2 humanoid upper limb positioning via forward kinematics. Both approaches calculate and structure all possible solutions for an end-effector pose within a robot workspace in advance using a powerful PC, in the off-line mode. Several levels of workspace and joint space discretization allow a user to select a required for his/her task level of the solution precision considering available onboard resources of the robot. Different discretization levels were evaluated at an offboard PC and at an onboard computer of the Robotis OP2 humanoid. The solutions with different discretization levels were compared in terms of memory consumption and precision. The solutions were initially obtained in the Gazebo simulation and then successfully validated with a real L. Zagidullin · T. Tsoy · E. Magid (B) Laboratory of Intelligent Robotic Systems, Intelligent Robotics Department, Kazan Federal University, Kremlin St. 35, Kazan 420008, Russia e-mail: [email protected] L. Zagidullin e-mail: [email protected] T. Tsoy e-mail: [email protected] K.-H. Hsia Graduate School Engineering Science and Technology, National Yunlin University of Science and Technology, 123 University Road, Section 3, Douliou, Yunlin 64002, Taiwan e-mail: [email protected] E. A. Martínez-García Institute of Engineering and Technology, Department of Industrial Engineering and Manufacturing, Autonomous University of Ciudad Juarez, Manuel Díaz H. No. 518-B Zona Pronaf Condominio, Chihuahua, 32315 Cd Juárez, Mexico e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_5

61

62

L. Zagidullin et al.

Robotics OP2 humanoid. The presented analysis might be useful for a discretization level selection under onboard memory limitations while dealing with manipulator kinematics.

5.1 Introduction Robots are created to replace humans in a variety of places and activities, including medical applications [1, 2] and robotic surgery [3, 4], conveyor production [5] and manufacturing tasks [6, 7], rescue operations [8, 9], and others. Typically, robots are initially constructed for particular applications, e.g., a welding operation requires an articulated manipulator [10], while a palletizing application would prefer a Cartesian manipulator [11]. However, employment of non-humanoid kinematics robots is not always possible or is not desirable due to a task definition or restrictions, e.g., the task execution by a non-humanoid robot might require significant additional spendings on infrastructure reorganization or adjustment [12]. On the contrary, a humanoid kinematic scheme allows immediate use of a human-centered infrastructure and tooling [13]. Moreover, the humanoid form has a significant positive impact on a human–robot interaction (HRI) acceptability and success [14, 15]. These provide clear advantages of humanoids in tasks that require interaction with a human or operation in typical for a human environments [16]. Designing, constructing, and programming of manipulators require to solve forward and inverse kinematics problems. The two mainstream approaches are a closed-form solution (algebraic or geometric [17]) and a numerical solution [18, 19]. In the past decades, neural networks-based approaches were also reported by a number of authors [20, 21] as a useful tool for solving the inverse kinematics problem. Direct replication of a biological structure might cause significant difficulties in further inverse kinematics calculations for a manipulator. Yet, creating a robotic arm, which does not have a natural prototype, allows to utilize primitive schemes that would simplify inverse kinematics calculations. A large number of degrees of freedom (DOF) or/and non-orthogonal intersections of joint axis and link conjunctions prevent using closed-form methods [22]. And while in such cases, a numerical approach remains the only viable option, it cannot guarantee a reasonable calculation time and convergence for any required manipulator configuration. This paper extends our previous work [23] and demonstrates alternative approaches to the inverse kinematics problem solution for a 3 DOF upper limb of a Robotis OP2 robot. Given a manipulator end-effector (EE) coordinates, it applies a brute-force method for off-line calculations and further results’ arrangements. The methods are based on the arm scheme analysis and employ pre-calculated arm configuration files. The computational feasibility of the methods was evaluated on a PC, and the proper performance was validated with a real Robotis OP2 humanoid robot.

5 Performance Evaluation of Multigrid Brute-Force Solutions of Inverse …

63

5.2 System Setup The Robotis OP2 (Fig. 5.1) is a humanoid robot created by Korean company ROBOTIS [24]. It is one of the most popular robotic platforms for training [25], research [10, 26], and various competitions [27]. It is broadly employed for human–robot interaction applications and as a test bed for practicing humanoid movements. This paper considers a 3DOF upper limb of the robot (Fig. 5.1, right). The main technical characteristics of the robot, which play important role in the inverse kinematics problem solution for the arm, are as follows: • • • • • •

20 MX-28T actuators (6 DOF leg × 2, 3 DOF arm × 2, and 2 DOF neck); CPU: Intel Atom Processor N2600 (dual core, 1.6 GHz); RAM: 4 GB; Storage: SSD mSATA module, 32 GB; Operating system: Ubuntu 12.04; Programming language used: Python 2.7.

Table 5.1 contains constant numerical values associated with physical dimensions of the upper limb. Table 5.2 presents the DH table for the upper limb. The DH table generates the following forward kinematics equations: ⎧ ⎨ x = 60s1 s2 + 60s1 − 16c1 − 129(c1 s3 − c3 s1 s2 ) , y = 60c2 + 129c2 c3 ⎩ z = −16s1 − 16c1 − 60c1 s2 − 129(s1 s3 + c1 c3 s2 )

(5.1)

where ci = cos(θi ), si = sin(θi ) and θ1 , θ2 , θ3 are angles of the corresponding joints (Fig. 5.1, right), and (x, y, z) is a target 3D space coordinate of the EE location. These

Fig. 5.1 Robotis OP2 robot (left) and its 3 DOF hand with joint angles marked as thetas (right)

64

L. Zagidullin et al.

Table 5.1 Constant values for the links of the robot hand

Constant

Value (mm)

L1

16.0

L2

60.0

d2

16.0

Lh

129.0

Table 5.2 DH parameters for Robotis Darwin OP2 robot hand i

α i−1

ai−1

di

θi

1

0

0

0

θ1

2

−90

L1

d2

θ 2 −90

3

90

L2

0

θ3

lead to the following inverse kinematics equations: ⎧ ⎨ xc1 s2 + ys1 s2 + zc2 − 16s2 − 60 = 16s3 + 129c3 . xc c + ys1 c2 − zs2 − 16c2 = 0 ⎩ 1 2 −xs1 + yc1 − 76 = −16c3 + 129s3

(5.2)

5.3 Algorithms Description This section presents two algorithms for numerical calculation and further use of inverse kinematics problem solution. Both algorithms use a brute-force approach, but while the first could be considered as a pure brute-force solution (referred as a brute-force algorithm, BFA), the second (referred as a matrix-based brute-force algorithm, MBBFA) has a modification that speeds up a further practical on-line use of an off-line solution on account of a higher memory consumption.

5.3.1 Brute-Force Algorithm The algorithm has two stages of execution: 1. Preparing a file of manipulator positions and joint angles (off-line). 2. Searching a suitable manipulator configuration (on-line). The first step is performed in an off-line mode separately for each robot arm. To build a set (a file) of exhaustive variations of manipulator positions, which satisfactorily covers the manipulator workspace, we set an empirically established joint angles

5 Performance Evaluation of Multigrid Brute-Force Solutions of Inverse …

65

sampling step h 1 . Using step h 1 for all joints, all possible instances of manipulator joint configurations are generated. Obviously, smaller values of h 1 improve a solution precision on account of memory and CPU usage. For all possible values of θ1 , θ2 , θ3 joint angles, selected in a brute-force manner with a sampling step h 1 , the EE coordinate values x, y, z are calculated using forward kinematics equations (Eq. 5.1), and a line with x, y, z, θ1 , θ2 , θ3 values is stored in the manipulator configurations file. After the manipulator configurations file is populated with all possible joint angles’ configurations for the selected step h 1 , , a discretization (thus, a solution) quality with h 1 is evaluated as follows. A random position of EE (x ' , y ' , z ' ) is generated, and the manipulator configurations file is searched in a brute-force manner for a line that contains a closest (to (x ' , y ' , z ' )) EE position (x, y, z) in terms of Euclidean distance, while storing the shortest distance as discretization error ed . This way, 100,000 random positions are generated, and an average discretization error ed1 for step h 1 is calculated over all these random positions. If the resulting ed1 value does not exceed a required precision threshold (i.e., has an acceptable for a user precision), it is stored in the end of the manipulator configurations file; the file is declared as usable, and we refer ed1 further as acceptable discretization error eda . Otherwise, it is recommended to decrease sampling step h 1 (e.g., by one degree) in order to improve the solution precision, and to repeat the procedure of manipulator configurations file generation. Finally, the resulting average discretization error that satisfies the threshold is stored as eda . Since the entire first step is performed one time in the off-line mode on a powerful PC, calculation time and resources limits are not considered as an issue. The second step is a search for a suitable configuration in the previously generated manipulator configurations file. This step is performed on the robot. An input for this step is a desired goal position of the EE (x g , yg , zg ). During this step, the file is searched in a brute-force manner for a best fit configuration of joint angles θ1 , θ2 , θ3 , which minimizes the Euclidean distance between (x, y, z) and (x g , yg , zg ). The solution is not always optimal in terms of precision since, in order to speed up the process, the search procedure stops as soon as a configuration with a distance less than eda (which is stored in the file) is detected; otherwise, the best fit is selected as a minimal value upon reaching the end of file. As a result, a string x, y, z, θ1 , θ2 , θ3 , which describes the EE position and a configuration of the manipulator joints is returned. It is important to emphasize that the second step should be as rapid as possible since the entire robot operation depends on time lags between issuing a motion command and a start of its execution. To improve the second step, an optimized approach was proposed, which is described in the next subsection.

5.3.2 Matrix-Based Brute-Force Algorithm Similarly to the previous algorithm, MBBFA has two stages:

66

L. Zagidullin et al.

1. Preparing a file of manipulator positions and joint angles (off-line). 2. Searching a suitable manipulator configuration (on-line). The first step is carried out in an off-line mode. Similarly to BFA, joint angles sampling step h 1 is empirically selected and all possible combinations of joint variables θ1 , θ2 , θ3 with step h 1 are generated. In addition, 3D workspace linear sampling step h 2 is empirically selected, the entire manipulator workspace is divided into cubes with edge h 2 , and a 3D matrix is created with each cell corresponding to a particular workspace cube. We further refer this 3D matrix as solutions matrix SM. For each configuration θ1 , θ2 , θ3 , coordinates x, y, z are obtained with equation (Eq. 5.1), and a corresponding cell [a1 , a2 , a3 ] of the SM is calculated with rounding each value down to the closest integer as ] [ ] [] [ ] [ [ ] yi zi xi , int , int , ax , a y , az = int h2 h2 h2

(5.3)

where i ranges from 0 to the maximal [ index of] all possible combinations of joint variables. Next, a corresponding cell ax , a y , az is populated with lines (x, y, z, θ1 , θ2 , θ3 )j where j can take a number of values from 0 to some integer n. Note that for a coarse discretization of the 3D space (i.e., with large values of h 2 ) number n of configurations that fall into the same cell due to discretization issue is high, while for a very dense discretization (i.e., with small values of h 2 ) some cells of SM might stay empty. Within each cell, (x, y, z, θ1 , θ2 , θ3 ) lines form an unordered list of all possible solutions that allow the EE to reach location (x, y, z) by setting joint variables to (θ1 , θ2 , θ3 ). Figure 5.2 illustrates the structure of SM. SM is stored as a manipulator configurations file. The second stage is conducted on the robot in the online mode. SM is loaded into the robot’s computer RAM. Next, when a particular command to the robot requires

Fig. 5.2 Visualization of solutions matrix SM structure

5 Performance Evaluation of Multigrid Brute-Force Solutions of Inverse …

67

to move the EE into goal location (x g , yg , zg ), its corresponding cell within SM is determined using equation (Eq. 5.3). If the cell contains a single solution, this solution is selected. Otherwise, in the case of multiple available solutions n that are stored in the cell, for each solution k = 0…n the Euclidean distance between (x k , yk , zk ) and (x g , yg , zg ) is calculated, and a solution with a minimal distance is selected.

5.4 Testing The algorithms were implemented in Python programming language. Unfortunately, this leads to increased memory consumption and reduced performance of the algorithms. Since the proof of concept demonstrated a positive result, one of our ongoing work tasks is a transfer the code into C++ language. All tests were performed using two computers—an internal robot’s onboard computer and an external personal computer (PC) depending on the usage of the algorithms, i.e., the ones that should be performed on the offboard PC were tested on the offboard PC, and the ones for the real robot were tested on the onboard computer of the robot. The external PC has the following characteristics: • • • • •

CPU: AMD Ryzen 5600H 6C/12T; RAM: 16 GB DDR4; Memory: M2 SSD, 512 GB; Operating system: Ubuntu 20.04; Programming language used: Python 3.9.

It is worth mentioning that while versions of Python were different for an external PC and the onboard computer, the code was insignificantly modified. The following tests were performed on the offboard PC: 1. Testing BFA (first step) for a size of external manipulator configurations files as a function of discretization step h1 2. Testing MBBFA (first step) for a size of external manipulator configurations files as a function of discretization step h2 with several discretization steps h1 3. Testing BFA (first step) for RAM consumption1 as a function of discretization step h1 4. Testing MBBFA (first step) for RAM consumption as a function of discretization step h2 with several discretization steps h1 . The following tests were performed on the onboard computer of the robot: 1. Testing BFA (second step) solution’s precision as a function of discretization step h1 1

The same test was not performed on the robot onboard computer since we believe that an amount of memory consumed does not depend on an execution environment under the condition of staying within the hardware limitations.

68

L. Zagidullin et al.

2. Testing MBBFA (second step) solution’s precision as a function of discretization step h2 with several discretization steps h1 . The following metrics were selected for testing: 1. RAM consumption during algorithm execution 2. Hard disk memory consumption by a manipulator configurations file 3. Execution error (EE displacement from a goal location).

5.5 Experimental Results Analysis Both algorithms were evaluated for memory consumption and accuracy of the obtained solution relatively to desired goal location (x g , yg , zg ) of the EE. Listed in Sect. 5.4, tests were executed on the PC (1.1–1.4) and on the robot (2.1–2.2), which directly corresponds to the algorithms’ flow. Figures 5.3, 5.4, 5.5, 5.6, 5.7 and 5.8 present the obtained results of the tests. We empirically limited values of h1 to integer values from 1° to 10°. BFA results (Fig. 5.3) demonstrated that an external manipulator configurations file size increases drastically when using sampling h1 ≤ 4°, while at a coarse grid with h1 ≥ 8°, the file size is less than 1 MB. Moreover, at a coarse grid with h1 ≥ 18°, the file size becomes less than 1 kB and further sampling rate decrease does not make any meaningful improvement. Obviously, RAM consumption dependence on the discretization step h1 (Fig. 5.4) behaves similarly, with unacceptably large values for sampling h1 ≤ 4° and slowly decreasing 16 MB (and less) after h1 ≥ 8°. It is obvious that as the configuration file size increases, an optimal configuration search time increases accordingly and a reasonable search time should be beyond 1 s. At the same time, it is interesting to note that an average positioning error between an obtained position and a goal position of the EE, being neglectable at h1 = 1, does Brute-force algorithm: a configurations file size

File size (MB), log scale

1000

489 61.2

100

18.4 7.9 10

4.1 2.3 1.5

1

7

8

1 1

2

3

4

5

6

0.7

0.54

9

10

0.1

Discretization step (degrees)

Fig. 5.3 Dependence of the external manipulator configurations file size on the discretization step h1 for BFA

5 Performance Evaluation of Multigrid Brute-Force Solutions of Inverse …

69

RAM consumption (MB)

Brute-force algorithm: RAM consumption 1000 900 800 700 600 500 400 300 200 100 0

909

339 109

1

2

53

32

23

18

16

14

13

4

5

6

7

8

9

10

3

Discretization step (mm)

Fig. 5.4 Dependence of RAM consumption on the discretization step h1 for BFA

Brute-force algorithm: a positioning error 4 3.21

3.5

3.72

2.76

3 Error (mm)

3.46

2.47 2.19

2.5

1.83

2

1.49

1.5

1.04

1 0.5 0.01 0 1

2

3

4

5

6

7

8

9

10

Discretization step (degrees)

Fig. 5.5 Dependence of a positioning error between an obtained position and goal position of the EE on the discretization step h1 for BFA

not change drastically when discretization step increases from h1 = 2 to h1 = 10 (Fig. 5.5). Since the robot EEs do not have grippers that allow grasping objects, the hands are mainly employed for rather abstract hand gestures that strengthen or replace verbal communication and humanoid reactions during HRI. In rare cases, the hands of Robotis OP2 robot are used to improve speed and stability of the robot during locomotion. In both cases, a high precision of the EE location is not a must, and our experience of using the robot in HRI [10] demonstrated that 10 mm positioning error for the EE is completely acceptable while a positioning difference of less than 2– 3 mm is even not visible by a human eye. A tradeoff for optimal memory consumption and solution accuracy concludes that a discretization step of h1 = 7° will be optimal for BFA.

70

L. Zagidullin et al.

Matrix-based brute-force algorithm: a configurations file size 182.4

File size (MB), log scale

100 169.2 37.2 23.7

20.6

19.5

19

18.7

18.6

18.5

18.5

21.2 21.1

10

7.2

h2 = 2 3.7

6.4

2.4

2.7 1

1.9

1.6

h2 = 5 1.5

1.4

1.3

0.4

0.3

0.9 0.6

1.4

0.5

0.8 0.5

0.4

h2 = 10 h2 = 20

0.3 0.2

0.1 1

2

3

4

5

6

7

8

9

10

Discretization step h1 (degrees)

Fig. 5.6 Dependence of the external manipulator configurations file size on the discretization step h1 for the matrix-based brute-force algorithm for four different discretization steps h2

Matrix-based brute-force algorithm: RAM consumption 885

RAM consumption (MB), log scale

1000

377

327

315

311

309

308

307

39

34

32

31

31

30

30

16

15

14

14

13

14

13

12

12

11

7

8

9

10

307 307

665 93

100

52 89 35

21 10

16

h2 = 2 h2 = 5 h2 = 10 h2 = 20

1 1

2

3

4

5

6

Discretization step h1 (degrees)

Fig. 5.7 Dependence of RAM consumption on the discretization step h1 for the matrix-based brute-force algorithm for four different discretization steps h2

Having a constant calculation time of MBBFA, we analyzed the influence of the Cartesian 3D space discretization step h2 on memory consumption. Four reasonable levels of Cartesian 3D space discretization h2 = 2, 5, 10, 20 mm were selected in a process of pilot experiments. Figure 5.6 demonstrates that while for a high level of joint space discretization, the Cartesian 3D space discretization level (h1 ≤ 1°) does

5 Performance Evaluation of Multigrid Brute-Force Solutions of Inverse …

71

Matrix based brute-force algorithm: a positioning error 7 6.17 5.7

6 4.76

Error (mm)

5

4.98 2.87 3

3.6

5.07

4.46

3.55

4

h2 = 2

3.86

h2 = 5

2.31 1.79

2

2.78

2.84

2.9

2.91

2.2

1.01

1.1

1.13

1.2

1.16

1.18

1.21

1.18

3

4

5

6

7

8

9

10

1.05 1 0.02 0.8 0 0.01 1 2

2.65

2.45

h2 = 10 h2 = 20

1.59

Discretization step h1 (degrees)

Fig. 5.8 Dependence of a positioning error on the discretization step h1 for the matrix-based brute-force algorithm for four different discretization steps h2

not significantly influence a manipulator configurations file size (182.4 MB for h2 = 2 mm vs. 169.2 MB for h2 = 20 mm), for a high level of joint space discretization, the file size reduces almost in 100 times between h2 = 2 mm and h2 = 20 mm. A similar behavior of RAM consumption is shown in Fig. 5.7 with 885 MB for h2 = 2 mm versus 665 MB for h2 = 20 mm at h1 = 1° and a 30 times difference at h1 = 10° (307 MB for h2 = 2 mm vs. 11 MB for h2 = 20 mm). Figure 5.8 demonstrates that an average positioning error between an obtained position and a goal position of the EE is neglectable at h1 = 1 for any 3D space discretization level. For h2 = 2 and h2 = 5, the error slightly gradually increases with h1 discretization increase and stays around 1 mm for the earlier and within 2–3 mm for the later. Yet, for h2 = 10 and h2 = 20 the error grows more rapidly with h1 discretization increase. Considering the same logics of HRI as for BFA, for MBBFA we recommend using 3D space discretization level h2 = 5 and to keep joint space discretization between 5 and 10°. If RAM consumption is a critical issue, an option of h1 = 5 with h2 = 10 or h2 = 20 could be considered. We believe that going beyond h2 = 5 does not make sense for HRI activities that do not involve object grasping. Additionally, we tested calculation speed of the algorithms on the robot, which is defined as a time between providing the algorithms with (x g , yg , zg ) and posting joint angles’ operation command (θ1 , θ2 , θ3 ) to the robot servos; the manipulator motion execution itself is not taken into an account. While for BFA a single operation calculation speed varied from 0.3 s for h1 = 10–41.5 s for h1 = 1, MBBFA demonstrated a significantly better performance—its access time to any SM cell within a manipulator configurations file was always beyond 1 ms and a solution selection within a cell took from 1 to 4 ms depending on the cell’s population density. Thus, in total MBBFA

72

L. Zagidullin et al.

calculation time was always about 5 ms, which significantly outperformed BFA. Yet, the exact time to fetch a required configuration from the manipulator configurations file depends on the RAM access time and its speed. For the recommended h1 and h2 variables, RAM consumption of BFA and MBBFA was almost the same. We emphasize that RAM consumption is the price that we pay for a faster solution search. It occurs due to a necessity to store the entire configuration file in RAM. Our preliminary analysis allowed to conclude that a high consumption occurred due to the interpreted Python language peculiarities of storing data in RAM. While onboard, 4 GB RAM of the robot allows using both approaches with their current versions of code release, this drawback could be reduced by using compiled programming languages, which could reduce the consumption of RAM for MBBFA. As it was mentioned before, one or our ongoing activities are transfer of the code into C++ language. As for the size of the manipulator configurations files, for the recommended h1 and h2 variables, memory consumption of BFA is 2–3 times higher than of MBBFA (Figs. 5.3 vs. 5.6), which allows MBBFA to employ higher values of the sampling rate h1 .

5.6 Discussion The methods presented above do not consider EE orientation of the Robotis OP2 robot. Two main reasons for such assumption are a lack of grippers, which significantly decreases the importance of the orientation, and a limited reachable workspace of the manipulators (Fig. 5.9). There are very few positions of EE within the workspace that allow a variety of orientations and therefore, in the case of MBBFA, if we extend SM to three additional dimensions (pitch, roll, and yaw), most of SM’s cells will be empty while the memory consumption will increase drastically.

Fig. 5.9 Workspace of the right arm of the Robotis OP2 humanoid (top view)

5 Performance Evaluation of Multigrid Brute-Force Solutions of Inverse …

73

An interesting research direction is decreasing a gap between a theoretically available mathematical precision of EE positioning, presented in this work, and the real robot EE positioning accuracy. Robotis OP2 uses Dynamixel MX-28 servos, and their manufacturer declares a 360° position control without dead zone and 4096 precise resolution by 0.088°. While operating the real robot, it turns out that such precision could not be achieved for a number of reasons, including friction, inertia, and battery charge level. Calibration of joints usually improves an accuracy of robot actions. If expensive calibration systems (e.g., Leica laser trackers [28]) are not available, such calibration could be carried out with offboard or onboard cameras of the robot using computer vision technologies [29]. Using the robot onboard camera to calibrate the joints requires preliminary calibration of the camera itself. In many cases, to achieve this goal, calibration patterns are used that have a known geometry and external unique features that are used to recognize the patterns using computer vision algorithms. A review of the literature showed that among the existing patterns, the most popular approach is to use of flat calibration patterns, e.g., classic checkerboard or circular patterns. Additionally to the classic patterns, there are only a few alternative approaches that use fiducial markers (FM), such as ArUco and AprilTag. The performance of FM-based calibration templates that could be located on a robot surface [30] is still not fully researched, and it is the ongoing work of the project, presented in this paper.

5.7 Conclusions This paper presented two off-line solution for a Robotis OP2 humanoid upper limb positioning via forward kinematics that calculate all possible solutions for an endeffector pose within a robot workspace in advance: a brute-force algorithm (BFA) and a matrix-based brute-force algorithm (MBBFA). Several levels of workspace and joint space discretization allow a user to select a required for his/her task level of the solution precision and calculation speed. Different discretization levels were evaluated at an offboard PC and on the onboard computer of the Robotis OP2 humanoid. The solutions were compared in terms of execution time, memory consumption, and precision. The solutions were initially obtained in the Gazebo simulation and then validated with a real ROBOTIS OP2 humanoid. The results analysis demonstrated that a tradeoff for optimal memory consumption and solution accuracy for BFA is achieved with a joint space discretization step of h1 = 7°. In turn, for MBBFA we recommend using 3D space discretization level h2 = 5 mm and to keep joint space discretization between 5° and 10°. If RAM consumption is a critical issue, an option of h1 = 5° with h2 = 10 mm or h2 = 20 mm could be considered. We believe that going beyond h2 = 5 mm for EE positioning is redundant for HRI activities that do not involve an object grasping or a precise object manipulating. MBBFA approach is preferable over BFA due to a significantly higher calculation speed. Acknowledgements This paper has been supported by RFBR, project number 20-38-90257.

74

L. Zagidullin et al.

References 1. Kolpashchikov, D., Gerget, O., Meshcheryakov, R.: Robotics in healthcare. In: Lim, C.P., Chen, Y.W., Vaidya, A., Mahorkar, C., Jain, L.C. (eds.) Handbook of Artificial Intelligence in Healthcare. Intelligent Systems Reference Library, vol. 212, pp. 281–306 2. Magid, E., Zakiev, A., Tsoy, T., Lavrenov, R., Rizvanov, A.: Automating pandemic mitigation. Adv. Robot. 35(9), 572–89 3. Freschi, C., Ferrari, V., Melfi, F., Ferrari, M., Mosca, F., Cuschieri, A.: Technical review of the da Vinci surgical telemanipulator. Int. J. Med. Robot. Comput. Assist. Surg. 9(4), 396–406 (2013) 4. Li, H., Liu, W., Wang, K., Kawashima, K., Magid, E.: A cable-pulley transmission mechanism for surgical robot with backdrivable capability. Robot. Comput.-Integr. Manuf. 49, 328–334 (2018) 5. Urrea, C., Jara, D.: Design, analysis, and comparison of control strategies for an industrial robotic arm driven by a multi-level inverter. Symmetry 13(86), 1–20 (2021) 6. Kheddar, A., Caron, S., Gergondet, P., Comport, A., Tanguy, A., Ott, C., Henze, B., Mesesan, G., Englsberger, J., Roa, M.A., Wieber, P.B.: Humanoid robots in aircraft manufacturing: the airbus use cases. IEEE Robot. Autom. Mag. 26(4), 30–45 (2019) 7. Voronin, V., Zhdanova, M., Semenishchev, E., Zelenskii, A., Cen, Y., Agaian, S.: Action recognition for the robotics and manufacturing automation using 3-D binary micro-block difference. Int. J. Adv. Manuf. Technol. 117, 2319–2330 (2021) 8. Murphy, R.R.: A decade of rescue robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5448–5449 (2012) 9. Magid, E., Ozawa, K., Tsubouchi, T., Koyanagi, E., Yoshida, T.: Rescue robot navigation: static stability estimation in random step environment. In: International Conference on Simulation, Modeling, and Programming for Autonomous Robots, pp. 305–316 (2008) 10. Gavrilova, L., Petrov, V., Kotik, A., Sagitov, A., Khalitova, L., Tsoy, T.: Pilot study of teaching English language for preschool children with a small-size humanoid robot assistant. In: 12th International Conference on Developments in eSystems Engineering (DeSE), pp. 253–260 (2019) 11. Tian, J., Zhao, J., Zhao, H., Li, Z., Hu, Z.: Research and development of palletizing robot structure and control system. In: IEEE International Conference on Real-time Computing and Robotics (RCAR), pp. 316–321 (2019) 12. Gribkov, A.A., Morozkin, M.S., Kuptsov, V.R., Pivkin, P.M., Zelenskii, A.A.: Industry 4.0 concepts in the machine-tool industry. Russ. Eng. Res. 41(7), 634–635 (2021) 13. Shabalina, K., Sagitov, A., Svinin, M., Magid, E.: Comparing fiducial markers performance for a task of a humanoid robot self-calibration of manipulators: a pilot experimental study. In: International Conference on Interactive Collaborative Robotics, pp. 249–258 (2018) 14. Chaminade, T., Zecca, M., Blakemore, S.J., Takanishi, A., Frith, C.D., Micera, S., Dario, P., Rizzolatti, G., Gallese, V., Umiltà, M.A.: Brain response to a humanoid robot in areas implicated in the perception of human emotional gestures. PLoS ONE 5(7), e11577 (2010) 15. Kahn Jr, P.H., Kanda, T., Ishiguro, H., Gill, B.T., Shen, S., Gary, H.E., Ruckert, J.H.: Will people keep the secret of a humanoid robot? Psychological intimacy in HRI. In: Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction, pp. 173–180 (2015) 16. Stasse, O., Flayols, T., Budhiraja, R., Giraud-Esclasse, K., Carpentier, J., Mirabel, J., Del Prete, A., Souères, P., Mansard, N., Lamiraux, F., Laumond, J.P.: TALOS: a new humanoid research platform targeted for industrial applications. In: IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), pp. 689–695 (2017) 17. Craig, J.J.: Introduction to Robotics: Mechanisms and Controls, 3rd ed. Addison-Wesley (2005) ˇ Filakovský, F., Bulej, V.: A novel approach for 18. Kelemen, M., Virgala, I., Lipták, T., Miková, L, a inverse kinematics solution of a redundant manipulator. Appl. Sci. 8(2229), 1–20 (2018) 19. Zhang, D., Hannaford, B.: IKBT: solving symbolic inverse kinematics with behavior tree. J. Artif. Intell. Res. 65, 457–486 (2019)

5 Performance Evaluation of Multigrid Brute-Force Solutions of Inverse …

75

20. Daya, B., Khawandi, S., Akoum, M.: Applying neural network architecture for inverse kinematics problem in robotics. J. Softw. Eng. Appl. 3(3), 230–239 (2010) 21. KöKer, R.I.: A genetic algorithm approach to a neural-network-based inverse kinematics solution of robotic manipulators based on error minimization. Inf. Sci. 222, 528–543 (2013) 22. Ali, M.A., Park, H.A., Lee, C.S.G.: Closed-form inverse kinematic joint solution for humanoid robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 704–709 (2010) 23. Zagidullin, L., Tsoy, T., Meshcheryakov, R., Hsia, K.-H., Magid, E.: Numerical solution approach for the ROBOTIS OP2 humanoid hand inverse kinematics. In: International Conference on Artificial Life and Robotics, pp. 682–685 (2022) 24. Robotis Company. http://en.robotis.com/. Accessed 3 March 2022 25. Morris, K.J., Anderson, J., Lau, M.C., Baltes, J.: Interaction and learning in a humanoid robot magic performance. In: AAAI Spring Symposium Series, pp. 578–581 (2018) 26. Galin, R., Shiroky, A., Magid, E., Meshcheryakov, R., Mamchenko, M.: Effective functioning of a mixed heterogeneous team in a collaborative robotic system. Inform. Autom. 20(6), 1224– 1253 (2021). https://doi.org/10.15622/ia.20.6.2 27. Klimov, A., Pugachev, P., Polyntsev, E., Prokazina, I., Shandarov, E.: Photon team description for RoboCup humanoid KidSize league (2018) 28. Theissen, N.A., Gonzalez, M.K., Barrios, A., Archenti, A.: Quasi-static compliance calibration of serial articulated industrial manipulators. Int. J. Autom. Technol. 15(5), 590–598 (2021) 29. Boby, R.A., Saha, S.K.: Single image based camera calibration and pose estimation of the endeffector of a robot. In: IEEE International Conference on Robotics and Automation, pp. 2435– 2440 (2016) 30. Tsoy, T., Safin, R., Magid, E., Saha, S.K.: Estimation of 4-DoF manipulator optimal configuration for autonomous camera calibration of a mobile robot using on-board templates. In: 15th Siberian Conference on Control and Communications, pp. 1–6 (2021)

Chapter 6

Neural Network Approach for Solving Inverse Kinematics Problem of Modular Reconfigurable Systems Aleksei Erashov , Anton Saveliev , and Dmitriy Blinov

Abstract The using of the modular principle in robotic systems allows implementing a system, which typical components are interchangeable. In view of the possible reconfiguration of modular robotic systems, the development of suitable algorithms and controllers is required, which will make it possible to control modular systems taking into account various formations. The features of this problem solution are the dependence of the result on the object’s kinematic structure, as well as the solutions multiplicity, that denotes the values variability of the intermediate links generalized coordinates of the robotic system for the same workspace target point. The purpose of this work is to develop algorithms using a neural network approach to solve the inverse kinematics problem of a modular robotic system. An algorithm for artificial neural network training is proposed to solve the inverse kinematics problem of different robot’s formations. Besides, an algorithm for automatic generation of a training dataset was also developed, which allows to take into account possible kinematic substructures of a certain robot formation. The developed algorithms were tested on models of manipulator and snake type formations with different numbers of links. The final training accuracy for 5000 examples lies in the range from 0.4 to 2.925°.

A. Erashov (B) · A. Saveliev St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, 39, 14th Line, 199178 St. Petersburg, Russia e-mail: [email protected] D. Blinov St. Petersburg State University of Aerospace Instrumentation, Bolshaya Morskaya 67, 190121 St. Petersburg, Russia © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_6

77

78

A. Erashov et al.

6.1 Introduction Modular robotic systems are a class of multi-link mechanisms with a variable kinematic structure. As a rule, this is achieved through using of modules with the same type of design. Robotic modules are self-powered or powered by a common module that acts as the main power source for the robotic system (RS). Often, these modules contain built-in computing and control devices. In the design of each module, various types of joints can be implemented, as a rule, rotary ones. Thus, one module can have several degrees of freedom, which for the RS as a whole indicates that by connecting several modules, system can get a redundant number of degrees of freedom, that is, more than six. This significantly increases the complexity of the analytical solution of the inverse kinematics (IK) problem of the whole system [1, 2]. From the point of view of modular robotic systems using, it is important to consider the possibility of reconfiguring the system, which will allow using this system to perform various types of tasks. For example, such tasks include the movement of goods, the manipulation of objects, and the study of hard-to-reach and dangerous objects, including participation in rescue operations [3, 4]. For multitasking systems, it is important to implement a method for solving the IK problem, which will allow finding a solution for various RS formations.

6.2 Related Works The applied methods for solving the IK problem in robotics are divided into analytical and numerical methods [5], as well as widely used methods and algorithms of machine learning [6, 7]. Methods for solving the IK problem based on the analytical solution are divided into algebraic and geometric. Both approaches use the analysis of the kinematic scheme. To carry out this analysis, it is possible to use various methods for describing the kinematic parameters of the robot, one of which is the widely used Denavit– Hartenberg method [5]. Algebraic methods include the definition of kinematic equations, according to which it is possible to find the position of the end-effector and links in the working space according to the known generalized coordinates of the robot, that is, to solve the forward kinematics (FK) problem. To solve the IK problem, the generalized coordinates of the robot are expressed from the obtained kinematic equations in terms of parameters that describe the position of the links in space. As a result of the application of geometric methods for solving the IK problem, analytical expressions can be found similar to those determined using algebraic methods [5]. Geometric solution consists of considering of the robot’s kinematic scheme in different planes, which allows using known geometric relationships to search for expressions by which the generalized coordinates of the robot are calculated. In paper [8], analytical solution of the IK problem of the KUKA LBR iiwa manipulator with 7° of freedom

6 Neural Network Approach for Solving Inverse Kinematics Problem …

79

is considered. This solution combines geometric and analytical methods. The solution of the IK problem is based on the limitations introduced by the authors on the bend angle in the elbow joint of the robot, as well as on the division of the working space to search for a collision-free trajectory. To analyze the kinematic scheme, the Denavit–Hartenberg parameters were used. For an anthropomorphic manipulator with a similar kinematic scheme, an analytical solution was proposed, which is based on the avoidance of singularities in the joints [9]. The proposed solution uses two additional parameters: arm angle to describe the proper movement of the robot’s elbow joint and orientation control to limit the number of IK solutions. Analysis of the simulation results showed that the proposed solution can be used to control the manipulator in real time. Analytical methods for solving the IK problem in general have a relatively low computational complexity; however, from the point of view of modular reconfigurable robotic systems, these methods require a reanalysis of the kinematic scheme and the determination of equations for solving the IK after reconfiguration or changing the modules. In addition, it is necessary to introduce additional restrictions that will allow avoiding the plurality of decisions of the IK problem. Numerical methods use numerical optimization methods, which allow finding a suboptimal solution, based on kinematic equations obtained using analytical methods at the stage of kinematic analysis. Among the numerical methods, there is the iterative algorithm new inverse kinematics algorithm (NIKA) [10], the objective function of which determines the positioning error of the manipulator’s end-effector relative to the target point. The advantage of NIKA is its ease of use due to the representation of the kinematics equations using the Denavit–Hartenberg parameters, which makes it possible to use the algorithm to find solution of the IK problem with different robot’s geometric characteristics. It is also developed the heuristic method forward and backward reaching inverse kinematic (FABRIK) [11], which allows, through iterative calculation of the end-effector position, to determine the generalized coordinates by solving the problem of finding a point on the line connecting the position of two neighboring links. Numerical Newton–Raphson method [12] allows to find the roots of a nonlinear equations system using a first-order approximation. This method has a slow convergence for cases with a significant number of nonlinearities in the original equations. However, most numerical methods are characterized by low convergence rate and high computational complexity [13]. Among the methods of machine learning in robotics, methods using artificial neural networks, methods using stochastic optimization algorithms, the JSM-method, as well as reinforcement learning methods are used. To solve the IK problem of a plane manipulator, a solution method using a feedforward neural network is proposed [14]. The developed solution consists of using a random value of angles in the joints of the robot to train the network, after which the dataset is supplemented with the values of the corresponding coordinates of the end-effector. The resulting dataset is used to train the model, which is then used to solve the IK problem. Another approach is considered in [15]. The authors proposed to consider the current angles in the joints of the robot, provided that the robot moves the end-effector along the

80

A. Erashov et al.

trajectory. One of the disadvantages of methods using artificial neural networks is the difficulty in obtaining a representative training dataset. In modular robotics, such stochastic optimization methods, for example, the genetic algorithm [16] and particle swarm optimization [17, 18], were applied. The use of genetic algorithms for solving the IK problem of manipulators showed a high accuracy of the solution [19]. In [20] for the synthesis of a control system for a walking robot, a JSM method is proposed. As a result of modeling, the author found that, compared with neural network methods, the JSM-method allows to get a higher learning rate and a high speed of obtaining a solution to the IK problem. Reinforcement learning methods for solving the IK problem use the mechanism of penalties and rewards for certain actions [6, 7, 21]. These methods use the function of the distance between the end-effector and the target point as a reward function, and iterative increments of generalized coordinates in the joints are considered as actions. The most popular reinforcement learning algorithm is deep Q-learning [22]. The study [7] proposes to use deep Q-learning to solve the IK problem to find the optimal position of a surgical manipulator with six degrees of freedom. As a criterion for the quality of the solution of the IK problem, the time the end-effector reached the target point was used. The considered numerical and analytical methods and methods of machine learning can be applied to solve the IK problem of a multi-link modular RS. Analytical methods are characterized by less computational complexity and high accuracy relative other methods. However, the features of the kinematic scheme can cause a multiplicity of solutions to the IK problem, and this group of methods also requires preliminary transformations of the kinematic equations, which requires more computational and time resources to obtain a problem solution. An analytical solution of the IK problem of a modular system can be obtained when this system consists of a small number of modules; however, changing the composition of the modules and reconfiguring the system into a state with a different kinematic structure require finding the solution of the IK problem again. Numerical methods are characterized by the uniqueness of the solutions of the IK problem, but this requires much more time to search in comparison with analytical methods. At the same time, a group of methods for solving the IK using machine learning is singled out, the computational complexity of which does not depend on the number of modules when system is working in real time. These methods, as well as numerical methods, do not require preliminary calculations and, due to preliminary training, reduce computational costs when controlling a modular multi-link system in real time. The application of machine learning methods, such as reinforcement learning methods or the JSM-method, is potentially applicable for solving the IK problem of reconfigurable modular RS. These methods can be applied to various modular RS formations without additional or significant modifications of control algorithms. Reinforcement learning makes the solution method scalable and can be applied to various formations of multi-link modular systems. Thus, for multi-link modular systems, even if they do not have a total degree of freedom greater than six, it is significantly difficult to solve the IK problem by

6 Neural Network Approach for Solving Inverse Kinematics Problem …

81

analytical and numerical methods, since these systems have a dynamic kinematic structure. In this regard, it is important to implement a method that will allow, without significant changes in the control algorithms of the RS, to solve the IK problem to control various formations of the modular RS. The development of such a method is the aim of this work.

6.3 Material and Methods The IK problem solution of multi-link modular RS is to search for such positions of n joints-modules (ϕ 1 , ϕ 2 , …, ϕ n ) that provide the required position of the end-effector or individual elements of the modular robot (x c , yc , zc ). In addition, the IK solution is complicated by the large variety of formations that this system can form through reconfiguration, with a different number of degrees of freedom. Most significantly in the context of solving the IK problem, this diversity affects formations with chain and mixed structures. Diagrams of various formations of the chain structure are presented in Fig. 6.1. Figure 6.1 shows the connections of the modules of the modular RS is considered in this work. The initial position of the end-effector is at the point with coordinates (x 0 , y0 , z0 ).

Fig. 6.1 Schemes of the modular RS chain type formations, which consist of a one module, b two modules, and c three modules

82

A. Erashov et al.

The use of artificial neural networks (ANNs) for solving IK problem refers to methods of supervised machine learning, during which an ANN model receives interconnected data and displays dependencies between them [23]. When solving the IK problem, the input data are the position matrix of the end-effector, and the output data are the states of the joints.

6.3.1 Algorithm of ANNs Learning for Solving IK Problem of Modular RSs There are three main tasks that need to be performed in order to accurately solve the IK using deep neural networks: to obtain the most representative dataset, to choose the most appropriate neural network architecture and training hyperparameters. The first and third tasks are solved within the framework of the developed ANN training algorithm for solving the IK problem of modular RSs, shown in Fig. 6.2. The input data of this algorithm are the next: direct kinematic models are mathematical descriptions of models for simulation; the size of the training dataset; ANN parameters for each of the formations; names or numbers of joints at the joints of modules; parameters for connecting to the simulation; and ANN macro-parameters. At the first step of the algorithm, a representative dataset is formed for the parent and child formations, that is, it is understood that the child formations are kinematic structures (subsets) contained in the general structure of the modular RS. This general structure, as noted above, is set by the user. The sample is generated using a separate algorithm, which is discussed in more detail below (Fig. 6.3). As a result of the execution of the sample generation algorithm, the generated random samples are written to files so that the obtained data can be reused. Further, all duplicate data pairs are excluded from the generated sample in the “duplicate removal” block. This is necessary to increase the representativeness of the dataset. This screening mechanism consists in the fact that the obtained coordinates are rounded to two decimal places, after which they are compared in pairs, in case of repetition of any result, one of the pairs is removed from the sample. The next step of the algorithm is the learning process of the ANN. At the first stage of training, the user specifies the parameters of the hidden layers of the neural network. Due to the increase in computational costs when removing duplicates and the increase in training time, it is not possible to obtain a large training sample, and therefore, at the verification stage, the estimates obtained can vary greatly depending on whether these estimates fall into the testing or training sets. This will affect the evaluation of the quality of the model. Therefore, this algorithm uses k-block crossvalidation [24] to set the optimal number of training epochs. In this algorithm, the resulting data array is divided into five blocks, while creating five copies of neural network models and training each ANN on four blocks with an estimate for the remaining blocks. After that, according to the obtained estimates, the average value is calculated, which is taken as a general test criterion. Finally, the best estimate of the

6 Neural Network Approach for Solving Inverse Kinematics Problem …

83

Begin Dataset generation Duplicate removal For each formation IK solving Saving model no

Is result correct?

yes

Get new ANNs parameters ANN compilation

Final learning

Checking by kblocks

Estimation of error End

Fig. 6.2 Diagram of an ANN learning algorithm for solving IK problem of modular RS Fig. 6.3 Block scheme of the dataset generation algorithm

Begin

yes

Is dataset ready?

no

Generation of random angles set Saving dataset

Forward kinematics solving

yes

Are there “child” formations?

Extraction of “child” formation End

no

84

A. Erashov et al.

training error is found and retrained on the full dataset. If the obtained results satisfy the required accuracy, then the algorithm proceeds to solve the IK problem of the next formation, otherwise the user is requested to make changes to the ANN parameters. As a measure of accuracy, this algorithm uses the average absolute deviation between the angles predicted by the ANN and the actual values of the angles at the joints obtained during the generation of the sample. After successful learning to solve the IK problem of all formations, weights and ANN models are stored for further use.

6.3.2 Dataset Generation In the developed dataset generation algorithm, there are two ways to obtain a sample: 1. Analytical is using Denavit–Hartenberg parameters. 2. Using simulation. The analytical method requires knowledge of the Denavit–Hartenberg parameters to construct the resulting matrices. The second method, which uses simulation, requires a robot model (for example, in the Universal Robot Description Forma is URDF). However, both methods using for creating a sample of the modular system parameters is accompanied by the following problems. In the case of the analytical method, due to the large number of matrix calculations, more computing power is required, and when using the simulation method, a model is required for each formation of a modular RS. In addition, when filtering out repeated pairs of parameters, we have to separately sort out the sets of angles for each formation. To solve the problems described above, an algorithm was developed for obtaining a representative dataset of pairs of joint states coordinates of end-effector. The block diagram of this algorithm is shown in Fig. 6.3. This algorithm is based on the principle of repeatability of the kinematic schemes of the module in multi-link modular systems. The algorithm allows to simultaneously obtaining datasets on several formations with a chain or mixed structure, consisting of modules with similar geometry. For this algorithm to work, it is required to specify the parameters or model of only one parent formation, as well as the number of joints in one module. Based on the data of the parent formation, its descendants are child formations will be extracted. Therefore, with the analytical method of obtaining a sample, intermediate resulting matrices will be used as descendants, and when using simulation, the positions of the intermediate links of the modular RS will be extracted. The input of this algorithm is the number of child formations, as well as the required size of the data array for training the ANN. The developed algorithm cyclically replenishes the training dataset until the desired sample size is reached. The corresponding check is carried out in the “Is dataset ready?” block. If the desired sample size is not reached, the generation of random sets of angle values at the RS joints continues, which are then used to solve the FK problem. After receiving the

6 Neural Network Approach for Solving Inverse Kinematics Problem …

85

solution, it is checked whether there are child formations left: if the “child” kinematic structures remain, then this formation is extracted, and the FK is solved until the child formations run out. If the child kinematic structures are over, then for the entire kinematic structure of the RS, steps are repeated to obtain data for training. At the end of the process of obtaining data for training, the dataset is saved, which is used in the subsequent steps of the main algorithm as shown in Fig. 6.2.

6.4 Experiments of Modeling an Autonomous Robotic System To test the developed solutions, experiments were carried out, which were divided into two stages. The first stage was the training of the ANN models, and the second one was the verification of the solution of the IK problem by random values of the angles in the joints. At the first stage of the experiments, the developed algorithms were applied to train the ANN, while the data for its training were generated using models of a mobile autonomous robotic system (MARS). Two types of formations were made from these modules: manipulator and snake. The first formation was divided into three (Fig. 4a), two (Fig. 4b), and one movable module (Fig. 4c). The “snake” formation had a similar partition: three (Fig. 4d), two (Fig. 4e), and one movable module (Fig. 4f). These formations of the RS, composed of MARS modules, have a chain kinematic scheme, while all modules have a similar geometry and kinematic structure. To start training, it is necessary to specify the size of the training dataset and the joint that will act as an end-effector. In the experiments, the size of the training set was 5000 elements, reflecting the possible states of the studied modular RS (Fig. 6.5). The joint located at the end of the formation was chosen as the end-effector. The trained ANN consisted of multiple hidden layers, the root mean square error was used as a loss criterion, while an optimizer was used that implements the Adam algorithm (stochastic gradient descent method based on adaptive estimation of the first and second order moments) [25, 26]. The hardware used in the experiments included an Intel Core i5-4690 K 3.5 Hz processor, 16 GB of RAM, and an NVIDIA GeForce GTX 970 4 GB graphics card. In the course of the learning algorithm, the parameters of the hidden layers were determined, which give the smallest average deviation error between the values of the angles in the joints. For formations consisting of three modules, these are three hidden layers of 256 neurons each with a pruning of 20%. For formations consisting of two modules, three hidden layers of 64 neurons each without pruning, and for a single module, three hidden layers of 32 neurons each without pruning were obtained. In the process of ANN training, the average absolute error was calculated after the training epoch for both formations. Curves of changes in this parameter were obtained. The graph of the change in the average learning error of the ANN for solving the IK of a modular RS, consisting of one module, is shown in Fig. 6.6.

86 Fig. 6.4 Investigates formations of the modular RS

Fig. 6.5 Visualization of generation dataset process

A. Erashov et al.

6 Neural Network Approach for Solving Inverse Kinematics Problem …

87

Fig. 6.6 Graph of the change in the predicting average angle error of a modular RS, consisting of one module

The values along the ordinates of all subsequent graphs are the result of normalizing the values of the average error, taking into account the standard deviation and the average value obtained after training the ANN models. Each normalized value is the difference between the mean error and the mean modulo errors divided by the standard deviation. Using the k-block check, it was determined that for a system of one MARS mobile module, the smallest error is achieved at the 79th training epoch. As can be seen from the presented graph, during the first epochs of training, the average absolute error decreased significantly, but after the 10th epoch, the rate of change in the error became much less. The denormalized value of the deviation between the values of the angles corresponds to an average of 0.4°. Figure 6.7 shows the curve of the change in the average absolute error when training the ANN to solve the IK problem of a system consisting of two modules. By checking for k-blocks, it was found that for a system of two MARS modules, the smallest error is obtained at the 89th training epoch. At the same time, during the first training epochs, the average absolute error decreased significantly. The calculated denormalized deviation between the values of the angles corresponds to an average of 0.4815°. The resulting average absolute error change curve for a system of three MARS modules is shown in Fig. 6.8. Using k-block testing, it was determined that the number of epochs needed to obtain the smallest error is 78 epochs. The resulting denormalized mean error was 2.925°. The second stage of proposed methods approbation was to check the accuracy of the IK problem solution using the obtained ANN models. For this, a test set T ϕ was generated from 1000 tuples t ϕ i = (ϕ 0 , …, ϕ W )i |i = 1…1000, where W is

88

A. Erashov et al.

Fig. 6.7 Graph of the change in the predicting average angle error of a modular RS, consisting of two modules

Fig. 6.8 Curve of the change in the predicting average angle error of a modular RS, consisting of three modules

the number of generalized coordinates of the modular system. In these tuples t ϕ i ∈ T ϕ , random values of the angles at the joints of each kinematic structure were recorded. Further, for each tuple of angles, the coordinates of the target point t ci = (x i , yi , zi ) were found through the analytical solution of the FK. To determine the analytical solution, a modified Denavit–Hartenberg technique was used [5]. Then,

6 Neural Network Approach for Solving Inverse Kinematics Problem …

89

Table 6.1 Absolute average deviations at two stages of experiments System parameter

One module

Two modules

Three modules

0.4

0.4815

2.925

Average angle deviation, grad

0.20

0.39

1.98

Average coordinates deviation (m)

0.0005

0.0015

0.0080

Stage I. learning Average angle deviation, grad Stage II. testing

these coordinates (x i , yi , zi ) were input to the ANN models to solve the IK problem, that is, to find the values of the angles (ϕ E0 , …, ϕ EW ), where ϕ Ej | j = 1…W is the prediction of the angle value obtained using the ANN. Finally, the accuracy of the IK solution was calculated, which was calculated as the absolute deviation between the given and obtained using the ANN angle values. The results obtained are given in Table 6.1 (see stage II). The results of the experiments showed that the average absolute deviation for the RS formations with different number of modules was 0.20°, 0.39°, and 1.98°, respectively. When expressed in terms of coordinates using the FK solution, the average deviation was 0.0005 m, 0.0015 m, and 0.0080 m, respectively. Thus, the proposed solution can be used to solve the IK problem of a modular RS that accepts various formations. In addition, the proposed solution allows to automatically obtain a training dataset for possible child formations of a multi-link RS in general.

6.5 Conclusion In this work, an algorithm for learning ANN designed to solve the IK of a modular multi-link RS is proposed. A feature of the designed solution is the possibility of automatically obtaining neural network models that will allow solving the IK for individual formations of modular RS. As a drawback of the proposed learning algorithm, one can note the manual selection of ANN hyperparameters in the learning process. The developed dataset generation algorithm makes it possible to obtain a training set for various child formations of a modular RS, which will allow solving the IK to control the modular system while maintaining a specific type of formation in case the number of modules has changed. In addition, the developed algorithm optionally makes it possible to obtain a sample for one specific formation without taking into account kinematic substructures and train the ANN model in order to solve the IK to control this RS formation. In general, the proposed algorithms have the potential to be used for training ANN models, which will allow solving the IK of a modular RS in various formations. The final training accuracy for 5000 examples lies in the range from 0.4–2.925°. When checking the accuracy of the IK solution,

90

A. Erashov et al.

the obtained average deviations did not exceed the deviations obtained at the stage of training ANN models. Further research will be aimed at automating the process of selection of ANN hyperparameters, at increasing the accuracy of the IK solution, as well as at testing the developed solutions for other formations of a modular multi-link RS. Acknowledgements This research is supported by RFBR project No. 20-08-01109_A.

References 1. Colomé, A., Torras, C.: Closed-loop inverse kinematics for redundant robots: comparative assessment and two enhancements. IEEE/ASME Trans. Mechatron. 20(2), 944–955 (2014). https://doi.org/10.1109/TMECH.2014.2326304 2. Singh, G.K., Claassens, J.: An analytical solution for the inverse kinematics of a redundant 7dof manipulator with link offsets. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and systems, pp. 2976–2982 (2010). https://doi.org/IROS.2010.5649095 3. Yim, M., David, D.G., Roufas, K.: Modular reconfigurable robots, an approach to urban search and rescue. In: Proceedings of the 1st International Workshop on Human-friendly Welfare Robotics Systems, pp. 69–76 (2000) 4. Yim, M., Shen, W.M., Salemi, B., Rus, D., Moll, M., Lipson, H., Chirikjian, G.S.: Modular selfreconfigurable robot systems [grand challenges of robotics]. IEEE Robot. Auto. Mag. 14(1), 43–52 (2007). https://doi.org/10.1109/MRA.2007.339623 5. Craig, J.J.: Introduction to robotics: mechanics and control (2005) 6. Ren, H., Ben-Tzvi, P.: Learning inverse kinematics and dynamics of a robotic manipulator using generative adversarial networks. Robot. Auton. Syst. 124, 103386 (2020). https://doi. org/10.1016/j.robot.2019.103386 7. Ansari, Y., Falotico, E., Mollard, Y., Busch, B., Cianchetti, M., Laschi, C.: A multiagent reinforcement learning approach for inverse kinematics of high dimensional manipulators with precision positioning. In: 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob), pp. 457–463 (2016). https://doi.org/10.1109/BIOROB.2016. 7523669 8. Gong, M., Li X., Zhang, L.: Analytical inverse kinematics and self-motion application for 7DOF redundant manipulator 7, 18662–18674 (2019). https://doi.org/10.1109/ACCESS.2019. 2895741 9. Tian, X., Xu, Q., Zhan, Q.: An analytical inverse kinematics solution with joint limits avoidance of 7-DOF anthropomorphic manipulators without offset. J. Franklin Inst. 358(2), 1252–1272 (2021). https://doi.org/10.1016/j.jfranklin.2020.11.020 10. Kucuk, S., Bingul, Z.: Inverse kinematics solutions for industrial robot manipulators with offset wrists. Appl. Math. Model. 38(7–8), 1983–1999 (2014). https://doi.org/10.1016/j.apm.2013. 10.014 11. Aristidou, A., Lasenby, J.: FABRIK: a fast, iterative solver for the inverse kinematics problem. Graph. Models 73(5), 243–260 (2011). https://doi.org/10.1016/j.gmod.2011.05.003 12. Merlet, J.P.: Interval analysis and robotics. Robot. Res. 147–156 (2010). https://doi.org/10. 1007/978-3-642-14743-2_13 13. Aristidou, A., Lasenby, J.: Inverse kinematics: a review of existing techniques and introduction of a new fast iterative solver (2009) 14. Duka, A.V.: Neural network based inverse kinematics solution for trajectory tracking of a robotic arm. Procedia Technol. 12, 20–27 (2014). https://doi.org/10.1016/j.protcy.2013.12.451

6 Neural Network Approach for Solving Inverse Kinematics Problem …

91

15. Almusawi, A.R.J., Dülger, L.C., Kapucu, S.: A new artificial neural network approach in solving inverse kinematics of robotic arm (Denso VP6242). Comput. Intell. Neurosci. (2016). https:// doi.org/10.1155/2016/5720163 16. Wu, W., Guan, Y., Li, H., Su, M., Zhu, H., Zhou, X., Zhang, H.: Task- oriented inverse kinematics of modular reconfigurable robots. In: 2013 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp. 1187–1192 (2013). https://doi.org/10.1109/AIM.2013.6584255 17. Du, Y., Wu, Y.: Application of IPSO algorithm to inverse kinematics solution of reconfigurable modular robots. In: 2011 International Conference on Mechatronic Science, Electric Engineering and Computer (MEC), pp. 1313–1316 (2011). https://doi.org/10.1109/MEC.2011.602 5711 18. Collinsm, T.J., Shen, W.M.: Particle swarm optimization for high-DOF inverse kinematics. In: 2017 3rd International Conference on Control, Automation and Robotics (ICCAR), pp. 1–6 (2017). https://doi.org/10.1109/ICCAR.2017.7942651 19. Galemov, R.T., Masalsky, G.B.: Combined search method for solving the inverse problem of the kinematics of a multi-link manipulator. Mechatronika, Avtomatizatsiya, Upravlenie 19(7), 464–473 (2018). (In Russ). https://doi.org/10.17587/mau.19.464-473 20. Dobrynin, D.: Simulation of trainable control system for quadruped robot. Electromech. Robot. 155–164 (2022). https://doi.org/10.1007/978-981-16-2814-6_14 21. Phaniteja, S., Dewangan, P., Guhan, P., Sarkar, A., Krishna, K.: A deep reinforcement learning approach for dynamically stable inverse kinematics of humanoid robots. In: 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1818–1823 (2017). https:// doi.org/10.1109/ROBIO.2017.8324682 22. Blinov, D., Saveliev, A., Shabanova, A.: Deep Q-learning algorithm for solving inverse kinematics of four-link manipulator. In: Proceedings of 15th International Conference on Electromechanics and Robotics “Zavalishin’s Readings”, pp. 279–291 (2021). https://doi.org/10. 1007/978-981-15-5580-0_23 23. Kriegeskorte, N., Golan, T.: Neural network models and deep learning. Curr. Biol. 29(7), R231–R236 (2019). https://doi.org/10.1016/j.cub.2019.02.034 24. Carmesin, J., Diestel, R., Hamann, M., Hundertmark, F.: k-Blocks: a connectivity invariant for graphs. SIAM J. Discret. Math. 28(4), 1876–1891 (2014). https://doi.org/10.1137/130923646 25. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: arXiv preprint arXiv: 1412.6980 (2014) 26. Erashov, A., Kamynin, K., Krestovnikov, K., Saveliev, A.: Method for estimating time of wireless transfer of energy resources between two robots. Inf. Autom. 20(6), 1279–1306 (2021). https://doi.org/10.15622/ia.20.6.4

Chapter 7

Algorithm of Trajectories Synthesis for Modular Wheeled Inspection Robot Ildar Nasibullayev

and Oleg Darintsev

Abstract Motion trajectories synthesis of mobile modular robots for complex spaces is based on two types of mathematical models: pattern motion trajectories or a system of high-dimensional differential equations. For the first option, a limited set of basic types of trajectories patterns is constructed, the combination of which allows to model complex robot movements with sufficient accuracy. In the second option, the significant computational resources are required for the numerical solution of the equations system, and only in the specific cases of trajectories it is possible to obtain analytical solutions. The present chapter proposes alternative mathematical and computer models using fast-count algorithms obtained as a result of a complex analysis of structural and kinematic features of a mobile modular robot. The resulting 2D models are the basis for solving more complex trajectory synthesis issues: constructing an arbitrary trajectory of a modular robot in a 3D working space; trajectories in a space with obstacles; and the construction of a dynamic model and parametric synthesis of the robots modules and joints structure ensuring the guaranteed passage along the required trajectory.

7.1 Introduction Modern industrial and municipal pipelines have a large length, a significant number of branches, and consist of pipes of various diameters. The use of robotic systems for inspection and repair of pipelines is an advanced and urgent task to keep pipelines operational. The existing inspection robots [1] are characterized by a variety of design solutions, which are reflected in the inspection technologies they provide. The trunk robots used in oil and product lines are practically standard and oriented toward I. Nasibullayev · O. Darintsev (B) Mavlyutov Institute of Mechanics, UFRC RAS, 71, Pr. Oktyabrya, Ufa 450054, Russia e-mail: [email protected] O. Darintsev Ufa State Aviation Technical University, 12, Karl Marx Str., Ufa 450008, Russia © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_7

93

94

I. Nasibullayev and O. Darintsev

pipes with a fixed diameter with small curvature or height difference [2]. Industrial inspection robots are used in complex topology pipeline plants, so the design incorporates sequentially connected active (ensuring the module movement with radially positioned tubes pressed against the pipe wall) and technological (for scanning, repairing, cleaning, etc., of the pipe wall) modules [3]. The modules joint connection allows the robots to move in pipes with small changes in the diameter of the cross section and to move into branches of pipes but only at the same size of cross section. Municipal (urban) pipelines are distinguished by a wider range of pipe diameters, many fittings, and a complex topology. Therefore, modular robots equipped with a special spacer structure are used to inspect such systems: the robot [4], built on active and passive modules, can move in pipes with a diameter of 130–200 mm. In [5], a two modules robot is presented and designed for inspection of pipes with a diameter of 300–500 mm, moving speed up to 18 sm/s. A special class of inspection systems are snake-like robots, which are moved by changing the configuration of modules, which limits the speed of movement. Thus, the work [6] represents a robot capable of crawling on the inner or outer surface of a pipe with a diameter of 75–300 mm, wherein its speed is limited to 2 sm/s. Inspection robots that are held on the surface of metal pipes by magnets are used, which, compared to modular robots, allows them to move through a system of pipes with a wider range of diameters. The work [7] presents a robot for metal pipelines with a diameter of >50 mm, pressed against the surface by permanent neodymium magnets, which limits its use, as ceramic and plastic pipes have become widespread in recent years. According to the method of movement, the most common inspection robots are represented by two classes. Wheeled robots can develop a speed enough to travel through long pipelines but have limited a cross-country capability, which is determined by the size of surface defects or obstacles. Compared to wheeled robots, crawler robots have a higher cross-country capability at a lower speed of movement and a higher energy consumption; there is also a high probability of damage to the surface on which the movement takes place. In order to increase the efficiency of inspection robots by multiplying speed, authors of the present work have proposed a new design of a modular wheeled robot which scans the inner surface of the pipe, moving along a helical trajectory [8]. For the inspection robots currently in use, there is no need to plan trajectories, as can be seen from the examples above, since most of them are push-through systems, unable to change even their orientation because they occupy almost the entire section of the pipe. Besides, the speed of movement of the robot is so low that the time to perform the calculation of the new trajectory almost does not lead to a significant decrease in the average speed. Therefore, the movement method specifics and high reconfiguration speeds require the development of dynamic models necessary for the synthesis of robot design, planning, and control algorithms. A 2D dynamic model of a modular wheeled robot is proposed by Pavlovsky [9] in the form of a system of differential equations, admitting two partial solutions— motion in a straight line and in a circle. In the following work [10], analytic solutions are obtained using special functions (Fresnel integrals) for moving the robot along trajectories (straight line, circle, and spiral) and defined moments of force, which

7 Algorithm of Trajectories Synthesis for Modular Wheeled Inspection …

95

must be applied to the wheels of the modules in order to ensure that they follow a given trajectory. In [11], a mathematical model of the dynamics of a modular robot (one or two rigidly connected active or passive modules) in a circular tube based on the Euler–Lagrange equations is constructed, and its experimental verification was carried out. The following motion parameters are determined: rolling and sliding friction forces, the inertia moment of the robot, the stopping time of the robot, the width of the stagnation zone, etc. The 3D computer model of the configuration of the robot modules is represented in the work [8] in the form of basic patterns of the displacement trajectory (plane, circle, helical line, and line along the pipe) and their combinations (transition from one configuration corresponding to a pattern to another). In the considered works, it was found that the driven modules deviate from the trajectory of the leading module depending on the selected movement pattern (a similar feature is present in macro systems, for instance, when driving a road train [12]). The use of special functions in analytical solutions when moving along trajectory patterns and the numerical solution of a system of differential equations when moving along an arbitrary trajectory require large computational resources, which makes it difficult to use models in the real time robot control system. This chapter shows mathematical and computer models, as well as the implementation of a quick-calculation algorithm for synthesizing the trajectories of the driven modules with arbitrary or specified movement of the leading module on the plane [13]. In the future, the model is planned to be used to develop a 3D model of the movement of an inspection modular wheeled robot along an arbitrary trajectory, including with the presence of obstacles.

7.2 Mathematical Model Figure 7.1 shows the geometry of the robot and the first module trajectory. The Cartesian coordinate system is used: the origin of the coordinate system O corresponds to the position of the geometric centers (GC) of the first module during the transition from the rectilinear trajectory L 1 to the circular one L c ; the axis Ox is directed along the movement in the first section of the trajectory L 1 , and the axis Oy is perpendicular, forming the right coordinate system. The model considers the movement of a robot consisting of n modules connected in series by ball joints, with the coordinates of GC modules (x j,i , yj,i ), where j = (1, n), i is the time step number. The first (leading) module moves along a given trajectory consisting of three parts: a rectilinear one with length L 1 ; an arc of a circle with length L c = Rα, where R is the radius of the circle and α is the central angle; and a rectilinear one with length L 2 . The time step number corresponding to the beginning of movement along these parts of the trajectory is denoted by i1 , i2 , and i3 . The geometric dimensions of the modules are determined by the distance from GC to the center of the joint L b (the removal of the joint relative to GC), the distance from GC to the extreme point of

96

I. Nasibullayev and O. Darintsev

Fig. 7.1 Robot geometry and the leading module trajectory

the wheel L w (2L w is the distance between the wheels), and the radius of the wheels Rw . The initial position of the modules was set as follows: x j,i1 = −L 1 − 2( j − 1)L b , y j,i1 = 0, β j,i1 = 0, j = (1, n),

(7.1)

where β j,i1 is the initial angle of rotation of the baseline L b the jth module from the Ox axis. When the leading module moves at a speed of v, the module will move ℓ = vτ over a period. The coordinates of the leading module GC when moving along rectilinear sections at the ith time step will be equal to: x1,i = x1,i−1 + ℓ cos β1,i , y1,i = y1,i−1 + ℓ sin β1,i ,

(7.2)

where the angle of rotation of the module is βj,I = 0 on the first section of the trajectory and βj,I = α on the third. At the i2 time step, the leading module switches from moving along a rectilinear trajectory to moving along the arc of a circle, where the coordinates of the center of the circle are set (x c , yc ), the direction of movement d (d = 1 is counter clock-wise and d = −1 is clock-wise) and the length of the arc β 0 , and the passage of which takes time τ: xc = x1,i2 − d · R sin β1,i2 , yc = y1,i2 + d · R cos β1,i2 , β0 =

ℓ . R

(7.3)

The coordinates of the GC and the angle of rotation of its baseline at the ith time step are determined as follows: x1,i = xc + d · R sin β1,i , y1,i = yc − d · R cos β1,i , β1,i = β1,i2 + i · β0 .

(7.4)

Note that after selecting the initial configuration (for example, in the form of (7.1)) Eqs. (7.2)–(7.4) can be combined in any order to create complex trajectories of the

7 Algorithm of Trajectories Synthesis for Modular Wheeled Inspection …

97

Fig. 7.2 Scheme of algorithms for moving the driven module TR (a) and RT (b)

leading module, consisting of both linear parts with different angles of inclination α to the axis Ox, and of the parts in the form of circular arcs with different radii R. The movement of each subsequent (driven) module is determined by the movement of the previous one according to one of two algorithms (see Fig. 7.2). In the first algorithm (let’s denote TR), after moving the j−1st module, the jth module must first move by some distance L j,i along its direction of motion βj,i−1 . This value is determined from the condition that the distance from the GC of the j−1st module to the center of the joint of the jth module is invariable and equal to the L b , i.e., the constraint equation is as follows: (Δx j,i − L j,i cos β j,i−1 )2 + (Δy j,i − L j,i sin β j,i−1 )2 = L 2b ,

(7.5)

where the displacement components of the jth module are determined by geometric dependencies: Δx j,i = x j−1,i − x j,i−1 − L b cos β j−1,i , Δy j,i = y j−1,i − y j,i−1 − L b sin β j−1,i . (7.6) Equation (7.5) has two roots, of which the desired value L j,i corresponds to: L j,i = Δx j,i cos β j,i + Δy j,i sin β j,i / −

) ) 1( 2 1( 2 Δx j,i − Δy 2j,i cos(2β j,i ) − Δx j,i + Δy 2j,i L 2b + Δx j,i Δy j,i sin(2β j,i ) + 2 2

(7.7)

and the new coordinates of GC: x j,i = x j,i−1 + L j,i cos β j,i−1 , y j,i = y j,i−1 + L j,i sin β j,i−1 .

(7.8)

The constraint Eq. (7.5) is the equation of a circle with a center coinciding with the GC of the jth module (x j,i , yj,i ) (8). To restore the joint connection between the modules, the rotation of the jth module relative to GC is performed by an angle:

98

I. Nasibullayev and O. Darintsev

( β j,i = arctan

) y j−1,i − y j,i − L b sin β j−1,i . x j−1,i − x j,i − L b cos β j−1,i

The second algorithm for moving the jth module (denote RT) is constructed as follows. The module is first rotated relative to its GC in such a way that the baseline points to the center of the joint of the j−1 module, i.e., the angle of rotation will be equal to: ( β j,i = arctan

) y j−1,i − y j,i−1 − L b sin β j−1,i . x j−1,i − x j,i−1 − L b cos β j−1,i

Then, the module “passes” the distance L j,i in the new direction after turning by the angle βj,i until the connection is restored in the joint. The displacement components are determined by (6), the distance L j,i by the formula (7.7), and the new coordinates of the GC module: x j,i = x j,i−1 + L j,i cos β j,i , y j,i = y j,i−1 + L j,i sin β j,i . With a decrease in the time step (τ → 0), the trajectories of the modules calculated according to these algorithms converge to a single trajectory. The maximum deviation ΔH j for the jth module tends to a constant value ΔH j,p but, depending on the chosen algorithm, the relative error: ) ( ε = 1 − ΔH j /ΔH j, p 100%

(7.9)

will have a different sign. The accuracy of the calculation can be improved by using a combination of TR and RT algorithms, which can be represented as a generalized TRT algorithm (Fig. 7.3). First, we move the jth module by the value λL j,i according to formula (7.7), where 0 ≤ λ ≤ 1. The new position of the GC of the jth module is determined by the coordinates: x˜ j,i = x j,i−1 + λL j,i cos β j,i−1 , y˜ j,i = y j,i−1 + λL j,i sin β j,i−1 . Then, we rotate the module by the angle as follows: ) y j−1,i − y˜ j,i − L b sin β j−1,i . = arctan x j−1,i − x˜ j,i − L b cos β j−1,i (

β j,i

To restore the connection in the joint / with the previous module, we move the ˜ jth module to the distance L j,i = Δx˜ 2j,i + Δ y˜ 2j,i − L b , determined from the Pythagorean theorem (Fig. 7.3), where Δx˜ j,i = x j−1,i − L b cos β j−1,i − x˜ j,i , Δ y˜ j,i = y j−1,i − L b sin β j−1,i − y˜ j,i .

7 Algorithm of Trajectories Synthesis for Modular Wheeled Inspection …

99

Fig. 7.3 Diagram of the algorithm TRT for moving the driven module

The properties of the TRT algorithm depend on the value of the weighting coefficient λ. If λ = 1, then we get the algorithm TR (each subsequent module moves along a trajectory with less curvature), if λ = 0, then—RT (with increasing j, the curvature increases). For the test trajectory under consideration (movement along a circle), the value λ = 0.5 ensures the movement of all modules along a trajectory with the same curvature. In general, the weighting coefficient λ is determined by the local curvature of the trajectory at the initial ρ1 and final ρ2 points of the section by the ratio λ = ρ2 /(ρ1 + ρ2 ). In the TR and RT algorithms, the movement per time step is determined by two coordinates (the start and end points of GC), and in the TRT algorithm by three (the start, intermediate, and end points of GC); therefore, the accuracy of the trajectory calculation in TR and RT is proportional to τ−1 , and in TRT—τ−2 . Table 7.1 presents the results of the simulation algorithms, TR, RT, and TRT versus time step τ: the maximum deviation of the ΔH of the trajectory of the leading module, the relative error ε is calculated by the formula (7.9), and CPU time t of the test problem with parameters: n = 100, L w = L b , L w = 0.5L b , R = 2L b , α = 225° (this angle corresponds to the maximum value of ΔH m ). The errors of the values of ΔH obtained by simulating by various algorithms were calculated relative to the TRT algorithm with a time step of τ = 10−5 s. The dashes in the table (–) mean that convergence of the numerical scheme has not been achieved for the time step under consideration. In the TR algorithm, convergence is achieved for τ ≤ 0.5 s, in the RT algorithm—τ ≤ 0.25 s, and in the TRT algorithm RT—τ ≤ 2 s. The calculation speed of the quadratic algorithm, compared with the linear one, is 1.7 times lower; however, this is compensated by higher accuracy and quadratic convergence with decreasing τ. For instance, to ensure the same calculation accuracy as in linear algorithms with τ = 10−5 s in a quadratic algorithm, it is enough to choose τ = 10−2 s, while the calculation speed will increase by 600 times.

100

I. Nasibullayev and O. Darintsev

Table 7.1 Convergence of numerical algorithms TR, RT, and TRT; ΔH p = ΔH n /L b , where p is the algorithm designation τ, s

ΔHTR , Lb

ΔHTR , Lb

ΔHTRT , Lb

εTR , %

εRT , %

εTRT , % tTR , s

tRT , s

tTRT , s

2

–

–

5.26770340

–

–

5.3

–

–

0.06

0.1

–

–

5.55503655

–

–

1.7 × 10−1

–

–

0.28

–

–

6.216

5.56348556

–

−11.7

2.7 × 10−2

–

0.4

0.64

0.5

5.365

5.834

5.56461894

3.59

−4.83

6.8 × 10−3

0.76

0.74

1.26

10−2

5.458

5.689

5.56490075

1.92

−2.22

1.73 × 10−3

1.4

1.4

2.4

10−3

5.554

5.577

5.56499613

0.204

−0.207

1.72 × 10−5

14

14

24

10−4

5.564

5.564

5.56499708

0.02

−0.02

1.69 × 10−7

142

141

241

10−5

5.565

5.565

5.56499709

0.002

−0.002

0

1441

1392

2420

7.3 Parametric Analysis of Robot Kinematics As a characteristic length, we choose the distance from GC to the joint L b . Consider a module with geometric parameters L w = L b and Rw = 0.5 L b . To determine the minimum radius of curvature of the trajectory, the geometry of the module should be considered. For the wheels on adjacent modules not to touch each other, it is necessary to perform the following relationship between the radius of the trajectory R, the length L b , the wheelbase L w , and the wheels radius Rw : sin α1 =

Lb Lw Rw Lb ⇒R> = , R − Lw R L b − Rw

where α1 is the angle between the distances from GC of neighboring modules to the center of the circle of radius R. For the selected geometric parameters of the modules, the radius of curvature of the trajectory is R ≥ 2L b . Consider the movement of a robot consisting of n = 100 modules along a circular arc with radii R = 2L b and R = 10L b and a central angle of 90°. The trajectories of the leading and driven modules are shown in Fig. 7.4. As the module number j increases, the deviation value ΔH j increases with the saturation ΔH j > ΔH j−1 . The position of the maximum deviation of the jth module (x m,j , −ΔH j ) is shifted in the opposite direction relative to the beginning of the curved section of the trajectory, i.e., x m,j < x m,j−1 . When entering the second rectilinear section L2, the module returns to the trajectory of the leading module when passing a path of the order of 10 L b . As R increases, the deviations ΔH j decrease.

7 Algorithm of Trajectories Synthesis for Modular Wheeled Inspection …

101

Fig. 7.4 Trajectory of the 1st (black lines), 10th (red lines), 50th (green lines), and 100th (blue lines) modules along an arc of 90° (gray area) with radii R = 2L b (a) and R = 10L b (b). The orange rectangle corresponds to the dimensions of the module

Figure 7.5 shows the dependence of the maximum deviations ΔH m for each 10th module of the robot versus the central angle of the arc α. With the growth of n, the value of ΔH m grows nonlinearly. The dependence of ΔH m with the growth of α has a spatial modulated character. The further the module is located from the leading one, the lower the modulation frequency ΔH m (α). The modulation amplitude decreases with increasing α, and if the leading and driven modules are on a trajectory with the same curvature, then the modulation is suppressed: ΔH m (α) → const for j < (αR)/(2L b ). With the growth of R, the magnitude and amplitude of the oscillations ΔH m decrease linearly. In the control systems of a technical device, it is necessary to change the operating parameters (factors) in real time. Simple analytical formulas for the control system in the form of a computing stand element are constructed using approximations of the results of full numerical simulation (response function). Since the problem to be solved depends on several parameters, we first determine the effect of these parameters on the deviation value ΔH m using the method of processing the results of a second-order computational experiment—orthogonal central compositional design (OCCD) [14]. We define the factors X r and the objective functions Y q . The input parameters are the number of modules X 1 = n and the radius of the arc of the curved section of the trajectory X 2 = R. As response functions, we will set the following: • the maximum deviation of the contact points of the wheels of the nth module Y 1 = ΔH m /L b for α = (0, 2π) and the corresponding central arc angle Y 2 = αm ; • additionally, we determine the maximum deviation of the wheel of the nth module Y3 = δH m /L b and the corresponding central angle of the arc Y 4 = δαm ; • processor simulation time for one set of factors Y 5 = t s .

102

I. Nasibullayev and O. Darintsev

Fig. 7.5 Dependence of the maximum deviation Δ H m for each 10th module versus the magnitude of the central angle of the arc α, R = 2Lb (a) and R = 10Lb (b)

Note that all objective functions are determined for each set of factors within a single computational experiment. For the convenience of data analysis, the natural parameters of X r are normalized on the interval [min(X r ), max(X r )] in the interval [−1, +1]: xr =

X r − X r,0 max(X r ) + min(X i ) max(X r ) − min(X r ) , |X r,l = , , |X r,0 = X r,l 2 2

where X r,0 is the central point and X r,l is the maximum deviation of the parameter from X r,0 . The response functions have the form as follows: ( ) ( ) Yq = a0,q + a1,q x1 + a2,q x2 + a3,q x1 x2 + a4,q x12 − γ2 + a5,q x22 − γ2 , where a0,q is the average value of the output parameter; coefficients a1,q and a2,q determine the linear contribution of the input parameters; coefficient a3,q is the coupled interaction; coefficients a4,q and a5,q determine the quadratic contribution of the input parameters; and γ2 = 2/3—parameter of the OCCD. A positive (negative) value of the coefficients means a positive (negative) contribution of the corresponding input parameter to the response function Y q . The extended matrix of the computational experiment is given in Table 7.2, and the values of the coefficients are given in Table 7.3. The leading factor for the response functions ΔH m , αm , and δH m is R (the contribution is negative), and for the calculation time t s is n (the contribution is positive). The significant factors for the response functions ΔH m and αm are n, and for t s is R. Taking into account the radius of the wheels gives an additional contribution to ΔH m of no more than 10% (|ar,3 /ar,1 | < 10), and in αm —less than 1% (|ar,4 /ar,2 | < 100).

7 Algorithm of Trajectories Synthesis for Modular Wheeled Inspection …

103

Table 7.2 Extended matrix of the OCCD x0

x1

x2

x1 x2

x1 2 −γ2

x2 2 −γ2

n

R/Lb

10Y1

100Y3

10Y4

1

−1

−1

1

0.33

0.33

4

2

4.22

82.1

4.12

7

52

1

1

−1

1

0.33

0.33

6

2

6.26

102.2

3.89

4

93

1

−1

1

−1

0.33

0.33

4

4

2.17

43.9

2.51

6

60

1

1

1

−1

0.33

0.33

6

4

3.22

55.2

2.35

5

108

1

−1

0

1

0.33

0.67

4

3

2.88

57.6

3.12

6

56

1

1

0

0

0.33

0.67

6

3

4.26

72.1

2.93

4

100

1

0

−1

0

−0.7

0.33

5

2

5.28

93.2

3.98

6

72

1

0

1

0

−0.7

0.33

5

4

2.72

50.2

2.41

5

83

1

0

0

0

−0.7

−0.67

5

3

3.60

65.5

3.01

6

77

Y2

Y5 , s

Table 7.3 Coefficients of approximation of the OCCD a0,q 10Y1

3.84453

a1,q

a3,q

a4,q

a5,q

0.74433

a2,q −1.27623

−0.24969

−0.03250

0.40251

−2.2000

69.11111

7.65000

−21.36666

−0.78333

6.06667

100Y1

3.14668

−0.09557

−0.78591

0.02160

0.02079

0.19080

10Y4

5.44446

−1.0

−0.17

0.50001

−0.33333

0.16668

1.75000

0.83333

0.33333

Y2

Y5

77.88889

22.16667

5.66667

The calculation time is an insignificant factor, due to these costs are carried out only when the approximation is performed, and an analytical formula will be used in the control/planning system.

7.4 Features of the Robot Dynamics When the first module moves at a constant velocity v 1 along the curved section of the trajectory L s and along the second rectilinear section L 2 , the trajectory of the driven modules becomes curved (see Fig. 7.4) and their velocities vj varies (see Fig. 7.6). With the growth of R, the ratio of the maximum speed of the driven module to the speed of the slave module is k = max(vj )/v1 . For instance, for R = 2L b the value is k ≈ 1.12, and for R = 10L b —k ≈ 1.05. When approaching the second rectilinear section, the velocity will asymptotically tend to the velocity of the first module vj → v1 . A change in the velocity of the modules vj and the local curvature of the trajectory ρj can lead to the skidding of the driven modules under the action of centrifugal force. The jth module will move without skidding if the centripetal acceleration F c does not exceed the sliding friction force F f : Fc < F f ⇒ mv 2j ρ j < μmg ⇒ v j
edz , ϑi0 = kϑ Vxi0 − Vxi + edz , Vxi0 − Vxi < −edz , ⎩ 0, −edz ≤ Vxi0 − Vxi ≤ edz ,

(8.53)

where edz is the dead zone introduced to filter high-frequency measurement noise [39]; Vxi is the UAV speed projection on the OX axis of the associated coordinate system; and kϑ is the controller parameter. In this case, the pitch angle is additionally limited by the values of ± ϑ max . Similarly, the requirements for the roll angle are formed (54) ( ) ⎧ ⎨ kγ( Vzi0 − Vzi − edz) , Vzi0 − Vzi > edz , γi0 = kγ Vzi0 − Vzi + edz , Vzi0 − Vzi < −edz , ⎩ 0, −edz ≤ Vzi0 − Vzi ≤ edz ,

(8.54)

where Vzi is the projection of the UAV speed on the OZ axis of the associated coordinate system; γi0 is the target roll angle; and kγ is the controller parameter. Both the roll angle and the pitch angle are additionally limited by the values of ± γ max . The restrictions are related to the control features. When the specified angles exceed some maximum values, the multicopter type UAV falls [40–42]. The desired steady state flight speed is determined by the velocity projection Vyi , which is equal to (55) Vyi = Vy0 .

(8.55)

Expressions (8.50) and (8.53)–(8.55) determine the driving influences for the UAV motion control system. Let us consider a variant of movement with swarm principles of group formation [43, 44]. The scheme of information exchange in such a group is shown in Fig. 8.6. According to the scheme presented in Fig. 8.5, at the initial moment of time, the task for all UAVs is received from the ground control point. The task includes the direction and speed of the flight of the group, as well as the required distance at which the UAVs must be located from each other.

8 Study of Algorithms for Coordinating a Group of Autonomous Robots …

127

Fig. 8.6 Information exchange in a swarm system

As before, the own coordinates of each UAV are determined using the signals of the external navigation system. In addition, all UAVs exchange their coordinates with neighboring UAVs of the group. The height of each UAV for the case of the swarm formation, as well as for movement with a leader, is given by expression (8.48). The desired direction of movement for all UAVs is calculated in accordance with expression (8.50). The desired coordinate along the OZ axis is calculated according to the expression (8.56) z i0 = z i−1 + rs , z 10 = 0, i = 2, Nr .

(8.56)

The desired coordinate along the OX axis in the case of the swarm formation is determined by the expressions (8.57)–(8.59) x10 = V0 t + kv (x2 − x1 ),

(8.57)

0 x Nr = V0 t + kv (x Nr −1 − x Nr ),

(8.58)

( xi0 = V0 t + kv

_ xi−1 + xi+1 − xi , 2

(8.59)

where i = 2, Nr − 1. The desired trim, roll, and yaw angles are calculated in accordance with expressions (8.50), (8.53), and (8.54), and the desired UAV speed vector is determined by expression (8.51).

128

V. Pshikhopov et al.

Thus, with swarm formation control, each UAV calculates its position along the OZ axis and the OX axis relative to neighboring UAVs, and not the leader UAV. The motion control system is based on the method of position-trajectory control. A detailed description of the UAV motion control algorithm synthesis procedure, as well as control system adaptation algorithms, are presented in [15, 28, 42]. The general view of the control algorithm has the following form (8.60) [

] ] [ A2 R A2 R˙ + T2 ψ˙ tr + T1 ψtr −1 M (K δ δ + Fd ) = − T3 ψv A4

(8.60)

where T1 , T2 , and T3 are controller matrices; R˙ is the matrix which elements are time derivatives of the elements of the matrix R ⎡ ⎤ ⎡ ⎤ 00 01 00 −ψ0 ⎢ ⎥ A2 = ⎣ 0 0 0 0 1 0 ⎦, A3 = ⎣ −ϑi0 ⎦, ψtr = A2 y + A3 , −γi0 00 00 01 [ ] A4 = 0 1 0 0 0 0 , A5 = [−V0 ], ψv = A4 y + A5 .

8.4 Results of the Study of Formation Algorithms A comparison of the leader control algorithm and the swarm algorithm was carried out using numerical simulation. The simulation was carried out with the following parameters of the UAV and the control system: initial positions of the UAV group (0; 100; 0)—leader, (0; 100; 50), (0; 100; 100), and (0; 100; 150); desired flight direction ψ0 = 0; desired flight altitude y 0 = 100 m; distance between UAVs rs = 50 m; cruising speed of the group V0 = 3 m/s; parameters of roll and pitch angle controllers kγ = 0.1, kϑ = 0.1; pitch and roll angles are limited to ± 0,3 rad; UAV maximum flight speed 4 m/s; UAV inertial parameters m = 5 kg, Jx = 5, Jy = 0.1, and Jz = 5; and maximum propeller thrust 100 N. Closed-loop control time constant is 2.0 s. Figure 8.7 shows the results of modeling the movement of a group of four UAVs with a leader at a navigation signal update interval of 1 s and a navigation standard deviation of 5.5 m. Table 8.1 shows the errors in maintaining a given formation by a group of UAVs with a leader at various update rates of navigation signals and various navigation errors. From Table 8.1, it follows that with an increase in the error of the navigation system, the error in maintaining a given position in the formation by a group of UAVs with a leader also increases. In this case, the greatest accuracy is provided through the height control channel, because it is maintained independently of each other. The

8 Study of Algorithms for Coordinating a Group of Autonomous Robots …

129

Fig. 8.7 Results of modeling a group of UAVs with a leader: a UAV trajectories; b UAV flight heights

Table 8.1 Dependence of the errors of the control system with a leader on the accuracy and frequency of updating navigation data Standard deviation of the navigation system, m OY axis error

OX axis error

OZ axis error

0,5

1,0

1,5

2,0

Δtn = 0,1 s

0,73

1,07

1,53

2,6

Δtn = 0,2 s

0,7

0,8

2,13

2,77

Δtn = 0,5 s

0,7

0,9

2,17

2,67

Δtn = 1,0 s

0,87

1,43

1,93

3,6

Δtn = 0,1 s

1,7

3,2

4,3

6,8

Δtn = 0,2 s

1,93

3,1

3,9

7,17

Δtn = 0,5 s

1,67

2,87

5,27

6,77

Δtn = 1,0 s

2,07

3,0

4,97

6,57

Δtn = 1,0 s

0,27

1,87

2,9

4,5

Δtn = 0,1 s

0,33

1,97

2,33

3,77

Δtn = 0,2 s

0,47

1,8

2,83

4,57

Δtn = 0,5 s

0,47

1,97

3,2

4,17

greatest error is observed in the control channels along the OX and OZ coordinates, which is associated with the group’s need to track the position of the leader. In this case, the error in maintaining the position along the OX axis is maximum, because it is necessary to keep track of the changing coordinate. The standard deviation [45] was chosen as the error of the navigation system, and the value averaged over implementations from the maximum errors during the flight was chosen as the error of the control system. Based on the data in Table 8.1, it can be noted that the error of the group control system with a leader weakly depends on the navigation data update interval until it approaches the time constant of the closed system. After that, the control system sharply loses quality and stability. So, in the simulated example, stability loss is

130

V. Pshikhopov et al.

observed at a navigation data update rate of about 1.5 s, which approximately corresponds to a closed system time constant of 2.0 s. Figure 8.8 and Table 8.2 show the results of modeling a swarm control system, in which the desired UAV positions along the OX and OZ axes are given by expressions (8.56)–(8.59). Comparison of the data presented in Tables 8.1 and s2 shows that the most significant difference between a swarm system, and a group control system with a leader is the higher positioning accuracy relative to the OX axis. In particular, for the standard deviation of the navigation system of 2.0 m, the swarm control system improves positioning accuracy by an average of 3.5 times. At the same time, the positioning error along the height and along the OZ axis does not change significantly. This is due to the fact that the swarm positioning principle is used specifically for positioning along the front (along the OX axis). It can be noted that an increase in the accuracy of maintaining the formation requires a more developed communication system or the availability of means on board the UAV, which makes it possible to determine the positions of neighboring UAVs with sufficient accuracy. In the case of a group with a leader, only the leader transmits his coordinates to the other members of the group. In the swarm method, UAVs transmit their coordinates to their neighbors. The joint operation of UAV communication systems requires the organization of a special communication network [46], which in some environments can create technical problems, for example, underwater [47]. Note that this paper does not consider modes of significant deviations from the desired positions. This is due to the fact that in such modes it is required to solve the problems of motion planning [48–51] and collision elimination.

a. UAV trajectories Fig. 8.8 Results of modeling a swarm group of UAVs

b. UAV flight heights

8 Study of Algorithms for Coordinating a Group of Autonomous Robots …

131

Table 8.2 Dependence of the errors of the swarm control system on the accuracy and frequency of updating navigation data 0,5

1,0

1,5

2,0

Δtn = 0,1 s

0,83

1,0

2,0

2,28

Δtn = 0,2 s

0,83

0,97

1,75

2,23

Δtn = 0,5 s

0,7

10,3

1,8

2,8

Δtn = 1,0 s

0,9

1,34

3,33

3,0

Δtn = 0,1 s

0,45

0,6

1,33

1,15

Δtn = 0,2 s

0,5

1,07

1,46

1,54

Δtn = 0,5 s

0,29

1,0

1,2

2,17

Δtn = 1,0 s

0,32

0,7

1,7

2,4

Δtn = 1,0 s

0,47

1,58

2,85

3,6

Δtn = 0,1 s

0,5

1,5

2,24

3,2

Δtn = 0,2 s

0,35

1,57

2,3

4,3

Δtn = 0,5 s

0,4

1,57

3,2

3,92

Standard deviation of the navigation system, m OY axis error

OX axis error

OZ axis error

8.5 Conclusion This article compares two approaches to controlling the movement of a group of UAVs when moving in formation—control with a leader and decentralized control. The comparison is made in terms of the accuracy of maintaining the formation, which is important when performing a number of practical tasks. The simulation results showed that the decentralized control system makes it possible to reduce the formation maintenance error by about 3.5 times compared to the control system with a leader. At the same time, the implementation of a decentralized control system requires that neighboring UAVs transmit information about their location to each other. In a system with leaders, only the leader transmits its coordinates to the group, and the slave UAVs are in silent mode. Thus, in a situation where it is possible to organize a communication network, the use of a decentralized control method seems to be more promising. Acknowledgements This work has been supported by the grant of the Southern Federal University No. SP02/S4_0708Prioritet_06.

References 1. Arteaga-Escamilla, C.M., Castro-Linares, R., Álvarez-Gallegos, J.: Leader-follower formation with reduction of the off-tracking and velocity estimation under visibility constraints. International J. Adv. Robot. Syst. 18(610) (2021)

132

V. Pshikhopov et al.

2. Sun, F., Li, H., Zhu, W., Kurths, J.: Fixed-time formation tracking for multiple nonholonomic wheeled mobile robots based on distributed observer. Nonlinear Dyn. 106, 3331–3349 (2021) 3. Dong, X., Yu, B., Shi, Z., Zhong, Y.: Time-varying formation control for unmanned aerial vehicles: theories and applications. IEEE Trans. Control Syst. Technol. 23(1), 340–348 4. Pack, D.J., DeLima, P., Toussaint, G.J., York, G.: Cooperative control of UAVs for localization of intermittently emitting mobile targets. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 39(4), 959–970 (2009) 5. Bezruk, G., Martynova, L., Saenko, I.: Dynamic method of searching anthropogenic objects in use of seabed with autonomous underwater vehicles. SPIIRAS Proc. 3(58), 203–226 (2018) 6. Shepeta, A.P., Nenashev, V.A.: Accuracy characteristics of object location in a two-position system of small onboard radars. Informatsionno-Upravliaiushchie Sistemy 2, 31–36 (2020) 7. Nenashev, V.A., Shepeta, A.P., Kryachko, A.F.: Fusion radar and optical information in multiposition on-board location systems. In: 2020 Wave Electronics and its Application in Information and Telecommunication Systems (WECONF), pp. 1–5 (2020) 8. Nenashev, V.A., Khanykov, I.G.: Formation of fused images of the land surface from radar and optical images in spatially distributed on-board operational monitoring systems. J. Imaging 7(251) (2021) 9. Nenashev, V., Khanykov, I.: Formation of a fused image of the land surface based on pixel clustering of location images in a multi-position onboard system. Inform. Autom. 20(2), 302– 340 (2021) 10. Lewis, M.A., Tan, K.-H.: High precision formation control of mobile robots using virtual structures. Auton. Robot. 4, 387–403 (1997) 11. Tan, K.-H., Lewis, M.: Virtual structures for high-precision cooperative mobile robotic control. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 132–139 (1996) 12. Morozova, N.S.: Virtual formations and virtual leaders in formation control problem for group of robots. Vestnik Sankt-Peterburgskogo Universiteta. Seriya 10. Prikladnaya Matematika. Informatika. Protsessy Upravleniya 1, 135–149 (2015) 13. Endo, T., Maeda, R., Matsuno, F.: Stability analysis of swarm heterogeneous robots with limited field of view. Inform. Autom. 19(5), 942–966 (2020) 14. Gaiduk, A.R., Martjanov, O.V., Medvedev, M.Y., Pshikhopov, V.K., Hamdan, N., Farhood, A.: Neural network based control system for robots group operating in 2-d uncertain environment. Mekhatronika, Avtomatizatsiya, Upravlenie 21(8), 470–479 (2020) 15. Pshikhopov, V., Medvedev, M.: Group control of autonomous robots motion in uncertain environment via unstable modes. SPIIRAS Proc. 5(60), 39–63 (2018) 16. Medvedev, M., Pshikhopov, V., Gurenko, B., Hamdan, N.: Path planning method for mobile robot with maneuver restrictions. In: 2021 International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME), pp. 1–7 (2021) 17. Carlos, A., Hebertt, S., Joés, A.: Stability of active disturbance rejection control for uncertain systems: a Lyapunov perspective. Int. J. Robust Nonlinear Control 27, 4541–4553 (2017) 18. Vorotnikov, V., Vokhmyanina, A.: Feedback liniarization method for problem of control of a part of variables in uncontrolled disturbances. SPIIRAS Proc. 6(61), 61–93 (2018) 19. Fel’dbaum, A.A.: On the distribution of roots of the characteristic equation of the control system. Avtomatika i Telemekhanika 9(4), 253–279 (1948) 20. Finaev, V.I., Medvedev, M.Y., Pshikhopov, V.K., Pereverzev, V.A., Soloviev, V.V.: Unmanned powerboat motion terminal control in an environment with moving obstacles. Mekhatronika, Avtomatizatsiya, Upravlenie 22(3), 145–154 (2021) 21. Park, B.-S.; Yoo, S.-J.: Adaptive secure control for leader-follower formation of nonholonomic mobile robots in the presence of uncertainty and deception attacks. Mathematics 9 (2021) 22. Hirata-Acosta, J., Pliego-Jiménez, J., Cruz-Hernádez, C., Martínez-Clark, R.: Leader-follower formation control of wheeled mobile robots without attitude measurements. Appl. Sci. 11(12), 5639 (2021) 23. Maghenem, M., Loria, A., Panteley, E.: Cascades-based leader-follower formation tracking and stabilization of multiple nonholonomic vehicles. IEEE Trans. Autom. Control, Inst. Electr. Electron. Eng. 65(8), 3639–3646 (2019)

8 Study of Algorithms for Coordinating a Group of Autonomous Robots …

133

24. Wang, Z., Wang, L., Zhang, H., Chen, Q., Liu, J.: Distributed regular polygon formation control and obstacle avoidance for non-holonomic wheeled mobile robots with directed communication topology. IET Control Theory Appl. 14(9), 1113–1122 (2020) 25. Sun, J., Chen, J.: A survey on Lyapunov-based methods for stability of linear time-delay systems. Front. Comp. Sci. 11, 555–567 (2017) 26. Hu, J., Bhowmick, P., Lanzon, A.: Group coordinated control of networked mobile robots with applications to object transportation. IEEE Trans. Veh. Technol. 70(8), 8269–8274 (2021) 27. Arnold, W.F., Laub, A.J.: Generalized eigenproblem algorithms and software for algebraic Riccati equations. Proc. IEEE 72(12), 1746–1754 (1984) 28. Pshikhopov, V., Medvedev, M.: Multi-loop adaptive control of mobile objects in solving trajectory tracking tasks. Autom. Remote. Control. 81(11), 2078–2093 (2020) 29. Thornton, S.T., Marion, J.B.: Classical dynamics of particles and systems. Brooks Cole 5, 672 (2003) 30. Götten, F., Finger, D.F., Havermann, M., Braun, C., Marino, M., Bil, C.: Full configuration drag estimation of short-to-medium range fixed-wing UAVs and its impact on initial sizing optimization. CEAS Aeronaut. J. 12, 589–603 (2021) 31. Milne-Thomson, L.M.: Theoretical Aerodynamics. Courier Corporation, p. 464 (2012) 32. Li, X., Qi, G., Zhang, L.: Time-varying formation dynamics modeling and constrained trajectory optimization of multi-quadrotor UAVs. Nonlinear Dyn. 106, 3265–3284 (2021) 33. Medvedev, M., Pshikhopov, V.: Path planning of mobile robot group based on neural networks. Lecture Notes in Artificial Intelligence, pp. 51–62 (2020) 34. Ren, X.X., Yang, G.H.: Noise covariance estimation for networked linear systems under random access protocol scheduling. Neurocomputing 455(30), 68–77 (2021) 35. Golnaraghi, F., Kuo, B.C.: Automatic Control Systems. 9th edn. Wiley, p. 944 (2010) 36. Diebold, F.: Elements of Forecasting, 4th edn. Thomson/South-Western, p. 366 (2007) 37. El-Sheimy, N., Hou, H., Niu, X.: Analysis and modeling of inertial sensors using Allan variance. IEEE Trans. Instrum. Meas. 57(1), 140–149 (2008) 38. Consolini, L., Morbidi, F., Prattichizzo, D., Tosques, M.: Leader-follower formation control of nonholonomic mobile robots with input constraints. Automatica 44(5), 1343–1349 (2008) 39. Cocetti, M., Tarbouriech, S., Zaccarian, L.: High-gain dead-zone observers for linear and nonlinear plants. IEEE Control Syst. Lett. 3(2), 356–361 (2019) 40. Liu, Y., Chen, C., Wu, H., Zhang, Y., Mei, P.: Structural stability analysis and optimization of the quadrotor unmanned aerial vehicles via the concept of Lyapunov exponents. Int. J. Adv. Manuf. Technol. 94(9), 3217–3227 (2018) 41. Ömürlü, V.E., Büyük¸sahin, U., Artar, R., Kirli, A., Turgut, M.N.: An experimental stationary quadrotor with variable DOF. Sadhana 38(2), 247–264 (2013) 42. Pshikhopov, V.K., Medvedev, M.Y., Gurenko, B.V.: Algorithms of terminal control of multicopters. Mekhatronika, Avtomatizatsiya, Upravlenie 20(1), 44–51 (2019) 43. Bayindir, L.: A review of swarm robotics tasks. Neurocomputing (2016) 44. Shi, Y.: Particle swarm optimization: developments, applications and resources. In: 2001 IEEE International Conference on Evolutionary Computing, vol. 1, pp. 81–86 (2001) 45. Bruce, P.C., Bruce, A.G.: Practical Statistics for Data Scientists, vol. 1. O’Reilly Media (2016) 46. Zhou, P., Fang, X., Fang, Y., He, R., Long, Y., Huang, G.: Beam management and self-healing for mmWave UAV mesh networks. IEEE Trans. Veh. Technol. 68(2), 1718–1732 (2019) 47. Li, N., Cürüklü, B., Bastos, J., Sucasas, V., Fernandez, J.A.S., Rodriguez, J.: A probabilistic and highly efficient topology control algorithm for underwater cooperating AUV networks. Sensors 17, 1022 (2017) 48. Kostjukov, V., Medvedev, M., Pshikhopov, V.: Method for optimizing of mobile robot trajectory in repeller sources field. Inform. Autom. 20(3), 690–726 (2021) 49. Medvedev, M., Kostjukov, V., Pshikhopov, V.: Optimization of mobile robot movement on a plane with finite number of repeller sources. SPIIRAS Proc. 19(1), 43–78 (2020) 50. Sánchez-Ibáñez, J.R., Pérez-del-Pulgar, C.J., García-Cerezo, A.: Path planning for autonomous mobile robots: a review. Sensors 21, 7898 (2021)

134

V. Pshikhopov et al.

51. Mac, T.T., Copot, C., Tran, D.T., De Keyser, R.: Heuristic approaches in robot path planning: a survey. Robot. Auton. Syst. 86, 13–28 (2016)

Chapter 9

Intelligent System for Countering Groups of Robots Based on Reinforcement Learning Technologies Vladimir Parkhomenko , Tatiana Gayda, and Mikhail Medvedev

Abstract Since the end of the last century, reinforcement learning has succeeded in the simplest computer games and modern games, and under special conditions of learning. The article describes the work on transferring actual strategic tasks to the game environment in order to approximate the function of their solution using reinforcement learning methods. The authors are faced with the task of battle of robots, for a successful solution of which it is necessary to make strategic decisions. The relevance of the problem lies in its difficult to formalize and the possibility of its transfer to other spheres of society, for successful functioning in which there are not enough methods based on linear logic. The task is formalized for model-free and model-based methods, off-policy and on-policy algorithms. Particular attention is paid to the development and understanding of the reward function and the neural network model. Article also presents comparison of simulation results under different conditions and methods of solution that affect the quality of the approximated policy. In the conclusion, authors give an analysis of achieved results, further methods for the development of the solved problem.

9.1 Introduction Machine learning is a class of artificial intelligence methods, characterized with the ability to automatically learn from experience and reproduce human cognitive functions using computer algorithms, which involve working with sample data (training data) to build models. Models, in their turn, predict future data or decisions. Machine learning (ML) algorithms have many applications, such as classification, detection, segmentation, regression [1–3], recommendation systems, etc. ML has replaced the generally accepted algorithms, which hardly handle the above-mentioned problems [4, 5]. V. Parkhomenko · T. Gayda · M. Medvedev (B) Southern Federal University, 105/42 Bolshaya Sadovaya Str., Rostov-on-Don 344006, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_9

135

136

V. Parkhomenko et al.

Artificial intelligence is divided into three categories: assisted, augmented, and autonomous. Machine learning belongs to the augmented intelligence category. The decision-making process is based on system dynamics [6, 7] and automated using numerous tools such as learning process, integrated data manipulation tools, decision modeling tools, and expert knowledge [8, 9]. Machine learning algorithms is divided to several methods depending on the approaches, types of input data and output data, as well as the type of problem statement. Those methods are supervised learning [10], unsupervised learning [11], semisupervised learning [12], reinforcement learning [13], self-learning [14], feature learning [15], and decision tree learning [16]. One of the promising approaches among the above mentioned is reinforcement learning (RL) [17] due to the absence of the need for labeled data, its adaptability and capability of making sequence of decisions. Since the last century beginning, programs based on reinforcement learning has been succeeding in winning simple computer games [18, 19], and under special learning conditions, in more sophisticated ones. The article describes the work on transferring actual strategic tasks to the game environment in order to approximate the function of their solution using reinforcement learning methods. Over the past decades, artificial intelligence (AI) technology has made significant progress. As an excellent testing platform for AI research, video games have provided strong support for the development of AI technologies such as traditional video games, games with incomplete information, etc. [20]. However, classical AI methods only consider video games in small scenes, and researchers only need to control one agent in these game environments. Games and AI have a long history, and gaming AI research is mainly focused on creating game agents. In academic gaming AI, we distinguish two main areas and related research activities: board games and video games. Checkers, chess, and go are classic board games that were conquered by AI scientists in 1994 [21], 1997 [22], and 2016 [23], respectively. Compared to board games, which have discrete turn-based mechanics and the full game state is visible to both players, video games are more complex. A notable video game AI milestone was reached in 2014 when a gaming AI program, developed by Google DeepMind played several classic Atari 2600 video games at a superhuman skill level [19]. A real-time strategy (RTS) like StarCraft was the next challenge [24]. OpenAI Five became the first AI system to beat world champions in an e-sports game. Playing Dota 2 presents some challenges for AI systems, such as long planning time horizons, incomplete information, and complex, continuous action spaces. OpenAI Five used existing reinforcement learning methods trained on approximately 2 million frames [25]. Due to the success of AlphaGo AI [26], the research community actively spread the belief that the effectiveness of the combination of Monte Carlo tree search (MCTS) and deep learning methods will allow to create an artificial intelligence system capable of beat any professional RTS players in the near future. Given the above achievements in the field of artificial intelligence, the authors are trying to solve the problem of robots countering against each other, for the successful

9 Intelligent System for Countering Groups of Robots Based …

137

solution of which it is necessary to make strategic decisions. The relevance of solving the problem lies in its difficult formalization and the possibility of its scalability to other areas of activity, where the methods based on linear logic cannot function successfully. The authors considered the problem statement for model-free and model-based methods, off-policy and on-policy algorithms [27, 28]. Particular attention is paid to the development and justification of the reward function and the neural network model. A comparison of simulation results under different conditions and solution methods that affect the quality of the policy being approximated are presented. In conclusion, an analysis of the achieved results is given, further directions for developing the stated problem are presented, for example, for multi-agent environments, possible methods for improving current results and solving the problem in less observable environments.

9.2 Problem Description A limited area in two-dimensional space, an enemy and an ally robots are given. Each robot has an affected area, indicated by a sector (Fig. 9.1a). The robot is considered hitted when it enters the opponent’s area of effect (it is considered when it enters the area of effect, a shot is fired that hits the opponent with a 100% probability). The goal of each robot is to hit the opponent. In Fig. 9.1b, the dotted arrow shows the necessary behavior of an ally to defeat the enemy. To solve the problem, a decision-making approach is required, which will take into account the need for a sequence of actions taken. When adding obstacles to the battlefield or additional allies/enemies objects, the complexity of the problem increases and requires consideration of a large number of factors when using traditional algorithms. There is a necessity to develop a universal solution that would easily satisfy the complexity increasement of the task.

Fig. 9.1 Problem statement

138

V. Parkhomenko et al.

A machine learning-based approach was chosen to solve the problem, which allows to work with hard-formalized tasks. In this article, the authors consider two different formulations of the same problem in the framework of reinforcement learning: discrete environments, where the agents use on-policy algorithms; continuous environments, where the agents use off-policy algorithms. In the case of discrete environments, the ally chooses at each step one of eight directions of movement, four of which allow movement vertically and horizontally, and the remaining four are axial. To solve the problem on-policy methods, control is carried out simultaneously by two variables: linear and angular velocities. The authors also considered the solution of the above stated problem in an environment with obstacles (Fig. 9.1b).

9.3 Algorithms To solve the above stated problem, model-free RL algorithms were chosen for the following reasons: 1. A reliable model of the environment is usually not available to the agent; 2. As a result of knowing the model, the agent performs well in training, but behaves not optimally in the real environment; 3. Learning the model is a complex task, so even intensive efforts (a lot of time and calculations) may not pay off; 4. Model-free methods are more popular and more carefully developed and tested than model-based methods. 5. Specific as an implementation of the model-free RL, the following algorithms were chosen: Soft Actor-Critic (SAC). To achieve high sampling efficiency and reliable performance, many works in the field of RL have been proposed. In particular, the SAC algorithm [29] achieves impressive performance on many complex continuous control validation tests and is an RL off-policy algorithm based on an entropy-maximizing reinforcement learning framework. Actor reuses past experience through off-policy optimization and acts in such a way as to maximize the expected reward and entropy to increase exploratory capacity [30]. Proximal policy optimization [31] (PPO)—refers to a family of policy gradient methods for reinforcement learning that consider data sampling through interaction with the environment and optimization of a “surrogate” objective function using stochastic gradient lifting. PPO has some of the benefits of trust region policy optimization (TRPO) but is much easier to implement and has more generalizing ability.

9 Intelligent System for Countering Groups of Robots Based …

139

9.4 Neural Network Architecture The architecture of the used neural network is shown in Fig. 9.2. The input data are the image, the coordinates of the ally and the enemy. The convolution part uses residential blocks [32]. Next comes the merging of layers and their subsequent processing by fully connected layers. The ally and enemy coordinates describe the position in the 2D environment and the orientation. Image

Ally coordinates

Enemy coordinates

Conv2d

Linear

Linear

MaxPool2d

ReLU

ReLU

Conv2d

Linear

Linear

MaxPool2d

ReLU

ReLU

ResBlock

Linear

Linear

MaxPool2d ResBlock MaxPool2d ResBlock MaxPool2d ResBlock MaxPool2d Conv2d Flatten

Сat Linear Linear Linear out_features = num_actions

Fig. 9.2 Architecture of the used neural network

140

V. Parkhomenko et al.

9.5 Preliminary Processing As a preliminary processing of the input data, the image and coordinates are normalized in the range [0, 1]. In order to increase the completeness of the information presented about the observed state of the environment, the current and two previous images were combined with different transparency coefficients. The output of the network model was transformed in accordance with the kinematics of the controlled robot using the following equations xn = xn−1 + Sx [ p], yn = yn−1 + S y [ p], Ω = S[ p]ω , where x and y are coordinates in space; Ω is the angle between the abscissa axis and the direction of the robot; n is modeling step; S is set of directional vectors; p is direction of movement; and S[p]ω is angle equal between the vector S[p] and the abscissa axis. In a continuous environment, kinematics is described by the following equations: xn = xn−1 + V ∗ cos(Ω) ∗ Δt, yn = yn−1 − V ∗ sin(Ω) ∗ Δt, Ωn = Ωn−1 + ω ∗ Δt, where V is linear velocity; ω is angular velocity; and Δt is modeling step.

9.6 Reward Function Often, in reinforcement learning tasks, rewards are given for achieving a goal or losing and events that are rare, yet other actions of the agent are left without due attention. The authors of the article have developed an additional reward function that applies to all non-terminal actions of the agent, defined by the following set of equations

9 Intelligent System for Countering Groups of Robots Based …

141

⎧ maxr , When reaching the target position ⎪ ⎪ ⎨ When hitting an obstacle obsr , r (x) = , ⎪ obsr , When leaving the map abroad ⎪ ⎩ r When taking a step /| | dist = − | Bx 2 + By 2 |, (

) Ax ∗ Bx + Ay ∗ By √ ϕ = arccos √ , Ax 2 + Ay 2 ∗ Bx 2 + By 2 ) ( dist ∗ k, r = ϕ ∗ kϕ + dist − kdist − ϕ + kγ Bx = x1 − x3 , By = y1 − y3 , Ax = cos( f ), Ay = sin( f ), where x 1 , x 2, y1 , and y2 are coordinates of the enemy and ally; f is enemy orientation angle; kϕ , kdist , k, and kγ are coefficients that determine the parameters of the reward function. When creating the reward function, the task was to encourage the actions of the agent to bypass the enemy from behind and penalize the collision. In the above equations, kϕ o determines the importance of getting behind enemy, kdist —the need to approach the enemy, kγ —the penalty for a collision, and k—the proportional coefficient. The graph of the reward function is shown in Fig. 9.3. In the top view (Fig. 9.3a), the arrow indicates the direction of the enemy. The spatial view is shown in Fig. 9.3b). This reward feature can be scaled across multiple allies and enemies.

Fig. 9.3 Reward graph

142

V. Parkhomenko et al.

9.7 Training The training was carried out taking into account the above features on the SAC algorithms (implementation for discrete media) and PPO. Figure 9.4 shows the results of the learning process in discrete and continuous environments with and without obstacles. The number of steps taken by the agent during the training period is showed on x-axis. In all PPO problems, the value of the reward converges to the asymptote. In a discrete environment, the result of SAC turned out to be unsatisfactory, unlike PPO. After completing the training, the agent was tested for 10,000 games. Testing was carried out for a situation with one enemy and one ally in an environment with and without obstacles (4–5 obstacles). The radius of the affect zone is 100 m, and the maximum speed is 35 km/h. The agent is trained using PPO and A2C algorithms. The results are shown in Table 9.1. Figure 9.5 shows the simulation results after completing the training. The agent bypasses the enemy from behind and then tries to cover him with the affect zone. As can be seen from the Table 9.1, PPO shows the best result in all environments. The generalized contentiousness of the SAC was not enough for learning in a discrete environment.

9.8 Conclusion As can be seen from the simulation results, the use of reinforcement learning methods makes it possible to successfully solve problems that require a sequence of decisions. Improving the available results is possible with the use of recurrent networks (RNN) and the use of attention technologies [33] which can cope with the task under uncertainty. In the article, the authors considered the solution of the problem of robots combat using reinforcement learning methods for continuous and discrete environments, with and without obstacles. A reward function was introduced to successfully approximate the desired policy function. Further, development of the work is possible in the field of application of multiagent reinforcement learning technologies in order to scale the presented solution to groups of robots, which will allow solving the problem in the group control approach.

9 Intelligent System for Countering Groups of Robots Based …

Learning in a Discrete Environment Without Obstacles Average reward per episode. Orange chart Average number of steps per episode. is PPO, blue chart is SAC. Orange chart is PPO, blue chart is SAC.

Learning in a continuous environment without obstacles Average reward per episode. The orange graph is the PPO agent, the blue graph is the SAC agent.

Number of steps per episode. The orange graph is the PPO agent, the blue graph is the SAC agent.

Learning in a discrete environment with obstacles Average reward per episode. The orange graph is the PPO agent, the blue graph is the SAC agent.

Number of steps per episode. The orange graph is the PPO agent, the blue graph is the SAC agent.

Learning in a continuous environment with obstacles Average reward per episode. The orange Number of steps per episode. graph is the PPO agent, the blue graph is The orange graph is the PPO agent, the SAC agent. the blue graph is the SAC agent.

Fig. 9.4 Learning results

143

144

V. Parkhomenko et al.

Table 9.1 Test results Discrete environment Outcome

With obstacles

Without obstacles

SAC (%)

PPO (%)

SAC (%)

PPO (%)

Won

85.83

91.99

91.25

97.03

Destroyed

13.5

6.96

6.64

2.7

Looping

0.59

1.05

2.11

0.27

Continuous environment Outcome

With obstacles

Without obstacles

SAC (%)

PPO (%)

SAC (%)

PPO (%)

Won

93.36

94.34

94.48

98.72

Destroyed

5.09

5.13

5.52

1.26

looping

1.55

0.53

0

0.02

Fig. 9.5 Simulation results

References 1. Gawehn, E., Hiss, J.A., Schneider, G.: Deep learning in drug discovery. Mol. Inf. 35(1), 3–14 (2016) 2. Vamathevan, J., Clark, D., Czodrowski, P., Dunham, I., Ferran, E., Lee, G., Li, B., Madabhushi, A., Shah, P., Spitzer, M., Zhao, S.: Applications of machine learning in drug discovery and development. Nat. Rev. Drug Discovery 18(6), 463–477 (2019) 3. Marcus, G.: Deep Learning: A Critical Appraisal. arXiv preprint arXiv:180100631 (2018) 4. Medvedev, M., Pshikhopov, V., Gurenko, B., Hamdan, N.: Path planning method for mobile robot with maneuver restrictions. In: Proceeding of the International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME) 7–8 October (2021). https://doi.org/10.1109/ICECCME52200.2021.9591090 5. Kostjukov, V.A., Medvedev, M.Y.E., Pshikhopov, V.K.: Method for optimizing of mobile robot trajectory in repeller sources field. Inf. Autom. 20(3), 690–726 (2021)

9 Intelligent System for Countering Groups of Robots Based …

145

6. Medvedev, M., Kostjukov, V., Pshikhopov, V.: Optimization of mobile robot movement on a plane with finite number of repeller sources. SPIIRAS Proc. 19(1), 43–78 (2020). https://doi. org/10.15622/sp.2020.19.1.2 7. Pshikhopov, V., Medvedev, M.: Multi-loop adaptive control of mobile objects in solving trajectory tracking tasks. Autom. Remote. Control. 81(11), 2078–2093 (2020) 8. Medvedev, M., Pshikhopov, V.: Path planning of mobile robot group based on neural networks. In: International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, pp. 51–62 (2020) 9. Gaiduk, A.R., Martjanov, O.V., Medvedev, M.Y., Pshikhopov, V.K., Hamdan, N., Farhood, A.: Neural network based control system for robots group operating in 2-d uncertain environment. Mechatron. Autom. Control 21(8), 470–479 (2020) 10. Liu, Q., Wu, Y.: Supervised learning. In: Web Data Mining, pp. 63–112 (2012). https://doi.org/ 10.1007/978-1-4419-1428-6_451 11. Alloghani, M., Al-Jumeily Obe, D., Mustafina, J., Hussain, A., Aljaaf, A.: A Systematic Review on Supervised and Unsupervised Machine Learning Algorithms for Data Science (2020). https://doi.org/10.1007/978-3-030-22475-2_1 12. Reddy, Y.C.A.P., Viswanath, P., Reddy, B.E.: Semi-supervised learning: a brief review. Int. J. Eng. Technol. 7(8.1), 81 (2018). https://doi.org/10.14419/ijet.v7i1.8.9977 13. Sharma, R., Prateek, M., Sinha, A.: Use of reinforcement learning as a challenge: a review. Int. J. Comput. Appl. 69, 28–34 (2013). https://doi.org/10.5120/12105-8332 14. Sallab, A., Rashwan, M.: Self-learning Machines using Deep Networks (2011). https://doi.org/ 10.1109/SoCPaR.2011.6089108 15. Kanishka Nithin, D., Bagavathi, S.P.: Generic feature learning in computer vision. Proc. Comput. Sci. 58, 202–209 (2015). https://doi.org/10.1016/j.procs.2015.08.054 16. Charbuty, B., Abdulazeez, A.: Classification based on decision tree algorithm for machine learning. J. Appl. Sci. Technol. Trends 2(01), 20–28 (2021) 17. Sutton, R.S., Barto, A.G.: Reinforcement learning: An introduction. MIT Press (2018) 18. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., Hassabis, D.: Human level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015) 19. Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., Hassabis, D.: Mastering the game of go with deep neural networks and tree search. Nature 529, 484–489 (2016) 20. Balducci, F., Grana, C., Cucchiara, R.: Affective level design for a role-playing videogame evaluated by a brain-computer interface and machine learning methods. Vis. Comput. 33, 1–15 (2016) 21. Cruz, R.M., Sabourin, R., Cavalcanti, G.D., Ren, T.I.: META-DES: a dynamic ensemble selection framework using meta-learning. Pattern Recogn. 48(5), 1925–1935 (2015) 22. Schaeffer, J., Lake, R., Lu, P., Bryant, M.: Chinook the world man-machine checkers champion. AI Mag. 17(1), 21–21 (1996) 23. Campbell, M., Hoane, A.J., Hsu, F.: Deep blue. Artif. Intell. 134(1), 57–83 (2002) 24. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Leg, S., Hassabis, D.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015) 25. Berner, C., Brockman, G., Chan, B., Cheung, V., D˛ebiak, P., Dennison, C., Farhi, D., Fischer, Q., Hashme, S., Hesse, C., Józefowicz, R.: Dota 2 with Large Scale Deep Reinforcement Learning. arXiv preprint arXiv:1912.06680 (2019) 26. Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., Chen, Y.: Mastering the game of go without human knowledge. Nature 550(7676), 354–359 (2017)

146

V. Parkhomenko et al.

27. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal Policy Optimization Algorithms. arXiv:1707.06347 (2017) 28. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous control with deep reinforcement learning. In: International Conference on Learning Representations (ICLR) (2016) 29. Christodoulou, P.: Soft Actor-Critic for Discrete Action Settings. arXiv preprint arXiv:1910. 07207 (2019) 30. Pu, Y., Wang, S., Yao, X. Li, B.: Context-Based Soft Actor Critic for Environments with Non-stationary Dynamics. arXiv preprint arXiv:2105.03310 (2021) 31. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal Policy Optimization Algorithms. arXiv preprint arXiv:1707.06347 (2017) 32. He, K., Zhang, X., Ren, S. Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016) 33. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

Chapter 10

LIRS-MazeGen: An Easy-to-Use Blender Extension for Modeling Maze-Like Environments for Gazebo Simulator Anzhelika Iskhakova , Bulat Abbyasov , Timophey Mironchuk , Tatyana Tsoy , Mikhail Svinin , and Evgeni Magid Abstract 3D modeling is a reconstruction technique used to create a 3D digital representation of real objects with varying difficulty. In robotics field, the modeling approach is often used for prototyping 3D robot models and designing virtual environments. But since 3D modeling requires essential skills and knowledge, modeling tools are hard to understand for beginners and therefore do not allow creating complex models rapidly. This paper presents a maze generation tool LIRS-MazeGen, which is an extension for the Blender modeling toolset. The extension is written in Python language and provides graphical user interface for creating 3D maze-like environments with varying difficulty in a quick manner. The tool allows creating four types of models: a regular maze, a bounded maze, an office type environment, and a warehouse environment. The output 3D models are saved into Gazebo world files and could be further easily loaded into Gazebo simulator. The generated mazes were validated using Hector SLAM and GMapping navigation algorithms with three types of mobile robots: Servosila Engineer, Husky, and TurtleBot3. The virtual tests demonstrated an acceptable level or real-time factor (RTF) that allows to use the generated Gazebo worlds in a comfortable for a user manner.

10.1 Introduction Designing objects in 3D simulators allows conducting experiments in similar to realworld conditions. In robotics, 3D modeling of objects plays an important role and is commonly used. An object that is created in a simulator can be considered useful if its model meets specifications of a physical object and does not create additional difficulties for a user due to the simulation complexity [1]. Only in this case getting A. Iskhakova · B. Abbyasov · T. Mironchuk · T. Tsoy · E. Magid (B) Kazan Federal University, Kremlin St. 35, Kazan 420008, Russia e-mail: [email protected] M. Svinin College of Information Science and Engineering, Ritsumeikan University, 1-1-1 Noji-Higashi, Kusatsu 525-8577, Shiga, Japan © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_10

147

148

A. Iskhakova et al.

benefits from a simulation, such as saving time, an ability of finding and correcting errors, keeping an intactness of physical equipment, is possible. In the robotics field, mazes and robots are closely related. Testing navigation, autonomy, and sensors of robots using mazes becomes possible. Moreover, mazes are beneficial environments that allow to simulate difficult for a robot features, e.g., elongated walls and multiple random obstacles, which are typical for complicated environments such as urban search and rescue [2]. Furthermore, simpler tasks in maze-like environments can also be assigned to robots, e.g., delivering drugs in a hospital or carrying a heavy load in a warehouse. This variety of tasks increases a relevance of using mazes in experiments and research tasks [3]. Blender software [4] allows 3D-simulating of objects and environments. The software usage requires essential skills and knowledge in 3D modeling. 3D modeling from scratch is a time-consuming process, which is hard to understand for beginners and therefore do not allow creating complex models rapidly. To solve this problem, we extended Blender software by developing a special plugin—LIRS-MazeGen (stands for Laboratory of Intelligent Robotic SystemsMaze Generator), which can automatically generate a random maze with a preset configuration within a short time. The plugin allows building four types of mazes with different levels of difficulty. The resulting world could be loaded into the Gazebo simulator [5] and used for various tasks. One of these tasks is simultaneous localization and mapping (SLAM) [6]. During a mapping procedure, a robot explores an environment and creates a map. A practical usability of mazes created by LIRSMazeGen was evaluated by running Hector SLAM [7] and GMapping [8] algorithms that were run on Servosila Engineer [9], Husky [10], and TurtleBot3 Burger [11] robots within the Gazebo simulator.

10.2 Related Work This section presents a review of recent literature on various software for automatic generation of environments in Gazebo. Lavrenov and Zakiev presented a new tool for creating a realistic 3D-landscape in Gazebo simulation based on the real sensory data [12]. It takes and filters a 2D grayscale image and generates a 3D map for loading into Gazebo simulator. The generation is based on a heightmap technique method, which causes a slow performance of lidar-based SLAM tasks due to collision computations. Abbyasov et al. presented LIRS World Construction Tool (LIRS-WCT) that automatically constructs landscapes for Gazebo simulator [13]. The tool converts a 2D grayscale image into a 3D Collada mesh model. Generation uses a solid model technique method. This approach reduces collision computation time. The resulting map can be directly imported into Gazebo as a mesh and used for different purposes. Another recent work by Gabdrahmanov et al. [14] presented an automatic generator LIRS-RSEGen that creates random step environment (RSE [15]), which simulates an overloaded debris environment and is popular in performance evaluation

10 LIRS-MazeGen: An Easy-to-Use Blender Extension for Modeling …

149

of urban search and rescue (USAR) robot models and worlds [16]. LIRS-RSEGen constructs worlds that could be further imported into the Gazebo.

10.3 Maze Generation Module The process of building a maze-like environment is captured in Fig. 10.1. The flowchart consistently reflects actions performed by a user. The process includes launching the Blender software, modeling mazes with LIRS-MazeGen plugin and transferring created mazes into Gazebo simulator. LIRS-MazeGen is installed in Blender as a plugin. It implies automatic modeling of four types of mazes using graphical user interface (GUI) [17], and exporting a world file to the Gazebo simulator, which contains a mesh model and necessary virtual environment settings. An automatic world file generation module takes a generated 3D maze model and creates a world populated with this model. A world file contains information about all simulation objects. For example, it describes a model size, a position, a texture, etc. SDF-element mesh is used to configure a 3D generated mesh file for an environment construction (see Algorithm A). After saving the generated world file, the virtual maze environment is ready for loading into the Gazebo simulator.

Fig. 10.1 Process flowchart of LIRS-MazeGen plugin

150

A. Iskhakova et al. Algorithm A. World file description



model://ground_plane

model://sun

0 0 0 0 0 0 true



file://maze.dae





10.3.1 Graphical User Interface One of advantages of LIRS-MazeGen tool is a simple GUI. After installing the plugin into the Blender software, the software offers a new menu displayed to a user. The menu simplifies a process of an automatic maze generation. It has five buttons: four allow generating a maze of particular type and one generates a world file with the constructed maze. “Classical maze” button generates a regular maze; “Inverted classical maze” button generates a bounded maze; “Warehouse maze” button generates a warehouse environment; and “Office type maze” button generates an office type environment (Fig. 10.2). “Export to the Gazebo mesh” button generates the world file and stores it in a user specified location (Fig. 10.3). After selecting a maze type with a corresponding button click, a user configures maze parameters. In case of regular and bounded mazes, it is necessary to set up a length and a width of the maze with X and Y coordinates (Fig. 10.4).

10 LIRS-MazeGen: An Easy-to-Use Blender Extension for Modeling …

151

Fig. 10.2 Deployed plugin menu

Fig. 10.3 Exporting a world file

Fig. 10.4 Setting parameters for regular and bounded mazes

For office type and warehouse environments, the number of parameters increases. These two types of mazes include room-like sections, which have additional parameters. In case of a warehouse maze generation (Fig. 10.5, left), a user configures minimum and maximum room sizes in X and Y directions and their number. The plugin suggests setting a size of a corridor that connects rooms in a warehouse. In case of an office type environment (Fig. 10.5, right), a user configures room sizes in X and Y directions and number of obstacles in the room. The plugin allows choosing a convex or a concave type of obstacles. Finally, “OK” button starts a maze generation. If required, a new random maze can be generated with the same steps without changing the parameters.

152

A. Iskhakova et al.

Fig. 10.5 Setting parameters for warehouse (on the left) and office (on the right) environments

10.3.2 Experimental Considerations To evaluate the GUI several dozens of experiments on automatic generation of random mazes in Blender software was carried out with the developed plugin. We set various parameters of mazes to obtain mazes of different sizes: from simplest models to compound ones. Small length and width values and small number of rooms provide small mazes, which are labeled as mazes of a simple difficulty level. An increase of these values allows generating mazes of medium difficulty level, while large values result in advanced level mazes. This way, a complexity of a maze (i.e., its type) hints on a number of paths or rooms inside the maze. For large and complicated mazes, it is natural to expect that an exploration task of a robot could be more complicated. Further, experiments with different mazes and robots were conducted with the consideration of the difficulty levels.

10.3.3 3D-Maze Modeling After a world file is generated, the constructed maze model could be opened in the Gazebo simulator. Gazebo is a popular well-designed free 3D robotics simulator that allows to design and emulate complex virtual environments and operate with robots. Gazebo simulator allows viewing generated by LIRS-MazeGen mazes: its physics engine parses a world file exported by LIRS-MazeGen tool and loads a 3D maze model into a configured virtual simulation. As was explained previously, generation of four types of mazes was implemented: a regular maze, a bounded maze, an office type environment, and warehouse environment. Each type has a different level of difficulty. This subsection contains figures of

10 LIRS-MazeGen: An Easy-to-Use Blender Extension for Modeling …

153

each maze type only in medium or advanced type of difficulty level to demonstrate possibilities of the plugin (Figs. 10.6, 10.7, 10.8, 10.9 and 10.10). Simple level maze examples are not included. Figure 10.6 shows a regular maze of the advanced difficulty level. The width and length are 35 and 40 m, respectively. The maze consists of a perimeter fence and deadlock paths. This is a classic type maze. It does not include room-type spaces, which can be seen in the office and warehouse type mazes. Figure 10.7 shows a model of a bounded maze of the medium difficulty level. The width and length are 20 and 25 m, respectively. In contrast to the regular maze, the bounded maze does not include a perimeter fence; therefore, a robot can enter the maze through several entrances.

Fig. 10.6 Regular maze of the advanced difficulty level in the Gazebo simulator

Fig. 10.7 Bounded maze of the medium difficulty level in the Gazebo simulator

154

A. Iskhakova et al.

Fig. 10.8 Office type maze of the medium difficulty level in the Gazebo simulator

Fig. 10.9 Simulated object models (two chairs and five tables) within an office type maze of the m dium difficulty level in the Gazebo simulator

Figure 10.8 shows an office type maze of the medium difficulty level. Office type mazes include different obstacles that imitate office furniture, e.g., tables, chairs, and cabinets. Furniture increases a complexity of a robot within the environment. Figure 10.9 illustrates simulated tables and chairs within the office type maze.

10 LIRS-MazeGen: An Easy-to-Use Blender Extension for Modeling …

155

Fig. 10.10 Warehouse type maze of the advanced difficulty level in the Gazebo simulator

To construct a warehouse environment with long corridors and a large number of rooms, which are located on both sides of the corridors, the algorithm selects N rooms in a row and sizes of the rooms and walls. Larger variable N produces a longer maze. Figure 10.10 shows 28 generated rooms (14 rooms in each row) in a warehouse environment. This type of maze has a high importance because warehouses are often modeled to simulate public structures such as hospitals, stores, hotels, or educational institutions that expect service robots to carry out different chores inside.

10.4 Maze Environment Validation Produced with LIRS-MazeGen environments were validated by running 2D lidarbased SLAM algorithms with UGVs. We used three different UGVs: Servosila Engineer, Husky, and TurtleBot3 Burger. Servosila Engineer is a mobile crawler-type robot made by Russian company Servosila for extreme conditions and emergency situations. Husky is an all-terrain medium sized four wheels robot. TurtleBot3 Burger of ROBOTIS Company is a compact, modular differential drive robot. To create a virtual environment map, hector_slam [18] and gmapping [19] 2D SLAM ROS algorithms were used. Both algorithms employ laser range finder (LRF) sensory data and odometry to build a 2D occupancy grid map. A robot with a LRF explored an environment and built a map of the area. Mapping and planning were visualized using a 3D visualization tool RViz [20].

156

A. Iskhakova et al.

Fig. 10.11 Servosila engineer robot in a regular maze of the medium difficulty level in the Gazebo simulator

Navigation tasks execution examples are shown in Figs. 10.11, 10.12, 10.13, 10.14, 10.15, 10.16, 10.17 and 10.18. In each experiment, a robot received a target point and started movement to this point while building a map of a maze. Robot’s movement was controlled via move_base [21], which was responsible for a path planning process. Servosila Engineer robot explored a regular maze of the medium difficulty level with gray-colored walls (Fig. 10.11) in the Gazebo simulator. The map was built by hector_slam algorithm, which is illustrated in RViz simulator (Fig. 10.12). The red region denotes a way that is safe for the robot exploration. Blue boundaries are the obstacles (walls), which limit robot locomotion. The green line is a local route planner’s path, and the red arrow is a global route planner’s path. Husky robot started at an entrance of a bounded maze of the advanced level (Fig. 10.13). Figure 10.14 demonstrates the environment map, constructed by the robot in the RViz simulator using GMapping algorithm. Light gray space presents already explored by the robot corridors, and their black boundaries imply detected walls. Figures 10.15 and 10.16 demonstrate the main difference between the office type environment and other mazes. The office maze contains simulated office objects that serve as additional obstacles for a robot, which the robot should recognize and Fig. 10.12 Running experiment with Servosila Engineer robot in the RViz simulator

10 LIRS-MazeGen: An Easy-to-Use Blender Extension for Modeling …

157

Fig. 10.13 Husky robot in a bounded maze of the advanced difficulty level in the Gazebo simulator

Fig. 10.14 Experiment with Husky robot (marked by the red rectangular) in the RViz simulator

negotiate with. Figure 10.15 shows Husky in the Gazebo simulator. Figure 10.16 illustrates a map of the office that was constructed by GMapping algorithm; the robot location is emphasized with a red square. The gray square places are rooms connected with corridors, while the black color marks the detected walls and obstacle boundaries. The small dot patches within the rooms imply detected legs of chairs and tables. TurtleBot3 Burger explored a warehouse environment that contained 20 rooms (ten rooms per row, Fig. 10.17). GMapping algorithm allows building a navigation map. The maze consisted of rooms and a straight common corridor with multiple entrances to the rooms (Fig. 10.18). A large room (on the left in the bottom of Fig. 10.18) that was explored by TurtleBot3 Burger was not completed in RViz simulator (a dark gray area in the room center) as a small robot requires extra time for a large area mapping. The robot location is emphasized with a red square.

158

A. Iskhakova et al.

Fig. 10.15 Husky robot and objects in an office type maze of the advanced difficulty level in the Gazebo simulator

Fig. 10.16 Husky robot constructs a map in an office type maze in the RViz

Effectiveness of a navigation task within four different environments, which were generated by LIRS-MazeGen extension, was evaluated using real-time factor (RTF) indicator of SLAM algorithms execution. While testing mazes all three robots successfully navigated within the generated environments and two of three of them demonstrated an acceptable behavior. TurtleBot3 Burger and Husky robots RTF was equal to 0.9, while the minimal acceptable RTF for a comfortable use of a robot in the Gazebo simulator is 0.3 [13]. RTF of Servosila Engineer robot was 0.1, which is

10 LIRS-MazeGen: An Easy-to-Use Blender Extension for Modeling …

159

Fig. 10.17 Gazebo simulation of TurtleBot3 Burger robot in a warehouse environment of the medium difficulty level

Fig. 10.18 Running experiment in a room of warehouse environment with TurtleBot3 Burger robot in RViz

below the recommended threshold. Servosila Engineer has caterpillar wheels that are modeled with more than 200 small moving parts [9]. Gazebo physics engine cannot handle collision calculation of tracks quickly. Thus, collision calculations affects RTF, and Gazebo simulation runs slowly.

10.5 Conclusions This paper presented LIRS-MazeGen—a new Blender extension for automatic generation of random mazes. The extension was written in Python language and provides

160

A. Iskhakova et al.

graphical user interface for creating 3D maze-like environments with varying difficulty in a quick manner. It allows to select the type of a maze and the level of difficulty depending on robot’s tasks. The tool allows creating four types of simple models: a regular maze, a bounded maze, an office type environment, and a warehouse environment. The resulting world could be directly imported into the Gazebo simulator as a mesh and used for various purposes: from testing navigation algorithms to complex USAR environment modeling. Constructed with LIRS-MazeGen environments of three different complexity levels were validated with Hector SLAM and GMapping algorithms that were run on Servosila Engineer, Husky, and TurtleBot3 Burger mobile robots within the Gazebo simulator. In all testing environments, robots demonstrated acceptable RTF level. Acknowledgements The reported study was funded by the Russian Science Foundation (RSF) and the Cabinet of Ministers of the Republic of Tatarstan according to the research project No. 22-21-20033.

References 1. Maria, A.: Introduction to modeling and simulation. In: Proceedings of the 29th conference on Winter simulation, pp. 7–13 (1997) 2. Magid, E., Pashkin, A., Simakov, N., Abbyasov, B., Suthakorn, J., Svinin, M., Matsuno, F.: Artificial intelligence based framework for robotic search and rescue operations conducted jointly by international teams. Smart Innovat. Syst. Technol. 154, 15–26 (2019) 3. Alamri, S., Alshehri, S., Alshehri, W., Alamri, H., Alaklabi, A., Alhmiedat, T.: Autonomous maze solving robotics: algorithms and systems. Int. J. Mech. Eng. Robot. Res. 10(12), 668–675 (2021) 4. Gschwandtner, M., Kwitt, R., Uhl, A., Pree, W.: BlenSor: Blender sensor simulation toolbox. In: International Symposium on Visual Computing, pp. 199–208 (2011) 5. Rivera, Z.B., De Simone, M.C., Guida, D.: Unmanned ground vehicle modelling in Gazebo/ROS-based environments. Machines 7(2), 1–21 (2019) 6. Labbé, M., Michaud, F.: RTAB-Map as an open-source lidar and visual simultaneous localization and mapping library for large-scale and long-term online operation. J. Field Robot. 36(2), 416–446 (2019) 7. Kamarudin, K., Mamduh, S.M., Yeon, A.S.A., Visvanathan, R., Shakaff, A.Y.M., Zakaria, A., Kamarudin, L.M., Rahim, N.A.: Improving performance of 2D SLAM methods by complementing Kinect with laser scanner. In: IEEE International Symposium on Robotics and Intelligent Sensors (IRIS), pp. 278–283 (2015) 8. Chan, S.-H., Wu, P.-T., Fu, L.-C.: Robust 2D indoor localization through laser SLAM and visual SLAM fusion. In: IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 1263–1268 (2018) 9. Dobrokvashina, A., Lavrenov, R., Martinez-Garcia, E.A., Bai, Y.: Improving model of crawler robot Servosila “Engineer” for simulation in ROS/Gazebo. In: 2020 13th International Conference on Developments in eSystems Engineering (DeSE), pp. 212–217 (2020) 10. Cadavid, H., Pérez, A., Rocha, C.: Reliable control architecture with plexil and ros for autonomous wheeled robots. In: Colombian Conference on Computing, pp. 611–626 (2017) 11. Amsters, R., Slaets, P.: Turtlebot 3 as a robotics education platform. In: International Conference on Robotics in Education (RiE), pp. 170–181 (2019)

10 LIRS-MazeGen: An Easy-to-Use Blender Extension for Modeling …

161

12. Lavrenov, R., Zakiev, A.: Tool for 3D Gazebo map construction from arbitrary images and laser scans. In: Proceedings of 10th International Conference on Developments in eSystems Engineering (DeSE), pp. 256–261 (2017) 13. Abbyasov, B., Lavrenov, R., Zakiev, A., Yakovlev, K., Svinin, M., Magid, E.: Automatic tool for gazebo world construction: from a grayscale image to a 3D solid model. In: International Conference on Robotics and Automation (ICRA), pp. 7226–7232 (2020) 14. Gabdrahmanov, R., Tsoy, T., Bai, Y., Svinin, M., Magid, E.: Automatic generation of random step environment models for gazebo simulator. Lect. Notes Netw. Syst. 324, 408–420 (2021) 15. Jacoff, A., Downs, A., Virts, A., Messina, E.: Stepfield pallets: repeatable terrain for evaluating robot mobility. In: Proceedings of the 8th workshop on performance metrics for intelligent systems, pp. 29–34 (2008) 16. Magid, E., Ozawa, K., Tsubouchi, T., Koyanagi, E., Yoshida, T.: Rescue robot navigation: static stability estimation in random step environment. Lect. Notes Comput. Sci. 5325, 305–316 (2008) 17. Oulasvirta, A., Dayama, N.R., Shiripour, M., John, M., Karrenbauer, A.: Combinatorial optimization of graphical user interface designs. Proc. IEEE 108(3), 434–464 (2020) 18. Kohlbrecher, S., Meyer, J., Graber, T., Petersen, K., Klingauf, U., von Stryk, O.: Hector open source modules for autonomous mapping and navigation with rescue robots. In: Robot Soccer World Cup, pp. 624–631 (2013) 19. Megalingam, R.K., Teja, C.R., Sreekanth, S., Raj, A.: ROS based autonomous indoor navigation simulation using SLAM algorithm. Int. J. Pure Appl. Math. 118(7), 199–204 (2018) 20. Pütz, S., Wiemann, T., Hertzberg, J.: Tools for visualizing, annotating and storing triangle meshes in ROS and RViz. In: 2019 European Conference on Mobile Robots (ECMR), pp. 1–6 (2019) 21. Silva Lubanco, D.L., Pichler-Scheder, M., Schlechter, T.: A novel frontier-based exploration algorithm for mobile robots. In: 6th International Conference on Mechatronics and Robotics Engineering, pp. 1–5 (2020)

Chapter 11

Modeling of Joint Motion Planning of Group of Mobile Robots and Unmanned Aerial Vehicle Aleksander Shirokov , Alexander Salomatin , Rinat Galin , and Vasiliy Zorin Abstract This paper studies the process of forming an algorithm for the actions of a group of mobile robots, which allows to effectively carry out a landing of an unmanned aerial vehicle on their formation in a given area. Building a communication field to transmit data from the drones to a group of robots is investigated, the planning of the local trajectory of the robots is described, and obstacles are taken into account. And the formation of robots taking into account obstacles and the creation of a formation of robots is considered. As a result of the review of current planning methods and their comparative analysis, the method that most effectively copes with similar problems is determined—the method of tangential avoidance. Authors of the paper adapt this method to the conditions of a problem to be solved and offer its modified version as a new method. To confirm the performance of the method, a series of experiments is conducted, which show that the developed method can successfully cope with the formation of trajectories of mobile robots at randomly determined locations of obstacles and robots.

11.1 Introduction In recent years, group application of robotics has been actively developing. A group of robots can solve a much wider range of tasks in a shorter time compared to a single robot. At the same time, only homogeneous or heterogeneous groups of robots can effectively cope with a number of tasks. For example, large-scale exploration and mapping of terrain, including on other planets, assembling complex structures underwater and in space. A. Shirokov (B) · A. Salomatin · R. Galin · V. Zorin V.A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences, Profsoyuznaya 65, 117997 Moscow, Russia e-mail: [email protected] A. Shirokov National Research University “MPEI”, Krasnokazarmennaya 14, 111250 Moscow, Russia © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_11

163

164

A. Shirokov et al.

A promising direction in the development of group robotics is the interaction of robots of different environments, in particular, aerial and terrestrial. Such interaction proves to be effective, since it can expand the functionality of the task being solved and optimize the process of its execution. For example, in [1], the use of unmanned aerial vehicles (UAVs) together with ground robots facilitates the task of building a three-dimensional terrain map. In addition to the task of mapping, ground robots can act as a platform for landing UAVs with the possibility of further maintenance or transportation [2, 3]. This is the issue the authors raise in this paper. Research shows that ready-made solutions already exist for landing UAVs on ground mobile vehicles, but the existing solutions are tied to specific, particular cases of problems, which are limited by the conditions set. In this case, they are considered from the assumption of the possibility to land a drone on a single object. Thus, the problem under consideration can be extended, and optimized by developing new methods and algorithms for controlling the collective interaction of robots in their group application under conditions of unknown dynamically changing situations in advance.

11.2 Problem Statement On a flat field of arbitrary shape, there are 3 robots platforms, which have a cylindrical shape, at random points. Each robot knows the coordinates of its location. There are also obstacles on the field in the form of solid height walls of different shapes, and the location of which is unknown. Above this field, there is a quadcopter, which has a restriction—it cannot land on the surface of the field or on an obstacle. The allowed surface for landing is the top of the ground robot. It is believed that the area of one robot is not enough to successfully land the drone on it, so the robots need to line up in a formation with enough area to land the quadrocopter on it. The condition of the task stipulates that the formation is sufficient for landing a drone when it is formed by three mobile drones. At the initial moment of time, the quadcopter sets a landing point. There are no obstacles in the vicinity of this point that could prevent the formation of the formation. The task is to form an effective action plan to conduct a successful landing of the quadcopter on the formation of robots in the specified location.

11.3 Review of Methods for Planning the Movement of Mobile Robots in a Group In the beginning, the robots must reach the landing zone, and in the process of movement toward the goal, the robot may encounter obstacles and other robots,

11 Modeling of Joint Motion Planning of Group of Mobile Robots …

165

which requires the robots to form paths that not only will be overcome in the shortest time, but also which will not cause collisions between the robots themselves, as well as the robots and obstacles. There are many methods that can be used to solve the problem of planning the movement of robots in a group [4–7]. There are two large groups of methods: global and local. Global methods (for example, methods based on cell decomposition, methods involving neural networks, methods on graphs) allow to plan a global path to a goal on the basis of global information about the environment. In turn, local methods allow the robot to avoid collisions with obstacles that have not been detected in advance or they are mobile, so that their current position cannot be established in advance. It is efficient to combine both types for successful planning. In the current task, the robots do not have global information about the environment; the information is updated as the robots move toward the goal, so local trajectory planning is studied, taking into account obstacles. The main methods of local search of the robot’s path of motion taking into account obstacles are as follows [8–13]: • Bug method. The robot moves toward the goal, and if it encounters an obstacle, the rule of exit to the path is applied, depending on the type of method, and then the robot continues to move toward the goal. In Bug1 method, the robot goes around the obstacle until it reaches the place closest to the target, and in Bug2 method, until the current slope of the straight line (from the current position to the target) coincides with the slope fixed initially in the starting point, after which the robot continues moving toward the target. There are also other modifications of the method, but they are used less often. • The method of potential fields. It is an analogy to the movement of a charged particle in an electrostatic field [14]. Obstacles generate repulsive forces, and the target point generates a significant attractive force. The direction and speed of motion are determined by the gradient of the potential field. • Vector field histogram method. Obstacles in this method are statistically represented by histogram grids, at first two-dimensional, then at the next stage polar one-dimensional [15]. The angle of motion of the robot is carried out in the direction of the sectors with the lowest density of obstacles. • The method of the close-distance diagram. During the motion the obstacles with increased and decreased safety associated with the freedom of movement of the robot in their vicinity are determined [16]. On the basis of these calculations a set of situations is formed and a decision tree is built to identify the current situation and make the right decision to control the robot. • Tangential avoidance method. The robot follows the direction tangent to the boundary of the obstacle to the point that is closest to it, whenever the obstacle is detected at a distance closer to the robot than the given one [17]. A comparative characteristic of the methods is presented below in Table 11.1. Trapped situations are complex situations, in which the robot finds itself in a position where it is difficult or impossible to move toward the goal due to external factors (e.g., passing through narrow areas, moving close to an obstacle, getting stuck in

166

A. Shirokov et al.

places with U-shaped obstacles, etc.). The degree of computational complexity was determined by analysis and comparison. For example, the computational complexity of the close-distance diagram method was defined as average, because many addition, multiplication and comparison operations were performed, and on their basis, the ranking into classes was made. The computational complexity of Bug methods was defined as very low, because in Bug2 method, only the slope of the straight line is calculated, while in Bug1 method, there are no formulaic calculations. For the other methods, the computational complexity was defined as low, because the number of iterations in these algorithms is small and the operations themselves are simple. In addition, each method can be characterized by its advantages and disadvantages, in connection with which the optimal method suitable for solving this or that problem can be chosen (see Table 11.2). In the problem to be solved, the tangential avoidance method is the most appropriate among the listed methods for selecting the path of the robot with obstacles, since it has several suitable prerequisites for this. First, it has a good accuracy of results both in terms of reaching the target location and in terms of selecting a path around obstacles. Second, it is versatile. Besides the fact that it can solve the problem with stationary obstacles, it is also focused on solving problems with obstacles that can move or deform. This is important, because the method allows to find paths even in cases where the free movement of the robot to the target is hampered not only Table 11.1 Comparison of the main methods of local search for the path of the robot, taking into account the obstacles Name of method

Trapped situations

Computational complexity

Number of adjustable parameters

Reaching the target point

Bug method

Yes

Very low

0

Not always

Method of potential fields

Yes

Low, depends on the size of the intensity and the number of obstacles

Depends on the number of parameter functions of the target point field potential and obstacle fields

Not always

Vector field histogram method

Yes

Low, depends on the number of set directions and the number of obstacles

Depends on the number of values for the given possible directions of movement

Always

The close-range diagram method

No

Average, depends on 1 the number of obstacles

Not always

Tangential avoidance method

No

Low, depends on the 2 number of obstacles

Always

11 Modeling of Joint Motion Planning of Group of Mobile Robots …

167

Table 11.2 Advantages and disadvantages of the main methods of local search for the path of the robot taking into account obstacles Name of method

Advantages

Disadvantages

Bug method

Very low computational complexity No configurable parameters

Poor accuracy of the method The robot may not reach the target, and even if it did, in most cases, the path of movement is far from optimal. Behavior in trap situations is not taken into account

Method of potential fields

Automatic selection of the trajectory; high efficiency of implementation

There may be abrupt changes in the direction of movement, which cannot be accurately handled by the robot’s control system; the robot may fall into a local minimum; uncertainty in the construction of potential functions related to the choice of function coefficients

Vector field histogram method

Insensitive to errors; reliability in reaching the target by the robot; copes with some trap situations (passing through narrow areas, no oscillations in the banded areas)

The need to consider the size of the robot and the limitations of the dynamics to achieve greater accuracy

The close-range diagram method

Avoids all trapped situations (e.g., getting stuck in places with U-shaped obstacles, driving among closely spaced obstacles); only one parameter to adjust

Not designed for non-circular robot shapes, special robot kinematics, and given dynamic constraints

Tangential avoidance method

The control system is stable, which means that the robot always reaches the goal of movement; no unnecessary maneuvers, accelerations and decelerations; fast method, high efficiency even in difficult situations; Fast method (no need to identify and analyze sets of possibilities before choosing)

In simple environments with the described environmental conditions, other methods may be more effective

by standing obstacles, but also other robots. In this case, the robots can be conditionally represented as dynamic obstacles and calculations can be made using the same formulas the method operates with. In addition, the method does not solve the problem for a specific situation, which depends on the distances between obstacles, robot shapes or dynamic constraints, but is aimed at solving a wide range of scenarios.

168

A. Shirokov et al.

11.4 Applying Tangential Avoidance to Calculate the Paths of Mobile Robots in a Group Let the area of the site where the robots should reach be a circle Br ∗ (x ∗ , y ∗ ). The center of the circle circle Br ∗ will be called the target point of robot movement. The robots themselves will be represented in the plane by circles Br (xi , yi ), i ∈ N , i ≤ 3. The centers of the circles will be assumed to be the centers of robots. It is also known that each robot is capable of moving with maximum speed vmax and has viewing angle ϕ = 360◦ and maximum detection distance d f . Each robot is able to detect obstacles located from its center at a distance d smaller than the maximum detection distance (d < d f ), and it cannot detect an obstacle if it is located behind another obstacle. It is necessary to find the paths along which the robot will reach the area of the site, moving to the target point. To calculate the robot path, we introduce additional notations: ρ—distance between the center of the robot and the target point; β—azimuth, the angle between the OX axis and the direction to the target; ψ—robot heading; α—heading angle, the difference between heading and azimuth; v—robot linear speed. These characteristics of the robot’s movement to the target point are shown in Fig. 11.1. In the tangential avoidance method, the robot is controlled based on two parameters: linear velocity v and angular velocity w, at which the distance to the target ρ and the course angle α tends to zero. The velocities can be calculated as follows: Fig. 11.1 Characteristics of the robot’s movement to the target point

11 Modeling of Joint Motion Planning of Group of Mobile Robots …

169

v = vmax tanh(ρ) cos α, tanh(ρ) w = kw α + vmax sin α cos α, kw > 0, ρ where vmax is the maximum speed of the robot, kw is some positive parameter determining the control action and the course angle α so: α = β − ψ − K ρ (dmin − d), dmin − d > 0, α = β − ψ, D − d ≤ 0, where K ρ is coefficient of the proportional component of the course angle, dmin is minimum distance to an obstacle and d is distance to an obstacle. For the problem under consideration, an adapted modification of the tangential avoidance method will be used, since the original method is designed for the robot to represent a point on the plane rather than a circle. We will assume that the distance between the robot and the obstacle in the formula is measured between two points: the center of the circle of the robot and the closest point on the obstacle, extended with a contour of width r. In this case, d < 0 will mean that the robot collides with the obstacle. An example of a contour for one obstacle and one robot is shown in Fig. 11.2. This means that in the modification of the method, the distance to the obstacle d will be compared with the sum of the minimum distance to the obstacle and the radius of the circle representing the robot. The heading angle will be calculated using the new formula: α = β − ψ − K ρ (D − d), D − d > 0, α = β − ψ, D − d ≤ 0, where D = dmin + r . A block diagram of the proposed method is shown below in Fig. 11.3. Using the marked actions, all three robots will arrive at the landing zone, moving along the calculated local paths of movement. The time in which the robots reach the landing zone will be determined dynamically, because, as mentioned earlier, the complete terrain map is unknown, and the surrounding environment is determined by the robots in real time. Moreover, it will Fig. 11.2 Obstacle extended with a contour taking into account the robot’s radius

170

A. Shirokov et al.

Fig. 11.3 Block diagram of the method of finding the path of the robot, taking into account the obstacles

generally be different due to the random distance of each robot from the landing point and the need for different deviations from the shortest trajectory due to obstacles of different numbers and sizes determined in the process of movement.

11.5 Experiments In order to clearly demonstrate how the method works, will conduct experiments. Form a map of obstacles randomly, on which robots are located at three given points. Let the number of obstacles is 75, and the distance at which the obstacles are visible is 30 m. The robots themselves are represented by circles with initially specified

11 Modeling of Joint Motion Planning of Group of Mobile Robots …

171

radius and coordinates of the circle center. The maximum speed of the robot is equal to 1. Consider circle B5 (50, 50) as the area the robots need to arrive at. Let the robots be represented as follows for the first three tests: B1 (12, 6), B1 (61, 15), B1 (96, 75), and for the other three: B1 (90, 5), B1 (2, 90), B1 (2, 40). After starting the simulation, the robots start moving to the marked area. As they move, the data affecting the route and speed of the robot at each computational step is collected, and the total time of the task is counted. In Fig. 11.4 and Table 11.3, the final results of the experiments with an open map of the terrain, where all the located obstacles can be seen, are shown.

Fig. 11.4 Results of experiments conducted

Table 11.3 Table of indicators of conducted experiments

Experiment number

Number of obstacles passed

Mission time, s

1

3

122

2

2

131

3

4

145

4

3

136

5

4

138

6

3

133

172

A. Shirokov et al.

Thus, the robots arrive in the specified area, after which their next step is to build together to form a landing platform.

11.6 Formation and Landing In the landing zone, the robot determines its number and according to it begins a maneuver to build. The formation of robots is in the form of an equilateral triangle, where robots are located in the vertices, and the center of the triangle is a given landing point. This formation is shown in Fig. 11.5. The first robot, arriving at the landing zone, is directed to a circular orbit of radius R, which is the circumcircle of the construction triangle, centered at the target point. Its position on the construction orbit becomes one of the vertices of the construction triangle. The second robot, arriving at the landing zone, determines by the position of the first robot the free vertices and occupies the closest of them. If the distance to both points is equal, one of them is selected according to the conflict resolution system. When a third robot arrives in the landing zone, it should determine the third construction point by the two standing robots and head for the free vertex of the triangle. If any robot arrives before the previous robot finishes its maneuver, it should wait until the previous robot finishes its formation and only then start its formation. After each robot occupies a stationary position, it sends a signal about its positioning to the drone. Fig. 11.5 Building robots near the landing point

11 Modeling of Joint Motion Planning of Group of Mobile Robots …

173

The described construction algorithm is shown in Fig. 11.6. It is also possible that the original number was incorrect, for example, because one robot is obscured by another, in which case you must redefine your number by assuming that the closer the robot is to the construction point, the smaller its number. In the case of the same distance to one construction point, the robot that has a smaller distance to another construction point acquires a smaller number and begins to move to a construction point that is unambiguously defined. The other robot waits for the completion of the maneuver of the previous robot and becomes on the remaining construction point. The algorithm of the entire system is shown in Fig. 11.7. In turn, the drone after sending the landing point coordinates over the ground robot heads to the landing point and hovers over it until it receives confirmation from all robots of its placement in the formation. After that, the drone begins landing maneuver on the formation of ground robots.

Fig. 11.6 Block diagram of the construction algorithm

174

A. Shirokov et al.

Fig. 11.7 Block diagram of the algorithm

11.7 Determining How to Build a Communication Field to Transmit Data from a UAV to a Group of Ground Robots For the purpose of ensuring interaction between groups of ground robots and the controlling UAV, it is necessary to organize stable communication channels. There are the following ways of communication organization: • Use of mobile radio telephone networks (hereinafter referred to as PRTS); • Use of satellite communication systems; • Use of aerial repeaters;

11 Modeling of Joint Motion Planning of Group of Mobile Robots …

175

• Use of terrestrial repeaters; • Use of direct radio channel. Of the methods listed above, the use of air and ground repeaters is less preferable because of the involvement of additional units of equipment. Satellite communication systems are expensive in terms of subscriber fees, as well as cumbersome and complex systems of pointing antennas to the satellite for UAVs, including in terms of power consumption. The most acceptable solution for providing a communication system between a constellation of ground robots and UAVs is to use a PRTS network, provided that it is in the network coverage area, which, depending on the radio frequencies used, is from 1 to 10 km [18]. If the condition is met, the UAV and each robot from the ground constellation are equipped with GSM/LTE modems with interfaces for the capabilities of the main control unit (miniPCIe, TCP, RS-232/485, etc.). The modules are equipped with SIM-cards; preference is given to the operator in whose coverage area the work or experiments are planned [19]. An alternative to the use of PRTS networks is the construction of radio communication using a direct radio data channel. This method has a number of limitations associated with: • The radio frequency band used (requiring a special permit to use radio frequencies and not requiring a permit) [20]; • Distance from the constellation of ground robots to the controlling UAV. If the radio frequency band in the 863–873 MHz range is used, the ground robots are equipped with a radio modem and microwave antenna. As an example of a line of equipment can be used as a domestic radio module MBee-868-2.0 [21], weighing 6.35 g and supporting data rates up to 500 kbit /s.This range and the radio module used can provide reliable reception within a radius of up to 1.8 km [22]. The use of radio frequencies in the 2.4 GHz band limits the range of the communication field to 200 m outside of urban areas in the line of sight when using compact radio modules such as MBee v3.0 [23].

11.8 Conclusion The research solved the problem of formation and landing of UAVs on it using a modified method of tangential obstacle avoidance within the framework of group control of mobile robots. This formation is formed by the interaction of a group of land mobile robots. Research was conducted in the area of robot motion planning, and the ways of constructing a communication field for data transfer from UAVs to a group of robots were analyzed. The main focus of the work was the development of a new algorithm for planning the local trajectory of the robots, taking into account the obstacles based on a modified method of obstacle avoidance. Moreover, to verify

176

A. Shirokov et al.

the performance of the algorithm, the authors of the paper conducted a series of experiments, in which it was reflected that the algorithm is able to generate motion paths for robots at any of their initial locations and any location of obstacles. The results of the work can be further used in solving the more general case of the problem, which involves a larger number of factors, such as the presence of dynamic obstacles, the closure of the robots’ departure area from the starting points, or the area of the landing platform formation.

References 1. Gabdullin, A., Buival, A., Lavrenov, R., Magid, E.: ROS modeling of interaction between UAVs and a ground-based unmanned robot to solve a route planning problem in a static environment. In: Unmanned Vehicles with Artificial Intelligence, pp. 21–30. Pero Publishing House, Moscow (2016) 2. Kulapin, V., Knyazkov, A., Egorikhin, A., Shevtsov, P.: Automatic control system for precision landing of unmanned aerial vehicles (UAVs) on a ground wireless charging platform. In: Proceedings of the International Symposium “Reliability and Quality”, pp. 244–246. Penza State University (2015) 3. Ngo, T.: Conceptual and graph-based models of interaction between an unmanned aircraft and a ground-based robotic service platform. Intell. Technol. Transp. 3(19), 35–41 (2019) 4. Tevyashov, G., Mamchenko, M., Migachev, A., Galin, R., Kulagin, K., Trefilov, P., Onisimov, R., Goloburdin, N..: Algorithm for multi-drone path planning and coverage of agricultural fields. In: Proceedings of 1st International Conference on Agriculture Digitalization and Organic Production, pp. 299–310. SPC RAS, St. Petersburg (2021) 5. Trefilov, P., Mamchenko, M., Romanova, M., Ischuk, I.: Improving methods of objects detection using infrared sensors onboard the UAV. In: Proceedings of 15th International Conference on Electromechanics and Robotics Zavalishin’s Readings, vol. 187, pp. 105–114. Springer, Cham (2020) 6. Kolpashchikov, D., Laptev, N., Manakov, R., Danilov, V., Gerget, O., Meshcheryakov, R.: Motion planning algorithm for continuum robots bending over obstacles. In: Proceedings of 2019 3rd International Conference on Control and Technical Systems, pp. 269–272. IoEEE Inc., St. Petersburg (2019) 7. Salomatin, A., Podvesovskii, A., Smolin, A.: Formation of unmanned aerial vehicle link configuration in the problem of cargo transportation scenario selection using geospatial data. Geocontext 1, 5–16 (2021) 8. Belonogov, A.: Analysis and selection of robot navigation systems for positioning under confined space conditions. In: Technical Sciences: Problems and Prospects: Proceedings of the IV International Scientific Conference, pp. 40–42. Svoye Izdatel’stvo, Saint Petersburg (2016) 9. Dubanov, A., Ayusheev, T.: Trajectory modeling in obstacle avoidance. Dyn. Syst. Mech. Mach. 7, 1–8 (2019) 10. Kalyaev, I.: Models and algorithms of collective control in groups of robots. Physical and Mathematical Literature, Moscow (2009) 11. Kazakov, K., Semenov, V.: A review of modern methods of motion planning. Proc. Inst. Syst. Program. 28(4), 2–54 (2016) 12. Liu, W.: Path planning methods in an environment with obstacles (review). Math. Math. Model. 01, 15–58 (2018) 13. Tolok, A., Petukhov, P.: A study of functional-voxel modeling method based on potential field means to path search problems. In: Proceedings of the 13th All-Russian Meeting on Control Problems, pp. 3173–3178. ICS RAS, Moscow (2019)

11 Modeling of Joint Motion Planning of Group of Mobile Robots …

177

14. Rubagotti, M., Della Vedova, M., Ferrara, A.: Time-optimal sliding-mode control of a mobile robot in a dynamic environment. IET Control Theory Appl. 5(16), 1916–1924 (2011) 15. Borenstein, J., Koren, Y.: The vector field histogram-fast obstacle avoidance for mobile robots. IEEE Trans. Robot. Autom. 7(3), 278–288 (1991) 16. Minguez, J., Montano, L.: Nearness diagram (ND) navigation: collision avoidance in troublesome scenarios. IEEE Trans. Rob. Autom. 20(1), 45–59 (2004) 17. Ferreira, A., Pereira, F., Frizera, R., Bastos, T.: A new approach to avoid obstacles in mobile robot navigation: tangential escape. ICINCO 19(4), 395–405 (2005) 18. Vegera, D., Zhiba, G., Pisarenko, V.: Evaluation of GSM signal propagation on a path with complex terrain and coniferous forest. Int. J. Open Inf. Technol. 9(10), 52–60 (2021) 19. Roskomnadzor Portal. https://is.gd/rDe1Zz. Accessed 8 Mar 2022 20. Decree of the Government of the Russian Federation of 21 December 2011 No. 1049-34: On Approval of the Table of Distribution of Radio Frequency Bands between Radio Services of the Russian Federation and Recognition of Some Resolutions of the Government of the Russian Federation as Null and Void. https://digital.gov.ru/ru/documents/3634/. Accessed 8 Mar 2022 21. Creating a Wireless Network Based on MBee S1G 2.0. http://sysmc.ru/solutions/wireless_mod ules_modems/modules/SYSMC_MBee_S1G_2.0/. Accessed 8 Mar 2022 22. Kalachev, A.: Domestic 868 MHz band wireless modules. Wireless Technol. 4(41), 34–39 (2015) 23. Creating a Wireless Network Based on MBee v.3.0. http://www.sysmc.ru/solutions/wireless_ modules_modems/modules/SYSMC_MBee_v3.0/. Accessed 8 Mar 2022

Chapter 12

Simulation of Controllable Motion of a Flying Robot Under the Action of Aerodynamic Force of a Bioinspired Flapping Wing Sergey Efimov, Oksana Emelyanova , and Sergey Jatsun Abstract The article discusses the issues of trajectory control of the movement of a flying robot due to control factors associated with the formation of the aerodynamic force of the biologically inspired movement of the flapping wing of natural analogs— insects. Controlling the movement of a robot with a flapping wing is a complex task and requires the development of mathematical models for the formation of aerodynamic forces resulting from wing oscillations. Under the kinematic and dynamic characteristics, we mean the position, speed and acceleration of the center of mass of the robot and transverse axes, and the forces arising from the wing. These studies will allow collecting information and estimating the forces of interaction of the wing with the air, and then using this information to simulate the movement and control the flight and hovering of a bioinspired mechanical analog. The analysis of the influence of complex laws of motion of the flapping wing, presented in the form of harmonic oscillations, on the formation of aerodynamic force, will allow us to determine the range of parameters at which traction and lifting forces are formed with various varying kinematic and geometric parameters of the bioinsperated flapping wing.

12.1 Introduction Small-sized robotic aircraft is widely used in the tasks of monitoring, remote sensing of space and collecting information for civil and military purposes. Traditionally, multi-rotor and aircraft types are used everywhere. At the same time, active research is being carried out to determine the optimal flight parameters of mechanisms that mimic insects in order to develop and replicate their performance characteristics. Such devices can be successfully used for the purposes of covert monitoring and tracking [1]. It is known that the power consumption of a flying robot with a flapping wing is much less than when using traditional multi-rotor type schemes. Therefore, the S. Efimov · O. Emelyanova (B) · S. Jatsun Southwest State University, 50 Let Oktyabrya Street, 94, 305040 Kursk, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_12

179

180

S. Efimov et al.

leading scientific centers of the world are developing to create small-sized flying robots that simulate the flight of insects and birds [2–4]. The development of bionics as an approach to the creation of technological devices, in which the idea and main elements of the device are borrowed from wildlife, became possible with the advent of microelectronics, miniature navigation systems, communications and other high technologies. An area in which the borrowing of ideas from nature is especially prominent is the creation of small-sized UAVs that imitate the movement of insects. However, the implementation of such principles of movement requires an understanding of the features of interaction and the availability of a mathematical description of the movement of a flapping wing in a resisting air medium, taking into account the properties of the drive [5, 6]. There are well-known dissertations devoted to the study of individual factors affecting the kinematics and dynamics of flapping propellers [7, 8]. For example, in the work of Tarasov 2015, the problem of interaction between a propulsor of the flapping wing type and a wave propulsor in the form of a shell, in liquid and gaseous media, is considered. The substantiation of the physical principles of the origin of the thrust force of propulsive propellers is carried out. Well-known foreign scientists T. Karman, I. Burgers, G. Birkhoff, V. F. Durand, J. Delourier made a great contribution to the theory of flapping propellers. Most often, mechanisms that provide one- and two-coordinate wing movement are considered [9–11]. It is shown that by controlling the rotation of the wings, that is, by adjusting the angles of their rotation, as well as the frequency and amplitude of flapping, the insect forms an aerodynamic force, the projections of which create lifting and traction forces. These forces allow insects to move in space along a given trajectory [2, 12]. In order to understand the mechanics of flapping insect flight, it is necessary to study flapping kinematics by combining it with aerodynamics and flight dynamics [13–15]. Important is the simulation of the insect wing drive. It is known that the chest and wings of an insect have the property of a drive and a spring is a damper, since it contains muscles consisting of elastic proteins-resilin. In a certain way, this biological mechanism makes it possible to implement the synchronous rotation of the wing in such a way as to create the optimal force at the moment, which ensures the planned movement of the insect. In addition, it can be said that a wing mounted on a natural elastic element will be an oscillatory circuit [12]. In connection with the above, we can say that today there is a problem associated with the development of a theory of controlled motion of a flapping wing apparatus based on the study of wing oscillations and their influence on the formation of aerodynamic forces. In turn, it is obvious that the modeling of aerodynamic forces arising during the flight of insects is determined by the kinematics and dynamics of the wing and depends on the inclination of the flapping plane, the laws of flapping and reciprocating rotational motion of the wing relative to its longitudinal axis, the shape and profile of the wing [15, 16]. Therefore, the purpose of this article is to simulate the controlled movement of a flying robot under the action of the aerodynamic force of a bioinspired flapping wing. To achieve this goal, it is necessary to establish a connection between the kinematics of the wing and the occurrence of lifting traction forces of the wing, depending on the adopted laws for changing the angles of rotation

12 Simulation of Controllable Motion of a Flying Robot Under the Action …

181

of the wing. Differential equations describing the movement of the robot in space will make it possible to simulate controlled flight modes.

12.2 Flapping Wing Kinematics In nature, the wing implements three types of rotation: ϕ is rotation around the longitudinal axis of the housing X 2 ; ψ is rotation of the vertical axis Z 2 , i.e., moving the wing forward and backward; α is rotation around the longitudinal axis of the wing, which tilts the wing to change its angle of attack this the weight of the flying platform is the force that occurs when the wing moves in the air (Fig. 12.1). We will assume that the wings of the robot move synchronously, so further we will consider the movement of one wing 1 as an independent rigid body of a rectangular shape (Fig. 12.1b). And of these rotations, a combination of movements along the angles ϕ and α, which make a significant contribution to the formation of lifting and traction forces [15, 17]. The robot moves in the absolute coordinate system OXYZ. The relative, moving coordinate system C 2 X 2 Y 2 Z 2 is associated with the body of the object under consideration. The origin of this coordinate system coincides with the center of gravity of the body C 2 . The C 2 X 2 axis is directed parallel to the longitudinal axis of the housing, the C 2 Y 2 axis is perpendicular to the C 2 X 2 Z 2 plane, and the C 2 Z 2 axis is perpendicular to the C 2 X 2 Y 2 plane, which is the plane of symmetry. The C 2 X 1 Y 1 Z 1 coordinate system is associated with the wing plane. The axis C 2 X 1 coincides with the axis of transverse rotation, and the axis C 2 Y 1 coincides with the edge of the wing and coincides with the longitudinal axis of rotation of the wing.

Fig. 12.1 a Various types of wing rotation in nature: 1, 3 is right and left wings; 2 is body; G is weight; F is force arising from the movement of the wing in the air; b scheme of rotation of the coordinate system of wing 1: υ C1 is speed of the center of mass C 1 of the wing; a, b is geometrical parameters

182

S. Efimov et al.

We will further denote the vectors by the symbols (0) , (1) , (2) defined in the corre(2) sponding coordinate systems. For example: F 1 the aerodynamic force created by the moving wing 1 in the moving coordinate system (2) . The distance between points C 2 and C 1 along the C 2 X 2 axis is equal to a, along the C 2 Z 2 axis it is equal to zero, and along the C 2 Y 2 axis –b; a and b are variables depending on the position of the point C 1 . For approximate calculations, we will take them as constant values. The matrices defining the vectors given in (1) coordinate system, in the C 2 X 2 Y 2 Z 2 coordinate system, taking into account two rotations, have the form (12.1): ⎤⎡ ⎤ 1 0 0 cos α1 0 sin α1 T12 = ⎣ 0 1 0 ⎦⎣ 0 cos φ12 − sin φ12 ⎦. 0 sin φ12 cos φ12 − sin α1 0 cos α1 ⎡

(12.1)

At this stage of research, we will consider the laws of change in the angles of rotation α1 (t) and φ12 (t), and, accordingly, their angular velocities α˙ 1 (t) and φ˙ 12 (t) in the form of smooth functions: α1 = α11 − α10 cos(ωt + γ ); α˙ 1 = α10 sin(ωt + γ );

(12.2)

1 0 0 φ12 = φ12 + φ12 sin ωt; φ˙ 12 = φ12 ω cos ωt,

(12.3)

where α11 , α10 is deviation from the zero (X 2 axis) level and the amplitude of the wing oscillations, relative to the longitudinal axis (Fig. 12.2); γ is phase shift angle; 1 0 φ12 , φ12 is deviation from zero (axis Y 2 ) level and amplitude of wing oscillations, relative to the transverse axis (Fig. 12.3); ω is frequency of oscillations of the wing. Formulas (12.2), (12.3) were obtained based on the analysis of experimental data [9, 11]. The laws of changing angles α1 = α1 (t) and φ12 = φ12 (t) depend on the type of the desired trajectory of the robot. An analysis of the dependences of the change in the angles of rotation of α1 (t) and φ12 (t) Eqs. (12.2), (12.3) shows that there is a phase shift in the angles of rotation

Fig. 12.2 Scheme of the location of the wing 1 relative to the longitudinal axis with the following parameters: a for α11 = 0◦ , α10 = 45◦ , γ = 0; b for α11 = 45◦ , α10 = 45◦ ; γ = 0; c for α11 = 30◦ ; α10 = 45◦ , γ = 0; d for α11 = −45◦ ; α10 = −45◦ , γ = 0

12 Simulation of Controllable Motion of a Flying Robot Under the Action …

183

Fig. 12.3 Scheme of the possible location of the initial position of the wing 1 relative to the 1 = 0◦ , φ 0 = 45◦ ; b for φ 1 = 30◦ , φ 0 = 45◦ transverse axis: a for φ12 12 12 12

of the wings, depending on their initial (zero) and subsequent deviation, relative to the longitudinal and transverse axes. This factor can be used to model the movement control of a flapping wing. The vector of control parameters for models (12.2), (12.3) can be represented as (12.4): ( ) 0 1 T ⊓ = ω, α10 , α11 , φ12 , φ12 .

(12.4)

Now let’s determine the radius-vectors that define the position of the center of mass point C 2 of the hull (link 2) in the OXYZ space (Fig. 12.1b): (0) (0) r OC = r (0) OC2 + r C2 C1 , 1

(12.5)

(0) where r OC = (X, Y, Z )T ; r C(0)2 C1 = T20 · r C(2)2 C1 , T20 is the resulting rotation matrix 2 (12.2) of the coordinate system relative to the inertial OXYZ, described in [7]. Then, (0) (2) r (0) OC1 = r OC2 + T20 r C2 C1 ,

In the coordinate system (Fig. 12.1b):

(2)

(12.6)

, the position of the center of mass C 1 is defined as rC(2)2 C1 = T12 r C(1)2 C1 ,

(12.7)

where r C(1)2 C1 = [−a, b, 0]T . Here a, b are geometric parameters that determine the position of the center C 1 in the coordinate system C2 X 2 Y2 Z 2 . To determine the speed of the center of mass of the wing (point C1 ), we differentiate expression (12.5) with respect to time:

184

S. Efimov et al. (2)

υ C1 = υ C2 + T˙20 r C(2)2 C1 + T20 r˙ C2 C1 ,

(12.8)

(2)

r (2) where T˙20 = dTdt20 ; r˙ C2 C1 = Cdt2 C1 is the speed of relative movement of the center of mass of the wing relative to the mass C2 of the center of the body. The projection equations for the velocity projection of the center of mass C 1 of wing υ C1 on the axes of coupled coordinate systems (1) and (2) according to Eq. (12.8), taking into account (12.6), (12.7) will have the following form:

( ) υ XC1 = b α˙ 1 cos φ12 cos α1 − φ˙ 12 sin φ12 sin α1 ) ( − a α˙ 1 cos α1 sin φ12 + φ˙ 12 cos φ12 sin α1 ;

(12.9)

υY C1 = −b φ˙ 12 cos φ12 + a φ12 sin φ12

(12.10)

( ) υ Z C1 = −b α˙ 1 cos φ12 sin α1 + φ˙ 12 sin φ12 cos α1 ) ( + a α˙ 1 sin α1 sin φ12 − φ˙ 12 cos φ12 cos α1 .

(12.11)

Dependences (12.9)–(12.11) allow us to determine the values of the projections of the velocity ( of ) the center of mass C 1 of the wing on the vector of control parameters υC1 = f 1 ⊓ . Thus, formulas for determining the position of the center of mass of the wing and body of the robot are obtained.

12.3 Model of Force in the Interaction of a Wing with Air (2)

To simulate the aerodynamic force that occurs when the wing moves in the air F 1 (2) and establishes a connection between the speed of the center of mass υ C1 and F 1 , ( ) 2 which is determined by the parameters of the vector ⊓:F 1 = f 2 ⊓ . In general, these vectors can be represented as projections (12.12): F1(2) = (F 1x , F 1y , F 1z )T , υ C1 = (υ C x , υ C x y , υ C z )T .

(12.12)

Let’s establish the relationship between the speed of the center of mass υ C1 of the (2) wing and the force F 1 in the form (12.13): 1 F1x = − C ρ|S X |υC x |υC x |; 2 1 F1z = − C ρ|S Z |υC z |υC z |. 2

| | 1 F1y = − C ρ|SY |υC y |υC y |; 2 (12.13)

12 Simulation of Controllable Motion of a Flying Robot Under the Action …

185

Fig. 12.4 The diagram of the robot wing with relative motion: a down, b up; 1 is the neutral line of the wing relative to the body; 2 the wing; 3 the body

The force F 1 depends on the aerodynamic constant C, which is determined experimentally for each wing, the air density ρ, and the size of the midsection area S = (S X , SY , S Z )T . Using an example, let us consider the schemes of the interaction of a force with a (2) resisting medium F 1 applied at point C 1 when the wing moves up (Fig. 12.4a) and down (Fig. 12.4b). Let’s represent the model of the change in the projection of the midsection area of the wing S in the form of smooth functions (12.14): S X = S0 sin α1 ; SY = S0 cos α1 sin φ12 ; S Z = S0 cos α1 cos φ12 .

(12.14)

Based on Eq. (12.14), taking into account (12.13) and (12.9)–(12.11), we will simulate the projections of the aerodynamic force in the form of f = F 1 /F 1max , created by the moving wing from time to time with the following parameters a = 0.2 m; b = 0.6 m; ω = 25.12 rad/s, C = 0.5; ρ = 1.29 kg/m3 (Fig. 12.5). The average value of the lifting force for the period is determined by the formula (12.15): 2π

(mid) f 1Z

ω = 2π

ω

(mid) f 1Z dt.

(12.15)

0

Variable parameters will be the amplitude of oscillations of the wings α1 (t) relative to the transverse axis (X 2 ) and ϕ12 (t) deviation from the zero (axis Y 2 ) level. Let us estimate the average value of the wing lift force over the period, in accordance with (12.15). Since the mathematical model of the aerodynamic force is 1 = 0; a multicriteria problem, for example, we fix the values of the angles: ϕ12 1 0 ◦ α1 = α1 = 45 ; γ = 0. Variable parameters are the oscillation frequency of the 0 (Fig. 12.6). wing ω and the amplitude of the oscillations of the wing ϕ12 An analysis of the graphs of the average values of the lift force of wing 1 over the period shows that the nature of the change in the projections of the force depends

186

S. Efimov et al.

Fig. 12.5 Dependences of the projections of the wing-air interaction force on time with the 1 = α 1 = 0; φ 0 = α 0 = 45◦ , γ = 0; b for φ 1 = 0, α 1 = 45◦ , following parameters a for φ12 1 12 1 12 1 0 = α 0 = 45◦ , γ = 0; c for φ 1 = −30◦ , α 1 = 30◦ , φ 0 = 60◦ , ← α 0 = 60◦ , γ = 0 φ12 1 12 1 12 1

(mid) over the period Fig. 12.6 Calculated dependences of the average value of the lifting force f 1Z 0 on the amplitude φ12 for different frequencies: 1 for ω = 12.56 rad/s; 2 for ω = 25.12 rad/s; 3 for ω = 50.24 rad/s; 4 for ω = 100.48 rad/s

12 Simulation of Controllable Motion of a Flying Robot Under the Action …

187

significantly on the parameters that determine the movement of the wing. The value of the lift force of the wing has the greatest value at the maximum frequency of oscillation of the wing. By changing the variable criteria, it is possible to obtain the forces that ensure the rise of the aircraft and movement along the trajectory.

12.4 Aircraft Motion Control The task of controlling the movement of an aircraft is to ensure the movement specified by the operator. For the synthesis of an automatic control system (ACS), a structure is developed and the parameters of the control system are determined. As calculations show, the most effective, from the point of view of the accuracy of movement of the control object along a given trajectory, is the control strategy implemented by a closed ACS with position control and based on the use of a benchmark model. The equations describing the position of the aircraft have the form [4, 7, 18]: ( ) ⎧ ⎨ V˙ = f(3 Y˙), F , Y =λ X ⎩ Ω = f 4 (γ )

(12.16)

( ) where f 3 Y , F the operator of the observed parameters; f 4 (γ ) is control operator; Ω = (ω X , ωY , ω Z )T is projection of the angular ( ) velocity on the axis of the moving coordinate system; γ = (φ, θ, ψ)T , λ = X is signal processing module; X = )T ( X C2 , YC2 , Z C2 , φ, θ, ψ , ϕ, θ, ψ is roll, pitch and yaw angles. ( )T We will consider the case when the position vector V = X˙ , Y˙ , Z˙ , ψ . The angles of roll ϕ, pitch θ and their derivatives will be insignificant due to the symmetry of the wings of the device, as well as the location of the center of gravity of the aircraft below the point of reference of external forces, which guarantees stability. The task of the ACS is to ensure the flight of a flying robot along a given trajectory, and the reference model acts as a given dynamic link that implements the requirement for the control system. Let us transform the differential equations of motion of the flying robot (12.16) to the benchmark model in the form (12.17): V˙ = AV + Bu,

(12.17)

[ ] where A, B[ are matrixes of object parameters of the form A = ai j (i, j = 1, n) , n ] = 4; B = bi j (i = 1, n; j = 1, m) , m = 3; u is control vector, u ∈ R m ; m ≤ n. Control actions 1 are formed by the corresponding electric drives according to the level of control voltages supplied to the electric drives, which are calculated in the control system controller (12.18):

188

S. Efimov et al.

u = (u 1 , u 2 , u 2 )T ,

(12.18)

where u 1 , is voltage to the drive, providing synchronous rotation of the wings at an angle φ12 relative to the longitudinal axis X 2 ; u 2 , u 2 is stresses on the drives that ensure the rotation of the wings 1, 3 (Fig. 12.1a) at the corresponding angles α1 , α3 relative to the transverse axis Y2 .The calculation of these voltages is made in the controller of the control system by the magnitude of the error Δ of the spatial position of the device (12.19): Δ = (Δx, Δy, Δz, Δψ)T ,

(12.19)

where Δx, Δy, Δz, Δψ is deviations in the corresponding coordinates and the angle of the actual position of the center of mass of the flying robot relative to the given one. For the vehicle to move along a given trajectory, it is necessary to provide a minimum of the functional of the form (12.20): ∞( ) T ΔRΔ + u Qu T dt, J=

(12.20)

0

where R, Q—weight matrices 4 × 4 and 3 × 3 respectively; Δ = r ∗ (t) − r (t) is position error. We will synthesize the control in the form:u = u(Δ). Let’s accept: u = K (Δ), where K-controller coefficient matrix, with dimension 3 × 4 (12.21): | | | κ11 κ12 κ13 κ14 | | | K = || κ21 κ22 κ23 κ24 || |κ κ κ κ | 31 32 33 34

(12.21)

To determine errors, the values of the desired generalized coordinates and the real coordinates of the flying robot are supplied to the controller block. To do this, the modules of the control system are interconnected by feedback and the information processing unit. For the synthesis of coefficients of the matrix K, we apply the LQR method (linear–quadratic regulator), described and applied in [6, 19]. The block diagram of the control system is shown in Fig. 12.7. The control system consists of three main modules: controlled electric drives, control actions and a dynamic model of the robot. The result of mathematical modeling of Eq. (12.16), taking into account (12.17– 12.19) are the dependency graphs shown in Fig. 12.8. Graphs illustrate three stages of flight: takeoff (t = 1–40 s); flight in a horizontal plane (t = 40–80 s) and descent (t = 80–120 s).

12 Simulation of Controllable Motion of a Flying Robot Under the Action …

189

Fig. 12.7 Structural diagram of the system of automatic control of the movement of a flying robot

Fig. 12.8 Dependences of X, Y, Z coordinates and yaw angle ψ: red gives specified parameters, blue gives real parameters

190

S. Efimov et al.

The use of the LQR method in numerical simulation shows sufficient robustness and allows for a fairly stable movement of the aircraft along the specified flight stages. The existing oscillation on the graph illustrating the change in the yaw angle ψ, as well as the non-uniformity of the movement of the aircraft along the X coordinate, are associated with the design parameters of the robot’s wings and the parameters of its control system. The obtained results will be used to solve the problems of modeling and calculation of bioinspectable flying analogs that meet the requirements of sustainable ascent and trajectory flight.

12.5 Conclusion The article deals with issues related to the controlled movement of an aircraft with a flapping wing. Particular attention is paid to modeling the two-coordinate rotation of the flapping wing, which simulates the dynamics of the flight of insects and the formation of the control aerodynamic force. A study was made of the influence of kinematic parameters on the nature of the formation of the lifting and traction forces of a flapping wing with the harmonic nature of oscillations. It is shown that five parameters provide control over the magnitude and direction of these forces, which allows the flying robot to fly along a given trajectory. The harmonic laws of changing the angles of rotation of the flapping wing and the laws of changing the midsection area of the wing S and their influence on the formation of the aerodynamic force are also studied. It has been established that the values of the lift force averaged over the period increase quite linearly with respect to the frequency of wing strokes. Therefore, when designing a bioinspired mechanical analog, it is important to take into account the influence of kinematic, dynamic, and features on the formation of aerodynamic forces, which, in turn, affect the errors that occur during flight along a given trajectory. To minimize deviations, a method for synthesizing the parameters of a multichannel controller based on LQR optimization is proposed. Acknowledgements This work has been supported by Project No 1.7.21/4 of the strategic project “Creation of robotic tools to enhance human functionality”.

References 1. Emelyanova, O., Polyakov, R., Efimov, S., Yatsun, S.: Mobile aircraft complex for early detection of fires. In: Fundamental and Applied Problems of Engineering and Technology, p. 38 (2018)

12 Simulation of Controllable Motion of a Flying Robot Under the Action …

191

2. Khan, Q., Akmeliawati, R.: Review on system identification and mathematical modeling of flapping wing micro-aerial vehicles. Appl. Sci. 11(4), 1546 (2021) 3. Tikhonravov, M.K.: The Flight of Birds and Cars with Flapping Wings. Oborongiz, Moscow (1949) 4. Jatsun, S., Efimov, S., Emelyanova, O., Leon, A.S.M., Davalos, P.J.C.: Modeling and control architecture of an autonomous mobile aerial platform for environmental monitoring. In: 2019 International Conference on Information Systems and Computer Science (INCISCOS), pp. 177–182 (2019) 5. Reynolds, A.M., Reynolds, D.R., Sane, S.P., Hu, G., Chapman, J.W.: Orientation in high-flying migrant insects in relation to flows: mechanisms and strategies. Philos. Trans. Royal Soc. B Biol. Sci. 371(1704), 20150392 (2016) 6. Hajiyev, C., Vural, S.Y.: LQR controller with Kalman estimator applied to UAV longitudinal dynamics (2013) 7. Polyakov, R.Yu.: Mobile instrument platform for the system of environmental monitoring of toxic air pollution. Doctoral dissertation, Kursk (2019) 8. Tarasov, A.E.: Aerohydrodynamic analysis of elastic elements of the driving type by the method of integral equations. Doctoral dissertation, Volgograd State University (2015) 9. Ji, B., Zhu, Q., Guo, S., Yang, F., Li, Y., Zhu, Z., Li, Y.: Design and experiment of a bionic flapping wing mechanism with flapping–twist–swing motion based on a single rotation. AIP Adv. 10(6), 065018 (2020) 10. Qiu, H., Dong, Y., Li, Q., Li, C., Wang, X., Qiu, Z.: Lift estimation of half-rotating wing in forward flight. In: 2017 8th International Conference on Mechanical and Aerospace Engineering (ICMAE), pp. 405–410 (2017) 11. Hosseinie, R., Roohi, R., Ahmadi, G.: Parametric study and performance analysis of a swinging sail wind machine. Energ. Convers. Manage. 205, 112452 (2020) 12. Deng, S., Wang, J., Liu, H.: Experimental study of a bio-inspired flapping wing MAV by means of force and PIV measurements. Aerosp. Sci. Technol. 94, 105382 (2019) 13. Helbling, E., Wood, R.J.: A review of propulsion, power, and control architectures for insectscale flapping-wing vehicles. Appl. Mech. Rev. 70(1) (2018) 14. Taha, H.E., Hajj, M.R., Nayfeh, A.H.: Flight dynamics and control of flapping-wing MAVs: a review. Nonlinear Dyn. 70(2), 907–939 (2012) 15. Ahmedov, T.: Aircraft and underwater vehicles with flapping propellers. Liters (2017) 16. Brodsky, A., Akhmedov, T., Galanin, I., Zeleev, R.: Apparatuses with flapping propellers and their natural analogues. Liters (2018) 17. Vorochaeva, L.Y., Efimov, S.V., Loktionova, O.G., Yatsun, S.F.: Motion study of the ornithopter with periodic wing oscillations. J. Comput. Syst. Sci. Int. 57(4), 672–687 (2018) 18. Yatsun, S.F., Loktionova, O.G., Vorochayeva, L.Yu., Emelyanova, O.V.: The design and control system of an ornithopter robot equipped with wings and a tail. Bull. South-West State Univ. 22(2), 18–26 (2018) 19. Argentim, L.M., Rezende, W.C., Santos, P.E., Aguiar, R.A.: PID, LQR and LQR-PID on a quadcopter platform. In: 2013 International Conference on Informatics, Electronics and Vision (ICIEV), pp. 1–6 (2013)

Chapter 13

Approaches to Optimizing Individual Maneuvers of Unmanned Aerial Vehicle Oleg Korsun , Alexander Stulovskii , Sergey Kuleshov , and Alexandra Zaytseva

Abstract Until late the control of aircraft-type aerial vehicles, manned and unmanned, in manual and automatic modes, was formed largely heuristically, as a result of the evolution of piloting methods, aerobatic figures, and automatic control modes. Mathematical methods of optimal control have been used very sparingly, and thus the study of basic flight maneuvers appears promising from this point of view. Therefore, this paper considers an approach to the optimization of flight maneuvers, which could be utilized onboard. Basis of the suggested approach consists of setting an optimization problem that determines the requirements for a particular maneuver, followed by the use of direct methods for finding the optimal control. The authors propose a modification of the direct method based on the representation of the desired control signals as Hermitian cubic splines and the solution of the resulting multidimensional parametric problem by means of particle swarm optimization. The applicability of the method is demonstrated on the example of solving the problem of the UAV’s acceleration optimizing. For this case, an algorithm for the optimal execution of the maneuver was determined in order to save fuel and reduce its execution time; the result is confirmed by mathematical simulation.

13.1 Introduction Currently, a wide range of different approaches is used in aviation and space technology to solve the problem of optimal control of dynamic systems [1]. They include classical theory based on the calculus of variations and the Pontryagin maximum principle [1, 2], dynamic programming [1], the method of inverse problems of dynamics [3], algebraic theory of control systems [4], etc. O. Korsun · A. Stulovskii State Research Institute of Aviation Systems, 7, Victorenko Street, 125167 Moscow, Russia S. Kuleshov · A. Zaytseva (B) St. Petersburg Federal Research Center of the Russian Academy of Sciences, 39, 14th Line V.O., 199178 St. Petersburg, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_13

193

194

O. Korsun et al.

Majority of these approaches requires, as classical control theory does, the solution of the two-point boundary value problem, which is computationally complex, or the linearization of control object, which is common in many applications of control theory. Although particular cases of nonlinear applications exist [2], this situation has caused active development of approaches based on intellectualization and “soft computing” [5]. Nowadays the use of adaptive systems becomes more widespread [6], as well as reconfiguration of control systems [4, 7], the use of algorithms based on fuzzy logic [8, 9] or neural network technology [10, 11] in control. Intelligent approaches are proposed to be used to detect the facts of loss of control on the part of the crew and prevent emerging negative consequences [12] and provide the flight safety [13]. Other applications also include flight information display and display systems [14, 15]. Since fully automatic systems in aviation in many cases do not provide the required quality of control, the vast majority of existing systems are man–machine. Much attention is given to their further development [16, 17], including the analysis of the dynamic properties of a human operator in the control loop [18] and methods for evaluating the characteristics of a closed loop in flight tests [19, 20]. Adaptive [21] and neural network approaches are used here [22]. At a higher hierarchical control level, there are expert information support systems for the crew, providing guidance for the crew about further flight performance [23, 24] and online analysis of the situation [25]. Thus, control problems are widespread in aircraft and appear at different hierarchical levels and scales. In most cases, control problems are solved numerically [26]. For this purpose in recent years, evolutionary and genetic algorithms have been widely used [27, 28]. The analysis enables to detect a class of maneuvers, which may be improved using the control theory. This class should include first of all basic flight modes, such as steady turns, climbs, descents with specified roll and pitch angles and some aerobatics. All maneuvers included in the class should be initialized at the command, performed automatically and provide the required quality of control according to specified criteria. Therefore, they are in their own way an intellectual extension of the controls which until recently was usually performed by the operator. It is advisable to synthesize algorithms for the automatic implementation of these modes in accordance with the methods of optimal control theory. To achieve this, it is proposed to apply the direct control method developed by the authors [29]. It does not require solution of two-point boundary value problem, which simplifies it numerical implementation. This paper further presents the general three-dimensional mathematical model of aircraft motion, a simplified set of equations derived from it and adapted for solving the control tasks, mathematical descriptions of these tasks as optimization problems, and the examples of solutions obtained by the proposed method. The main characteristic of the method is the reduction of the initial task to the problem of single-criteria parametric optimization in multidimensional space. The numerical solution is provided by particle swarm optimization (PSO) algorithm. Approaches

13 Approaches to Optimizing Individual Maneuvers of Unmanned Aerial …

195

for the adaptation of the proposed individual maneuvers to the control of the group of unmanned aerial vehicles (UAVs) are also suggested.

13.2 Mathematical Model of the Aircraft and Problem Formulation For the solution of particular problem, it is necessary to formulate a model of control object. Let us take as a starting point complete nonlinear model of the spatial motion of an aircraft. The model is valid under an assumption that the axes of the associated coordinate system coincide with the main axes of inertia [30]: ) (a ) ) 1 (( ax y − ω y sin β sin α + + ωx sin β cos α cos β V V ) (a ) (a az y x β˙ = cos β − sin β − ω y cos α + sin β + ωx sin α, V V V

α˙ = ωz −

V˙ = ax cos α cos β − a y sin α cos β + az sin β, Jy − Jz Sl ω y ωz + q m x , Jx Jx ) ( Pright − Pleft z eng keng ωz Jz − Jx Sl ω x ωz + q m y + + , ω˙ y = Jy Jy Jy Jy ) ( Pright + Pleft yeng keng ω y Jx − Jy Sb A ωx ω y + q mz − − , ω˙ z = Jz Jz Jz Jz

ω˙ x =

(13.1)

ϑ˙ = ω y sin γ + ωz cos γ , ( ) γ˙ = ωx − tan ϑ ω y cos γ − ωz sin γ , ψ˙ =

) 1 ( ω y cos γ − ωz sin γ , cos ϑ

X˙ g = V (cos α cos β cos ϑ cos ψ + sin α cos β(sin ϑ cos γ cos ψ − sin γ sin ψ) + sin β(sin ϑ sin γ cos ψ + cos γ sin ψ)), H˙ = V (cos α cos β sin ϑ − sin α cos β cos ϑ cos γ − sin β cos ϑ sin γ ), Z˙ g = −V (cos α cos β cos ϑ sin ψ + sin α cos β(sin ϑ cos γ sin ψ + sin γ cos ψ) + sin β(sin ϑ sin γ sin ψ − cos γ cos ψ)),

196

O. Korsun et al.

) qS ( −cx + c p − g sin ϑ, m qS c y − g cos ϑ cos γ , ay = m qS az = cz + g cos ϑ sin γ , m ax =

where α and β are the angles of attack and sideslip in radians; ωx , ω y and ωz are the angular velocities relative to the body related axes of coordinates in rad/s; ϑ, γ and ψ are the angles of pitch, roll and yaw in radians; V is the actual airspeed in m/s; H is the flight altitude in meters; X g , Z g are the flight coordinates in the Earth related normal system of coordinates in meters; m x , m y and m z are the coefficients of the aerodynamic moments in the body related system of coordinates; cx , c y and cz are the coefficients of the aerodynamic forces in the body related system of coordinates; Jx , Jy and Jz are the inertial moments about the body related axes of coordinates in kg m2 ; m is the aircraft mass in kg; l and bA are the wing span and the length of the mean aerodynamic chord in meters; S is the equivalent wing area in m2 ; q = ρ H V 2 /2 is the dynamic velocity pressure in Pa; ρ H is the air density ( at the flight) altitude in kg/m3 ; g is the gravitational acceleration in m/s2 ; c p = Pright + Pleft /q S is the coefficient of the engine thrust; Pright and Pleft are the thrust of the right and left engines in N; keng in the inertial momentum of the engines in kg·m2 ; yeng and z eng are the coordinates of the engine relative to the body related axes of coordinates in meters. Having determined a sufficiently complete mathematical model of spatial motion for an aircraft or an aircraft-type UAV, we can proceed to solution of the considered problem—optimization of flight maneuvers. Unlike the classical methods of automatic control theory, direct control methods do not require model linearization. However, it is advisable to develop simplified model of smaller dimensions, primarily in order to reduce the necessary amount of computational time and resources. Let us introduce the model suitable for analyses of motion restricted to the vertical plane. This assumption allows significant simplification of original model (13.1). Let us consider following system of differential equations: g qS Px c ye (α) − sin α(t) + (sin α(t) sin ϑ(t) mV (t) mV (t) V (t) + cos α(t) cos ϑ(t) cos γ (t)),

α˙ = ωz −

Px qS cos α(t) + g(− cos α(t) sin ϑ(t) V˙ = − cxe (α) + m m + sin α(t) cos ϑ(t) cos γ (t)), H˙ = V (t)(cos α(t) sin ϑ(t) − sin α(t) cos ϑ(t) cos γ (t)).

(13.2)

13 Approaches to Optimizing Individual Maneuvers of Unmanned Aerial …

197

Algebraic expression for the angular velocity of pitch supplements system of Eq. (13.2): ˙ cos γ (t). ωz = ϑ(t) Introduced system (13.2) is sufficient for describing maneuvers in the vertical plane, taking into account nonlinearities. For its use, we require the values of the aerodynamic lift and drag coefficients only. The pitch angle and thrust could be considered as control signals. The advantage of this choice is that both signals changes slowly over time. Thus, their modeling by way of, for example, a cubic spline, could be done without large number of nodes. The general formulation of optimal control problem is conventional [2] and does not include restrictions. Let us consider following nonlinear time-varying control object model: x˙ = f(x, u, t).

(13.3)

where x and x˙ are the vectors of state coordinates and their time derivatives; u is control vector to be found; f(·) is the known vector-valued function of vector arguments. The initial conditions x(t0 ) = x0 are assumed to be known. The targeted functional to minimize is: (T J (x, u) =

F(x, u, t)dt,

(13.4)

t0

where t0 , T are the beginning and the endpoint of time interval under consideration. Let us assume that control should ensure the minimum deviation of the state coordinates from their desired values over the time interval. Then functional could be specified as follows: (T J (x,∼ x, u) =

x(t))dt, x(t))T K(x(t) − ∼ (x(t) − ∼

(13.5)

t0

where ∼ x is the desired values of state coordinates and K is the matrix of weighting coefficients. For the problem of optimizing the aircraft speed increment the functional (13.5) includes desired values of airspeed and altitude. Since calculations are performed in discrete time, the integral in the expression can be replaced by the sum: F(x, u, t) =

N ( ∑ ( ( ) ) ) ∼(ti ) 2 + k2 H (ti ) − H ∼(ti ) 2 , k1 V (ti ) − V i=0

(13.6)

198

O. Korsun et al.

∼, H ∼ are the desired values of airspeed where k1 , k2 are the weighting coefficients, V and altitude.

13.3 Modification of Direct Control Method Proposed modification of direct control method finds optimal control ∼ u based on model Eq. (13.2) and functional (13.6). Each scalar control signal appearing in the u is projected to a basis of linearly independent functions. That is achieved by vector ∼ use of third-order Hermitian splines [31]. Here we give a brief description of them. Let us choose M values t j ( j = 1, 2, . . . , M) called spline [ nodes ] from the time interval t ∈ [t1 , t M ]. Then for each point t on the interval ti , ti+1 , the value of the spline is found by following formula: S(t) = ϕ1 (τ ) f i + ϕ2 (τ ) f i+1 + ϕ3 (τ )bi f˙i + ϕ4 (τ )bi f˙i+1 , where f i and f˙i are the values of the function and its derivative at node i, ϕ1 (τ ) = (1 − τ )2 (1 + 2τ ), ϕ2 (τ ) = τ 2 (3 − 2τ ), ϕ3 (τ ) = τ (1 − τ )2 , τ = (t − ti )/bi . bi = ti+1 − ti , ϕ4 (τ ) = −τ 2 (1 − τ ), These splines are continuously differentiable scalar functions [31]. This ensures smoothness of obtained controls. The values of the approximated function and its first derivative at the nodes of the spline constitute a vector of unknown parameters that fully determines a unique spline. If the control signals are subject to intense changes or a long time period is considered, then approximation accuracy could be obtained by an increase in the number of nodes, i.e., the number of required parameters. As a result control signals are represented by the vector of parameters and solution of the problem (13.2), (13.6) require minimization of functional in the parameter space: a˜ = argmin J (x,∼ x, u(a)),

(13.7)

a∈A

where a is the vector of parameters and A is search domain in the parameter space. This kind of optimization problem can be solved numerically using genetic or population algorithms. These methods are especially useful in cases where the dimension of the search space is large. Since the parameter space in the given problem has dimensions of the order of 20–30, and also to improve the scalability of the proposed method, we used an algorithm belonging to this family—the particle swarm algorithm [32, 33]. The general concept of particle swarm algorithms is as follows. Initially the search space is set which dimensions correspond to the number of unknown parameters.

13 Approaches to Optimizing Individual Maneuvers of Unmanned Aerial …

199

Each point of this space corresponds to a certain vector of parameter values that determines specific solution of the problem. Afterward particles are placed into the search space. Each of the particles has coordinates inside the space and velocity vector. Coordinates, magnitude and direction of velocity vector are ascribed randomly. Placement of the particle corresponds to a specific solution and determines the value of the targeted functional. At each step of the algorithm, the particles move, under the influence of their initial velocity, as well as information about the position of those swarm particles that are known to have the best value of the targeted functional. Weight coefficients determine the relative importance of these influences. The choice of the coefficients permits to adjust of algorithms ensuring the required coverage of the search space and the thoroughness of the search of promising solutions areas [34]. Summing up the intermediate result, the proposed method consists of the direct minimization of the targeted functional (13.6) taking into account the mathematical model of the control object (13.2). To achieve this goal, the control signals are represented as cubic Hermitian splines. The resulting multidimensional parametric optimization problem is solved using a particle swarm algorithm.

13.4 General Methodological Foundations of UAV Formation Control The considered individual aircraft maneuvers are the basis for the control of the UAV formation. At this hierarchical level, it is required to determine the overall scheme for the coordinated flight control. As a rule, main controlled processes for the group are circling rendezvous, performed after takeoff, group flight and formation reconfiguration. After takeoff, all aircraft-type UAVs implement a coordinated circling rendezvous in order to align their relative position. All multi-rotor UAVs, after takeoff and climb, maintain their current position in anticipation of the formation of the group. After alignment, the aircraft use a hybrid formation control system that provides the desired formation and movement along specified routes. During the flight, the formation can smoothly change in accordance with the proposed method of changing the configuration of the circuit. The selected distributed coordinated grouping control method consists of three parts: • algorithm of circling rendezvous (for aircraft-type UAVs) or maintaining current position at specified point (for multi-rotor UAVs), • algorithm of coordinated route tracking control for the leading UAV, • algorithm of coordinated “leader-follower” control for slave UAVs.

200

O. Korsun et al.

According to principles of consensus-based circling rendezvous, the UAVs sequentially take off and prepare for the upcoming formation, thereby evading collisions on their way. The coordinated route-following control approach is based on the maneuvering of the leading UAV in the group, and the leader–follower coordination control principle directs the slave UAVs along the calculated route. This algorithm is suitable for aircraft-type UAVs, while applying it to multi-rotor UAVs will significantly reduce their energy efficiency. During the mission, a group of multi-rotor UAVs takes off from the launch pad at the same time. Having taken off, each UAV gains the required height and hangs at a specified point under the dome of the optical navigation system. As soon as all the UAVs have reached the required height, their realign themselves into the determined formation. This method also solves the problem of reconfiguring the UAV group into a prescribed formation. The leader UAV changes formation configuration using a coordinated path planning scheme, and other UAVs change their position relative to it. All UAVs are equipped with an identical set of onboard instruments. Each UAV carries a global positioning system receiver, a local optical navigation receiver for communicating with the lead UAV, a flight controller, an onboard computer and a telemetry system. To control the flight task and monitor the task performance, the ground control center of the UAV is used, which is equipped with special software and allows setting input parameters for the group. The UAV ground control center is managed by one operator, which mainly performs online trajectory plotting for leading UAV, flight status monitoring and high-level command launching. Thus, the entire communication system of agents in the group and the group with the operator consists of an optical local navigation system in the air and a UAV ground control point. The proposed hierarchical modular architecture includes five top-down modules, namely: a module for interaction with operator, a communication module, a coordination module, as well as high and low level controls (Fig. 13.1). The module for interaction with the operator is necessary to display the UAV status, information about telemetry and the geographic environment on the screen of the ground control station. It also provides various interaction tools for control of the group. The communication module ensures the transfer of information between the UAVs and the ground control point. The remaining two modules describe the operation at the level of an individual UAV. Each module is implemented on each UAV at the level of coordination. The high-level control unit is deployed on the onboard processor and can perform resource intensive tasks such as path planning, formation control and target tracking. Tasks of such a high level are performed through the cycle “observation-orientationdecision-action”. Path plotting belongs to this level of control, and it is optimized according to a pre-selected quality criterion.

13 Approaches to Optimizing Individual Maneuvers of Unmanned Aerial …

201

Fig. 13.1 Levels of coordinated control algorithm for group of aircraft-type UAVs

13.5 Results of the Study Let us consider the problem of an aircraft speed increment at the shortest time. As an additional requirement it is specified that the altitude should be maintained constant throughout time interval. Let us study the problem in following way: altitude and airspeed in the functional (13.6) would acquire zero weights from the beginning of the maneuver (1.5 s) to some specified moment in time. Then the moment of weight coefficients’ change will shift closer to the endpoint in successive implementations. Due to this, the solutions will move away from the optimal response time, since the algorithm closely follows the starting point of non-zero values of the coefficients. On the other hand, we will obtain a family of solutions that reach a desired speed at different times. The values of altitude and speed are given for speed increment from 160 to 180 m/s in Fig. 13.2.

O. Korsun et al.

Fig. 13.2 Values of altitude a and airspeed b obtained through variation of weighting coefficients in the problem of speed increment from 160 to 180 m/s

H, m

202 4880 4860 4840 4820 4800 4780 4760 4740

Full interval 8s 16 s 24 s 0

10

t, s

20

30

185

V, m/s

180 175

Full interval

170

8s

165

16 s

160 155

24 s 0

10

t, s

20

30

The fuel consumption estimations corresponding to the solutions of speed increment problem are given in Table 13.1. Analyzing Fig. 13.2, we can conclude that in terms of the quality of the solutions obtained, the full functional allows to achieve good results, slightly worse than functional with delays approximately equal to the stabilization time. However, a more detailed examination of Table 13.1 clearly shows that these solutions possess a noticeable advantage in terms of stabilization time and fuel consumption. Let us now consider the thrust projection in this task, shown in Fig. 13.3. Table 13.1 Fuel consumption estimations (speed increment from 160 to 180 m/s) Fuel consumption on the whole interval, kg

Fuel consumption to the speed 180 ± 0.5 m/s, kg

Stabilization time, s

Full functional

7.41

4.97

10.97

8s

7.07

4.25

8.31

16 s

7.76

6.04

25000 20000 P, N

Fig. 13.3 Values of thrust projection obtained for the problem solution

15.1

15000 10000 5000 0

0

2

4

6

8

10

12 14 t, s

16

18

20

22

24

13 Approaches to Optimizing Individual Maneuvers of Unmanned Aerial …

203

Acceptable solution were limited in this case by maximum value of thrust equal to 22,500 N. It allows us to formulate following algorithm for maneuver implementation. Initially, the thrust is brought to the maximum available value, and when the desired speed is reached (as can be seen by comparing Figs. 13.2 and 13.3), it is reduced to the value necessary to maintain this speed value. Thus, the problem of speed increment can be optimally solved in two stages. At the first stage, the full functional is used. At the second stage, we introduce a delay corresponding to the stabilization time obtained at the first stage. As can be seen in Fig. 13.2, this provides sufficiently smooth trajectories, as well as achieving visible advantage in terms of time and fuel consumption (Table 13.1) over the initial solution.

13.6 Conclusion An approach to flight maneuvers optimization is considered. It consists of setting the optimal control problem and applying the direct method to obtain an individual aircraft control. To acquire the control signals they are approximated by cubic Hermitian splines. Values of corresponding parameters are determined by numerical solution of the optimization problem using PSO. Applicability of the approach is shown by the example of the speed increment maneuver. An algorithm for its optimal performance is found allowing for the reduction of time and fuel consumption. All the results are confirmed by mathematical simulation. Acknowledgements This work has been supported by the grant of the Russian Fund for Fundamental Research project 20-08-00449a performed at the State Research Institute of Aviation Systems and by state research No FFZF-2022-0005 performed at the SPC RAS.

References 1. Pupkov, K.A., Egupova, N.D. (eds.): Methods of Classical and Modern Theory of Automatic Control. Publishing House of Bauman Moscow State Technical University (2004) 2. Chernousko, F.L., Ananyevsky, I.M., Reshmin, S.A.: Methods of Control of Nonlinear Mechanical Systems. Fizmatlit (2006) 3. Krutko, P.D.: Inverse Problems of Dynamics in the Theory of Automatic Control. Mashinostroenie (2004) 4. Bukov, V.N.: Embedding Systems. Analytical Approach to the Analysis and Synthesis of Matrix Systems. Publishing House of Scientific literature by N.F. Bochkareva (2006) 5. Kalyaev, I.A. (ed.): In: Materials of the 14th All-Russian Multi-conference MKPU-2021. Southern Federal University Publishing House (2021) 6. Sheridan, T.B.: Adaptive automation, level of automation, allocation authority, supervisory control, and adaptive control: distinctions and modes of adaptation. IEEE Trans. Syst. Man Cybern. Part A Syst. Humans 41(4), 662–667 (2011)

204

O. Korsun et al.

7. Santamaria, E., Barrado, C., Pastor, E., Royo, P., Salami, E.: Reconfigurable automated behavior for UAS applications. Aerosp. Sci. Technol. 23(1), 372–386 (2012) 8. Pashchenko, F.F. (ed.): Fuzzy Models and Control Systems. LENAND (2017) 9. Boubertakh, H.: Knowledge-based ant colony optimization method to design fuzzy proportional integral derivative controllers. J. Comput. Syst. Sci. Int. 56(4), 681–700 (2017) 10. Kondratiev, A.I., Tyumentsev, Yu.V.: Application of neural networks for the synthesis of flight control algorithms. I. Neural network method of reverse dynamics for aircraft flight control. News of higher educational institutions. Aviat. Equip. 2, 23–30 (2013) 11. Kondratiev, A.I., Tyumentsev, Yu.V.: Application of neural networks for the synthesis of flight control algorithms. II. Adaptive adjustment of the neural network control law. News of higher educational institutions. Aviat. Equip. 3, 34–39 (2013) 12. Smaili, M.H., Breeman, J., Lombaerts, T.J.J., Mulder, J.A., Chu, Q.P., Stroosma, O.: Intelligent flight control systems evaluation for loss-of-control recovery and prevention. J. Guid. Control Dyn. 40(4), 890–904 (2017) 13. Martins Ana, P.G.: A review of important cognitive concepts in aviation. Aviation 20(2), 65–84 (2016) 14. Sebryakov, G.G., Tatarnikov, I.B., Tyuflin, Yu.S., Skryabin, S.V., Tarnovskii, A.V.: Principles of creating universal visualization systems of modeling complexes for training tasks, situational analysis and coaching. Bull. Comput. Inform. Technol. 3, 48–50 (2006) 15. Ackerman, K.A., Talleur, D.A., Carbonari, R.S., Xargay, E., Seefeldt, B.D., Kirlik, A., Hovakimyan, N., Trujillo, A.C.: Automation, situation awareness display for a flight envelope protection system. J. Guid. Control Dyn. 40(4), 964–980 (2017) 16. Sebryakov, G.G.: Problems of designing semi-automatic guidance systems for aircraft. Bull. Comput. Inform. Technol. 10, 2–7 (2007) 17. Sebryakov, G.G.: Characteristics of human operator activity in dynamic tracking and guidance systems of aircraft. Bull. Comput. Inform. Technol. 11, 2–8 (2007) 18. Sebryakov, G.G., Nabatchikov, A.M., Burlak, E.A.: Conceptual model of a control object in the formalization of human operator activity in a dynamic tracking loop. In: Materials of the Sixth All-Russian Multi-Conference on Management Problems, vol. 2, pp. 95–100 (2013) 19. Korsun, O.N., Semenov, A.V.: Evaluation of aerobatic characteristics of aircraft based on the results of flight experiment, identification and modeling. Bull. Comput. Inform. Technol. 7, 2–7 (2007) 20. Korsun, O.N., Tikhonov, V.N.: Determination of flight characteristics based on modeling of expert assessments in the “pilot-airplane” system. Inform.-Measuring Control Syst. 6(2), 45–50 (2008) 21. Ting, C.H., Mahfouf, M., Linkens, D.A., Nassef, A., Nickel, P., Roberts, A.C., Roberts, M.H., Hockey, G.R.J.: Real-time adaptive automation for performance enhancement of operators in a human-machine system. In: Proceedings of the Mediterranean Conference on Control Automation (2008) 22. Evdokimenkov, V.N., Kim, R.V., Krasil’shchikov, M.N., Sebryakov, G.G.: The use of the neural network model of pilot control actions for their individually-adapted support. J. Comput. Syst. Sci. Int. 54(4), 609–620 (2015) 23. Zheltov, S.Y., Fedunov, B.E.: Operational goal setting in anthropocentric objects from the viewpoint of the conceptual model called Etap: I. Structures of algorithms for the support of crew decision-making. J. Comput. Syst. Sci. Int. 54(3), 384–398 (2015) 24. Zheltov, S.Y., Fedunov, B.E.: Operational goal setting in anthropocentric objects from the viewpoint of the conceptual model called Etap: II. Operation modes of the on-board real-time advisory expert system and its dialog with the crew. J. Comput. Syst. Sci. Int. 55(3), 380–393 (2016) 25. Levitsky, S.V.: System analysis of long-range air combat for the development of algorithmic support of an onboard system for supporting tactical decisions of a pilot. In: Vasiliev, S.N. (ed.) Intelligent Control Systems, pp. 123–134. Mechanical Engineering (2010) 26. Rao, A.V.: Survey of numerical methods for optimal control. Adv. Astronaut. Sci. 135, 497–528 (2010)

13 Approaches to Optimizing Individual Maneuvers of Unmanned Aerial …

205

27. Karpenk, A.P.: Modern Search Engine Optimization Algorithms. Publishing House of Bauman Moscow State Technical University (2014) 28. Diveev, A.I.: A numerical method for network operator for synthesis of a control system with uncertain initial values. J. Comput. Syst. Sci. Int. 51(2), 228–243 (2012) 29. Korsun, O.N., Stulovskii, A.V.: Direct method for forming the optimal open loop control of aerial vehicles. J. Comput. Syst. Sci. Int. 58(2), 229–243 (2019) 30. Bushgens, G.S. (ed.): Aerodynamics, Stability and Controllability of Supersonic Aircraft. Nauka (1998) 31. Zav’yalov, Yu.S., Kvasov, B.I., Miroshnichenko, V.L.: Spline-Function Methods. Nauka (1980) 32. Eberhardt, R.C., Kennedy, J.A.: Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks, pp. 1942–1948 (1995) 33. Olsson, A.E.: Particle Swarm Optimization: Theory, Techniques and Applications. Nova Science Publishers (2011) 34. Faris, H., Mirjalili, S., Aljarah, I., Mafarja, M., Heidari, A.A.: Nature-Inspired Optimizers: Theories, Literature Reviews and Applications. Springer, Cham (2020)

Chapter 14

Neural Network Technologies in the Tasks of Estimating and Forecasting the Resource of Power Supply Systems in Robotic Complexes Nikolay Poluyanovich , Dmitriy Burkov , and Marina Dubyago Abstract The problem of electromagnetic compatibility (EMC) is related to ensuring reliable operation of a set of primary (power) and secondary circuits (electronic devices). The problem of insulation heating was considered on the basis of a mathematical model of the theory of thermal fluctuations in the destruction of dielectrics at different temperatures. The influence of temperature on the long-term electrical insulation strength of the cable line was investigated. It is shown that the dependence of the time to destruction can be explained on the basis of the theory of thermal fluctuations of destruction. A method for determining the parameters of the equation of the cell life curve is considered. As well as the dependence of the layer-by-layer propagation of magnetic induction on the radius for homogeneous insulation and in the presence of a section of the main insulation with an air cavity. The results obtained confirm the assumptions about the possibility of using the theory of thermofluxation in determining the PCL resource. A layer-by-layer analysis of the distribution of the electric field in the cable revealed distortions of the electric field and showed its dependence on the design features of the cable and the size of the defect.

14.1 Introduction and Problem Statement The study of electromagnetic processes in high-voltage power cable networks is an important task related to maintaining the required level of insulation heating to ensure uninterrupted electrical supply to consumers. One of the main objects of the electric power industry, during the operation of which dangerous magnetic fields of industrial frequency arise, are power transmission lines of various design and high voltage. In connection with the development of electronic computing technology, it has become possible to use numerical methods for calculating electromagnetic N. Poluyanovich (B) · D. Burkov · M. Dubyago Southern Federal University (SFEDU), 105/42 Bolshaya Sadovaya Str., Rostov-on-Don 344006, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_14

207

208

N. Poluyanovich et al.

fields described by partial differential equations, which allow us to solve with great accuracy those problems that were previously solved using analytical techniques under certain assumptions. One of the tasks for which an analytical method can be applied with a sufficient degree of accuracy is the calculation of the magnetic field along the routes of electric transmission lines. The calculation of magnetic fields along the routes of high-voltage cable lines has become relevant in recent years. The reason for this was the widespread introduction of cable lines (CL) with insulation made of cross-linked polyester (SPE), the operating currents of which are higher than the operating currents of cables with magically impregnated insulation and oil-filled cables. In addition, cables with SPE insulation are made, among other things, in a singlelegged version, which allows using various ways of laying them. These two reasons cause an increase in the intensity of magnetic fields along the routes of cable lines, and, as a consequence, the need for their calculation and reduction. The influence of electromagnetic fields on the electrical conductivity of the insulation material of the SCL is due to a small number of free deposits associated with the presence of impurities or oxidation products resulting from heating and aging of the cable insulation during operation. In the idle mode of a cable under voltage, the force acting on electric charges is determined by the intensity of the electric field.

14.2 Related Works During the operation of cable lines, due to the manifestation of the skin effect and the proximity effect, the current is concentrated on the near-surface sections of the conductive cable cores, while the central parts of the cores experience less load. In a Milliken conductor, due to the presence of several (usually from 3 to 6) segments of a small cross-section, the manifestation of the surface effect and the proximity effect is reduced, the electric current is more evenly distributed over the cross-section of each segment [1]. A similar situation is observed when the cable is in an external magnetic field. Such a case is realized, for example, by induction heating of the structural elements of the cable. In particular, induction heating of a conductive core is one of the main operations in the technological process of cable production to increase the rate of vulcanization of cross-linked polyethylene and the productivity of the technological line for cable production. The study of electromagnetic processes and the determination of the power distribution and current density in the conductive core of the cable are an important task, which is a necessary step in solving issues related to ensuring the required level of heating to achieve high-quality performance of the cable structure. The analysis of electromagnetic processes in segmented veins using analytical expressions was carried out in earlier works [2, 3]. The influence of frequency on the

14 Neural Network Technologies in the Tasks of Estimating …

209

course of electromagnetic processes in cables is studied in the article [4]. Electromagnetic processes in a cable line consisting of three cables with segmented cores with a given current flowing through them are calculated and analyzed in [5]. The influence of electromagnetic fields on the electrical conductivity of insulating materials is due to a small number of free charges associated with the presence of impurities or oxidation products that occur as a result of heating and aging of insulation during operation. In the idle mode of a cable under voltage, the force acting on electric charges is determined by the intensity (magnitude of the voltage) of the electric field.

14.3 Investigation of the Influence of Temperature and Electric Field Strength on the Breakdown of Polymer Materials of the Thermofluctuation Theory One of the main influencing factors on the service life of the main insulation (MI) of the power cable is the electric field strength, which has a direct dependence on the voltage applied to the cable core. It is known that the dependence of the time to breakdown on the temperature and the voltage of the electric field can be described on the basis of known empirical equations of power, exponential, universal equations. However, the use of these equations does not provide a physical explanation of the laws of electrical aging and practically encounters great difficulties associated with the absence in the literature of the values of the insulation parameters included in these equations, as well as the absence of conditions under which the studies were conducted [6]. Ra6ota is devoted to the study by computer simulation of electromagnetic processes in the insulating materials of a carapace with polyethylene insulation at a voltage of 10 kV. The core has a cross-section (240 mm2 ) and is located in an alternating magnetic field of a cylindrical inductor.

14.3.1 Problem Statement and Its Solution A single-phase power cable with a multi-core aluminum core and cross-linked polyethylene insulation is considered (Fig. 14.1). The method of integrated temperature sensors was used to monitor thermal fluctuation processes in the SCL. In [7], a mathematical model of the layer-by-layer distribution of thermal processes in a power cable is considered. In this model, the cable is considered as a system of homogeneous bodies, which significantly simplifies the calculation. However, this model does not take into account such a factor as the effect on cable heating—dielectric losses in insulation, namely, CR.

210

N. Poluyanovich et al.

Fig. 14.1 Power cable with cross-linked polyethylene insulation

The mathematical model [7] is refined, which allows calculating the temperature distribution across layers (Fig. 14.1) and a formula is derived that allows determining θ° in the core, taking into account the power released by the active component of the leakage current [8]. Figure 14.2 shows the electric field and equipotential distribution within the cable in the AC cycle, where the conductor potential is at its maximum value. The core material aluminum is conductive and non-magnetic. A strong electric field on the surface of the conductor and a weak one on the insulating sheath of the

Fig. 14.2 Electric field and equipotential distribution through XLPE cable cross-section

14 Neural Network Technologies in the Tasks of Estimating …

211

cable insulation. The data shows the rotational symmetry of the electric field around the conductor due to the symmetry of the cable.

14.3.2 Calculation of the Time to Breakdown Based on the Thermofluctuation Theory The paper shows that the explanation of the regularities of the electrical aging of polymer dielectrics can be performed on the basis of the thermofluctuation theory. According to this theory, the process of destruction of the dielectric is considered as the rupture of a chemical bond under the action of thermal fluctuations of molecules, taking into account the effects of various forces (electric field tension, mechanical load, etc.), which facilitate the rupture of a chemical bond [9, 10]. A simplified model of polyethylene insulation as a slice of CH2 molecules in which breakdown can occur under the influence of an electric field. Imagine a square (Fig. 14.3) as a coordinate system where the breakdown coordinates are determined randomly. A special case of insulation aging in a high-voltage electric field (EP) is considered. Under the influence of EP, aging occurs due to the development of the CR in isolation. Periodically repeated CRS destroy the insulation by the appearance of microcracks in it, which eventually leads to its breakdown. To determine the degree of degradation of insulation leading to breakdown, the distribution law of the appearance of defects in the dielectric is used. It is necessary to determine the criterion for the occurrence of insulation breakdown that is, the degree of destruction of the material according to the insulation radius for the occurrence of breakdown. According to the obtained dependence, the main part of insulation breakdowns occurs in the range from 300 to 500 broken H

H

H

H

Н

Н

Н

H

Н

Н

C

C H

H

H

H

Н Н

Н H

Н

H

H

Н Н

C

C

C

C H

H

H

H

C

H

H

Н Н

Н

H

C

H

C

Н

C

C

Н

C

C

H

С

C C

Н

C H

H

C

C H

C Н

H

C C

H

C

C

Н

C

Н Н

C

Н

Н

C

C

C

Н H

H

C H

Н

C

Н H

H

C H

H

H

Fig. 14.3 Structural model of the isolation of the XLPE in the form of a slice of CH molecules

212

N. Poluyanovich et al.

molecules. Thus, to break through the insulation, it is enough to break through 4% of the mole. With each CR, a larger number of molecules can break through [11]. During the breakdown, the bonds of the carbon atom with neighboring carbon atoms are destroyed and free hydrogen atoms are formed, which can cause the appearance of bubbles. (−CH2 − CH2 −)n → C + 2H·, 2H· → H2 (bubbles). The resulting unbound hydrogen is knocked out as a positively charged H+ion, which, in turn, promotes ionization and further destruction of the molecular structure of polyethylene as a whole and subsequently, capturing a free electron, becomes neutral, and carbon becomes a conductive microparticle. In this case, neighboring carbon atoms form bonds with each other. A variant of the transformation of ionized radicals (and) is possible, then free hydrogen forms bonds with neighboring carbon atoms in undisturbed molecules and forms terminal groups. When such a bond is destroyed under the influence of an electric field, a free hydrogen and an end group with a neighboring carbon atom are formed. −CH2 − CH2 − · · · − CH2 + ·H → −CH2 − CH2 − · · · − CH3 or + −CH2 − CH2 − · · · − CH3 → −CH2 − CH2 − · · · − C− H2 + H − −CH2 − CH2 − · · · − CH3 + ·H → −CH2 − CH2 − · · · − CH2 + H2 . To verify this assumption, we conducted studies of the influence of temperature and the strength of the applied electric field on the time before the breakdown. Segments of a single-core power cable were used as samples for the study (Fig. 14.1) with insulation made of cross-linked polyethylene, with a diameter of 19 mm and a thickness of the main insulation of 4.5 mm. Measuring the temperature (θ ) in the cross-section of the cable allows us to characterize not only their gradient, but also to detect thermal fluctuations caused by partial discharges (PD). The calculation of the time to breakdown (14.1) is known on the basis of the thermal fluctuation theory according to [10]. τ = τ0 e

D·ϕ(x) 2KT

,

(14.1)

where τ 0 is the time constant, c; D is the energy of breaking the chemical bond, J; ϕ(x) is a function (14.2) of the acting loads (x) causing a decrease in the potential barrier. [ ] √ 1√ 1 + ϕ(x) = 1 − 2x − xln 1 − 2x − 1 , (14.2) x x

14 Neural Network Technologies in the Tasks of Estimating …

x=

/ )2 1 ( −bT Ae βηE + (γ σ )2 , D

213

(14.3)

where A, γ are the parameters of Eq. (14.3) determined experimentally; η is the coefficient taking into account the increase in the electric field strength due to the shape of the electrodes; β is the coefficient taking into account the increase in the electric field strength due to the heterogeneity of the structure of the dielectric material; σ is the mechanical load, (N/m); E is the intensity of the applied electric field, (V/m); b is the coefficient taking into account the change in the elastic modulus of the material with temperature, 1/K. However, for mathematical modeling, it is proposed to choose the model of aging of G. S. Kuchinsky’s insulation (14.4), since it is based on taking into account the energy components of destructive effects, which are the insulation temperature (as a parameter of thermal and thermo-oxidative destruction) and the power of partial discharges, which is a function of the electric field strength. τcπ = AE−n exp

(

) Wa , RT

(14.4)

where A = 5.3 × 1015 ; n = 3 are dimensionless coefficients that need to be specified for each type of cable separately, focusing on 30 years of cable operation; T = 26 + 273, K;—core temperature; W = 84 kJ/mol;—activation energy is taken from [12]; U = 7.76 × 103 is taking into account the connection of the cable according to the “star” scheme, with neutral grounding through a low-resistance resistor, a voltage of 95% of the nominal on the cable core is accepted. Consequently, the effective value of the voltage applied to the insulation is 5.7 kV, and the voltage amplitude is 8.16 kV. Let us take for calculation a voltage that is 95% of the nominal that is, 7.76 kV [12]; R = 8.314 × 10–3 kJ/(mol K). In the developed model of insulation aging, it is proposed to use a universal gas constant R based on the fact that in our model of insulation aging, “inclusions” are gas cavities. The electric field strength in the cable insulation will depend on the voltage of the cable core and the radius according to the formula (14.5): U ( ).

E(r ) = ln

R2 R1

(14.5)

To calculate the electric field strength generated in the cable, a range of distances along the main insulation (MI) is selected. r = r 1 + [0.1·r 1 : 0.001·r 1 : r 2 ]. Taking into account the passport service life of the studied cables with XLPE insulation, the parameters of the aging model were selected in such a way that at the maximum permissible temperature of 90 °C, the service life was about 30 years. The development of the G. S. Kuchinsky insulation aging model (14.4) is to refine the coefficients A and n for XLPE insulated cables so that the aging model being developed can be used to determine the residual life and predict insulation wear.

214

N. Poluyanovich et al.

The substantiation of the regularity of the processes of development of the PD, the mechanism of aging and electrical breakdown was carried out in [13]. There, the dependences of determining the geometric characteristics of “inclusions” are proposed, namely, between the amplitude of the PD in the inclusion and its discharge area. In [13] it was found that the ignition voltage (U v ) makes it possible to determine the thickness of inclusions in accordance with the proposed formula (14.6), and the apparent amplitude of the PD (q) is their discharge area. Ubv = Um

εε0 Si εh i , q = Ubv. , h + (ε − 1)h i h − hi

(14.6)

where S i = π ·A·b, a, b is geometric parameters of the ellipse, U m is voltage in the cable core, U bv is breakdown voltage of the gas switch-on; ε0 = 8.85 × 10−12 F m−1 , εmi = 2, 3, S i is gas inclusion area, hi is inclusion thickness, h is radial distance of the main insulation. PD is a measure of the degree of electrical aging, temperature and other influences that affect the formation and development of inclusions in insulation [13].

14.3.3 Experimental Investigation A investigation of the dependence of the electric field strength and the time to breakdown has been carried out. Based on the thermofluctuation theory, the results of calculating the time to breakdown are given in Fig. 14.4. The analysis of the obtained dependences (Fig. 14.4) shows that with an increase in the electric field strength, the average time to breakdown decreases. With an

Fig. 14.4 Dependence of the time to breakdown on the electric field strength at different temperatures

14 Neural Network Technologies in the Tasks of Estimating …

215

increase in temperature from T 1 = 80 °C to T 2 = 100 °C, the dependence curve lies lower. As the temperature increases, as can be seen from Fig. 14.4, the average time to breakdown decreases sharply exponentially in accordance with the thermofluctuation theory. Thus, we can say that the obtained results confirm the assumptions given earlier [13–15] about the possibility of using the thermal fluctuation theory in determining the resource of power cable lines.

14.4 Investigation of Magnetic Induction in an Inhomogeneous Dielectric The layer-by-layer change of magnetic induction in the material of the cable APvPu g–1 × 240/25–10 is investigated. Due to the peculiarities of the electrical insulation structure, the heterogeneity of the dielectric itself, bulk charges can accumulate in the structure of the insulating material, leading to a distortion of the internal electric field and a decrease in the breakdown voltage of the dielectric. The average breakdown strength of a homogeneous electric field is taken as a measure of the electrical strength of the dielectric E bv , calculated by (14.7): E bv =

Ubv , d

(14.7)

where U bv is breakdown voltage, B; d is dielectric thickness, m. According to GOST 21515-76, breakdown is the phenomenon of formation of a conducting channel in the dielectric under the action of an electric field. Insulation capacity without defect for a section of the size r (14.8): C=

ε0 εr r 2 . r

(14.8)

Equivalent capacity of a site with a defect in the area of weak fields (14.9): CΣ =

ε0 εr r 2 Cis C0 , = Cis + C0 r + r0 (εr − 1)

(14.9)

where r 0 is the size of the defect along the application of the test voltage, εr is relative permittivity of the insulation dielectric, ε0 is dielectric constant. The relative permittivity of the defect, for simplicity, is taken as one, since the defect is filled, as a rule, with air. From the above expression, it is obvious that the capacity of the defect site is lower than the capacity of the defect-free site. In this case, the extreme case is the equality of the relative permittivity of the insulation εr = 1. Calculated field strength in the defect (14.10):

216

N. Poluyanovich et al.

E 0 = Er

εr , ε0

(14.10)

where E r is field strength in the dielectric, ε0 is dielectric constant, εr is relative permittivity of the insulation dielectric. Taking into account the fact that the control is/carried out at voltages close to the breakdown strength of the dielectric, the ratio εr ε0 , as a rule, is in the range from 2 to 6, and the electrical strength of the defect is a multiple of the electrical strength of the insulation, then when the voltage is monitored in the defect area, high-intensity electrical discharges will occur and the defect area will have high conductivity. In this case, the entire test voltage will be applied to the entire insulation section r − r0. The electrical capacity of the defect will lose its physical meaning, and the equivalent capacity of the insulation section with the defect will increase and be equal to (14.11): CΣ =

ε0 εr r 2 . r − r0

(14.11)

The electric field strength is a measure of the electrical strength of a dielectric. Considering the relationship between tension and magnetic induction (14.12): H=

B , μ0 μr

(14.12)

where μ0 is magnetic constant, μ0 = 4π × 10–7 (H/m), μr is relative magnetic permeability of a substance. Of interest are studies of the layered propagation of magnetic induction for a defect-free section of insulation and a section with a defect where the capacity is lower than the capacity of the defect-free section.

14.4.1 Case of Homogeneous Insulation Source data: cable core current I = 285 A; μ0 = 4π × 10–7 ; μm = 0.999994 is relative magnetic permeability of copper; μbi = 0.0421 relative magnetic permeability of basic insulation; μa = 1.00000037 relative magnetic permeability of air; r 1 = 9.5 mm. core radius; r 1 = 13 mm the radius of the basic insulation; r 3 = 14.5 mm is screen radius; rk = 17.5 mm is total cable radius. Consider the layer-by-layer distribution of magnetic induction B(r) for the case when the insulation does not contain inclusions. To consider the distribution of magnetic induction in the internal structure of a power cable, we use the following formula (14.13): B=

I · μbi · μ0 . 2·π ·r

(14.13)

14 Neural Network Technologies in the Tasks of Estimating …

217

Fig. 14.5 The dependence of the layer-by-layer propagation of magnetic induction on the radius is homogeneous isolation

Figure 14.5 shows the dependence of the layer-by-layer propagation of magnetic induction on the radius, determined by the formula (14.13). The profile of the change in (k) showed a sharp increase in magnetic induction in the layer representing the screen of the power cable line.

14.4.2 Case of Inhomogeneous Insulation Source data: cable core current I = 285 A; μ0 = 4π × 10–7 ; μm = 0.999994 is relative magnetic permeability of copper; μbi = 0.0421 relative magnetic permeability of basic insulation; μa = 1.00000037 relative magnetic permeability of air; r 1 = 9.5 mm. core radius; r 1 = 13 mm is the radius of the basic insulation; r 3 = 14.5 mm is screen radius; rk = 17.5 mm is total cable radius; vkl = 0.5 × 10–3 is inclusion radius. As is known, structural inhomogeneities of the physical structure cause a local increase in the field strength in the dielectric and lead to premature breakdown of insulation [16]. We investigate the layered propagation of magnetic induction B(r), depending on the radius, for the case of inhomogeneity in the main insulation of a power cable line with an area of 7 × 10–7 m2 and located at a distance from the center (m = 0.011 m). Figure 14.6 shows the dependence of the layer-by-layer propagation of magnetic induction on the radius, including B(r). We will analyze the dependence of the layer-by-layer propagation of magnetic induction, which allows us to identify distortions of the electric field caused by the structural features of the cable (SCL screen), as well as the presence of inhomogeneities in its electrical insulating layer (Fig. 14.6).

218

N. Poluyanovich et al.

Fig. 14.6 The dependence of the layer-by-layer propagation of magnetic induction on the radius, in the presence of inclusion

This will reveal the dependence of magnetic induction on the geometric characteristics of the “inclusions”. As can be seen, the bulk charges accumulated in the insulating material lead to distortion of the internal electric field of the power cable line. The large gradient of change in magnetic induction (B) at the location of the inhomogeneity (m = 0.011 m) increased by 23.7 times due to a change in the dielectric constant of the material in the power cable line.

14.5 Conclusion The results of numerical simulation of the dependence of the time before the breakdown of a power cable line on the electric field strength at different temperatures showed the influence of temperature and electric field strength on the time before the breakdown. Namely, with an increase in the electric field strength, the average time to breakdown decreases exponentially in accordance with the thermal fluctuation theory. The results obtained confirm the assumptions about the possibility of using the thermal fluctuation theory in determining the resource of a power cable line. Analysis of the layer-by-layer distribution of the electric field in the cable revealed distortions of the electric field and showed its dependence on the features of the cable design and the size of the defect. Bulk charges can accumulate and lead to distortion of the internal electric field. The presence of inhomogeneities in the electrical insulation material reduces its service life. Based on the conducted studies and measurements, the “Methodology for assessing the technical condition of the power cable line [13] was developed based

14 Neural Network Technologies in the Tasks of Estimating …

219

on the results of a diagnostic examination by gentle and non-destructive methods” with certain evaluation criteria.

References 1. Letu, F.: High-quality conductive cores for high-voltage and ultra-high-voltage cables (type “Milliken”). Carousels and Wires 1(308), 18–21 (2008) 2. Eimin, L.S., Gamayunov, O.L.: Induction heating of aluminum balings in the production of o6 casings of caramel. Vestn Samara State Tech. Univer. Series: Tech. Sci. 37(8), 47–50 (2005) 3. Benato, R.: Multiconductor analysis of underground power transmission systems: EHV AC cables. Electric Power Syst. Res. 79(1), 27–38 (2009) 4. Suzuki, H., Kanaoka, M.: Theoretical investigation on skin effect factor of conductor in power cables. IEEJ Trans. Power Energy 126(8), 807–820 (2006) 5. Steinbrich, K.: Reduction of operating frequency for power cables. Euro. Trans. Electric. Power 12(3), 235–238 (2007) 6. Shidlovsky, A.P., Shcherba, A.A., Podoltsev, A.D., Kucheryavaya, H.H., Zolotarev, B.M.: Analyzers of electromagnetic processes in segmented conductive veins of power cables at a voltage of 330 kV. Tech. Electrodyn. 6, 7–13 (2008) 7. Polyakov, D.A.: Monitoring of the residual insulation life of 6(10) kV cable lines made of cross-linked polyethylene. In: Dissertation for the Degree of Candidate of Technical Sciences (2017). (In Russ.) 8. Dubyago, M.N., Poluyanovich, N.K.: Improvement of methods of diagnostics and forecasting of electrical insulation materials of power supply systems, vol. 192. Southern Federal University Press (2019) 9. Maratkyzy, M., Badanova, A.A.: Studies of the effect of temperature on the long-term electrical strength of wires with polypropylene insulation. Energy, electromechanics and energy-efficient technologies through the eyes of youth. In: Materials of the III Russian Youth Scientific SchoolConference, pp. 89–92 (2015). (In Russ.) 10. Dmitrievsky, V.S.: Calculation and design of electrical insulation. In: Energoizdat, vol. 392 (1981) 11. Electrical insulation strength of cables. http://electricalschool.info/main/kabel/2524-elektrich eskaya-prochnost-izolyacii-kabeley.html. Accessed 25 Jan 2022 12. Merkulov, V.I., Arefyev, K.P., Leonov, A.P.: Determination of the parameters of the equation of the curve of life on the basis of thermofluctuation theory. In: Physical and Technical Problems in Science, Industry and Medicine: a Collection of Abstracts of the VII International Scientific and Practical Conference, pp. 114–115 (2015). (In Russ.) 13. Dubyago, M.N., Poluyanovich, N.K., Pshikhopov, V.H.: Method of investigation of thermal fluctuation fluctuation processes in problems of diagnostics and forecasting of insulating materials. Bull. Don State Tech. Univer. 17 No. 3(90), 117–127 (2017) 14. Zaitsev, E.S.: Development of mathematical models and algorithms for monitoring the capacity of 110–500 kV cable lines. In: Dissertation for the Degree of Candidate of Technical Sciences, vol. 178 (2016) 15. Dubyago, M.N., Pshikhopov, V.H., Poluyanovich, N.K.: Evaluation and forecasting of insulation materials of power cable lines. News of the SFU. Tech. Sci. 7(168), 230–237 (2015) 16. Poluyanovich, N.K., Dubyago, M.N., Shurykin, A.A., Burkov, D.V.: Estimation of partial discharge energy in a mathematical model of thermal fluctuation processes of a power cable. Ufa State Aviation Technical University, pp. 127–132. Publishing complex USATU, Ufa (2019)

Chapter 15

Instantaneous Common-Mode Voltage Reduction of Three-Phase Multilevel Voltage Source Inverter Under Quarter-Wave-Symmetric Space Vector PWM with Full Set of Vectors Nikolay Lopatkin Abstract The three-phase power supplying is now widely used for robotics equipment energizing, including traditional three-phase induction alternating current (AC) motors and new kinds of brushless motors. A three-phase inverter can provide a highperformance control of a high-power industrial robotic manipulator. In particular, the precision powerful drives of manipulators with 5–7 degrees of freedom (used, for example, in collaborative robots) require a significant reduction in the torque pulsations. This can be readily achieved by using a three-phase multilevel voltage source inverter (MLVSI) with a pulse width modulation. Unfortunately, a pulse nature of inverters voltages and currents contributes to the problem of a common-mode voltage (CMV), which is the source of many harmful phenomena, noticed in AC motors. The paper proposes a way to reduce the CMV instantaneous values of a three-phase MLVSI to the opposite in sign levels of one third of a base (unit level) value of the supplying direct voltage (and the zero level). The approach is based on the searching the switching states with the pointed CMV levels by one of them for each of the full space vector diagram vectors and calculating the phase-to-ground executed voltages needed to accomplish the mode.

15.1 Introduction A three-phase power supplying is now widely used for robotics equipment energizing. It is necessary for both traditional three-phase induction AC motors and new kinds of brushless motors, like electronically commutated brushless DC electric motors. A three-phase inverter can provide a high-performance control of a highpower industrial robotic manipulator. The precision powerful drives of manipulators N. Lopatkin (B) Shukshin Altai State University for Humanities and Pedagogy, 53 Korolenko Str., 659333 Biysk, Altai Region, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_15

221

222

N. Lopatkin

with degrees of freedom values from 5 to 7, used, for example, in cobots (collaborative robots), require a significant reduction in the torque pulsations, which can be achieved without a bulky filter precisely by using a three-phase voltage source inverter with a pulse width modulation. In addition to the better output power quality, the lowered requirements to the individual voltage ratings of power switches and the comparatively low values of the switches commutation frequency and corresponding losses are the obvious advantages of a multilevel inverter over a two-level inverter. A three-phase multilevel voltage source inverter (MLVSI) as a wide class of power electronics devices plays now a pivotal role in electromechanics power supply systems from the point of view of speed and torque control performances improvements of powerful AC variable-voltage and variable-frequency drives. Despite a variety of conventional MLVSI topologies and control solutions, developing MLVSI innovative circuitry and modulation techniques remains one of trends of both the power electronics itself and the AC drive systems [1–12]. It can be explained by growing performances demands of industrial applications along with a significant backlog of existing MLVSI technologies industrial adoption. Also, the MLVSI cost parameters optimization leads, in particular, to designing innumerable circuits’ configurations with a reduced part count, see, for instance, recent conference papers [13–15]. Anyway, the MLVSI projects are now more than just topical ones but becoming increasingly important for the secondary three-phase AC power supplying. Unfortunately, a pulse nature of modern-voltage-source-inverters-generated voltages and currents makes particularly critical in inverter-based systems a problem of a common-mode voltage (CMV) [16]. The CMV in both single-phase and three-phase AC power systems is the source of many undesirable phenomena, such as commonmode leakage currents through parasitic elements of transformerless systems like photovoltaic inverters, shaft voltage and bearing currents (Fig. 15.1) in AC motors, etc., entailing reduction of the equipment service life, efficiency and reliability, decline in the quality of the generated voltage and a risk of electric shock to service personnel. Certainly, solving the CMV problem requires huge research and engineering resources. Only the diagnosis alone of each of the CMV consequences generates complex engineering and scientific problems, see for example [17]. Fig. 15.1 Bearing fluting as obvious example of bearing currents action [17]

15 Instantaneous Common-Mode Voltage Reduction of Three-Phase …

223

Regarding three-phase MLVSI circuits, the CMV reduction and elimination is the problem, common to all topologies and attracting many researchers. Like to solutions for single-phase inverters, all the CMV-mitigating measures in three-phase inverterbased systems can be roughly divided into circuitry, control and mixed solutions [7]. Among the applied hardware means there are special-designed external passive and active common-mode filters and special inverter topologies [18–20]. But typical common-mode noise filters are bulky, expensive and reducing a system efficiency. The special topologies are mostly oriented to low level inverters, since MLVSI circuits a priori have a reduced amplitude and voltage transition step of the CMV. The control solutions are discussed in the next sections. As will be shown, applying the only space vectors with the zero value of a three-phase CMV considerably increases higher harmonics distortions of the MLVSI output voltages and currents at low values of the amplitude modulation index. Details of the technique, which is keeping use of the full set of the space vectors while fitting switching states combinations of them to decrease the CMV values to the acceptable low level, are explained in this work.

15.2 Zero-Common-Mode-Voltage Space Vectors Applying The control solutions are generally based on advanced pulse width modulation (PWM) techniques [6, 10, 20–27], including modified sinusoidal PWM technologies. In case of MLVSI applying, space vector PWM (SVPWM) techniques provide a rich choice of switching states and generated voltage space vectors (SVs), therefore the CMV can be completely eliminated due to selecting and using particular SVs. Additionally, the rising number of MLVSI levels eliminates the problem of the output power quality declining and makes it possible to reduce AC motors’ electromagnetic torque pulsations. An extraordinary trinary hybrid 81-level inverter with zero common-mode voltage, designed for motor drive, is presented in [3]. So, the most radical control approach to the CMV problem is the CMV elimination through the use of SVs with the zero value of a three-phase CMV, here and after called the zero-common-mode-voltage space vectors (ZCMVSVs). The three-phase CMV is the voltage u nG across a load neutral point n and a whole MLVSI-based three-phase AC power supplying system ground point G. Let’s introduce being executed voltage signals: these are the resulting control signals after applying the being considered control technique, and their waveforms must be replicated in appropriate voltages of a power circuit. In other words, they take values that we have chosen from among the accessible values related to the existing set of the MLVSI space vectors and we should provide these values thanks to the applying control technique. The executed signals are here designated by “exe” in the subscripts. Also asterisks in the superscripts designate the relative (normalized) values, u ∗X = uUXd , u ∗X Y = uUXdY , here Ud is the direct voltage of the unit (base) level.

224

N. Lopatkin

The instantaneous value of the executed three-phase CMV is calculated according with the values of phase-to-ground executed voltages: u ∗nGexe (t) =

∗ u ∗AGexe (t) + u ∗BGexe (t) + u CGexe (t) . 3

(15.1)

The grounding node G position is the different for two three-phase MLVSIs groups. Each member of the first group has a separated per phase supplying and forms a three-phase star-connected AC source with a grounded neutral point (a common star point). Figure 15.2a shows the cascaded H-bridge (CHB) topology MLVSI as the typical and the most frequently used member of this category of MLVSI. Each member of the other MLVSIs circuits group, using the same power source DC voltage levels for all the three phases, has the central point of the sectioned DC voltage source (DC bus mid-point) as the system ground point G. The most known and used member of this MLVSIs category is the so-called diode clamped or neutral point clamped (NPC) topology (see Fig. 15.2b), which is winning the competition with the flying capacitors (FC) topology in this group.

Fig. 15.2 Common-mode voltage in three-phase inverters feeding star-connected induction motors: a in cascaded H-bridge five-level circuit; b in neutral point clamped five-level circuit

15 Instantaneous Common-Mode Voltage Reduction of Three-Phase …

225

The full set of space vectors of three-phase seven-level inverter and the set of its ∗ ZCMVSVs (SVs meeting the condition  ∗ u nGexe  (t) = 0) are shown in the obliqueangled delta voltages’ coordinates u AB , u ∗BC as the vertices of the corresponding modulating triangles in Fig. 15.3a and b, respectively. The ZCMVSVs have been considered with respect to the elimination of the instantaneous three-phase CMV in many publications, see, for instance, [28–31]. However, this improvement has reached at the cost of unavoidable degradation of the quality of the generated output power. This is quite obvious if the MLVSI voltage’s harmonic distortions are compared for the common-mode-voltage-eliminating space vector pulse width modulation (CMVE-SVPWM) [30, 31], applying the ZCMVSVs, and the quarter-wave-symmetric SVPWM (QWS-SVPWM) [32], which is the baseline technique of the CMVE-SVPWM, but applying the entire set of SVs. The PSIM-simulated dependences of the MLVSI phase-to-neutral output voltage’s (1) harmonics factor K hu (THD) and first-order integrated voltage harmonics factor K hu (weighted THD) [33] on the phase voltage amplitude modulation index maY are presented in Fig. 15.4 for the CMVE-SVPWM and QWS-SVPWM.

Fig. 15.3 Space vector diagrams with triangles of the three nearest voltage vectors of three-phase seven-level inverter: a full set of space vectors; b zero-common-mode-voltage space vectors

Fig. 15.4 Harmonic distortions indicators versus phase voltage amplitude modulation index: a voltage’s harmonics factor (THD); b first-order integrated voltage harmonics factor

226

N. Lopatkin

Here m aY = UUd = U ∗ , where U and U ∗ are the value and normalized value of the reference voltage space vector magnitude. The conventional frequency modulation index m f can be expressed via the clock-cycle f c and modulating (MLVSI output voltage) f frequencies or via the clock-cycle Tc and modulating T periods as follows: m f = ffc = TTc . The low specified m f values reflect the concept of the semiconductor switches’ dynamic power losses reducing for the overall efficiency increasing. The MLVSI load current THD value in the first-order generalized load circuit can be estimated via the known value of the first-order integrated voltage harmonics factor of the MLVSI output voltage [32]. As can be seen from Fig. 15.4, the price of the common-mode-voltage eliminating (1) is a large increase in the K hu and K hu values in the maY range up to approximately 2. This range of the amplitude modulation index value corresponds to the MLVSIs up to the levels’ number of 5, which is the alleged frontier of the MLVSIs modern industrial adoption. The reasonable problem solution could be keeping use of the full set of the space vectors while fitting switching states combinations to decrease the CMV values to the acceptable low level.

15.3 Common-Mode Voltage Reduction Under Applying of Full Set of Space Vectors The reducing of the CMV values via MLVSI control methods instead of its full elimination focusing is one of the subjects of many publications with more or less encouraging conclusions [7, 10, 22–27]. However, some of the offered techniques are not sufficiently effective, and almost all other ones are challenging to implement for the increased number of the MLVSIs levels. Since the number of space vectors, which can provide any particular CMV value, is small, the constant CMV value modes are not able to generate output power of high quality in terms of the higher harmonics content. Thus, the only alternative remains to be tried to implement, namely available CMV reduction under applying of full set of SVs. The CMV instantaneous values limitation would mitigate the CMV problem and simplify measures to protect against the undesirable phenomena. Each voltage space vector as the exact position of the modulating triangles’ vertices on the diagram of Fig. 15.3a is uniquely defined by two of the three values of the normalizeddelta voltages  as the oblique-angled coordinates and therefore should be designated u ∗AB , u ∗BC . The SVs positioning for balanced load modes can also be performed via two  of the three values of the normalized phase-to-neutral voltages, for example u ∗Cn , u ∗An , as in [30]. It should be noted that the phase and delta MLVSI output voltages for a three-phase balanced load mode are connected by the well-known matrix relation:

15 Instantaneous Common-Mode Voltage Reduction of Three-Phase …

⎡ ⎤ ⎤  ∗ 2 1 u ∗An 1 ⎣ u ∗ ⎦ = · ⎣ −1 1 ⎦ · u AB . Bn u ∗BC 3 u ∗Cn −1 −2

227

⎡

(15.2)

At last,  the sets of three  values of the normalized phase-to-ground voltage coordinates u ∗AG , u ∗BG , u ∗CG should be treated as the switching states’ definitions. So, depending on its proximity to Fig. 15.3, a diagram center, some particular voltage space vector can have multiple such switching states. It should be noted that any integer shift, simultaneous for all of the u ∗AG , u ∗BG and u ∗CG values, is producing respective switching state with no changes in values of the delta and phase  MLVSI load voltages, i.e., holding the same space vector. The switching state u ∗AG , u ∗BG , u ∗CG with a minimum absolute value of the CMV should be chosen for applying at any given point of time. As a result of a preliminary analysis of Fig. 15.3a voltage space vectors through the prism of (1), the accessible normalized CMV values u ∗nG min (with minimum



absolute value), namely, −1 3, 0 and 1 3, are shown for each SV of the four-level inverter in Fig. 15.5. Thus, the needed for further calculations being executed CMV values can be stored in a lookup table beforehand. Let’s recall the phase and delta voltages’ axes natural positions in the MLVSI voltage space vectors’ diagram, see Fig. 15.6. In contrast to the presented here directions, both the u ∗AB and u ∗BC values are projected to the other axes in Fig. 15.3 (and in Fig. 15.5), namely the oblique-angled 60° axes, for ease of SVs presenting and processing. To better recognize and use periodic patterns in u ∗nG min values, it is reasonable to compare the latters with space vectors’ coordinates of one of the phase-to-neutral voltage. The voltage space vectors’ diagram of Fig. 15.5, rotated clockwise by 30°, is presented in Fig. 15.7 with matched axis of the phase “B” voltage. Applying functions of the integer and fractional parts of values makes it possible to calculate u ∗nG min values and to define necessary values of the phase-to-ground executed voltages, avoiding lookup table use. Fig. 15.5 Accessible normalized common-mode voltage values of four-level inverter

228

N. Lopatkin

Fig. 15.6 Natural axes positions of phase voltages and delta voltages of three-phase four-level inverter

Fig. 15.7 Mapping between accessible normalized common-mode voltage values with minimum absolute value and phase-to-neutral phase “B” voltage coordinates of space vectors

Mapping between the u ∗nG min and u ∗Bn values in Fig. 15.7 has helped to determine the following relationship:  u ∗nG min = u ∗Bn − u ∗Bn +



∗  3u Bn + b + 1 3 · , 3u ∗Bn + b + 1 3

(15.3)

15 Instantaneous Common-Mode Voltage Reduction of Three-Phase …

229

where x means the rounding down x to the closest integer number, taking into account the sign (the “floor” function), so x is the integer part of x; b is a biasing number for the shift of the 3u ∗Bn + b + 1 to the range of positive integer values, so b must meet the conditions: 1. b = 3n, n is a natural number, 2. b ≥ 2(N − 1), here N is the number of the MLVSI levels. In accordance with the approach of the executed voltages’ signals, such the reproduced in power circuit control signals are being generated continuously and producing waveforms, some of which are directly control power switches. The above mentioned QWS-SVPWM [32] shapes its main executed voltages’ signals, namely, the delta voltages’ signals u ∗ABexe and u ∗BCexe , as follows:  u ∗X Y exek (tkc ) = u ∗X Y refsk + f X Y exek (tkc ),

(15.4)

where u ∗X Y exek (tkc ) is the delta voltage signal on the clock-cycle interval k, tkc is the current time from the start of the clock-cycle with number k, tkc = t − (k − 1) · Tc , t is the total current time; u ∗X Y refsk and f X Y exek (tkc ) are, respectively, the sampled for interval k value of the reference delta voltage signal and the value of a conditional pulse function, which is possessing the only values 0 and 1 [32]. Substitution of u ∗Bn from (2) to (3) determines the executed accessible, minimum in absolute value, normalized CMV values at every moment of time in terms of the executed delta voltages’ signals, being the resulting values of the applied MLVSI control algorithm’s main operations, namely it defines the instantaneous function:    ∗ A(t) u (t) − u ∗ABexe (t) + , u ∗nG min exe (t) = − BCexe 3 A(t) A(t) =

(15.5)

u ∗BCexe (t) − u ∗ABexe (t) + b + 1 , 3

where {x} is the fractional part of x, {x} = x − x. And finally, we should specify the values of the executed phase voltages with reference to the ground point G (the inverter neutral point), which directly control the MLVSI phase leg’s power switches and provide the condition of the minimum CMV absolute value: u ∗ABexe (t) − u ∗BCexe (t) 3 3u ∗nG min exe (t) − u ∗ABexe (t) + u ∗BCexe (t) , = 3

u ∗BGexe (t) = u ∗nG min exe (t) −

u ∗AGexe (t) = u ∗BGexe (t) + u ∗ABexe (t),

(15.6) (15.7)

230

N. Lopatkin

u ∗CGexe (t) = u ∗BGexe (t) − u ∗BCexe (t).

(15.8)

15.4 Computer Simulation A series of computer experiments have been performed in PSIM by using model of the three-phase seven-level cascaded H-bridge inverter with the quarter-wavesymmetric space vector PWM control [32], where the latter has been supplemented by the proposed technique for reduction of the instantaneous common-mode voltage. All the PSIM-simulations confirm that the CMV value is limited within the expected range,  ∗ u

  ∗  = u

nGexe (t)

 1  ≤ , |u nG (t)| ≤ Ud , 3 3

nG min exe (t)

(15.9)

while changing the amplitude modulation index over the entire acceptable linear range. In other words, each voltage space vector from the full set of vectors of the MLVSI space vector diagram has in absolute value CMV normalized  its minimum  1 , 0, . Such the CMV values limitation mitigates value, which belongs to the set −1 3 3 the consequences of the CMV problem. The simulated waveforms, corresponding to Ud = 100 V and the control parameters m f = 24 and m aY = 2.6, are shown in Fig. 15.8. Since the above described technique is the supplement to the QWS-SVPWM algorithm with the full set of the being applied space vectors, all the higher harmonics distortions’ dependences in [32] remain true for it. The offered technique is promising for MLVSI with high numbers of the levels, since the more is the MLVSI levels’ number, the less is the three-phase CMV normalized magnitude while the normalization is by the output voltage amplitude.

15.5 Conclusion Three-phase multilevel voltage source inverters are promising for providing highpower industrial robotics equipment, including drives of robotic manipulators, with the high-quality three-phase supplying. The paper considers some control solutions of the three-phase common-mode voltage problem of three-phase multilevel voltage source inverters, in particular the approach of the zero-common-mode-voltage space vectors applying is assessed. Alternatively, a way to reduce the CMV instantaneous value of a three-phase multilevel voltage source inverter (of any arbitrary topology and any arbitrary number of generated equal voltage levels) to the level of one third of a base (unit level) value of the supplying direct voltage with no exceptions from the initially accessible space vectors’ set is proposed. It should be noticed that

15 Instantaneous Common-Mode Voltage Reduction of Three-Phase …

231

Fig. 15.8 Simulated waveforms of executed voltages’ signals and power circuit voltages’ signals of QWS-SVPWM-controlled MLVSI under proposed common-mode voltage minimization technique

the specified CMV level remains the same for the growing values of the output voltage. The approach is based on searching the switching states with the pointed (calculated) CMV levels by one of them for each being applied vector of the full space vector diagram and further calculating the phase-to-ground executed voltages needed to accomplish the mode. The provided equations are proved by illustrations and computer simulations in PSIM. The technique fits perfectly into the context of the recently proposed quarter-wave-symmetric space vector PWM, but can be customized for any control algorithm, calculating values of inverter’s output voltages. The provided common-mode voltage instantaneous values minimization would simplify measures to protect against the undesirable phenomena, in particular, the appropriate filters would have less rated voltages and currents, better weight and size parameters.

232

N. Lopatkin

References 1. Kouro, S., Malinowski, M., Gopakumar, K., Pou, J., Franquelo, L.G., Wu, B., Rodriguez, J., Pérez, M.A., Leon, J.I.: Recent advances and industrial applications of multilevel converters. IEEE Trans. Industr. Electron. 57(8), 2553–2580 (2010). https://doi.org/10.1109/TIE.2010.204 9719 2. Gonzalez, S.A., Verne, S.A., Valla, M.I.: Multilevel Converters for Industrial Applications. CRC Press (2013) 3. Luo, F.L., Ye, H.: Advanced DC/AC Inverters: Applications in Renewable Energy. CRC Press (2013) 4. Ge, B., Peng, F. Z., Li, Y.: Multilevel converter/inverter topologies and applications. In: Power Electronics for Renewable Energy Systems, Transportation and Industrial Applications, pp. 422–462 (2014) 5. Dos Santos, E.C., da Silva, E.R.C.: Advanced Power Electronics Converters: PWM Converters Processing AC Voltages. Wiley-IEEE Press, Wiley Inc, USA, NJ, Hoboken, Piscataway (2015) 6. Leon, J.I., Kouro, S., Rodriguez, J., Wu, B.: The essential role and the continuous evolution of modulation techniques for voltage-source inverters in the past, present, and future power electronics. IEEE Trans. Industr. Electron. 63(5), 2688–2701 (2016). https://doi.org/10.1109/ TIE.2016.2519321 7. Wu, B., Narimani, M.: High-Power Converters and AC Drives, 2nd edn. Wiley-IEEE Press, Wiley, Hoboken (2017) 8. Gupta, K.K., Bhatnagar, P.: Multilevel Inverters: Conventional and Emerging Topologies and Their Control. Academic (2017) 9. Kumar, A.R., Deepa, T., Padmanaban, S., Holm-Nielsen, J.B.: Low-Switching Frequency Modulation Schemes for Multi-Level Inverters. CRC Press (2021) 10. Jiang, D., Shen, Z., Li, Q., Chen, J., Liu, Z.: Advanced Pulse-Width-Modulation: With Freedoms to Optimize Power Electronics Converters. Springer, Singapore (2021). https://doi.org/10.1007/ 978-981-33-4385-6. 11. Kabalci, E. (ed.): Multilevel Inverters: Introduction and Emergent Topologies. Academic (2021) 12. Kabalci, E. (ed.): Multilevel Inverters: Control Methods and Advanced Power Electronic Applications. Academic (2021) 13. Thakre, K., Mohanty, K.B., Nayak, A.K., Mishra, R.N.: Hybrid topology for multilevel inverter with reduced circuit switches using carrier based PWM scheme. In: 7th Power India International Conference (PIICON) Proceedings, pp. 1–5. IEEE, Bikaner, India(2016). https://doi.org/ 10.1109/POWERI.2016.8077163. 14. Kahwa, A., Obara, H, Fujimoto, Y.: Design of 5-level reduced switches count H-bridge multilevel inverter. In: 15th International Workshop on Advanced Motion Control (AMC) Proceedings, pp. 41–46. IEEE, Tokyo, Japan (2018). https://doi.org/10.1109/AMC.2019.837 1060. 15. Vineeth, K, Nirmal Mukundan, C.M., Jayaprakash, P.: A new topology of asymmetrical multilevel inverter with reduced switch count for electric drive applications. In: International Conference on Power Electronics, Drives and Energy Systems (PEDES) Proceedings, pp. 1–6. IEEE, Jaipur, India (2020). https://doi.org/.1109/PEDES49360.2020.9379568. 16. Mutze, A.: Bearing Currents in Inverter-Fed AC-Motors (Berichte aus der Elektrotechnik). Shaker Verlag GmbH, Germany (2004) 17. Önel, I., Benbouzid, M.: Induction motor bearing failure detection and diagnosis: Park and Concordia transform approaches comparative study. IEEE/ASME Trans. Mechatron. 13(2), 257–262 (2008). https://doi.org/10.1109/TMECH.2008.918535 18. Mei, C., Balda, J.C., Waite, W.P.: Cancellation of common-mode voltages for induction motor drives using active method. IEEE Trans. Energy Convers. 21(2), 380–386 (2006). https://doi. org/10.1109/TEC.2005.859983 19. Han, D., Lee, W., Li, S., Sarlioglu, B.: Comparative performance evaluation of common mode voltage reduction three-phase inverter topologies. In: Applied Power Electronics Conference

15 Instantaneous Common-Mode Voltage Reduction of Three-Phase …

20.

21. 22.

23.

24.

25.

26.

27.

28.

29.

30.

31.

32.

233

and Exposition (APEC) Proceedings, pp. 2625–2629. IEEE, San Antonio, TX, USA (2018). https://doi.org/10.1109/APEC.2018.8341387 Szymanski, J., Zurek-Mortka, M., Sadhu, P.K., Goswami, A.: Mitigation methods of ground leakage current caused by common-mode in voltage frequency drives. In: Sikander, A. et al. (eds.). Proceedings of 2nd International Conference on Energy Systems, Drives and Automations (ESDA 2019). Lecture Notes in Electrical Engineering, vol 664, pp. 1–11. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-5089-8_1 Holmes, D.G., Lipo, T.A.: Pulse Width Modulation for Power Converters. IEEE Press, Hoboken (2003) Loh, P.C., Holmes, D.G., Fukuta, Y., Lipo, T.A.: Reduced common mode carrier-based modulation strategies for cascaded multilevel inverters. In: Conference Record of the 2002 IEEE Industry Applications Conference, 37th IAS Annual Meeting (Cat. No.02CH37344), vol 3, pp. 2002–2009, Pittsburgh, PA USA (2002). https://doi.org/10.1109/IAS.2002.1043807 Bang, N.L.H., Nho, N.V., Tam, N.K.T., Dung, N.M.: A phase shifted PWM technique for common-mode voltage reduction in five level H-bridge cascaded inverter. In: International Conference and Utility Exhibition on Green Energy for Sustainable Development (ICUE) Proceedings, pp. 1–7. IEEE, Pattaya, Thailand (2014) Sibi Raj, P.M., Rashmi, M.R.: Reduction of common mode voltage in three phase inverter. In: International Conference and on Technological Advancements in Power and Energy (TAP Energy) Proceedings, pp. 244–248. IEEE, Kollam, India (2015). https://doi.org/10.1109/TAP ENERGY.2015.7229625 Bharatiraja, C., Munda, J.L., Bayindir, R., and Tariq, M.: A common-mode leakage current mitigation for PV-grid connected three-phase three-level transformerless T-type-NPC-MLI. In: 5th International Conference and on Renewable Energy Research and Applications (ICRERA) Proceedings, pp. 578–583. IEEE, Birmingham, UK (2016). https://doi.org/10.1109/ICRERA. 2016.7884401. Chen, H., Zhao, H.: Review on pulse-width modulation strategies for common-mode voltage reduction in three-phase voltage-source inverters. IET Power Electron. 9(14), 2611–2620 (1– 10) (2016). https://doi.org/10.1049/iet-pel.2015.1019 Balogun, A., Agoro, S., Okafor, F., Adetona, S., and Ojo, O.: Common-mode voltage reduction and elimination in a space vector modulated three-phase five-level multistring inverter. In: IEEE PES/IAS Power Africa Conference Proceedings, pp. 672–676. IEEE, Abuja, Nigeria (2019). https://doi.org/10.1109/PowerAfrica.2019.8928909 Rodriguez, J., Pontt, J., Correa, P., Cortes, P., Silva, C.: A new modulation method to reduce common-mode voltages in multilevel inverters. IEEE Trans. Industr. Electron. 51(4), 834–839 (2004). https://doi.org/10.1109/TIE.2004.831735 Nguyen, T.-K.T., Nguyen, N.-V., Prasad, N.(R.)R.: Eliminated common-mode voltage pulsewidth modulation to reduce output current ripple for multilevel inverters. IEEE Trans. Power Electron. 31(8), 5952–5966 (1–15) (2016). https://doi.org/10.1109/TPEL.2015.2489560 Lopatkin, N.: Common-mode voltage elimination of three-phase multilevel voltage source inverter by means of quarter-wave-symmetric space vector PWM approach. In: Ronzhin, A., Shishlakov, V. (eds.). Electromechanics and Robotics. Smart Innovation, Systems and Technologies, vol 232, pp. 299–310. Springer, Singapore (2022). https://doi.org/10.1007/978-98116-2814-6_26 Lopatkin, N., Fenskiy, S.: Integrated voltage harmonics factors estimation of multilevel voltage source inverter with common-mode-voltage-eliminating space vector PWM. In: XVIII International Scientific Technical Conference Alternating Current Electric Drives (ACED) Proceedings, pp. 1–6. IEEE, Ekaterinburg, Russia (2021). https://doi.org/10.1109/ACED50605.2021. 9462278 Lopatkin, N.: Quarter-wave symmetric space vector PWM with low values of frequency modulation index in control of three-phase multilevel voltage source inverter. In: Ronzhin, A., Shishlakov, V. (eds.). Proceedings of 15th International Conference on Electromechanics and Robotics “Zavalishin’s Readings”, Ufa, Russia, 15–18 Apr 2020. Smart Innovation, Systems

234

N. Lopatkin

and Technologies, vol 187, pp. 445–457. Springer, Singapore (2021). https://doi.org/10.1007/ 978-981-15-5580-0_36 33. Zinoviev, G.S.: Power Electronics. Textbook for undergraduate students. 5th edn. Jurajt, Moscow, Russia (2015)

Chapter 16

System of Decentralized Control of a Group of Mobile Robotic Means Interacting with Charging Stations Vladimir Kostyukov

and Viacheslav Pshikhopov

Abstract When solving the problem of organizing energy consumption within a group of mobile robotic objects, stationary charging stations can be used. When solving the problem of optimizing such energy consumption, the problem of increasing the efficiency of interaction between the elements of the group with such stations arises. This problem can be solved by using a decentralized method of managing a system that includes a group of these moving objects and recharging stations. This article develops the concept of interaction between mobile and stationary objects, which implies the possibility of choosing by each interaction agent the corresponding companion. Such a choice is made taking into account the current state of the system and an assessment of the history of interaction results. The developed concept is detailed for a system that includes unmanned aerial vehicles and their recharging stations. In this case, the energy efficiency of the charging process and the time taken by the unmanned aerial vehicles to reach the target point are key indicators when choosing pairs of interacting elements. An optimization procedure has been developed that finds the number of the charging station that is most suitable for a given mobile object for interaction.

16.1 Introduction At the present time, the transition from the use of single mobile robotic means (MRM) to the use of their groups has intensified, in connection with which a number of conceptual, theoretical, and technical problems arise. First of all, these are the problems associated with the management of elements within the group, the organization of reliable information exchange between these elements [1], ensuring the effective functioning of the group as a whole in conditions of limited energy resources at its disposal [2, 3]. Accompanying here are the tasks of coordinating the interactions of mobile robotic means with stations for their maintenance, which can perform V. Kostyukov (B) · V. Pshikhopov Southern Federal University, 105/42 Bolshaya Sadovaya Str, Rostov-on-Don 344006, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_16

235

236

V. Kostyukov and V. Pshikhopov

the functions of: (a) points of destination/dispatch of goods; (b) replenishment of consumed MRM energy; (c) relaying signals from the MRM board; (d) monitoring the state of the MRM, including remotely; (e) repair of MRM; (f) other. When solving the problem of optimizing energy consumption within the MRM group, the problem arises of increasing the efficiency of interaction of the group’s elements with specialized, including stationary, stations for recharging them. This problem can only be solved by considering a combined system, which includes mobile objects that require constant replenishment of the energy they spend, and stationary objects—charging stations, potentially capable of providing this replenishment, as an integrated control object. In [2], an interactive model of interaction between the charging station and the UAV of the group is considered, which allows balancing the supply and demand for electricity, as well as the safe and reliable purchase of UAV energy from the station. In [3], the problem of maximizing the operation time of each UAV group is solved based on the solution of the corresponding traveling salesman problem. Models of interaction between group UAVs and charging stations considered in articles [2, 3] do not take into account local factors affecting the efficiency of UAVs performing their tasks and caused by preparation for interactions with individual stations, and these interactions themselves do not take into account the evolution of these factors over time. In addition, these models are based on an unlimited supply of energy at each station, which can only take place when such stations are connected to a centralized power grid, thereby excluding recharging stations based on renewable energy sources from consideration. There are known works in which optimization problems of energy consumption of a group of UAVs are solved, performing a joint task without taking into account stationary objects, with the exception of an operator for receiving and sending target information. So, in the work [4], the issue of increasing the range of data transmission from UAV surveillance through the use of peripheral UAVs, acting as repeaters, is considered. The paper discusses the issue of optimizing the energy consumption of all UAVs of such a group in order to increase the range and quality of transmitted information. Despite the presence of a large practical backlog in the field of organizing and servicing the movement of a group of UAVs when they perform various missions, as well as the corresponding patents (see, for example, patents [5–7]), there is currently no general way of considering the optimal interaction of mobile and stationary objects some distributed technical, including energy, system. Mainly, centralized methods of coordination of UAV actions in a group have been developed [7]. In the patent [7], a generalized centralized method for controlling the functioning of a group of mobile autonomous objects of different nature is considered for the implementation of tasks such as transportation of goods and recharging and repair at service stations, taking into account feedback from sensors measuring the current values of the spatial field quantities characterizing the environment of functioning of a group of mobile robots. The basis of the proposed [7] method for planning the trajectories of movement of each PRS group and managing the interaction of PRS with docking stations is the

16 System of Decentralized Control of a Group of Mobile Robotic Means …

237

centralized decision-making by the operator based on: goals and objectives of the PRS group functioning; the current data coming from the measuring equipment of the stations and, possibly, each PRS group; other data coming from other sources and related to the current or future processes of the operation of the PRS group and docking stations. The disadvantages of the method claimed in [7] are as follows: 1. Lack of the possibility of decentralized control in the system, which significantly limits the scope of the method of this patent. The degree of this limitation grows with an increase in the number of system elements, the number of potentially possible modes of their functioning, and the area of its action zone. In addition, the reliability of the operation of the entire system and the likelihood of successful completion of the mission are reduced due to the significant risk of failure of the control center in the absence of redundancy of its functions. 2. Inability to use the accumulated information about the previous states of the system to optimize the process of subsequent interactions of moving and stationary objects of the system according to a given criterion of its functioning, i.e., lack of self-learning of the system for the best solution of the tasks assigned to it on the basis of its accumulated experience in the past. Completely centralized control of such a system is justified in those cases or those modes when operator control is required, coordinating the actions of all elements of the system at the upper level of its control. However, with an increase in the number of elements in a group and an increase in the complexity of accounting for the entire complex of processes that determine the functioning of this group, a combination of centralized and decentralized management methods becomes a higher priority solution. The complex of problems of decentralized management of such a group includes the task of organizing the optimal interaction of its elements in order to achieve a certain goal of its functioning by the group. In the case of organizing energy exchange between mobile objects of the system and its stationary elements—charging stations, solving this problem plays a key role in developing the concept of optimal energy consumption of this group. In this regard, it becomes necessary to develop, firstly, the concept of interaction between stationary and mobile objects of a generalized technical system based on decentralized adaptive control principles with a memory effect. Secondly, the specification of such a concept is required in relation to energy systems, where the interaction between the relevant agents is reduced to replenishing the energy of mobile objects. Third, in connection with the increased requirements imposed on the degree of autonomy of robotic systems, the specified energy system with charging stations based on renewable energy sources requires detailed consideration. The interaction of elements of homogeneous robotic systems, for example, mobile objects of a group, may not require the introduction of various modes of functioning of these elements. So, for example, if the interaction between such elements implies only information exchange, then the control system of each of the objects will deal

238

V. Kostyukov and V. Pshikhopov

with one mode of operation associated with the tasks of planning a route to the target point and avoiding collisions with obstacles of various kinds. In the case of the presence of energy exchange interactions between the elements of a robotic system, as a rule, it is necessary to single out in the general case at least two states for each object of the system, the first of which is reduced to the interaction of the object with another, and the second—in its functioning without interaction. A striking example of this is heterogeneous energy systems in which subsets of energy sources and receivers can be distinguished. The most important here are interactions within each or some of the source-receiver pairs. The set of pairs admissible for interaction can be defined in different ways, while the topology of connections between dissimilar elements can be, first, dynamic, i.e., allow rebuilding of relations between interacting elements in the graph of their connections over time. Secondly, it can allow the simultaneous interaction of one source with several receivers, or vice versa, one receiver with several sources. In the case of a dynamic topology, it is necessary to determine the law of its rebuilding, which can be centralized or decentralized. The second approach is most promising for systems with a large number of elements, the centralized coordination of actions of which is possible under serious space–time constraints. The possibility of simultaneous interaction of more than two dissimilar objects of the system is determined by its specifics. For example, an energy system for supplying electricity to private consumers, enterprises, utilities, infrastructure, etc., may have only one source of energy, but it may also allow energy consumption from two substations. In the case of a system for recharging UAVs, two options are possible: In the first, each recharging station can serve only one UAV at a time; secondly, the station can serve several UAVs simultaneously. There are known approaches where various decomposition methods are used to optimize the analysis of exchange processes in energy systems, which reduce to some partitions of the sets of the original system into subsets with the establishment of links between the latter [8–10]. At the same time, some of the initial connections between elements are neglected as weak with one threshold or another. The methods of decomposition are widely considered, allowing to distinguish fast and slow processes in dynamic systems [11–15]. The most applicable methods of decomposition are based on the selection of strong (slow) and discarding weak (fast) factors that determine the dynamics of the system [11–13]. A simplified model obtained by discarding fast components describes slow processes, called motion, in the system [13]. The possibility of applying this approach to decomposition is due to the ability to distinguish movements in a system of different flow rates. Different frequencies of the selected processes can be generated by the internal features of the system or introduced artificially by including links with high gains, discontinuous characteristics, etc., into the system [13]. The most general system of decentralized management of the interaction of heterogeneous elements is one in which, firstly, for each potential pair of interacting elements, the possibility/expediency of interaction is determined according to some target criterion; secondly, this determination is made by the control systems of the elements themselves included in the pair; third, this choice takes into account all the

16 System of Decentralized Control of a Group of Mobile Robotic Means …

239

background history of the previous choices of companions for interaction that were made by each of the elements of the pair. Below we consider a control system for the interaction of elements that meets these properties.

16.2 Synthesis of a Method for Decentralized Control of the Process of Functioning of a Generalized Doubly Connected Technical System 16.2.1 Problem Statement Let us consider the process of functioning of a generalized doubly connected technical system—a distributed generalized network (DGN), which includes a static distributed subnetwork (SDS) and a dynamic distributed subnetwork (DDS). The SDS includes stationary objects (hereinafter called docking stations or simply stations) that perform certain functions of tracking the movement and/or servicing mobile robotic means included in the DDS. The key feature of the DGN functioning process is the interaction between its elements to accomplish assigned tasks or achieve a specific goal. For example, the purpose of functioning can be: (a) organization of cargo traffic between a number of points of receipt/delivery of goods; (b) organization of terrain monitoring; (c) mapping; (d) carrying out a rescue operation; (e) conducting a combat operation in a certain territory; (f) carrying out a clearance/demining operation, etc.; (g) other. The interaction between the elements of the DGN can include: information exchange; loading and unloading of components transported by MRM at the station; energy exchange between the SDS element and the MRM, as a rule, reduced to recharging the MRM at the station; other. Stationary objects can be stations performing one or several of the following functions: (a) loading and unloading of goods/cargo transported by MRM; (b) replenishment of the energy losses of individual MRM—recharging due to the centralized supply of electricity or the generation of energy of one nature or another using renewable sources; (c) repair of MRM; (d) other. These stations can be additionally equipped with instrumentation, in the general case, recording the current characteristics of the state of the MRM, the station itself, the environment in the vicinity of the station (for example, using a meteorological station). SDS objects, in addition to docking stations, may include various transmitting and receiving equipment with an appropriate control unit, which generates, receives, transmits, or retransmits signals of any nature (radio, optical, acoustic, etc., signals) for the exchange of information between all or part of the elements of a distributed generalized network. Mobile objects can be: (a) UAVs; (b) autonomous robotic boats; (c) autonomous robotic underwater vehicles; (d) autonomous robotic aeronautical platforms; (e) etc.

240

V. Kostyukov and V. Pshikhopov

A variant of the implementation of the DGN is a distributed generalized energy network (DGEN); in addition, DGN can be a DGEN and performs additional functions of servicing the MRM. An important particular case of DGEN is a distributed network, which includes MRM recharging stations and an MRM group; the task of the functioning of such a network is to fulfill a given mission by a group of MRM with the possibility of recharging with energy at the stations of the network. It is necessary to develop a method for the decentralized interaction of mobile and stationary objects of a certain technical system, which provides: (1) the possibility for each agent to select a potential interaction of the corresponding companion; (2) taking into account the prehistory of the process, when for making a decision— choosing an object for interaction, both current information about these objects at a given point in time and the estimated results of previously made elections that have shown their appropriate effectiveness in relation to a certain target criterion are used. The practical significance of this method lies in increasing the efficiency of the autonomous functioning of a group of mobile robotic means, in particular, reducing the likelihood of failure of its elements, reducing the time it takes to complete the mission by each element of the group and the group as a whole, reducing the participation of the operator or centralized control link in coordinating the actions of the group, an increase in the efficiency of solving typical problems with an increase in the self-learning time of the system, including due to adaptive adjustment of the configuration of the static elements of the system or/modes of operation of the MRM, the possibility of the operation of the MRM group in conditions of limited energy resources of charging stations. Integration of the sought method with the principles and methods of planning and controlling the movement of each mobile robotic means, as well as methods of controlling the functioning of each station, should ensure the autonomous execution of the mission by the MRM group with the achievement of a given target quality of this execution under given constraints.

16.2.2 Control System for a Distributed Generalized Network, Including Mobile Robotic Vehicles and Docking Stations Consider a control system for a distributed generalized network (Fig. 16.1), including mobile robotic means and docking stations, and based on the method of decentralized interaction of mobile and stationary objects of this DGN, to which the requirements of the previous paragraph are imposed. Before the start of the functioning of the distributed generalized network, the goal is formed, and the tasks of the corresponding mission are developed in block 1; then, a rule is established according to which the operator or/the automated centralized command link can intervene in the control process of the DGN at each or some of

16 System of Decentralized Control of a Group of Mobile Robotic Means …

1 The purpose and objectives of the DGN functioning

Operator connection algorithm to the DGN management process

2

Development and planning of interactions between elements of the static and dynamic subsystems of the DGN Planning the movement of the DDS elements and the functioning of the SDS elements Regulatory control of individual elements of the DGN Adaptive priority changes in the interactions of SDS elements with DDS elements Adaptive reconfiguration of the SDS elements and changing the modes of movement of the DDS elements

241

3

4

5 6

7

Fig. 16.1 Functional diagram of a distributed generalized network control system, including mobile robotic means and docking stations, in which the claimed method is implemented

the stages of this control. In one extreme case, only, the operator has the right to make decisions at all stages—a centralized control mode is obtained, in the other— the influence of the operator/centralized link is completely excluded, at least until the end of the mission. In general, the operator/centralized link can intervene in the process with one or another frequency. Then, in block 3, the interaction of pairs of elements of the DDS and the SDS is planned, based on the algorithm below; in block 4, the movements of the MRM and the operation of stations are planned, providing, without taking into account random and a priori uncertain disturbances within the system, the implementation of the map of planned interactions outlined in block 3. Block 5 provides low-level regulatory control for each of the DGN elements, which ensures the minimum possible deviation of the real trajectories of movement of elements in the phase spaces of their state variables from the corresponding program trajectories laid down in block 4. Implementation of self-learning, the acquisition of “experience” by the DGN elements in relation to the criteria the best performance of the tasks assigned to them is provided by block 6, where the weight coefficients are adjusted—the parameters of the algorithm for the decentralized selection of interacting heterogeneous elements of the DGN. In the case when it is possible or expedient to change the configuration, the arrangement of the SDS elements or the setting values of each of the stations, as well as to adjust the characteristic modes of the MRM movement to optimize the given quality criteria of the DGN, block 7 is used; the results of such changes made

242

V. Kostyukov and V. Pshikhopov

in block 7 according to some adaptive algorithm correct the sequential operation of blocks 3–6; however, unit 7 can be turned on no more often than after a time interval necessary and sufficient for the accumulation of relevant information about the operation of the DGN in the previous configuration and with the previous values of the setting values of its elements. Such a control system is applicable for an arbitrary distributed generalized system, regardless of the degree of decentralization of its control, the requirements imposed on the elements of the DGN, as well as the degree of uncertainty of the operating environment.

16.2.3 Method of Decentralized Interaction of Mobile and Stationary Objects of the DGN Let in a certain territory there is a set of fixed points of destination, receipt and delivery of goods and/or charging stations, etc., between which transport communication is carried out using mobile robotic objects. Interaction with a part of the stations can be planned in advance for the PRS, and the possibility of interaction with the rest of the stations is generally determined by the claimed method. In this case, the objects of the dynamic/static distributed subsystem will be referred to the class Cl, and the objects of the opposite static/dynamic subsystem—to the class Cl. In the process of movement, the MRM can use the services of one of the stations located in the designated area. Likewise, stations can also select individual MRMs to interact with based on the information available to them based on experience gained. In the general case, in order to make a decision on the interaction of a given pair, it is necessary and sufficient to fulfill a strict condition for each of the elements of this pair to choose the opposite element of this pair, in the way that is laid down in the proposed method. It is also possible that the decision on interaction can be made only by objects of one of the classes—SDS or DDS. Let us introduce the concept of the relative efficiency of a class object for a class object in the current situation with numbers, when the control system needs to make a decision on interaction with some object of the opposite class in relation to the implementation of a certain task or to achieve the goal of functioning of individual PRS or a distributed dynamic subnetwork as a whole. We will call such tasks basic; they can be: recharging of individual missile defense systems while moving along arbitrary or regular trajectories; organization of the exchange of goods/cargo between the missile defense system and points of departure/destination; other tasks. Each basic task induces the corresponding relative efficiency of a given object of one class in relation to some other object of the opposite class for their possible interactions at a given moment in time. Let us introduce the concept of the relative efficiency of a class Cl object ObjCl,q for a class Cl object ObjCl, p in the current situation with number s, when the control

16 System of Decentralized Control of a Group of Mobile Robotic Means …

243

system needs to make a decision on interaction with some object of the opposite class in relation to the implementation of a certain task or achievement of the goal of functioning of individual MRM or a distributed dynamic subnetwork as a whole. We will call such tasks basic; they can be: (1) recharging of individual MRM in the process of movement along arbitrary or regular trajectories; (2) organization of the exchange of goods/cargo between the MRM and the points of departure of destination; (3) other tasks. Each basic task induces the corresponding relative efficiency of a given object of one class in relation to some other object of the opposite class for their possible( interactions at a given moment in time. ) Let ϕClCl p, q, s p be a functional that quantitatively expresses the relative efficiency of the potentially possible interaction RClCl ( p, q) of the pth object of the class Cl with the qth object of the class Cl at a step s p in relation(to a given ) basic problem solved by the pth object of the class Cl; let, further, ϕClCl q, p, sq be a functional that quantitatively expresses the relative efficiency of the same interaction, but with respect to the basic problem solved by the qth object of the class Cl at a step sq , and the steps s p and sq refer to the same time. These functionals must take into account: firstly, the current situation, imposing spatiotemporal, informational, energy, etc., restrictions on the ability of objects to interact ObjCl, p and ObjCl,q ; secondly, the experience accumulated to date in the interaction of ObjCl, p objects with class Cl objects and ObjCl, p objects with class Cl objects. A variant is possible when, for example, each object of a class Cl becomes available a similar previous experience of all other objects of this class. ( ) Let’s consider the factors influencing the value of the functional ϕClCl p, q, s p . These factors will be called partial. They can be: (1) the degree of power consumption of the station for a given MRM in a given situation, expressed as the difference between the declared energy by the station, potentially available for this MRM, and the energy demand of the MRM itself, taking into account its movement to this station; (2) the additional time spent on reaching the station, subsequent charging and the return process to the target trajectory, expressed as the actual recharge time plus the time to reach the station itself; (3) the accuracy of estimating the travel time to the MRM, which is determined, among other things, by the features of the location of the infrastructure and the influence of natural factors in the small vicinity of the station, can be expressed as the difference between the estimated and real time according to clause 2; (4) the possibility of technical maintenance by this station of this MRM; (5) the possibility of accepting the cargo carried by this MRM at this station; (6) other factors and related quantities. Let us assign to each kth partial factor (k = 1, K ) a partial functional ϕ(k) ( p, q), ClCl depending on (a) the vector X pq (s) of all considered quantities affecting the efficiency of the interaction RClCl ( p, q) and can be measured by the pth object of the class Cl; (b) the vector X qp (s) of all considered quantities affecting the efficiency of the interaction RClCl ( p, q) and can be measured by the qth object of the class Cl. To evaluate such a functional, produced by the pth object of the class Cl a priori, before interacting with the qth object of the class Cl, one can write:

244

V. Kostyukov and V. Pshikhopov

[ ( ) ( ) ∗ ( )] ∗(k) ∗ p, q; X p, q, s = ϕ s p , X qp s p . ϕ∗(k) p pq ClCl ClCl

(16.1)

The ( basis )of the proposed method is the presentation of the functional ϕClCl p, q, s p in the following general form: ( ) ϕClCl p, q, s p = F[A, δ].

(16.2)

[ { } ( ) ∗ ( )] ∗ X s , s , k = A ≡ ϕ∗(k) X 1, K , p p pq pq ClCl

(16.3)

Here,

( ) p, q, s p , where K × 1 is a vector of estimates of all partial functionals ϕ∗(k) ClCl determined according to (16.1) on the basis of measurements made ObjCl, p and ObjCl,q ; { )} (k) ( p, q, s p δ ≡ δClCl { [ ]} = f ϕ(k) p, qopt ( p, u); X pqopt ( p,u) (u), X qopt ( p,u) p (u) , ClCl 1, s p , k = 1, K ,

(16.4)

where 1 × K is a vector of weight coefficients depending on the s p × K –array of the values of the functionals ϕ(k) ( p, q) at all previous stages of interactions ClCl ObjCl, p with objects of the class Cl, determined on the basis of the measured vectors X pqopt ( p,u) (u), X qopt ( p,u) p (u). This vector integrates information about the effectiveness of the influence of each kth factor on the result of these interactions of the object ObjCl, p in the past. Elements ∗ ∗ of opposite classes must exchange the values of the vectors X pq (s), X qp (s) and X pqopt ( p,u) (u), X qopt ( p,u) p (u), u = 1, s through the appropriate information channel for the correct selection of a suitable object ( for interaction. ) Expressions for the functional ϕClCl q, p, sq are obtained from the corresponding equalities (16.2)–(16.4) by substitutions p ↔ q, Cl ↔ Cl, including in subscripts. The functions of many variables F and f are determined by the characteristics of the DGN and the tasks it performs. The distinctive part of the proposed method corresponds to the algorithm for decentralized regulation of interactions between each pair of mobile and stationary network objects, the generalized block diagram of which is shown in Fig. 16.2; this algorithm is implemented in on-board computers of all or some elements of the distributed generalized network. Let us put for definiteness Cl = A, Cl = B. Figure 16.2 introduced the following designations: 1—incrementing the number of the element of the pair at the first position; 2—incrementing the element number of the pair at the second position; 3—checking the condition “are there currently

16 System of Decentralized Control of a Group of Mobile Robotic Means …

245

Fig. 16.2 Algorithm for decentralized regulation of interactions between each pair of mobile and stationary network objects

246

V. Kostyukov and V. Pshikhopov

pairs, the interaction between which needs to be implemented?”; 4—implementation of previously planned interactions; 5—correction of the weights of all elements of the system, determined by the interacting elements; 6—checking the conditions for enforcing the interaction of this pair; 7—forced assignment of a pair for interaction; 8—implementation of information exchange between the elements of the pair; 9—checking the condition “the current number of the element of the pair at the second position is less than the total number of elements of the second class (B)”; 10—calculation of the value of the target functional determined by the element in the first position; 11—calculation of the value of the target functional determined by the element in the second position; 12—checking the condition “the number of the element in the second position is equal to the total number of elements of the second class (B)”; 13—optimization of the functional for the element in the first position with obtaining the number of the corresponding optimal element in the second position (second class); 14—checking the condition “the number of the element in the first position is equal to the total number of elements of the first class (A)”; 15—optimization of functionals for each element in the second position with obtaining the numbers of the corresponding optimal elements in the first positions (class A); 16—determination of all pairs of elements that satisfy the cross criterion for optimal selection; 17—adding pairs determined by the forced selection condition; 18—scheduling interactions of selected pairs. The logic of the algorithm corresponding to the block diagram in Fig. 16.2, next. After incrementing the number of both elements of the pair (blocks 1 and 2) in blocks 3 and 4, the existence of pairs is checked, the interaction between the elements of which is to be implemented at a given time step of the processor time of the on-board computers (the times of the on-board computers of the various elements of the DGN run synchronously), and this interaction is implemented; on the basis of the results of these interactions in block 5, the values of all partial functionals (16.1) are measured, δ for objects p and q are calculated using and then, the updated of the weights ) values ) (k) ( (k) ( p, q, s p and δBA q, p, sq ; the implementation of the forced assignment (16.4): δAB of pairs to be interacted by the operator is performed in blocks 6 and 7 (if such an assignment takes place). If after that the last element of the class A is not reached, i.e., the condition p = N A of block 14 is not met; then, the return to the point of incrementing the number p occurs; otherwise, the actions described below are performed. If the condition of block 6 is not met, then in block 8, the condition of the possibility of carrying out information exchange by the pair (p, q) is checked; if such an exchange is not possible, then a transition is made to the next element through blocks 10 and 14 of checking the conditions q < N B and p = N A , or—a transition to the stage of data processing after cycles (block 15 and subsequent). If the ( ) specified exchange is possible, then it is performed, and functionals ϕAB p, q, s p in blocks 9 and 11 are calculated based on its results. If further the condition q = N B( of block ) 12 is satisfied, then in block 13, the optimization of the functional ϕAB p, q, s p with respect q is performed to obtain the optimal element qopt for a given p; then, after checking the condition of block 14, a transition occurs to the next number p, or to block 15 for optimizing the functional

16 System of Decentralized Control of a Group of Mobile Robotic Means …

247

( ) ϕBA q, p, sq with respect p to obtaining an optimal element popt for a given q, excluding the forcibly selected pairs. Based on the results of the operation of blocks ' that satisfy the condition: 13 and ( 15,) block 16 selects ( all) pairs ( pm , qm ), m = 1, M ' popt q, sq = p ∧ qopt p, s p = q (in the number M of pieces, if M ' = 0, such pairs are absent). Finally, in block 17, forcedly selected pairs are added with the formation of a complete set of pairs ( pm , qm ), m = 1, M, M ≥ M ' , the interaction of the elements of which is planned in block 18. At this time step, it may appear M = 0. Algorithms that imply the ability to choose only for objects of one class A or B can be obtained from the considered algorithm. If the elements of class B are forbidden to choose the elements of class A that are most suitable for interaction, associated with the calculation then from the block diagram in Fig. 16.2, the blocks ( ) and optimization of functionals of the type ϕBA q, p, sq should be excluded, as well as the corresponding weight coefficients and partial functionals are not calculated; similarly, if such a prohibition is imputed to the elements of class A, then, on the with the calcucontrary, from the block diagram in Fig. 16.2, the blocks ( associated ) lation and optimization of functionals of the type ϕAB p, q, s p should be excluded; the corresponding weight coefficients and partial functionals are also not calculated.

16.3 Decentralized Control Method for a DGN Containing a Group of UAVs and a Station for Recharging Them Suppose that the UAV group for the DGN is the DDS, and the SDS is a set of stations with which each UAV can dock—in one way or another to perform DGN tasks, and each station can be equipped with an automatic UAV recharging module (RM). As a basic task to be implemented by the DGN, let us define the organization of the speed-optimal transport stream between the destinations—the stations themselves. We will assume that the UAV group is homogeneous; some stations are equipped with UAV recharging modules, while others are not. Each UAV, as soon as its energy reserve drops below the threshold value, can land at one of the stations equipped with RMs; at other stations, the UAV can land to deliver or pick up cargo for transportation; it is possible that the destination station is used by the UAV as a charging station. Below we consider a special case when the decision on the interaction between the station and the UAV is made by the latter (this corresponds to the second mode of operation of the proposed method), thereby adjusting their routes between specified destinations in order to minimize the fulfillment of their missions. Such an adjustment becomes possible after some time, required to determine an adequate set of weighting factors (16.4) for each station by each UAV based on the tasks it performs. The decentralized management discussed above is of particular relevance for power charging stations based on renewable energy sources. It is advisable to install

248

V. Kostyukov and V. Pshikhopov

such stations, firstly, in hard-to-reach places where centralized power supply is difficult or economically ineffective; secondly, in places where the presence of a human operator of servicing stations/mobile objects for one reason or another should be reduced or excluded. In both of these cases, the absence of centralized power generation and the requirement to reduce operator participation dictates the need to use the above-stated decentralized control method, which implies, in particular, the optimization of the process of power generation by stations equipped with RMs and the modes of operation of the UAV. Each recharging module contains: (1) a carrier platform, which can be stationary or mobile, and on which the equipment indicated below is installed; (2) a complex power plant, consisting, in the general case, of a wind power plant and photoconverting elements, the joint operation of which is regulated by a single control unit; (3) module for charging/changing UAV batteries or wireless charging module; (4) a module for analyzing the energy state and predicting the operation of this MP; (5) a receiving and transmitting module for exchanging information signals with individual MRM of the groups. Each station receives recharge request signals from UAVs in its vicinity and analyzes the feasibility of each such recharge, based on its current energy reserve, and also, possibly, relying on the forecast of its power generated in the near future. In response, the station sends to each UAV that has sent it a request, a response signal, which transmits information about the energy that may be available for this UAV at this station at the current time. Further, the on-board computer of this UAV selects the most suitable station with a number q = qopt (s) and sends this station a signal of the intention to land on it. After receiving this signal, the station with the number qopt (s) makes a reservation of the corresponding part of its energy for charging the named UAV. And the exchange of information of this station with all other UAVs that may apply later will be made taking into account this reserve, as well as taking into account information about all previously recorded reserves. The components of the vector X pq (s) are as follows: E n,i (s)—the need for the pth UAV in charge energy, taking into account the reaching of the qth station; Tpq (s) —the real time that the UAV will spend to reach the nearest target point of the route, taking into account the preliminary arrival of the qth station and recharging at it. The vector X pq (s) here consists of one element: E pq (s)—the real energy available for a given UAV at the qth station at the time of its reaching, taking into account the calls to this station by other UAVs. The values X pq (s) and X pq (s) are relevant only when choosing a pair ( p, q) as subject to interaction. After making a decision on the choice of a certain charging station, a corresponding signal is generated, which is transmitted to the selected station with the help of the receiving–transmitting ( ) module of this MRM. The functional ϕAB p, q, s p depends on A, δ by (16.2) as follows: (

)

ϕAB p, q, s p =

⎧ (1) δAB ( p, q, s)

π − arccos

[

∗

∗ E pq (s) − E n,pq (s)

E max

]⎫ +

16 System of Decentralized Control of a Group of Mobile Robotic Means …

⎧

249

[

]⎫ ΔT + π − arccos ∗ = Tpq (s) + ΔT [ [ ∗ ] ] ∗(2) ∗ ∗ T E E + δ p, q, s)ϕ , = δ E ( p, q, s)ϕ∗(1) (s), (s) ( (s) T pq n,pq pq AB AB (2) δAB ( p, q, s)

(16.5)

(1) where the partial functionals ϕ(1) AB and ϕAB are given by the expressions:

[ ϕ(1) AB E pq (s),

[

]

E n,pq (s) = π − arccos

E pq (s) − E n, pq (s) E max

[ ] ϕ(2) AB Tpq (s) = π − arccos

[

] [] ,

] ΔT , Tpq (s) + ΔT

(16.6)

(16.7)

where s is the ordinal number of the landing made by the given pth UAV; ΔT is ∗ upper bound estimate for Tpq (s); E max is the maximum possible energy that can be allocated to one UAV when recharging. The weight vector δ = [δ E , δT ] is updated according to the following rule: δ E, pq (s = 1) = δ E00 /N B , δT , pq (s = 1) = δT 00 /N B , / δ E, pq (s + 1) = ηpq (s + 1) / δT , pq (s + 1) = ξpq (s + 1)

M Σ

(16.8)

ηpq (s + 1),

j=1 M Σ

ξpq (s + 1),

(16.9)

j=1

⎧

[ ] E pq (s), E n, pq (s) , atq = qopt (s); ϕ(1) AB ηpq (s + 1) = ηpq (s) + , 0, otherwise, ⎧ (2) [ ] ϕAB Tpq (s) , atq = qopt (s) . ξpq (s + 1) = ξpq (s) + 0, otherwise.

(16.10)

(16.11)

The weight coefficients δ E, pq characterize the average degree of demand for free energy accumulated at the qth station for the pth UAV, taking into account all previous (up to s-step) experiences of receiving/losing/not receiving energy by this UAV at this station; The weight coefficients δT, pq characterize the comparative average time spent on recharging the pth UAV at the qth station in the past up to the s-step. Coefficients δ E, pq (s),δT, pq (s), for all p, q, s, satisfy the normalization conditions: NB Σ q=1

δ E, pq (s) = δ E,00 = const,

250

V. Kostyukov and V. Pshikhopov NB Σ

δT, pq (s) = δT,00 = const, δ E,00 + δT ,00 = 1.

(16.12)

q=1

As a result of maximizing functional (16.5), we obtain a station with a number qopt (s), which is chosen as an object for the nearest interaction of a UAV with number p. The efficiency of the adaptive selection of a suitable recharging station according to the claimed algorithm, specified by expressions (16.5)–(16.12), is increased each time by analyzing the information accumulated to the current time about the recharging experiments of this UAV. This information is accumulated in the values ηpq (s) and ξpq (s). The decrease in the weight δ E, pq (s) for the q-station in comparison with its initial value in (16.8) can be caused by the following factors: (1) rare use of the station by the pth UAV due to the peculiarities of the regular trajectories used by it; (2) periodic and rather abrupt changes in the operating conditions of the plant, leading to a significant ∗ decrease in real energy E pq (s) in relation to the corresponding declared E pq (s); (3) unforeseen changes on the route of the UAV to this station, which can occur with the UAV itself, as a result of which the value E n, pq (s) can sharply increase, for example, as a result of spontaneous wind gusts, deterioration of visibility conditions, and the appearance of random obstacles; (4) inaccuracy in calculating the value of the energy that the station can potentially provide to this UAV; (5) interference in the ∗ communication channel, distorting information about the value E pq (s) received on board the UAV; (6) other reasons. The decrease in weight δT , pq (s) is due to factors (1) and (3) from the above list, and is also determined by the peculiarity of the relative position of the area of passage of the UAV routes in relation to the location of the station. The computing unit of each UAV learns to choose the most suitable, with respect to a certain criterion, station in the conditions of the given territory of operation, the location of individual stations and regular trajectories of the UAV. When these conditions are forced to change, training must be repeated. At the same time, the experience of the training performed allows you to correct the specified conditions, if possible and/or expedient, of course, with the involvement of forces external to the DGN, including the operator. The inventive method makes it possible for individual UAVs to exchange accumulated experience, decentralized or with the help of periodic connection of the operator, which is advisable if they carry out the same type of tasks or move on average along the same regular trajectories. The decentralized adaptive selection of pairs of interacting objects considered here is especially effective for energy distributed systems, including charging stations for autonomous mobile objects, including UAVs, based on renewable sources of wind and solar energy. The effectiveness of the method should increase with an increase in the number of system elements and an increase in the number of interactions carried out by each given element of the system by the current moment, i.e., training time.

16 System of Decentralized Control of a Group of Mobile Robotic Means …

251

16.4 Simplified Model of UAV Energy Consumption The current need of each UAV for recharging is determined by: (a) the target trajectory of the UAV; (b) cruising speed (c) maximum thrust that engines can develop in the direction of travel; (d) high-speed wind conditions in the traffic area; (e) limiting threshold energy Epr. When a multi-rotor UAV moves, it generates a thrust force perpendicular to the aerodynamic plane of rotation of all propellers, equal to [16]: FT = 0.5C T πρ R 4

n Σ

ωi2 ,

(16.13)

i=1

where R is the radius of each screw, ωi is the angular speed of its rotation of the ith screw, ρ is the air density, n is the number of screws, C T is a dimensionless coefficient of thrust of each propeller. In the case of movement of the vehicle in a fixed mode with a constant ground speed and at a fixed height, the vertical component of the force FG goes to counteract the force of gravity G = mg: FG = FT cos ϑ = G = mg,

(16.14)

where ϑ is the pitch angle of the vehicle. Almost the entire horizontal component FT is used to work against the forces of frontal aerodynamic drag FA,x generated by the incoming air flow in the local coordinate system (CS): Fl = FT sin ϑ = FA,x = 0.5cx ρVr2 Schar ,

(16.15)

| | where Vr = |V − V w | is the speed of the vehicle relative to the oncoming air flow, V , V w is the vector of the absolute speeds of the vehicle and the wind in the basic, terrestrial coordinate system; Schar —the characteristic area of the apparatus; cx is the dimensionless drag coefficient in the velocity coordinate system of the vehicle. Here, for simplicity, but without losing the generality of the results obtained below, we will neglect the powers spent on overcoming the resistance along the OZ axis of the high-speed CS, as well as on parrying short-term, impulsive wind disturbances along all axes of the high-speed CS. We will, however, take into account the steadystate components of wind speeds V w at various local sections of the vehicle’s motion, considering them to be parallel to the horizon everywhere. Equations (16.14) and (16.15) make it possible to obtain the pitch angle ϑ0 required to provide the specified gravity and drag forces: ( ϑ = ϑ0 = arc tan

) 0.5cx ρVr2 Schar , mg

From expression (16.14), it follows that when the inequality is satisfied:

(16.16)

252

V. Kostyukov and V. Pshikhopov

mg > FT max , cos ϑ0

(16.17)

where FT max is the maximum allowable power developed by the propellers of the apparatus, normal flight with a given counteraction of the oncoming air flow becomes impossible without reduction. In this case, it is obviously necessary to reduce the pitch angle to a value ϑ0' that is its upper limit ϑ ≤ ϑ0' . mg . FT max

cos ϑ0' =

(16.18)

The introduced limitation ϑ ≤ ϑ0' based on equalities (16.15) and (16.18) leads ' to the following expression for the required speed modulus V = V ' V n , V n = V /V , minimally deviating from the modulus of the original ground speed V while maintaining the original direction V :

V ' = Vw cos ϕ +

[ | | |

/ 2(mg)2 2 FT max 2 2 ρcx Schar w sin ϕ

,

(16.19) '

where ϕ is the angle between vectors V w , V . Obviously, this speed V exists only if: / 2(mg)2 2 FT max 2 2 ρcx Schar w sin ϕ

.

(16.20)

Failure to fulfill (16.20) means that the movement of the vehicle along the direction V n = V / V is impossible at a given speed of the wind disturbance V w . However, if you choose a new direction of the vector of the absolute velocity of the vehicle, specified by the following angle ϕ ' : [

1 ϕ ' = a sin Vw

/

/ 2

xchar

] 2(mg)2 FT max

,

(16.21)

then condition (16.19) will be fulfilled as equality, and we will get the minimum angle ϕ correction in absolute value. After that, the required minimum change in the absolute speed module of the vehicle can be calculated using the formula (16.19): V '' = Vw cos ϕ ' . Thus, the general rule for correcting ground speed is as follows:

(16.22)

16 System of Decentralized Control of a Group of Mobile Robotic Means …

V corr

253

⎧ V , atCond1 ; / ⎪ ⎪ ] [ / ⎪ ⎪ 2(mg)2 ⎪ 2 FT max ⎨V − Vw2 sin2 ϕ , at Cond1 ∧ Cond2 ; V cos ϕ + ρcx Schar = V w , (16.23) ⎪ ( ( ) )] [ ⎪ ' ' ⎪ ⎪ V sin (ϕ − ϕ ) + Vx cos(ϕ − ϕ ) ⎪ ⎩ VVw cos ϕ ' y , otherwise Vy cos ϕ − ϕ ' − Vx sin ϕ − ϕ '

where the following conditions are introduced: ⎫ ) ⎧ ( 0.5cx ρVr3 Schar ' ≤ ϑ0 , Cond1 ≡ ϑ0 = arc tan mg ⎫ ⎧ / ⎬ ⎨ 2 FT2 max − (mg)2 Cond2 = ≥ Vw2 sin2 ϕ . ⎭ ⎩ ρcx Schar

(16.24)

If the speed is known, then the total required force generated by the propellers can be calculated: / ) ( | |2 (16.25) FT = (mg)2 + 0.5cx ρ |V corr − V w | Schar . The total power generated by all propellers can be calculated using the formula: P = 0.5m k πρ R

5

n Σ

ωi3 ,

(16.26)

i=1

where m k is the dimensionless torque coefficient of each screw. Assuming that in cruise mode, all propellers have the same angular velocity, and combining (16.13) and (16.26), we get: / 2FT . π C T ρn / 2FT m k FT mk . Rω = P= CT C T R π C T ρn 1 ω= 2 R

(16.27)

(16.28)

The method developed here for correcting the ground speed of flight depending on the current absolute wind speed at a given gravity of the vehicle and the maximum thrust force of its propellers, as well as calculating the corresponding angular speed of rotation of the propellers and the required generated power according to formulas (16.23)–(16.28) allows one to determine the key flight characteristics of a multi-rotor UAV when working out target trajectories in local sections of measured steady-state wind disturbances.

254

V. Kostyukov and V. Pshikhopov

The energy remaining in the spacecraft by the time t that has passed after the start of the flight is found by the formula: &t E(t) = E 0 −

P(t)dt,

(16.29)

0

where E 0 is the initial supply of this energy. Here, for simplicity, we neglect the efficiency of all the transmission links of the apparatus, standing between the initial energy stored in its batteries and the energy spent on counteracting the external environment during movement [17].

16.5 Required Energy Characteristics of a Recharging Station Based on a Wind Power Plant The energy capacity of each charging station is determined by the following parameters: (a) average power generation Pin ; (b) the average power of the battery discharge for recharging the UAV–Pout . Power can be calculated using the approximate formula: PS (t) = 0.5ρ Am Vw3 c p ηg ηm ,

(16.30)

where Vw is the speed of the wind acting on the wind turbine; Am is a value of the midsection of the installation in the direction perpendicular to the undisturbed wind flow; c p is a coefficient of use of wind energy; ηg , ηm are coefficients of generator efficiency and energy transfer from the wind turbine rotor to the generator; ρ is an air density. The coefficient c p depends, first of all, on the shape and dimensions of the part of the wind turbine that directly interacts with the incoming flow; in the general case, this part includes the rotor of the installation, and its static elements intended to form a flow with the required aerodynamic properties. This coefficient can depend on the mode of operation of the wind turbine; in particular, it can be a function of the speed of the wind flow and/or the degree of its turbulence (the theoretical upper limit is c p = 0.593, usually the values actually achieved in practice—up to 0.45). The midsection area is, as a rule, proportional to the diameter of the wind turbine rotor D, regardless of the type of installation (horizontal or vertical-axial). If there are photoconverting elements with power at the station, the formula for the total generated power will take the form: PS (t) = 0.5ρ Am Vw3 c p ηg ηm + PSolar (A S , L), where A S is the area of the solar panels, L is the current illumination.

(16.31)

16 System of Decentralized Control of a Group of Mobile Robotic Means …

255

Suppose that charging stations can have the same average dependence of the generated power on the wind speed PS (Vw0 ), where Vw0 is an average wind speed during Tchar is the characteristic period of operation of the energy system under consideration, when the average wind speed is likely not to change; for example, can be taken as Tchar one day or one hour. The total energy generated by one station for the period is E S,T,0 = PS0 (Vw0 )Tchar .

(16.32)

Let us estimate this function from above, taking into account the need to ensure guaranteed recharging of this group of UAVs. Let each UAV need maximum energy to implement its local mission: &Txap E UAV, max,i (Tchar , Vw0 ) =

Phard,i (Vw0 , t)dt, i = 1, .., N ,

(16.33)

0

where Phard,i (Vw0 , t) is the function of the fastest, heaviest energy consumption of the ith UAV at an average wind speed in the corresponding extreme mode, N is the number of UAVs in the group. Then, to ensure guaranteed recharging of all UAVs, the equality must be satisfied: E S,T,0 = Echarw0 UAV,max ,

(16.34)

[ ] Echarw0 max E UAV, max,i (Tchar , Vw0 ) UAV, max .

(16.35)

where i∈[1,N ]

From equality (16.37) taking into account (16.35), it follows: PC⊓0 (Vw0 ) =

kS N Echarw0 UAV,max , Tchar M

(16.36)

where M is the number of charging stations, k S is the coefficient of exceeding the average load of each station, taking into account the degree of possible non-uniformity in the distribution of serviced UAVs between stations due to the peculiarities of the localization of stations, UAV trajectories and current weather conditions. Let the average energy E UAV,0 (Tchar ) consumed by each UAV for the period Tchar be known, taking into account averaging over all possible wind regimes, and E charge,0 is the average portion of energy that each UAV must receive from the charging station, moreover, E charge,0 < E charge,m , where E charge,m is the corresponding maximum energy. Then, the average number of landings for each UAV will be:

256

V. Kostyukov and V. Pshikhopov

] [ kland,T,0 = E UAV,0 (Tchar )/E charge,0 ,

(16.37)

where [·] is the sign of the integer part of the number. The average number n T,0 of UAVs served by each station is equal to: ] [ n T,0 = E S,T ,0 /E charge,0 .

(16.38)

The average period between two adjacent recharges of two UAVs at one station ( ) Tcharge,0 = Txap / n T,0 − 1 .

(16.39)

If the contact principle is used at the station to replenish the spent energy of the UAV, then two options can be implemented. In the first, the station recharges the UAV battery using its own battery; in the second, the discharged UAV battery is replaced with a new one of the same type, charged at the station. For the first case, the battery charge Q of the station, which is used to recharge the UAV battery, must be calculated using the formula: ( ) Q = k0 E charge,m / kbattery U S,n ,

(16.40)

where U S,n is the nominal voltage of the battery of the station, kbattery is the coefficient of the allowable discharge of the battery, meaning the fraction of its residual charge, below which the consumer should not discharge the battery in order to prevent a sharp decrease in its quality; k0 —safety factor for battery charge (usually k0 = 0, 1÷0, 2). For the second method of contact recharging, formula (16.39) expresses only a part of the full charge of the UAV battery used as a removable, replaceable element. The total charge of such a battery can be calculated using a formula similar to (16.39) when replacing in the latter: (a) E charge,m for E UAV —the total energy that must be contained in the battery of each UAV to perform its mission between adjacent recharges; (b) U S,n —for the nominal voltage of the UAV battery. The safety factor k0 should be chosen in such a way that, firstly, the deviation of the real energy E S,T received by each station during the period Tchar from the expected calculated value E S,T , downward, and, secondly, an increase in the intensity of UAV landings with a possible increase in the average required power recharging for each of them compared with E charge,0 ,—did not violate the inequality: ( ) ( ) E S,T Txap ≥ E ΣUAV Txap ,

(16.41)

where E ΣUAV is the total energy required to recharge all UAVs landing at the given station.

16 System of Decentralized Control of a Group of Mobile Robotic Means …

257

16.6 Simulation Let us simulate the decision-making of individual UAVs according to the decentralized principle discussed above, taking into account the prehistory of interaction processes between static and dynamic elements of the system. Let the static subsystem of DGN be represented by M = 18 stations localized at points {Am }, m = 1, M, and some of them, namely, a subset U A,S = {A1 , A3 , A10 , A11 , A12 , A15 , A16 , A17 , A18 }, are simultaneously UAV recharging stations. Let there are three UAVs, which received track routes from the operator in the following form: for UAV-1:A1 → A15 → A5 → A11 → A10 → A1 → A10 → A8 → A6 → A15 →A9 → A10 → A1 → A7 → A8 → A15 → A9 → A2 → A1 → A10 ; for UAV-2: A3 → A10 → A9 → A8 → A6 → A15 → A11 → A10 → A3 → A7 →A6 → A15 → A9 → A2 → A3 → A10 → A11 → A9 → A6 → A9 . for UAV-3: A4 → A10 → A9 → A7 → A6 → A8 → A11 → A10 → A4 → A3 →A10 → A9 → A6 → A8 → A11 → A4 → A10 → A9 → A7 → A6 . Also, the time interval for monitoring the movements of the UAV is set: Tchar = 1, 3 h. In the process of performing route tasks, UAVs have the opportunity to recharge at destination stations included in the set U A,S . This case is referred to as scheduled recharging. In addition, they can also be recharged at stations from the set U A,S that are not included in the route, then we will talk about forced recharging. Note that the UAV can evaluate all charging stations, including those that are among the destination stations of its route. For these latter, efficiency weights will be calculated, and in this experiment, they will not affect the choice of these stations only if the corresponding forced selection conditions are met. This means that the UAV can make a free decision to use such a charging station, provided that this station is not the nearest target point of its route, and it is not in a sufficiently small vicinity of it. It is necessary to simulate the movement of the UAV between destinations with possible landings at additional charging stations. A simplified dynamic model was used to simulate the movement of each UAV in the group. It contains the following external influences: gravity, aerodynamic force of the incoming air flow neglecting the lateral component, control force, the value of which is set inertialessly and so as to compensate for the effect of the total aerodynamic force. If the maximum value of the control force of the vehicle is not sufficient for movement at a given cruising speed, then this speed decreases to the closest value that provides parrying of the aerodynamic force when moving along a given section of the trajectory. The aerodynamic force takes into account the wind speed in the selected local areas of the territory. Initial values of the parameters of each UAV: vehicle weight m = 2.8 kg; ρ = 1.1225 kg/m3 air density; Schar = 0.06 m2 —UAV characteristic area; UAV drag coefficient at zero angle of attack cx = 0.1; initial cruising speed Vk = 20 m/s; maximum energy stored in the UAV battery E m = 50 W h; the maximum value

258

V. Kostyukov and V. Pshikhopov

of the energy of the UAV, upon reaching which the search for a recharging station occurs, E pr = 20 W h; radius of information visibility, assumed to be the same for all UAVs and stations−−Rinf . The power of one charging station at an average wind speed Vw0 = 5 m/s is taken equal to PS0 = 50 W. During the observation time Tchar , different UAVs will generally perform different numbers of recharges: s1 , s2 , s3 . Of interest is the dynamics of changes in the weight coefficients of the energy and time factors for each station from the point of view of each of the three UAVs when the number of the recharging stage changes within, u 1 = 1, 2, .., s1 , u 2 = 1, 2, .., s2 , u 3 = 1, 2, .., s3 , respectively. For the first UAV, the real route will include, in addition to the initial nodal points, additional charging stations. The sequence A(1) S of visits to all charging stations for the UAV-1, including both forcibly determined, which are previously route nodes, and freely chosen, according to simulation results, is as follows: A(1) S = [A16 A11 A10 A1 A17 A15 A17 A7 A17 A15 A10 A1 ], ⇒ s1 = 12. In a similar way, we get the corresponding sequences for the second and third UAVs: A(2) S = [A10 A16 A15 A15 A11 A10 A11 A16 A15 A11 A10 A10 ], ⇒ s2 = 12; A(3) S = [A14 A16 A17 A11 A10 A14 A18 A18 A10 A11 A15 A18 A18 A17 A17 ], ⇒ s3 = 15. According to calculations: average values of the thrust force and its horizontal projection for the first UAV: Fu,x,k = 1.5H , FT ,k = 27.3H ; the average power consumed by it throughout the entire mission P0 = 21, 68 W; total energy consumed by all UAVs during the period Tchar = 1.3 h: E ΣUAV (Tchar ) = 8.2 · 105 W s, the required power of one station to recharge the UAV according to the simulation: PS0 = 34 W. The calculation according to the developed method of decentralized control in relation to the considered energy system “UAV group + charging stations” showed a reduction in the total time of the mission by the UAV group by 6.204% compared to planning without memory effect, i.e., excluding the estimated results of previous recharges. This second case corresponds to maximizing functional (16.5) with constant weighting factors for all stages of recharging. Figure 16.3 shows the target (highlighted in red) and real trajectories in the considered approximation (highlighted in blue) of all UAVs during the development of the above route missions; it can be seen that UAVs are periodically forced to land at charging stations, which were not in the initial assignment lists. This leads to additional branches from the target trajectories in the corresponding real trajectories. Here, red circles also indicate stations that do not have charging modules, and

16 System of Decentralized Control of a Group of Mobile Robotic Means …

259

green small circles—charging stations; green large circles mark areas with additional steady-state additive wind disturbances relative to the average wind speed Vw0 . Figure 16.4a shows the dependences of the weight coefficients determined by the first UAV, for various numbers of forced recharging stages up to the last reference s1 = 12, δ E,1 j (u), u = 1, 2, .., s1 , corresponding to the selected interval Tchar = 1.3 h; in Fig. 16.4b shows similar dependences, but for the weights of the time factor δT ,1 j (u), u = 1, 2, .., s1 . From a comparison of the graphs shown in Fig. 16.4a, it can be seen that in terms of energy, all stations for the first UAV are not the same. Despite the fact that in the particular case we are considering, all stations are energetically equivalent. However, the value of the charge energy available for a given UAV at a charging station is influenced by the accumulation of queues for charging at various stations. The latter is obviously determined by the localization of the stations and the nature of the route target trajectories of the UAV. In addition, for unused UAV charging stations, their

a) for UAV-1

b) for UAV-2

c) for UAV-3

Fig. 16.3 Target and real trajectories of three UAVs

Fig. 16.4 Dependences of the weight coefficients determined by the first UAV based on the recharging experience acquired for each current stage, the lower branch is for A16 ; the upper branch is for A17

260

V. Kostyukov and V. Pshikhopov

Fig. 16.5 Dependences of the weight coefficients determined by the third UAV based on the recharging experience acquired for each current stage—the lower branch—for A14 , the upper branch—for A18

weight coefficients will automatically generally decrease in comparison with those involved, which follows from formulas (16.11)–(16.16). From a comparison of the graphs shown in Fig. 16.4b, we can conclude that the weight δT,1,16 (u) of station A16 ultimately decreases significantly compared to stations A10 and A11 , which can be explained by the presence of a local wind load section (local additional wind flow with a uniform speed V w3 = 5[−0.707; − 0.707] m/s) in a small vicinity of point A16 , which complicates and slows down movement between stations A7 , A9 , A10 , and A11 . The weight of the A14 station is also decreasing, but to a lesser extent than that of the A16 station. At the same time, the weight of the A17 station increases significantly, since the UAV is reoriented precisely to this station after the very first time-ineffective recharging at the A16 station. In Fig. 16.5 shows similar dependences δ E,3 j (u), δT ,3 j (u), u = 1, 2, .., s3 obtained on board the third UAV, at s3 = 15. From a comparison of the graphs shown in Fig. 16.5a, b, it can be concluded that the weights of station A14 , located in the zone of action of an additional wind flow with a speed V w3 = 3[0.707; − 0.707] m/s, as a result, significantly decrease, and the values of similar weights for station A18 , to which a partial reorientation occurs when planning recharging of the third UAV after the very first ineffective recharging at station A14 , significantly increase in comparison with the corresponding average weights at the last iteration. The efficiency of the considered self-training of each UAV group in terms of optimizing the choice of charging stations will be the greater, the more regular the UAV trajectories are, and the longer the operation time of the entire group in a given territory.

16 System of Decentralized Control of a Group of Mobile Robotic Means …

261

16.7 Conclusion The technical and economic advantage of the decentralized method of coordinating the actions of the DGN elements presented in the article in relation to the existing centralized methods, as well as to possible decentralized methods without selftraining, is to minimize the time for solving the tasks assigned to the MRM group, as well as failures of the MRM during the mission, which is achieved by optimizing the choice of a pair of interacting stations and MRM in each specific case, based on both the current characteristics of all elements of the system, and the history of all previous interactions for each of the elements. One of the most important cases of this interaction is automatic recharging/refueling, including UAVs, both in stationary and non-stationary conditions, while an ORS group of a given size can function completely autonomously for a long time in a certain geographic territory. At the same time, the location of individual autonomous charging stations in this territory depends on the shape, size, area of the latter, and the specified/possible modes of operation of the MRM group on it. This solution, when applied to a distributed power grid, which includes a group of mobile robotic objects and their charging station, in contrast to the existing ones, makes it possible to significantly extend the autonomous operation of the entire MRM group in a certain area. In the case when, for example, a one-time operation of a group of UAVs is supposed, for example, for a single survey of an uninterrupted infrastructure object, then not the entire network of charging stations can be used, but only one or, at most, two such stations; at the same time, the time spent on the recharging process is reduced by at least 5 times compared to manual recharging of one UAV by one operator, and at least 10 times compared to manual recharging of two or more UAVs by one operator. When a group of MRMs with dozens of elements is working, the time spent on recharging can be reduced by at least two orders of magnitude; at the same time, there is a significant saving in the use of routine human labor. The property of decentralized control of the interaction between the MRM and the docking stations, which is essential for the presented method, makes it, in contrast to the existing ones, especially effective in those conditions when the operator’s work is impossible, unsafe, or ineffective: mountainous terrain, water surface, difficultto-pass forest areas, etc., places of radiation contamination, fire centers, northern regions, etc., rural areas; in conditions of dense urban development, making the use of conventional transport ineffective. The positive effect of the application of the considered method can be associated with the operation in urban or rural conditions, where the manual type of recharging, although it can be used, is much less effective than the automatic recharging system. Moreover, the gap in the efficiency of these two types of recharging will grow strongly, firstly—with an increase in the number of MRM to be serviced, secondly, with an increase in the area of the territory that should be covered by this DRS group in the course of its operation, and, thirdly, with an increase in the time of functioning of the

262

V. Kostyukov and V. Pshikhopov

specified group in a given territory due to the effect of self-learning when making decisions. The developed method seems to be very popular in cases of significant danger of the external environment, in which the operator is forced to work to coordinate the actions of the DGN, as well as with a relatively large area of the territory covered by DGN, as well as with a strong uncertainty of the operating environment, which in turn complicates the use of centralized methods of coordination actions of the elements of the DGN. Acknowledgements This work has been supported by the grant of the Russian Science Foundation No. 22-29-00370 performed at the Research and Design Bureau of Robotics and Control Systems.

References 1. Gupta, L., Jain, R., Vaszkun, G.: Survey of important issues in UAV communication networks. IEEE Commun. Surv. Tutorials 18(2), 1123–1152 (2015) 2. Hassija, V., Chamola, V., Krishna, D.N.G., Guizani, M.: A distributed framework for energy trading between UAVs and charging stations for critical applications. IEEE Trans. Veh. Technol. 69(5), 5391–5402 (2020) 3. Li, L., Wu, J., Xu, Y., Che, J., Liang, J.: Energy-controlled optimization algorithm for rechargeable unmanned aerial vehicle network. In: 2017 12th IEEE Conference on Industrial Electronics and Applications (ICIEA), pp. 1337–1342 (2017) 4. Hanyu, A., Kawamoto, Y., Kato, N.: On improving flight energy efficiency in simultaneous transmission and reception of relay using UAVs. In: 2019 15th International Wireless Communications and Mobile Computing Conference (IWCMC), pp. 967–972 (2019) 5. Ippolito, M.: Electrically charging system for drones. WO Patent 2016/113766, 7 January 2016 (2016) 6. Wang, M.: Systems and methods for UAV battery exchange. US Patent 9139310B1, 22 September 2015 (2015) 7. Gentry, N.K., Hsieh, K., Nguyen, L.K.: Multi-use UAV docking station systems and methods. US Patent 9387928B1, 12 July 2016 8. Voronov, A.A.: Introduction to the dynamics of complex controlled systems. Science, Moscow (in Russ.) (1985) 9. Ghadami, R., Shafai, B.: Decomposition-based distributed control for continuous-time multiagent systems. IEEE Trans. Autom. Control 58(1), 258–264 (2012) 10. Borrelli, F., Keviczky, T.: Distributed LQR design for identical dynamically decoupled systems. IEEE Trans. Autom. Control 53(8), 1901–1912 (2008) 11. Pervozvanskiy, A.A., Gaytsgori, V.G.: Decomposition, aggregation, and approximate optimization. Science, Moscow (in Russian) (1979) 12. Abgaryan, K.A.: Splitting of a singularly perturbed multitamp system. Bull. Acad. Sci. Armenian SSR, Math. 5, 327–337 (1979) 13. Fradkov, A.L.: Adaptive management in complex systems: searchless methods. Science, Moscow (in Russ.) (1990) 14. Kamachkin, A.M., Shamberov, V.N.: Decomposition method in multidimensional nonlinear dynamical systems. Voronezh State University Bulletin. Series: Syst. Anal. Inf. Technol. 1, 47–55 (2012) (in Russ.) 15. Abgaryan, K.A.: Asymptotic splitting of equations of a linear automatic control system. Reports USSR Acad. Sci. 166(2), 301–304 (1966)

16 System of Decentralized Control of a Group of Mobile Robotic Means …

263

16. Yuriev, B.N.: Aerodynamic Calculation of Helicopters. State Defense Industry Publishing House, Moscow (in Russian) (1956) 17. Kostyukov, V.A., Medvedev, M.Y., Poluyanovich, N.K., Dubygo, M.N.: Features of electromechanical control of a complex power plant with a vortex-type wind-conversion device. In: Cyber-Physical Systems: Industry 4.0 Challenges, pp. 159–166 (2020)

Chapter 17

Method for Optimizing the Trajectory of a Group of Mobile Robots in a Field of Repeller Sources Using the Method of Characteristic Probabilistic Functions Vladimir Kostyukov

and Viacheslav Pshikhopov

Abstract To solve the problem of avoiding the influence of disturbance sources by a group of ground-based robotic platforms, the method considered in the previous works of the authors is proposed based on calculating the probabilities of successful passage of the elements of the group and their trajectories under the influence of sources. These probabilities can be found by estimating characteristic probabilistic function (CPF) parameters describing their effect on moving objects in the small vicinity of points in the source domain over short time intervals. The proposed method is modified by additional optimization of the resulting spatial trajectory along the length for each robotic platform (RP), taking into account the specified degree of permissible deviation from the original curve. The methods that make it possible to find the target trajectories of the leading and driven elements of the RPs group, the probability of successful passage of which exceeds a given target value has been developed. These methods are generalized to the case when the optimization criterion is the probability of successful passing of only a part of the RPs group. The results of modeling are considered and discussed, confirming the effectiveness of the proposed method for planning the trajectories of the movement of robots forming a group in the field of repeller sources.

17.1 Introduction The control of a group of robotic platforms (RP) in the implementation of a given mission in motion in an environment with obstacles and sources of disturbances generally implies a complex solution of two main tasks: (a) planning the target trajectories of each element of the group in the phase space of external R m (m is the number of external coordinates) or internal coordinates based on the requirements for solving mission problems in conditions of operation in a priori not completely defined environment that includes obstacles and sources of disturbance (detection V. Kostyukov (B) · V. Pshikhopov Southern Federal University, 105/42 Bolshaya Sadovaya Str., Rostov-on-Don 344006, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_17

265

266

V. Kostyukov and V. Pshikhopov

radar, enemy’s fire, electronic warfare, etc.); (b) regulation of the control actions of each RP in order to carry out its movement with a given accuracy along a previously constructed trajectory of the phase space, taking into account the selected physical and mathematical model of the movement of a group element as an object of control in a given external environment. Note that the design features and the propulsion-steering mechanism of the RP, as well as its energy determine the class of admissible trajectories of movement of each individual RP in the phase space. For ground-based RPs, the class of the corresponding admissible motion trajectories is narrower than the corresponding class for aircraft RPs, taking into account the limitations of the latter to movement on the surface [1, 2]. This article is devoted to the consideration of the method of program trajectory control of a group of ground-based wheeled RPs when moving in an environment that includes obstacles and sources of disturbances. The generalized algorithm of the developed method includes the following main stages: (a) construction of the initial trajectory of the conditional leading element of the group when moving in the specified environment without taking into account the influence of sources (or with preliminary rough consideration of this influence) when performing a given stage of the mission; (b) construction of the initial trajectories of movement of all other elements of the group associated with the trajectory of the previous stage; (c) implementation of source avoidance maneuvers generation of corrected trajectories of the leading and driven elements of the group. The first stage (a) implies the use of such methods of suboptimal planning of movements as A* , D* , tree method, potential fields method, etc. Because of the use of these methods, a trajectory is formed, which usually includes sections with strong oscillations. To smooth the latter, an effective technique should be developed that takes into account the scale of these oscillations and their frequency. Further, it is necessary to establish a certain high-speed mode of movement along the constructed spatial trajectory, which is determined by the operational characteristics of the RPs of the group. Then, at step (b) it is necessary, based on a given simple pattern of formation, to determine the corresponding trajectories of movement of all other elements of the group in an acceptable approximation. Finally, at stage (c), it is necessary to correct the initial trajectories of the elements of the group, taking into account the provision of a given probability of non-detection (non-defeat) from the sources of influence of all elements of the group, or a certain subset of the group, or a given number of its elements. The path planning can be carried out by methods of spatial decomposition, which involve partitioning the initial space into discrete cells [3]. The methods of adaptive spatial decomposition have also become widespread [4, 5]. Spatial decomposition allows you to apply discrete search methods based on graph theory. Algorithms of the A* and D* families have gained the greatest popularity [6, 7]. Algorithm A* is an extension of Dijkstra’s algorithm, which reduces computational complexity by choosing a heuristic path cost function. At the same time, the algorithm is demanding on memory and its computational complexity increases sharply with

17 Method for Optimizing the Trajectory of a Group of Mobile Robots …

267

the growth of the number of partition cells. To reduce the memory requirements of the original A* algorithm and its computational complexity, its modifications have been developed [8]. The method of potential fields is computationally more efficient. It allows you to plan a path in the environment when new obstacles appear, as well as to obtain, on average, smoother trajectories than by previous methods [9]. The main limitation of the method of potential fields is that the robot gets into local minima, from which the robot must be derived by separate algorithms [10]. The main problem in constructing the formation of the group in the first approximation, without taking into account the influence of disturbance sources, is the requirement to comply with the given structure of the formation in some approximation, taking into account obstacles, as well as kinematic and dynamic limitations of the elements of the group, caused by the peculiarities of their propulsion-steering apparatus and mass-inertial characteristics. These restrictions lead to a forced distortion of the configuration of the group structure when passing through curved sections of the elements’ trajectories. Therefore, the problem arises of a correct transition to the trajectories of the slave elements of the group from the previously constructed trajectory of the leader. In connection with the above problems that arise at the stage of program control of the movement of a group of robots in an environment with disturbance sources, it is proposed to use an upgraded algorithm of potential fields that which contains a procedure for bypassing local minima. To construct smoothed trajectories, a special technique is proposed for minimizing the obtained spatial piecewise linear curve along the length, followed by rounding the sections at the vertices with circular arcs. Further, to organize the movement of the group in the first approximation, it is proposed to align the full-time intervals of the RP movement with each other due to the corresponding change in the functions of their trajectory velocities according to the method developed in [10], followed by rounding of the sections at the vertices by arcs of circles [11]. Finally, to effectively solve the problem of calculating the probability of successful passage of sections of motion trajectories in the fields of disturbance sources, it is proposed to use the technique developed in [12, 13]. It is based on the concepts of the characteristic probabilistic function of the source and includes the procedure of adaptive local optimization of trajectory segments, which implements an effective escape from the action of sources by a moving mobile robot with the achievement of a given probability of successfully passing the target trajectory by means of an appropriate iterative procedure. This technique is modified in this article by additional optimization of the resulting spatial piecewise broken line along the length, taking into account the specified degree of permissible deviation from the original curve.

268

V. Kostyukov and V. Pshikhopov

17.2 The Problem Statement A group of RPs on a plane is considered, which moves in an environment with obstacles and conflict areas (Fig. 17.1). The obstacles are shown in Fig. 17.1 with black rectangles. Conflict areas are sectors bounded by circles and a given angle. It is required to develop a planning algorithm that ensures the implementation of avoidance maneuvers from obstacles and sources of conflict areas of all RPs group, provided that in one way or another the initial trajectory of the conditional leading element of the group is constructed when moving in the specified environment with obstacles and the trajectory of the corresponding driven elements of this group, taking into account the given tuning pattern and kinematic and dynamic limits of RPs group. At the same time, it is necessary to propose effective algorithms for planning the trajectories of the leading RPs group and a method for smoothing the obtained trajectory to reduce oscillations. The problem will be solved in the following practically important approximation: we assume that the distances between the elements of the group at each moment of the time interval of motion in a medium with sources are much less than the minimum distance from some RPs group to the nearest source at the corresponding times, at least for the family of corrected trajectories after calculation of evasive maneuvers.

B

Fig. 17.1 To the problem statement

O (4)

O (3) RP-1

-2 RP

A

-3 RP

O (2)

O (1)

17 Method for Optimizing the Trajectory of a Group of Mobile Robots …

269

17.3 Methods for Constructing a Program Trajectory of a Ground-Based Robotic Platform Under the Influence of Disturbance Sources 17.3.1 Method for Optimizing Local Sections of the Trajectory Based on the Characteristic Probabilistic Function When a RP group moves under the conditions of interference sources, to minimize the influence of the latter on the group, the method of calculating the probabilities of successfully passing the RP trajectory in the field of these sources can be effectively used. It is based on the concept of the characteristic probabilistic function (CPF) of a source, which gives the field of elementary probabilities of successful passage of small trajectories in the vicinity of a given source at various points in its area of action [12]. We will further consider a special case of sources with a uniform azimuthal distribution of the CPF: ]} { [ ( exp −εT exp −(d/d p )2 , for M ∈ U S , (17.1) q0S (d, α, T ) = 1 otherwise where U S is the circular area of action of the source S, which is characterized by the center O, directing vector of the midline n S , radius d0 , and angle ∆α S contracting this sector; d p is the effective source radius, which may not be equal to the sector radius d0 ; ε is the characteristic frequency of the source impact on the RP; an increase in ε reduces the probability of passing this segment. The peculiarity of the formulation of the problem of optimizing the trajectory of motion under the conditions of sources of disturbance and the method of its solution in [12] differ significantly from the methods of applying risk functions to evaluate suitable trajectories [14, 15], first of all, by directly calculating the probabilities of successful passage/failure of a particular trajectory. The works [13, 14] also developed a heuristic approach to the modernization of the initial RP motion trajectory in order to increase the probability of successful passage of a new modified trajectory in the field of such sources. The essence of the approach lies in local, independent of each other, shifts of the nodal points of the original piecewise broken trajectory to their new positions by optimizing special functionals that characterize the position of a point in the source area, the direction and speed of the RP at a given point, as well as the source’s CPF. It is shown that the application of a special iterative procedure in relation to the deviation of the RP’s trajectory from the action of one source strictly achieves the target probability of successfully passing the changed trajectory in a finite number of steps, if such a trajectory exists at all. For the case of many sources, this approach is a good way to find the global maximum of the probability of successfully passing the trajectory in the field of these sources.

270

V. Kostyukov and V. Pshikhopov

17.3.2 Method for Reducing Oscillations and Trajectory Length When Planning Motion in a Region with Sources Since the used local optimization of different nodes of a piecewise polygonal line does not directly imply and does not take into account the real correlation of their displacements to achieve local and global minima of the indicated probability, the method implies a special procedure for averaging the positions of the nodes of the original polygonal line and their corresponding shifted positions obtained after local optimizations. Namely, for the entire piecewise polygonal line obtained at some iteration after local optimization, the following functional is additionally optimized: ] N −1[( )2 ( ∑ )2 ( ) ' xi − xi + yi' − yi G T r ' = −δG i=2

− δL

N −1[ ∑

(

xi' − xloc,i

)2

)2 ] ( + yi' − yloc,i

(17.2)

i=2

Here, the first term is)the sum of the squared deviations of the coordinates of each ( target point Ai' xi' , yi' from the corresponding source point Ai (xi , yi ) before optimization (optimization by this functional), and the second term ( ' is' ) the sum of the ' A squared deviations of the coordinates of each target point i x i , yi from the point ( ) Aloc,i xloc,i , yloc,i obtained by solving the local optimization problem for Ai (x i , yi ). The target points in their totality form the required piecewise polygonal line T r ' . The coefficients 0 ≤ δG, δL ≤ 1 characterize the influence of factors minimizing the deviation from the original curve and from the curve obtained after local optimizations of the nodal points. The initial nodal points, by assumption, are devoid of interference oscillations and irregularities in the formation of the original piecewise polygonal line. However, the corresponding points after local optimizations may have such oscillations. The level of the latter obviously strongly correlates with a significant decrease in the probability of successful RP passage along such a piecewise linear trajectory. Including due to the increase in the length of the section of this trajectory passing in the field of action of the sources. Therefore, the averaging procedure expressed by the optimization requirement (47), makes it possible, on average, to reduce the level of oscillations of the nodes of the modified curve and smooth it. At the same time, binding for such averaging to the nodal points of the original curve has a drawback—we start from the behavior of the original function to be optimized. Therefore, it is more ( promising) to use the smoothing method, which is based solely on the points Aloc,i xloc,i , yloc,i , i = 2, . . . , N − 1 obtained after local optimizations. But a similar technique was developed by us in Sect. 17.2 for smoothing the

17 Method for Optimizing the Trajectory of a Group of Mobile Robots …

271

oscillations of piecewise linear functions obtained after using D* -based motion planners, potential fields, etc. Thus,( the following ) functional is proposed for smoothing the oscillations of points Aloc,i xloc,i , yloc,i , i = 2, . . . , N − 1: N −1[ ∑ ( ' ( ) )2 ( )2 ] ( ) xi − xloc,i + yi' − yloc,i G T r ' = G T r ' = δL i=2

+ δG

N −1[ ∑ (

' xi+1 − xi'

)2

)2 ] ( ' + yi+1 − yi' ,

(17.3)

i=2

where the coefficients 0 ≤ δ L , δG ≤ 1 have the same meaning as in (17.3). The expression (17.3) formally coincides with the corresponding functional used in [10] to effectively smooth the trajectory obtained after applying the discrete motion planner. The difference lies in the fact that, in contrast to [10], here we use the original piecewise polygonal line constructed on the basis of points obtained after the local optimization procedure according to the method of [13]. Therefore, the minimum of functional (17.3) is expressed by the following analogous equalities: X opt = A−1 B X , Yopt = A−1 BY ,

(17.4)

where [A]N−2×N−2 ⎡ δ L + 2δG −δG 0 ⎢ −δG δ L + 2δG −δG ⎢ ⎢ 0 −δ2 δ L + 2δG ⎢ ⎢ 0 0 −δG ⎢ =⎢ ⎢ ... ... ... ⎢ ⎢ 0 · 0 ⎢ ⎣ 0 0 · 0 0 0

⎤ ... 0 0 0 ⎥ ... · 0 0 ⎥ ⎥ ... 0 · 0 ⎥ ⎥ ... ... ... ... ⎥ ⎥ ⎥ · · 0 0 ⎥ ⎥ 0 · δ L + 2δG −δG ⎥ 0 −δG δ L + 2δG −δG ⎦ 0 0 −δG δ L + 2δG (17.5) ]T [ BX = δL xloc,2 + δG xloc,1 , δL xloc,3 , . . . , δL xloc,N−2 , δL xloc,N−1 + δG xloc,N , ]T [ BY = δL yloc,2 + δG yloc,1 , δL yloc,3 , . . . , δL yloc,N−2 , δL yloc,N−1 + δG yloc,N . (17.6) 0 0 −δG ... ... ... ... ...

It can be shown that, at the found stationary point (17.4), the sufficient condition for a local maximum in the minors of the Hessian matrix is satisfied for any values of δL , δG such that 0 ≤ δL , δG ≤ 1,δL + δG = 1.

272

V. Kostyukov and V. Pshikhopov

17.4 Correction of Individual Trajectories of Ground-Based Robotic Platforms Forming a Group Under the Influence of Disturbance Sources In the case of optimizing the movement of a group of M robotic platforms according to the criterion of the maximum probability of successful passage, two characteristic optimization problems arise. The first task means optimizing the probability of successfully passing the section with sources by all elements of the group and is associated with maximizing the following functional [12, 13]: Q[{T r }] =

M ∏

qm (T rm ) =

m=1

M ∏ m=1

[ exp −

K ∑

] ε

(k)

·I

(m,k)

,

(17.7)

k=1

where {T r } is the set of trajectories of all RPs of the group, qm (T rm ) is the probability of successful passage of the individual trajectory T rm by the mth RP of the group. The values I (m,k) are given by the formula: ⎧ ⎡ ( ) ⎤ (k) (k) 2 )⎨ dm,l sin ϕm,l (k) ( ⎦ ηl Am,l , Am,l+1 exp⎣− ≡ (k) ⎩ 2v d p l=1 |) |)]} ( [ ( ) (| ) (| | (k) | | (k) | (k) (k) , × sign ρ˜2,m,l erf |ρ˜2,m,l | − sign ρ˜1,m,l erf |ρ˜1,m,l | √

I (m,k)

N∑ m −1 π d (k) p

(k) ρ˜1,m,l =

(k) (k) dm,l cos ϕm,l

d (k) p

(k) (k) , ρ˜2,m,l = ρ˜1,m,l +

∆rm,l d (k) p

,

(17.8)

(17.9)

| | (k) where dm,l = | Ok Am,l | is the distance from| the center of | the k-th source to the point Am,l of the m-th RP of the group; ∆rm,l = | Am,l Am,l+1 | is the length of the segment (k) Am,l Am,l+1 ; ϕm,l is the angle between vectors Ok Am,l and Am,l Am,l+1 . Here ε(k) , d (k) p is the characteristic frequency and the effective range of the k-th source with an area in the form of a circular sector, and d (k) p may not be equal to the radius of this sector d0(k) . The second task means maximizing the probability of successfully passing a section with sources only by a certain number of elements of the group. Consider partitioning the set of all RPs of the group into subsets of m elements, where m = 1, 2, . . . , M. Then for each m, the set of possible combim , and within each such subset nations is numbered as follows: u = 1, 2, . . . , C M Pm,u with the number u, we number each of its elements as follows: u = m ' (m, u, 1), m ' (m, u, 2), . . . , m ' (m, u, m), where m ' (m, u,) is a function of three natural arguments m, u,, (= 1, 2, . . . , m), that accesses each element of Pm,u. Let us expand the set of values of the third argument of the function m ' (m, u,) so that

17 Method for Optimizing the Trajectory of a Group of Mobile Robots …

273

when m Fmg, ϕ = θ ≈ 0. During the flight, it is necessary to orient the USAV relative to the given trajectory by the heading angle χ (the angle between the diametral plane of the aerial vehicle and the direction to the trajectory) by turning around the mass center by the yaw angle ψ, then it is necessary to turn the device by the angle β, which is defined as the tangent of the inclination angle between the straight-line p and the OY axis (see Fig. 19.9a). Thus, the heading angle for the USAV χ = ψ is equal to the yaw angle, and is determined by the expression: χ = a tan 2

y1 , x1

(19.20)

where x 1 and y1 are direction of the moving coordinate system. External disturbances affect the dynamics of the SUAV flight, which can lead to deviations from the desired trajectory (Fig. 19.9b). It is proposed to carry out the return of the aerial vehicle to the trajectory by turning the SUAV to the heading angle χ, and then it is necessary to calculate the path deviation vector according to the formula: | | | ∆x | | | ( ) (19.21) ∆ = || ∆ y || = T10 r iCPn , |∆ | z

308

A. S. Martinez Leon et al.

Fig. 19.9 SUAV trajectory diagram: a heading angle correction λ and b correction of movement along the horizontal plane

where r iCPn = rOC − r iOPn is vector representing the path of the USAV to the ith point | | | cos χ sin χ 0 | | | of the given trajectory; T10 = || − sin χ cos χ 0 || is matrix of transition from moving | 0 0 1| to inertial coordinate system. As a result of eliminating path deviations, the USAV should fly to a given point of the trajectory Pn (x n , yn ), and then it is necessary to similarly orient the aerial vehicle relative to the given trajectory at the heading angle χ. Next, let us consider an example of modeling the motion of the USAV along the proposed trajectory, while it is necessary to set the parameters of the straight-line segments, time intervals, and corresponding boundary conditions (Table 19.1). The virtual modeling was carried out in the MATLAB mathematical package. The parameters, used as input data when simulating the movement of the SUAV, are given in Table 19.2. Table 19.1 Trajectory planning boundary conditions Parameters of the USAV trajectory planning Stages

Takeoff

Conditions

Time interval (s)

Coordinates (m)

Intermediate points (m)

Speed (m/s)

Acceleration (m/s2 )

–

P˙0 = 0 P˙1 = 0 P˙0 = 0 P˙4 = 0

P¨0 = 0 P¨1 = 0 P¨1 = 0 P¨4 = 0

P˙4 = 0 P˙5 = 0

P¨4 = 0 P¨5 = 0

Initial

t0 = 0

P0 (0,0,0)

Final

t 1 = 10

P1 (2,0,5)

Horizontal flight

Initial

t1 = 0

P1 (2,0,5)

P2 (6,4,5)

Final

t 4 = 40

P4 (14,0,5)

P3 (10,4,5)

Landing

Initial

t 4 = 40

P4 (14,0,5)

–

Final

t 5 = 50

P5 (16,0,0)

19 Control System of Small-Unmanned Aerial Vehicle for Monitoring Sea …

309

Table 19.2 SUAV simulation parameters Parameters of the USAV motion simulation Name

Value

Units

Weight

1.5

kg

Length of the frame beams

0.225

m

Lifting force coefficient

8.048 × 10–6

N m s2

Resistance coefficient

2.423 × 10–7 ⎤ ⎡ 0.0035 0 0 ⎥ ⎢ ⎢ 0 0.0035 0 ⎥ ⎦ ⎣ 0 0 0.005

N m s2

Matrix of the main axial moments of inertia I x , I y , and I z

kg m2

Fig. 19.10 Spatial motion of the SUAV

The main results of modeling the USAV motion along a given trajectory are shown in Figs. 19.10, 19.11 and 19.12.

19.6 Vision System In this section, the problem of monitoring maritime transport objects by using convolutional neural networks (CNNs) of the YOLO class is considered. The vision system

310

A. S. Martinez Leon et al.

Fig. 19.11 Diagrams of the USAV mass center deviation from the specified one under the action of disturbing influences: a–c the USAV movement along the OX, OY, and OZ axes; d–f the change in the aircraft angles ϕ, θ, and ψ; I—the aerial vehicle takeoff; II—flight along the horizontal plane OXY; and III—landing

(VS) includes an Intel RealSense RGB-D video camera and a Raspberry Pi 4 microcontroller. The process of recognizing and determining the location coordinates of sea vessels, detected during the monitoring, is performed on board the USAV in real time. According to Fig. 19.13, a S × S grid is applied to the original image obtained as a result of shooting, then this image is fed to the input of a convolutional neural network (CNN). The internal structure of the CNN includes convolutional layers, pooling layers, and activation functions [26]. At the output, we get a multidimensional image matrix (tensor). Each cell is represented by the vector σi = [p, x, y, w, h, c]T , where p—some probability that the object is located in this cell; x and y—coordinates of the corresponding grid cell center; w and h—width and height of the bounding box relative to the entire image; and c—class of the monitoring object (sea vessels). Further, to apply the bounding boxes on the image, the following actions must be performed: 1—reset cells that do not satisfy a certain threshold value p < η (η = 0.5); 2—select the cell with the highest value according to the probability criterion p as the center of the monitoring object; and 3—remove intersecting boxes corresponding to the same object by using the non-maximal suppression algorithm [28].

19 Control System of Small-Unmanned Aerial Vehicle for Monitoring Sea …

311

Fig. 19.12 Diagrams of the SUAV control modes formation: a a diagram of the resulting lift force U 1 dependence on time t; b a diagram of the external moments MX (U 2 ) dependence on time t; c a diagram of the external moments MY (U 3 ) dependence on time t; d a diagram of the external moments MZ (U 4 ) dependence on time t; I—aerial vehicle takeoff; II—flight along the horizontal plane OXY; and III—landing

The process of the CNN training is carried out through the use of OpenCV. The image dataset consists of 500 units. In this case, the number of training epochs is 100. The main results obtained during the training outcome are shown in Fig. 19.14.

19.7 Conclusions In this paper, an assessment of the state of the coastal territories monitoring issue is carried out. The structure of a software and hardware complex to perform tasks of monitoring the activities of ships on the coastal territory of Ecuador using an intelligent flying platform made according to a quadcopter scheme is proposed. Based on a nonlinear dynamic model of a small-sized aerial vehicle, some algorithms have been developed to control orientation, altitude maintenance, and movement along a given

312

A. S. Martinez Leon et al.

Fig. 19.13 General scheme of the VS implementation algorithm for the detection of maritime transport objects

Fig. 19.14 Results of water transport detection using the CNN of the YOLO class

trajectory based on a classical PID controller, and it is also proposed to use an adaptive control strategy with a predictive model (NMPC) capable of optimizing control parameters received by the onboard computer depending on changes in external disturbances affecting the stabilization and positioning accuracy of the SUAV when performing the flight path. In addition, the simulation of the SUAV motion was carried out in the MATLAB environment to study the proposed algorithms for controlling the movement of the aerial vehicle, the parameters of the controller to ensure the quality indicators of the control system within the specified limits were determined.

19 Control System of Small-Unmanned Aerial Vehicle for Monitoring Sea …

313

References 1. Wang, X., Huang, Zh., Sui, G., Lian, H.: Analysis on the development trend of future UAV equipment technology. Acad. J. Eng. Technol. Sci. 1(2), 114–121 (2020) 2. Fan, B., Li, Y., Zhang, R., Fu, Q.: Review on the technological development and application of UAV systems. Chin. J. Electron. 2(29), 199–207 (2020) 3. Jatsun, S., Emelyanova, O., Bezmen, P., Martinez Leon, A.S.: Hardware/Software architecture for research of control algorithms of a quadcopter in the presence of external wind Loads. Electromech. Robot. J. 165–177 (2022) 4. Jatsun, S., Efimov, S., Emelyanova, O., Martinez Leon, A.S.: Modeling and control architecture of an autonomous mobile aerial platform for environmental monitoring. In: 2019 International Conference on Information Systems and Computer Science (INCISCOS), pp. 177–182. IEEE, Quito, Ecuador (2019) 5. Correia, A., Simões-Marques, M.: Machine learning in coastal incident detection, identification and classification with UAVs. In: 16th Iberian Conference on Information Systems and Technologies (CISTI), pp. 1–6 (2021) 6. Loya, H., Enríquez, V., Franklin Salazar, L., Sanchez, C.: Analysis and determination of minimum requirements of an autopilot for the control of unmanned aerial vehicles (UAV). In: International Conference on Computer Science, Electronics and Industrial Engineering (CSEI), pp. 129–142 (2019) 7. Zurita, C.M.A., Wilbert, G.A., Enríquez, C.V.X.: Toward the development of surveillance and reconnaissance capacity in Ecuador: geolocation system. J. Dev. Adv. Defense Secur. Proc. MICRADS 2019, 123–136 (2019) 8. Valencia, E.A.V., Palma, K.A., Changoluisa, D., Higalgo Dias, V.H.: Wetland monitoring through the deployment of an autonomous aerial platform. In: IOP Conf. Ser. Earth Environ. Sci. 1(32), 012002 (2019) 9. Mellinger, D., Kumar, V.: Control and planning for vehicles with uncertainty in dynamics. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 960–965 (2010) 10. Alves, A.N., Ferreira, M.A.S., Colombini, E., Simoes, A.S.: An evolutionary algorithm for quadcopter trajectory optimization in aerial challenges. In: Proceedings of the IEEE 2020 Latin American Robotics Symposium (LARS), 2020 Brazilian Symposium on Robotics (SBR) and 2020 Workshop on Robotics in Education (WRE), pp. 1–6 (2020) 11. Emelyanova, O., Kazaryan, G., Martinez Leon, A.S., Jatsun, S.: The synthesis of electric drives characteristics of the UAV of “convertiplane-tricopter” type. MATEC Web Conf. 99, 249–263 (2017) 12. Bao, N., Ran, X., Wu, Zh., Xue, Y.: Research on attitude controller of quadcopter based on cascade PID control algorithm. In: 2017 IEEE 2nd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), pp. 1493–1497 (2017) 13. Martinez Leon, A.S., Jatsun, S., Emelyanova, O.: Control of the electric drives of a multirotor system type convertertiplane. J. Fundam. Appl. Probl. Tech. Technol. 1, 83–93 (2020) 14. Suresh, H., Sulficar, A., Desai, V.: Hovering control of a quadcopter using linear and nonlinear techniques. Int. J. Mechatron. Autom. 6, 120–129 (2018) 15. Doukhi, O., Fayjie, A.R., Lee, D.J.: Intelligent controller design for quad-rotor stabilization in presence of parameter variations. J. Adv. Transp. (2017) 16. Martins, L., Cardeira, C., Oliveira, P.: Feedback linearization with zero dynamics stabilization for quadrotor control. J. Intell. Robot. Syst. 1(101), 1–17 (2021) 17. Flores, A., Scipion, D.A., Saito, C., Apaza, J.: Unmanned aircraft system for Andean Volcano monitoring and surveillance. In: Proceedings of the 2019 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pp. 297–302 (2019) 18. Jatsun, S., Morocho, L.M.M., Emelyanova, O., Martinez Leon, A.S.: Controlled adaptive flight of a convertiplane type tricopter in conditions of uncertainty for monitoring water areas. In: 2020 International Multi-Conference on Industrial Engineering and Modern Technologies (FarEastCon), pp. 1–7. IEEE (2020)

314

A. S. Martinez Leon et al.

19. Jatsun, S., Emelyanova, O., Martinez Leon, A.S.: Design of an experimental test bench for a UAV Type convertiplane. IOP Conf. Ser. Mater. Sci. Eng. 714(1), 012009 (2020) 20. Jatsun, S., Emelyanova, O., Lushnikov, B., Martinez Leon, A.S.: Hovering control algorithm validation for a mobile platform using an experimental test bench. IOP Conf. Ser. Mater. Sci. Eng. 1(1027), 012008 (2021) 21. Dim, C., Nabor, F., Santos, G., Schoeler, M., Chua, A.Y.: Novel experiment design for unmanned aerial vehicle controller performance testing. IOP Conf. Ser. Mater. Sci. Eng. 1(533), 012026 (2019) 22. Mansouri, S.S., Lindqvist, B., Agha-mohammadi, A., Nikolakopoulos, G.: Nonlinear MPC for collision avoidance and control of UAVs with dynamic obstacles. IEEE Robot. Autom. Lett. 4(5), 6001–6008 (2020) 23. Nascimento, I.B.P., Ferramosca, A., Pimenta, L.C.A., Raffo, G.V.: NMPC strategy for a quadrotor UAV in a 3D unknown environment. In: 19th International Conference on Advanced Robotics (ICAR), pp. 179–184 (2019) 24. Zenkin, A., Berman, I., Pachkouski, K., Pantiukhin, I.: Quadcopter simulation model for research of monitoring tasks. In: Proceedings of the 26th Conference of Open Innovations Association (FRUCT), pp. 449–457 (2020) 25. Krzysztofik, I., Koruba, Z.: Analysis of quadcopter dynamics during programmed movement under external disturbance. J. Nonlinear Dyn. Control. 177–185 (2020) 26. Hu, Y., Wu, X., Zheng, G., Liu, X.: Object detection of UAV for anti-UAV based on improved YOLO v3. In: Chinese Control Conference (CCC), pp. 8386–8390 (2019) 27. Liu, M., Wang, X., Zhou, A., Fu, X., Ma, Y.: UAV-YOLO: Small object detection on unmanned aerial vehicle perspective. J. Sens. 8(20), 1–12 (2020) 28. Zhao, D., Li, X.: Ocean ship detection and recognition algorithm based on aerial image. In: Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC), pp. 218–222 (2020) 29. Sa, I., Corke, P.: System identification, estimation and control for a cost effective open-source quadcopter. In: Internal Conference on Robotics and Automation, pp. 2202–2209. IEEE, Minnesota (2012) 30. Casado, R., Bermúdez, A.A.: Simulation framework for developing autonomous drone navigation systems. J. Electron. 1(10), 7 (2021)

Chapter 20

Development of an Algorithm for Coverage Path Planning for Survey of the Territory Using UAVs Valeria Lebedeva

and Igor Lebedev

Abstract Increasingly, unmanned aerial vehicles (UAVs) are used for surveying areas in search of a specific object, mapping, compiling an orthophotomap, or a three-dimensional terrain model, as well as for processing agricultural fields. At present, the development of algorithms for constructing a trajectory that covers a certain area and takes into account the limited energy resource on board the UAV remains an urgent task. The paper proposes an algorithm for calculating the trajectory for uniform coverage of the territory in the form of a polygon, which provides a complete study of the site by the UAV technical vision system, taking into account its parameters. The paper presents an analysis of existing algorithms for planning the coverage trajectory and proposes a new algorithm that takes into account obstacles in the surveyed area. The resulting trajectory is optimized by reducing the number of maneuvers and smoothing them out. As a result of the experiments, the flight time of the UAV along the trajectories calculated by the developed method on a site with an area of 950,000 m2 was reduced by 10% compared to the flight time along the trajectory calculated by the standard algorithm.

20.1 Introduction Surveying the territory is necessary in many areas, including environmental studies and precision farming. To solve the problem of surveying or monitoring the territory, unmanned aerial vehicles (UAVs) are often used. The results of the work of the UAV in solving the problem of surveying the territory are a reconstructed map, a threedimensional model, or a set of photographs in a certain spectrum. The generated maps are usually built from a set of overlapping geo-referenced images recorded in flight by the UAV. They contain data on the state of the crop and other important V. Lebedeva (B) · I. Lebedev St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, 39, 14th Line, 199178 St. Petersburg, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_20

315

316

V. Lebedeva and I. Lebedev

field parameters. Waypoint GPS navigation allows the UAV to independently move to certain points that are pre-set in the UAV navigation program. When surveying various areas, the UAV should follow a continuous and smooth trajectory and at the same time, avoid areas that are not of interest. Therefore, the development of algorithms that provide the construction of such trajectories for UAVs remains an urgent task at present. There are three main types of UAVs: aircraft type (planes and wings), helicopters, and multi-rotors (quadcopters, hexacopters, and octocopters) [1]. Fixed-wing aircraft has advantages in speed and altitude and is suitable for surveillance over wide areas. However, due to their high flight speed, they require high-speed sensors to take pictures [2]. In addition, fixed-wing aircraft is difficult to hold in place and take off or land in small areas. Helicopters excel in trajectory planning and maneuverability [3]. However, they require special maintenance and specialized knowledge. Multirotor UAVs have the same advantages as helicopters: They are able to hover over a point, have a wide range of flight speeds, and are easy to control [4]. Multi-rotor UAVs have a simple design, consisting of four or more engines on a cruciform frame, which also carry the main control controller, sensors, and other auxiliary modules. Position and altitude changes are achieved by controlling the speed of the motors. Multi-rotor UAVs have good maneuverability, besides, the carrying capacity of some models reaches up to 15 kg of payload. Multi-rotor UAVs do not need catapults and specially prepared platforms for takeoff and landing. Due to its high maneuverability combined with the simplicity of control methods, multi-rotor UAVs are widely used in reconnaissance missions now [5]. The goal of a flight planner when surveying a territory is to build such an UAV trajectory in order to completely cover the area of interest, i.e., it is necessary to solve the coverage problem (CPP—coverage path planning) [6]. CPP is a particular problem in robotics. The solution to the CPP problem is the planned trajectory of the robot. To solve the CPP problem, methods are used to determine the path to completely cover a certain area. The goal of a flight planner when surveying a territory is to build such an UAV trajectory in order to completely cover the area of interest, that is, it is necessary to solve the coverage problem (CPP—coverage path planning) [6]. CPP is a particular problem in robotics. The solution to the CPP problem is the planned trajectory of the robot. To solve the CPP problem, researchers use path-finding methods to completely cover a certain area. The authors from [7] classify the existing methods of trajectory planning for covering the study area in accordance with the cell decomposition technique used. Based on this classification, there are methods without workspace decomposition and methods using exact and approximate decomposition. The solution to the problem of surveying a regular-shaped area without obstacles using a single UAV is usually based on methods without decomposition of the working area. To study such areas, simple geometric patterns of movement are sufficient. The most common patterns are the back-and-forth and the spiral [8]. Missions carried out in a territory with obstacles with the help of one UAV require the decomposition of the working space, i.e., additional decomposition of a given

20 Development of an Algorithm for Coverage Path Planning for Survey …

317

Fig. 20.1 Example of an approximate decomposition

area into smaller areas—cells. The decomposition process can be applied to areas of various shapes (convex and non-convex polygon). To solve the coverage problem, two types of decomposition are used: exact cellular and approximate decomposition. Approximate decomposition involves splitting a given section depending on the obstacles located on the site, or chaotic splitting into convex polygons if the site is represented as a non-convex polygon. After splitting the section into convex polygons, the planner builds trajectories in each area using known motion patterns (reciprocating motion and spiral) [9, 10] Fig. 20.1. Such a decomposition of the working area is often applicable in multi-agent systems, where each agent is assigned its area of operation [11–13]. When using approximate decomposition in a system with one agent, it becomes necessary to additionally construct trajectories connecting all covering trajectories with each other. It is also necessary to establish the order of examination if the selected area has a complex shape. In other words, you need to find such an entrance and exit from each area in order to reduce the length of the total trajectory. This problem can be solved by optimizing the process of constructing the trajectory, for example, using a genetic algorithm [14] or an ant colony algorithm [15]. Exact decomposition is the imposition of a grid over the entire study area. Each cell is assigned a status: occupied by an obstacle (1) or free (0). And based on the available free cells, a trajectory is built that connects all the free cells of the grid [16, 17]. The cell can only be used once. An example of a precise cellular decomposition is shown in Fig. 20.2. In this work, a complete decomposition of the surveyed area is used. With it, you can examine a site of any geometric shape. Obstacles can also be of any shape. The cell size depends on the camera parameters and the frame overlap percentage or can be calculated using a predetermined formula. The construction of the trajectory takes place on a simplified representation of the surveyed area. One of the main problems associated with the movement of UAVs is the rather limited flight duration for performing missions to cover a given area during aerial photography. During maneuvers, UAVs must slow down, rotate, and accelerate, increasing the flight time and, consequently, energy consumption [18]. Therefore, it is necessary to develop a new algorithm for constructing the coverage trajectory, which

318

V. Lebedeva and I. Lebedev

Fig. 20.2 Example of a complete cellular decomposition and the found coverage path

minimizes the length of the trajectory, the flight time, or the number of maneuvers to reduce the energy consumption of the UAV. The paper discusses the problem of reducing the consumption of UAV energy resources for aerial photography of a territory with pre-marked obstacles on it. The territory is represented as a polygon with a given number of corners. To solve the problem of reducing energy consumption in a mission to cover a given area, a new algorithm for constructing a coverage trajectory based on a complete decomposition of the survey area is proposed. The algorithm takes into account the presence of obstacles on the site and also finds the shortest trajectory with a minimum number of maneuvers in order to reduce flight time. The found trajectory is smoothed in order to save the energy resource of the UAV. Also, the paper proposes to maintain the orientation of the UAV and not change course while moving along the trajectory. This further reduces the use of energy during maneuvers and increases the operating time of the UAV. The work is composed as follows: Sect. 20.1 is devoted to the description of the requirements put forward for the developed algorithm; a block diagram and description of the developed algorithm for calculating the pavement trajectory are presented in Sect. 20.2; the results of testing the developed algorithm and their discussion are in Sect. 20.3.

20.2 Requirements for the Algorithm for Constructing a Covering Trajectory Based on the review of the literature, it follows that the constructed trajectory of the UAV must completely cover the given area. Let us assume that during the movement of the UAV along the constructed trajectory, photographic material is collected. A set of images can be used for terrain reconstruction, map building, or orthomosaic. Therefore, it is necessary to define some rules for collecting photographic material.

20 Development of an Algorithm for Coverage Path Planning for Survey …

319

Consecutive images must have a specified overlap percentage. The larger the overlap, the more accurate the 3D model, map, or plan will be. The quality of a 3D model, map, or plan will be better if images of adjacent terrain are taken at the same time. Otherwise, uncorrelated shadows or visual differences may appear, resulting in a more difficult reconstruction process and poorer quality results. It is essential that all images are taken in the same orientation as this simplifies the process of 3D reconstruction and image stitching, which facilitates the search for feature points in adjacent images. Also, shooting in one orientation reduces the number of maneuvers, which leads to a decrease in the consumption of energy resources and an increase in the flight time of the UAV. We list the input data for the algorithm that solves the problem of covering a given area: 1. the number of corners of the examined area and their coordinates, 2. coordinates of takeoff and landing points. They are the start and end points, respectively, 3. the number of obstacles within the surveyed area and their location, 4. camera parameters: length (l) and width (w) of the frame. Specified horizontal (h) and vertical (v) percent overlap between images, as shown in Fig. 20.3.

Fig. 20.3 Camera parameters: length l and width w. Overlap options: h and v denote the percentage of horizontal and vertical overlap of images

320

V. Lebedeva and I. Lebedev

The output data are the found trajectory in the form of an array of point coordinates. The found trajectory must be continuous, covering the entire surveyed area. In addition, the trajectory must be smoothed and minimized in length. This is necessary to save the energy resource on board the UAV. The length of the trajectory depends on how the workspace is divided, that is, on the parameters of the camera or other scanning device, the method of smoothing the trajectory and the number of maneuvers. If the trajectory satisfies the above requirements, then the CPP problem is considered solved.

20.3 Development of an Algorithm for Constructing a Covering Path for a Polygon The most important point when using the UAV in the survey task is to reduce energy consumption in order to increase the flight time. As indicated in [19], the energy consumption can be reduced by reducing the number of turns. At a fixed altitude, the turn time for a multi-rotor UAV increases because the aircraft must come to a complete stop before moving in the other direction. When maneuvering, the UAV loses time while it slows down and accelerates after changing the course angle. Therefore, the process of decomposition of a given section is followed by the process of creating a trajectory based on reciprocating motion with a minimum number of turns that the UAV makes to cover the territory. The algorithm developed in the work is responsible for the process. Multi-rotor UAVs can move in any direction. This simplifies turns, because a multi-rotor UAV can change direction without changing its course orientation. In order to reduce the time spent on each turn, the multi-rotor type UAV will not make a heading turn. Due to the advantages of a multi-rotor type UAV, its orientation can be fixed, and in order to make a turn, it will move to the side and back, while not rotating around the vertical axis, i.e., yaw (Fig. 20.4). A feature of this solution is the orientation of images, which will be the same for all received images, facilitating the tasks of recreating a three-dimensional model, stitching images, and drawing up a terrain plan. Fig. 20.4 Example of the movement of a multi-rotor type UAV with a fixed orientation

20 Development of an Algorithm for Coverage Path Planning for Survey …

321

Figure 20.4 shows an example of the movement of a multi-rotor type UAV with a fixed orientation along the vertical axis. The orientation direction along the vertical axis is indicated by the red arrow. The black arrow indicates the direction of the UAV movement along the trajectory represented by the blue arrow. The lack of rotation around the vertical axis simplifies the maneuver of turning the UAV. The simplified maneuver does not waste the UAV’s energy resources, in comparison with the complete rotation of the UAV around the vertical axis. To create a reciprocating path, you need to know d—the distance (Fig. 20.3) between two rows. The calculation of this distance depends on the specified vertical overlap and camera parameters. Let v be the vertical overlap and w be the width of the camera frame. The row spacing, denoted as d, is the (vertical) distance between two frames. Taking into account the vertical overlap, d is calculated as follows: d = w · (1 − v). The number of turns n that must be performed in a given polygon depends on the values of d, w, and ls—the length of the optimal line sweep direction. Let us define an intermediate value z = ls − w/2, where w/2 represents half the size of the camera’s coverage area. Graphically, these parameters are presented in Fig. 20.5. Let d define the distance separating the rows. [z/d] denotes the number of rows needed to cover the polygon using the reciprocating pattern. However, if the distance between the last row and the top vertex (z mod d) is greater than w/2, the polygon is not completely covered by the last row, and another row is required. Each segment needs two pivots, resulting in a total number of pivots given by ⎧ n=

Fig. 20.5 Display of the values required to calculate the number of turns

2 · [z\d], if z mod d ≤ w/2 . 2 · ([z\d] + 1), if z mod d > w/2

322

V. Lebedeva and I. Lebedev

Fig. 20.6 Optimal a and non-optimal b line sweep direction

The number of rotations depends on z, since d is given when setting the problem. The parameter z also depends on the length of ls. This implies the conclusion that the number of turns depends on ls, and to reduce the number of turns (maneuvers), the length ls should be as short as possible. In other words, it is necessary to determine the optimal line sweep direction. As indicated in [20], the calculation of the optimal line sweep direction is a wellknown and already solved problem. There may be more than one line sweep, but the optimal one allows you to build a path that minimizes the number of turns. The following is a brief description of the method for calculating the optimal sweep direction. When working with a convex polygon, the optimal line sweep direction is calculated as follows: Initially, a segment is built from each edge of the polygon to the farthest vertex, and its length is measured. The segment with the shortest length indicates the direction of the optimal sweep (i.e., there will be fewer turns on the coverage path in this direction). The direction of the line points to the corresponding vertex. Figure 20.6 shows the optimal (minimum length) and non-optimal line sweep directions. Once the optimal line sweep direction is obtained, there are four coverage options to create a reciprocating path. The coverage option is determined according to two criteria. The first is taking into account the current direction of the line sweep or the opposite; and the second is how the coverage path is constructed: clockwise (first turn is to the right) or counterclockwise (first turn is to the left). All pavement options contain the same and minimum number of turns, because they are based on the same line sweep. Therefore, it is necessary to solve the problem of finding a coverage option that minimizes the total length of the trajectory, which is calculated as the sum of the lengths of all trajectories: D = d1 + d1 + d3 , where d 1 is the length of the trajectory from the starting point to the survey site, d 2 is the length of the trajectory from the survey site to the landing point, and d 3 is the length of the reciprocating trajectory. The following is a block diagram (Fig. 20.7) and a description of the algorithm for constructing a trajectory for covering a given area. To calculate the coverage trajectory by the developed algorithm, it is necessary to enter the following data: the number of corners of the site and their coordinates,

20 Development of an Algorithm for Coverage Path Planning for Survey …

323

Fig. 20.7 Block diagram of the algorithm for constructing the trajectory of the coating

the number of obstacles, the coordinates of obstacles, the coordinates of the takeoff point (starting point) and landing point (end point), frame size and the percentage of required overlap. Based on the known values of the camera parameters, the size of one cell is calculated. Next, you need to divide the area under examination into cells of the previously calculated size. After the decomposition of the workspace and based on the known data on obstacles, each cell is assigned its own status. If the cell is free, then its status is 0, if the cell contains an obstacle or part of it, then its status is 1. Thus, the map presentation is simplified for further UAV navigation on it. After decomposition, the direction of the optimal sweep is calculated. Knowing the sweep direction, a graph is constructed that connects the centers of all free cells. The graph has start and end nodes that have one edge, and all other nodes must have no more than two edges. Thus, the algorithm does not allow the construction of a trajectory through the same cell. The constructed graph is taken as the found trajectory. The trajectory consists of an array of cell center coordinates. Finally, the found trajectory is smoothed by cubic interpolation [21, 22]. The final smoothed

324

V. Lebedeva and I. Lebedev

trajectory is displayed on the screen using the program for creating flight missions— QGroundControl [23]. QGroundControl is an open-source software that allows you to control the flight mission of an UAV in simulation and in the real world.

20.4 Result of Operation of the Algorithm Gazebo modeling environment was chosen to conduct experiments and test the performance of the proposed algorithm. To test the proposed method, the standard UAV model “iris” was used. The experiments were carried out on a computer with the following parameters: processor Intel Core i5-8250U (1.6 GHz), memory (RAM) 8 GB DDR4-2400 MHz, video card nVidia GeForce MX 150 2 GB GDDR5, main hard drive SSD M.2 256 GB (3300 MB/s—reading from disk; 1200 MB/s writing to disk). To test the performance of the developed algorithm for constructing a covering trajectory, 6 experiments were compiled. All input data are presented in Table 20.1. In all experiments, the camera frame was represented as a rectangular cell with given sides. To simulate the UAV flight according to the calculated trajectory, the following parameters were set: UAV speed—7 m/s, flight altitude—35 m, battery charge percentage at the beginning of the flight mission—100%, minimum UAV battery charge—10%, maximum UAV flight duration 18 min. Table 20.2 shows the results of measurements of the length of the trajectory for two sections of the experiments in three cases. The first case describes the construction of the trajectory without additional calculation of the optimal sweep direction. The second case is the construction of a trajectory with the optimal sweep direction, but Table 20.1 Input data for the algorithm Experiment number

Takeoff coordinates

Landing coordinates

Survey site vertex coordinates

Territory area, m2

The Frame coordinates width w, m of the tops of the obstacles on the site

1

[0, 0]

[0, 125]

[10, 10] [100, 20] [100, 100] [40, 50] [10, 120]

10,200

–

5

2

[600, −600] [−550, 0]

[−250, 0 150, 0 150, −100 −250, − 100]

100

[500, −450] 950,000 [500, 500] [−500, 500] [−500, − 450]

20 Development of an Algorithm for Coverage Path Planning for Survey …

325

Table 20.2 Experimental results Parameter: path length Experiment number

Trajectory length without calculating the direction of the optimal sweep, m

Path length without smoothing, m

Trajectory length with optimal sweep direction and smoothing, m

1

2720

2680

2490

2

9850

9531

9300

Parameter: flight time Experiment number

Flight time along the Time of flight along the trajectory without trajectory without calculating the direction smoothing of the optimal sweep

Time of flight along the trajectory with optimal sweep direction and smoothing

1

7 min 52 s

6 min 24 s

5 min 20 s

2

15 min 51 s

15 min 6 s

14 min 12 s

Parameter: percentage of charge after flight along the constructed trajectory Experiment number

Percentage of battery charge after flight along the trajectory without calculating the direction of the optimal sweep, %

Percentage of battery charge after flying on a non-smoothed trajectory, %

Percentage of battery charge after flying a path with optimal sweep direction and smoothing, %

1

82

84

88

2

26

30

32

without smoothing. The last case illustrates the length of the final trajectory with the calculated optimal sweep direction and smoothing. To evaluate the results of the experiments, three parameters were chosen: the length of the trajectory, the flight time along the trajectory, and the percentage of battery charge at the end of the flight mission. For comparison, the results of experiments were selected, where the trajectory was built without the optimal direction of sweep and smoothing, and where the trajectory was built by the algorithm developed in the work. Comparing the results of experiments, it was possible to establish that, using the developed algorithm, the length of the trajectory in the first section was reduced by 9%, and in the second by 6%, compared with the length of the trajectory constructed without calculating the optimal direction of sweep and smoothing. Also, using the developed algorithm for constructing the coverage trajectory, the flight time in the first section was reduced by 30%, and in the second—by 10%. The percentage of battery charge after flying on two legs increased by 7% and 19%, respectively. The obtained results indicate that the developed algorithm really reduces the energy consumption during the flight of the UAV. The reduction is achieved due to the presence of a block for calculating the optimal direction of the sweep and smoothing the trajectory. Smoothing the trajectory allows the UAV to maintain a constant speed. And also reducing the number of maneuvers reduces the power consumption of the

326

V. Lebedeva and I. Lebedev

Fig. 20.8 Visualization of trajectories for the first experiment: a, b, c—the result of the construction of trajectories; d, e, f—the actual flight path taken from the log files

UAV and increases the duration of the aircraft in the air. Figure 20.8 shows examples of calculating trajectories for the first section. Figure 20.8a–c show the result of the developed algorithm for constructing a covering trajectory. Figure 20.8a illustrates an example of calculating the coverage trajectory without calculating the direction of the optimal direction. Figure 20.8b shows the trajectory with the calculation of the optimal sweep direction. It can be visually seen that the number of turns has decreased. Figure 20.8c shows the trajectory with the calculated optimal sweep direction and smoothing. Figure 20.8d–f show the actual flight trajectories corresponding to the calculated trajectories in Fig. 20.8a–c. The actual trajectories were obtained using Gazebo’s UAV motion simulation. The calculated trajectories recorded in a file were sent to the UAV model’s pixhawk flight controller in Gazebo. All flights are recorded in the flight log. The actual trajectories were visualized through PX4 Flight Review (Fig. 20.8d–f). Figure 20.9 is a screenshot from the simulation. Figure 20.9 shows the simulation environment where the flight simulation took place. The figure also shows the trajectory along which the UAV moves. The trajectory is calculated by the developed algorithm. This trajectory is built using the optimal sweep direction and is also smoothed by cubic interpolation.

20 Development of an Algorithm for Coverage Path Planning for Survey …

327

Fig. 20.9 UAV flight simulation along the coverage trajectory

The results of experiments with the UAV model prove that the calculated trajectory satisfies all the requirements put forward to reduce energy consumption during aerial photography of a given area. Namely, the trajectory constructed by the developed algorithm is continuous, has a minimum number of turns, is smoothed, optimal in length, takes into account the presence of obstacles, and is also passable for a multi-rotor UAV.

20.5 Conclusion A flight mission for a UAV to cover a given area is often encountered in territory survey tasks. To perform such flight missions in the shortest possible time, it is necessary to develop an algorithm that calculates trajectories with a minimum length and a minimum number of maneuvers. The development of an algorithm for constructing a coating trajectory is related to the issue of energy consumption, since the energy reserve on board the UAV is limited. The algorithm for constructing the coverage trajectory developed and presented in this paper reduces the number of maneuvers by calculating the sweep direction. Also, the number of maneuvers is reduced due to the constant orientation of the multitorque type UAV. Smoothing the constructed trajectory allows the UAV to keep the speed constant. The aircraft no longer needs to slow down and pick up speed at designated turn points. This aspect also allows you to save the energy resource of the UAV and increase the duration of the flight mission in the air. To confirm the effectiveness of the developed algorithm, experiments were carried out in two areas. For each section, the coverage trajectory was calculated without the optimal sweep, the coverage trajectory with the calculation of the optimal sweep, and the smoothed trajectory. As a result of the comparison, it was found that the path length on the two legs was reduced by 9 and 6%, the flight time was reduced by an average of 30% and 10%, respectively, and the battery charge at the end of the missions increased by 7 and 19%. These results show that the constructed

328

V. Lebedeva and I. Lebedev

optimal trajectory increases the operating time of the UAV in the air. Therefore, the results obtained indicate the effectiveness of the developed algorithm for covering a given area. Also, the advantage of the developed algorithm is its use in areas with pre-marked obstacles.

References 1. Watts, A.C., Ambrosia, V.G., Hinkley, E.A.: Unmanned aircraft systems in remote sensing and scientific research: classification and considerations of use. Remote Sens. 4(6), 1671–1692 (2012) 2. Coombes, M., Fletcher, T., Chen, W.H., Liu, C.: Decomposition-based mission planning for fixed-wing UAVs surveying in wind. J. Field Robot. 37(3), 440–465 (2020) 3. Yuan, Y., Chen, R., Thomson, D.: Propeller design to improve flight dynamics features and performance for coaxial compound helicopters. Aerosp. Sci. Technol. 106, 106096 (2020) 4. Sandino, J., Maire, F., Caccetta, P., Sanderson, C., Gonzalez, F.: Drone-based autonomous motion planning system for outdoor environments under object detection uncertainty. Remote Sens. 13(21), 4481 (2021) 5. Olson, K.G., Rouse, L.M.: A beginner’s guide to mesoscale survey with quadrotor-UAV systems. Adv. Archaeol. Pract. 6(4), 357–371 (2018) 6. Nam, L.H., Huang, L., Li, X.J., Xu, J.F.: An approach for coverage path planning for UAVs. In: IEEE 14th International Workshop on Advanced Motion Control (AMC), pp. 411–416 7. Choset, H.: Coverage for robotics—a survey of recent results. Ann. Math. Artif. Intell. 31(1), 113–126 (2001) 8. Cabreira, T.M., Brisolara, L.B., Ferreira, P.R., Jr.: Survey on coverage path planning with unmanned aerial vehicles. Drones 3(1), 4 (2019). https://doi.org/10.3390/drones3010004 9. Torres, M., Pelta, D.A., Verdegay, J.L., Torres, J.C.: Coverage path planning with unmanned aerial vehicles for 3D terrain reconstruction. Exp. Syst. Appl. 55, 441–451 (2016) 10. Pham, T.H., Bestaoui, Y., Mammar, S.: Aerial robot coverage path planning approach with concave obstacles in precision agriculture. In: Workshop on Research, Education and Development of Unmanned Aerial Systems (RED-UAS), pp. 43–48 (2017) 11. Muñoz, J., López, B., Quevedo, F., Monje, C.A., Garrido, S., Moreno, L.E.: Multi UAV coverage path planning in urban environments. Sensors 21(21), 7365 (2021) 12. Acevedo, J.J., Arrue, B.C., Maza, I., Ollero, A.: Distributed approach for coverage and patrolling missions with a team of heterogeneous aerial robots under communication constraints. Int. J. Adv. Rob. Syst. 10(1), 28 (2013) 13. Albani, D., Nardi, D. Trianni, V.: Field coverage and weed mapping by UAV swarms. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4319–4325 (2017) 14. Pehlivanoglu, Y.V., Pehlivanoglu, P.: An enhanced genetic algorithm for path planning of autonomous UAV in target coverage problems. Appl. Soft Comput. 112, 107796 (2021) 15. Famouri, M., Hamzeh, A.: A new complete heuristic approach for ant rendezvous problem. In: 4th International Conference on Computer and Knowledge Engineering (ICCKE), pp. 364–369 (2014) 16. Nam, L.H., Huang, L., Li, X.J., Xu, J.F.: An approach for coverage path planning for UAVs. In: IEEE 14th International Workshop on Advanced Motion Control (AMC), pp. 411–416 (2016). https://doi.org/10.1109/AMC.2016.7496385 17. Cho, S.W., Park, H.J., Lee, H., Shim, D.H., Kim, S.Y.: Coverage path planning for multiple unmanned aerial vehicles in maritime search and rescue operations. Comput. Ind. Eng. 161, 107612 (2021) 18. Di Franco, C., Buttazzo, G.: Coverage path planning for UAVs photogrammetry with energy and resolution constraints. J. Intell. Rob. Syst. 83(3), 445–462 (2016)

20 Development of an Algorithm for Coverage Path Planning for Survey …

329

19. Li, D., Wang, X., Sun, T.: Energy-optimal coverage path planning on topographic map for environment survey with unmanned aerial vehicles. Electron. Lett. 52(9), 699–701 (2016) 20. Huang, W.H.: Optimal line-sweep-based decompositions for coverage algorithms. In: Proceedings ICRA of IEEE International Conference on Robotics and Automation, vol. 1, pp. 27–32 (2001) 21. Nam, L.H., Huang, L., Li, X.J., Xu, J.F.: An approach for coverage path planning for UAVs. In: IEEE 14th International Workshop on Advanced Motion Control (AMC), pp. 411–416 (2016) 22. Wang, C., Liu, P., Zhang, T., Sun, J.: The adaptive vortex search algorithm of optimal path planning for forest fire rescue UAV. In: IEEE 3rd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), pp. 400–403 (2018) 23. Ramirez-Atencia, C., Camacho, D.: Extending q ground control for automated mission planning of UAVs. Sensors 18(7), 2339 (2018)

Chapter 21

Computer Vision System of Robot Control for Monitoring Objects in Radioactive Areas Andrei Malchikov

and Petr Bezmen

Abstract Computer vision system (technical vision) is a modern technology integrated into the general structure of industrial automation that contributes to obtaining information about the properties of the observed objects. Computer vision plays a special role in a robotic systems used in unacceptable conditions for humans, such as radiation. The authors consider aspects of the using of in-pipe crawling robot equipped with a computer vision system inside the technological cavities with a potentially increased level of ionized radiation. The paper presents the algorithm to scan objects inside cavities with the use of the computer vision system with controlled camera orientation. This algorithm makes it possible to conduct the pipe diagnostics in conditions of increased background radiation. The mathematical model of the robot, the structure of the control system, and the image recognition implementation of a scanned object are presented. The results of numerical simulation confirmed the effectiveness of the use of the proposed algorithms of the robot control system when scanning objects inside a pipe. The authors propose the using of robot links for orientation of diagnose equipment relative to a scanned object. The paper discusses that the information from the computer vision system is used as feedback for the robot control.

21.1 Introduction Currently, in all industrialized countries, intensive work is underway to create mobile robots for moving and performing of technological and inspection operations in places which are inaccessible or hard to reach for humans, as well as in areas with radioactive environment that is not safe for a human. Wheeled, caterpillar, walking devices are widely used, however, the further expansion of the scope of mobile devices, primarily in pipelines, including those filled with some mud, requires the creation of fundamentally new vehicles without any protruding part and equipped A. Malchikov (B) · P. Bezmen Southwest State University (SWSU), Kursk, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_21

331

332

A. Malchikov and P. Bezmen

with a sealed body [1–3]. Automation of visual control of the inner surface of pipes is an urgent task, since in practice, an operator does the visual inspection is subject to physical and emotional fatigue, his work depends on experience, therefore, automation of visual control, as a means of eliminating subjectivity and increasing accuracy, is an important aim [4–5]. One of the promising methods of inspection automation is the use of robots with the so-called bionic mode of movement based on peristaltic crawling principles, like worms or caterpillars [6–9]. Such robots can act as a carrier platform for a computer vision system, which will ensure automatic or semi-automatic operation of a robotic device that is integrated into the overall structure of industrial automation and facilitates the obtaining information about the properties of observed objects. In this paper, we consider the actual problem of stage-by-stage inspection of objects on the pipeline inner surface with the use of technical vision tools.

21.2 Scheme and Principle of the Mobile Robot Operation As a transport platform, the six-link mobile robot with a peristaltic principle of movement inside a limited space of pipelines is considered in the paper. The mobile robot consists of six articulated body parts 1, 2, 3, 4, 5, 6 (Fig. 21.1). Hinges 7, 8, 9, 10, 11 with electric drives 12, 13, 14, 15, 16 are installed between the links, providing angular displacement of the links-housings 1–6 relative to each other. In hinges 7–11 and at the ends of links 1 and 6, there are support elements 17, 18, 19, 20, 21, 22, 23 with built-in contact sensors. The contact of the support elements 17, 18, and 19 is achieved by turning the housing links in the hinges 7, 8, 9, 10, 11 under the action of the electric drives 12, 13, 14, 15, 16. In this case, the moment of the support elements’ contact is registered by built-in contact sensors and transmitted to the robot control system. The attached implements (diagnostic equipment, for example, camera) can be placed both on the intermediate links and on the extreme links of the device. The device moves by alternately wedging and moving the front and rear pairs of links. In detail, the

Fig. 21.1 Multi-link robot design scheme

21 Computer Vision System of Robot Control for Monitoring Objects … Fig. 21.2 Diagnostic equipment orientation inside the pipe relative to the scanned object

333

(a)

(b)

construction and the principle of movement are described in [3]. The most stable position of the robot is one in which three links are wedged at once, providing reliable fixation of the robot inside the pipe, while the rest of the robot body not involved in holding can be considered as a sequential manipulator orienting the attached equipment inside the pipeline. At the same time, the redundancy of such a manipulator allows to diagnose the point objects inside the pipe from several positions, with the aim to obtain a more complete picture of the object under study (Fig. 21.2). The obtaining data on the location, size, and shape of objects inside the pipe using a computer vision system can be applied to more accurate diagnostic tools. Thus, in-pipeline monitoring can be divided into separate stages: initial detection, identification and determination of geometric parameters, additional diagnostics– interaction with the object. This phased approach will improve the energy efficiency of monitoring (due to the targeted use of energy-intensive tools) and increase the speed of diagnostics (identification of object parameters can proceed during the movement of the robot). An important role in performance of diagnostics is played by the implementation of a tracking system for controlling the position of the robot’s links relative to some desired object. A feature of such control system is the implementation of feedback using recognition algorithms and computer vision methods.

21.3 Mathematical Model and Principle of the Control System Operation To study the system operation, the mathematical model of the robot was developed. This model is for the mode of the pipe surface scanning (the diagnostic mode). Thus, we consider the scheme shown in Fig. 21.3. In the scheme (Fig. 21.3), the following designations are adopted: 1–3 are movable links, due to which the computer vision (CV) orientation is carried out relative to the object for scanning; Oi , (i = 1 . . . 4) is the ends of links; Ci , (i = 1 . . . 3) is the mass centers of links; Mi , (i = 1 . . . 3) is the robot electric drives’ torques;

334

A. Malchikov and P. Bezmen

Fig. 21.3 Mathematical model scheme of a six-link robot in the diagnostic mode

m i g, (i = 1 . . . 3) is gravity forces acting on the robot links; ϕi,i+1 , (i = 0 . . . 2) is absolute angles of the links’ rotation; xi Ci yi , (i = 1 . . . 3) is the moving coordinate systems. We take the vector of generalized coordinates as q = |ϕ1 , ϕ2 , ϕ3 |T and use the Lagrange’s equations of second kind. Hence, it appears that the system of differential equations which describes the robot dynamics can be written in the following form: ˙ + G(q) = Q, M(q)q¨ + V (q, q) ⎡ ⎢ M(q) = ⎣

m 1 l12 + (m 2 + m 3 )l12 3 (m 2 +2m 3 )l1 l2 cos(φ2 − φ1 ) 2 m 3 l1 l3 cos(φ3 − φ1 ) 2

( m2 2

) + m 3 l1l2 cos(φ1 − φ2 ) m 2 l22 + m 3l22 3 m 3 l2 l3 cos(φ3 − φ2 ) 2

m 3 l1 l3 2 m 3 l2 l3 2

⎤ cos(φ1 − φ3 ) ⎥ cos(φ2 − φ3 ) ⎦, m 3 l32 3

) ( ⎤ φ˙ 22 m22 + m 3 l1l2 sin(φ1 − φ2 ) + φ˙ 32 m 3 l12l3 sin(φ1 − φ3 ) V (q, q) ˙ = ⎣ φ˙ 12 (m 2 + 2m 3 ) l12l2 sin(φ2 − φ1 ) + φ˙ 32 m 3 l22l3 sin(φ2 − φ3 ) ⎦, φ˙ 12 m 3 l12l3 sin(φ3 − φ1 ) + φ˙ 22 m 3 l22l3 sin(φ3 − φ2 ) ) ⎤ ⎡ ⎤ ⎡ ( M1 − m 3( + m 2 + )m21 gl1 cos φ1 G(q) = ⎣ − m 3 + m22 gl2 cos φ2 ⎦, Q = ⎣ M2 ⎦. M3 −m 3 g l23 cos φ3 ⎡

The controlled movement of the three-link mechanism is achieved by generating the required torques Mi , (i = 1 . . . 3) of the robot drives. In this case, the torques are defined as Mi = k G cm ηi i , where k G and η are the total gear ratio and the efficiency of the mechanical transmission, respectively, cm is the torque coefficient of the motor, i i is the currents

21 Computer Vision System of Robot Control for Monitoring Objects …

335

Fig. 21.4 Block diagram of the control system

determined by the Kirchhoff equation: L

di i + ri i + ce φ˙ i k G = Ui , dt

where L and r are the inductance and resistance of the motor windings, respectively, ce is the speed coefficient of the motor. The supply voltages Ui are generated by the robot control system, the block diagram of which is shown in Fig. 21.4. In Fig. 21.4, the following designations are adopted: IK is the block for solving the inverse problem of kinematics; LB is the decision block that can implement the controller according to the robot link deviation angle or according to the coordinates of the object center in the frame depending on current mode of the control system; Ri (i = 1 . . . 3) is the controllers of relative position of the links ϕi,i+1 , (i = 0 . . . 2); Acti (i = 1 . . . 3) is the electric drives of the links; Enci (i = 1 . . . 3) is link rotation angle sensors; Cam is video camera; CV is the image recognition system which receives the coordinates of the center and the dimensions of the rectangle that encloses the image of an object. The internal control loops of the relative position of the links are formed based on the readings of the position sensors as follows: ( ∗ ) ( ∗ ) Ui = k p φi−1,i − φi−1,i + kd φ˙ i−1,i − φ˙ i−1,i , where k p and kd are the proportional and differential coefficients of the PD position controller, respectively. ∗ The required rotation angles φi−1,i (i = 1 . . . 3) of the links can be formed as a result of solving the inverse problem of kinematics with trajectory control, when the required laws of the equipment position change relative to the fixed robot part are

336

A. Malchikov and P. Bezmen

programmed; at the same time, the information about the camera position relative to the object is not used. This mode can be used directly for detecting objects, or for semi-automatic video shooting, when the camera orientation is directly controlled by the operator. The obvious disadvantage of this approach is the need for direct participation of an operator in the process of object scanning, as well as the associated errors. The use of automatic monitoring will not only exclude an operator from the scanning process but also increase the accuracy of the procedure. In this case, the control problem is to ensure the required orientation of the camera or other equipment located on the extreme link of the robot relative to the visual object at a known position of the fixed part of the robot. A feature of the control system is that the feedback channel data ensure the camera orientation relative to the scanned object and are formed by the CV methods. Let us consider the object scanning algorithm using the example shown in Fig. 21.5. As can be seen from Fig. 21.5, the information about location and properties of an object in the frame can be used to obtain the true shape of this object and to aim the diagnostic equipment at an object. With the use of the information about the robot position relative to an object and the links’ rotation angles, we can implement various scanning algorithms, including the one in which an object must remain in the camera frame center when the camera moves along a given trajectory. In this case,

Fig. 21.5 Options of the robot links’ location during the scanning procedure

21 Computer Vision System of Robot Control for Monitoring Objects …

337

Fig. 21.6 General block diagram of the decision block

the robot can be represented as a flat two-link mechanism, the free apex of which moves along a given trajectory, and the third link is oriented toward an object. The general block diagram of the decision block is presented in Fig. 21.6. An important problem is to ensure the camera orientation accuracy relative to an object and relative to the robot part fixed in a pipe. It can be achieved by optimization of the controller parameters via numerical simulation of the system operation.

21.4 Formation of the Data About a Studied Object Let us consider the algorithm of the data formation with CV in detail. In this case, the recognition task is reduced with aim to find the object contour and determine the object properties. In this paper, it is proposed to use the Viola-Jones method and the Haar features [10]. As features (masks) we use the set of geometric shapes with a black and white pattern. These masks help to define the boundaries of an object, such as: a contaminant accumulation, a crack, a rupture, or a deformed section of a pipe. The essence of the Viola-Jones method is to apply masks to different parts of the frame received from the camera. Next, the program determines whether the desired object can be in the frame. The proposed algorithm involves tuning a classifier that searches for an object in the frame and setting of the recognition threshold values. The classifier tuning is achieved by training on the images of the desired object (areas of increased deterioration, ruptures, cracks, etc.). The recognition threshold value is responsible for signaling about the presence of the desired object in the camera frame. During CV operation, an image from the robot’s video camera is transmitted to the classifier. The classifier imposes a mask on the frame and separately adds the brightness of the pixels which fall into the mask white part and the brightness of the pixels which fall into the mask black part. Next, the difference between the first and second values is searched. The result is compared with the previously set threshold. If the result is less than threshold value, it means that there is no desired object in the studied part of the frame, and the algorithm ends its work. Otherwise, if the result is greater than threshold value, the classifier moves to the next part of the frame. When the recognition algorithm is functioning, we assign a set of characteristics to each object: shape (it often determines the nature of the object), size in the frame (if

338

A. Malchikov and P. Bezmen

we know the distance to some object, we can estimate the object real size), location in the frame (these data are used in the control loop). The recognition program is based on the library of computer vision and image processing algorithms OpenCV [11]. During CV operation, the image from the camera is converted into a binary format. Then, the CV kernel matrix is defined. After pre-processing of the frame, CV searches for the desired object which corresponds to the features loaded into the classifier. The result of the output is the upper left point coordinate of the square that frames the studied object in the image, as well as width and height of the square. The information about the location and size of the object is used to form an error of the control system. In that operation mode, when the camera moves along a certain trajectory, and the camera must be oriented to the scanned object, an important role is played by the algorithm for determining the rotation angle error. If we know the distance to the object and the displacement of the object center relative to the frame center, we can calculate the required rotation angle of the robot link with the camera as follows: ϕ3 = arctan(Δy/s), where Δy is the displacement of the recognized object center relative to the frame center, s is the distance to the object.

21.5 Mathematical Modeling of the Control System in the Object Scanning Mode To test the performance of the proposed algorithms and to adjust the coefficients of the controller, numerical simulation was performed via solving of the system of equations that describes the robot controlled motion dynamics. The parameters of the mathematical model are shown in Table 21.1. The modeling task is as follows. Let us set the point O3 trajectory in the form of a circle fragment: x O3 = A x cos(wt) + Δ A x y O3 = A y sin(wt) + ΔA y , } { where b = A x , A y , w, ΔA x , Δ A y is the vector of parameters specifying the trajectory fragment, for the modeled case the parameters are b = {−0.14, 0.14, 0.3, 0.35, 0}. Other ways to set the motion trajectory, as well as the method for solving of the inverse problem of kinematics used in the work, are described in detail in [12]. The problem of the end link O3 O4 drive control (see Fig. 21.3) lies in ensuring of an orientation of the link with the camera: In this orientation, the camera is directed to the center of the object, i.e., regardless of the movement of other robot links. To obtain a mathematical model of the feedback channel, experimental studies of the object recognition system functioning were carried out inside a pipeline fragment (Fig. 21.7).

21 Computer Vision System of Robot Control for Monitoring Objects … Table 21.1 Parameters of the mathematical model

Parameter name

Designation

339 Value

Mass of each link

m i (i = 1 . . . 3) 0.18 kg

Length of each link

li (i = 1 . . . 3)

0.21 m

Mass of the camera

–

0.07 kg

η

0.75

The parameters of each drive – reducer efficiency – total gear ratio

kG

220

– motor torque coefficient

cm

8.11 mN m/A

– motor speed coefficient

ce

1180 rpm

– motor winding inductance

L

0.113 mH

– motor winding resistance

r

2.83 Ω

Fig. 21.7 Experimental studies of the recognition system: a original image from the camera, b processed image

During the experiment, the camera moved relative to the object, and the angular deviation value was calculated by the feedback channel and was recorded. The recognition error is shown in Fig. 21.8. According to natural experiments, the maximum error of the angular deviation recognition is 0.62°, and the average value is 0.12°. It is obvious that the distribution

Fig. 21.8 Recognition error plot

340

A. Malchikov and P. Bezmen

of the error has a random character. On the basis of the obtained data, we form a function that simulates a feedback channel. Let us show a set of the mechanism positions obtained during the trajectory motion modeling of the two-link mechanism with the orientable link equipped with the camera. For the trajectory shown in Fig. 21.9, we obtain the time diagrams of the links’ relative rotation angles (Fig. 21.10). As can be seen from the simulation results, there is a static error due to the action of gravity force, as well as minor fluctuations at the beginning of the movement. Let us estimate the influence of the positioning error of the camera’s link attaching point on the positioning error of the camera relative to the scanned object (Fig. 21.11). The presence of a random error change in the feedback channel obtained due to object recognition inserts an oscillatory component into the link movement; however, the error value is less than 1.5°, which allows to use the proposed algorithm to implement the controlled robot movement for the scanning of objects.

Fig. 21.9 Positions of the mechanism during the simulation: 1 is the two-link positioning mechanism, 2 is the trajectory of the point O3 , 3 is the link equipped with the camera, 4 is the scanned point object

21 Computer Vision System of Robot Control for Monitoring Objects …

341

Fig. 21.10 Time diagrams of the links’ relative rotation angles: the desired and actual angles during moving along the trajectory, rad

Fig. 21.11 Deviations of the camera angular position relative to the scanned object

21.6 Conclusion Monitoring of the industrial infrastructure facilities communications state with the help of robots equipped with a computer vision system is showing great promise. Within the paper, the algorithm to scan objects inside pipelines with the use of a multi-link mobile robot is proposed. The algorithm makes it possible to conduct the pipe diagnostics in conditions of increased background radiation. The use of robot links for orientation of diagnose equipment relative to a scanned object is proposed. Furthermore, the information from the computer vision system is used as feedback for the robot control. The developed mathematical model of the in-pipe robot facilitated to carry out the numerical simulation that is proved the applicability of the proposed technique for in-pipe monitoring under radiation conditions.

342

A. Malchikov and P. Bezmen

References 1. Fjerdingen, S.A., Liljebäck, P., Transeth, A.A.: A snake-like robot for internal inspection of complex pipe structures (PIKo). In: 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE (2009) 2. Hopkins, J.K., Spranklin, B.W., Gupta, S.K.: A survey of snake-inspired robot designs. Bioinspiration Biomimetics 4(2), 021001 (2009) 3. Jatsun, S., Loktionova, O. Malchikov, A.: Six-link in-pipe crawling robot. In: Advances on Theory and Practice of Robots and Manipulators, pp. 341–348. Springer, Cham (2014) 4. Muramatsu, M., Namiki, N., Koyama, R., Suga, Y.: Autonomous mobile robot in pipe for piping operations. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No. 00CH37113), vol. 3 (2000) 5. Rizzini, D.L., Kallasi, F., Aleotti, J., Oleari, F., Caselli, S.: Integration of a stereo vision system into an autonomous underwater vehicle for pipe manipulation tasks. Comput. Electr. Eng. 58, 560–571 (2017) 6. Saga, N., Nakamura, T.: Development of a peristaltic crawling robot using magnetic fluid on the basis of the locomotion mechanism of the earthworm. Smart Mater. Struct. 13(3), 566 (2004) 7. Carbone, G., Malchikov, A., Ceccarelli, M., Jatsun, S.: Design and simulation of Kursk robot for in-pipe inspection. In: SYROM 2009, pp. 103–114. Springer, Dordrecht (2010) 8. Darintsev, O., Migranov, A.: Analytical review of approaches to the distribution of tasks for mobile robot teams based on soft computing technologies. Inform. Autom. 21(4), 729–757 (2022). https://doi.org/10.15622/ia.21.4.4 9. Jatsun, S., Vorochaeva, L., Yatsun, A., Savin, S., Malchikov, A.: Bio-inspired adaptive control strategy for a snake-like robot. In: 2015 19th International Conference on System Theory, Control and Computing (ICSTCC) (2015) 10. Peleshko, D., Soroka, K.: Research of usage of Haar-like features and AdaBoost algorithm in Viola-Jones method of object detection. In: 2013 12th International Conference on the Experience of Designing and Application of CAD Systems in Microelectronics (CADSM) (2013) 11. Bradski, G., Kaehler, A.: OpenCV. Dr. Dobb’s J. Softw. Tools 3, 120 (2000) 12. Jatsun, S., Malchikov, A., Yatsun, A., Leon, A.S.M.: Simulation of a walking robot-exoskeleton movement on a movable base. J. Artif. Intell. Technol. 1(4), 207–213 (2021)

Chapter 22

Method for Searching Deployment Zones of Ground Seismic Sensors by a Heterogeneous Group of UAVs in an Environment with a Complex Topography Roman Iakovlev , Valeria Lebedeva , Ivan Egorov , Vitaly Bryksin, and Andrey Ronzhin Abstract In this paper, the problems of automation and robotization of field seismic operations using unmanned aerial vehicles (UAVs) are discussed. An original method of aerial monitoring of terrain by a group of UAVs for the purpose of searching deployment zones for ground seismic sensors by a heterogeneous group of UAVs in an environment with a complex topography is proposed. The method consists of several main stages: the construction of an orthophotomap of the observed terrain using a group of fixed-wing UAVs, obtaining the target scheme of seismic sensors placement indicating the boundaries of potential regions of interest, building a local altitude map for each region of interest using a group of multirotor UAVs, and searching for the zone for seismic sensor placement on each of the given regions of interest. According to the results of the developed method approbation, the averaged over the regions of interest share of the successful missions of searching deployment zones for seismic sensors was 0.746, which illustrates the high efficiency of the proposed solution. In addition, the work revealed an inverse dependence of the share of successful searching results on the altitude of the UAV during the flight mission.

R. Iakovlev · V. Lebedeva (B) · A. Ronzhin St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, 39, 14Th Line, 199178 St. Petersburg, Russia e-mail: [email protected] I. Egorov LLC R-Sensors, 4/1, Likhachevsky Proezd, Dolgoprudny, 141701 Moscow Region, Russia e-mail: [email protected] V. Bryksin Immanuel Kant Baltic Federal University (IKBFU), 14, Aleksandra Nevskogo Str., 236041 Kaliningrad, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_22

343

344

R. Iakovlev et al.

22.1 Introduction Automation and robotization of field seismic operations during geodetic surveys contribute to the reduce of the preparation activities and costs of exploration, minimize damage to the environment, and create more comfortable conditions for service personnel [1]. One of the topical problems in this domain is searching and identification of zones of seismic sensors deployment when conducting experiments in a nondeterministic environment. This problem becomes most relevant when conducting seismic studies in complex natural-climatic conditions: in tundra, taiga, mountainous areas, etc. [2]. One of the promising directions in solving the above problem is the use of groups of unmanned aerial vehicles (UAV), capable of analyzing the structural and geometric properties of the external environment through the use of depth cameras, as well as sensors of the visible spectrum [2, 3]. Within the considering application, the use of geophones based on molecularelectronic technology as seismic sensors seems especially promising [4, 5]. These seismic sensors fully satisfy the requirements for operation in conditions of extreme temperature range [6, 7], they are notable for increased resistance to overloading during transportation and installation on the ground. Also, an important distinguishing characteristic of seismic sensors based on molecular-electronic technology is preservation of operability at any angle of inclination from the vertical. The development of a method for searching deployment zones for seismic sensors by a heterogeneous group of UAVs involves solving the following group of problems: planning of UAV trajectories, both at the stage of building a general orthophotomap of the observed area and during the analysis of the spatial characteristics of the regions of interest; assessment of spatial characteristics and construction of local altitude maps of the observed regions by UAVs. The work contains the following sections: Sect. 22.1 is devoted to a review of current research in the field of robotization of seismic works; Sect. 22.2 contains a detailed description of the proposed method of aerial monitoring of terrain by a group of UAVs in order to find the deployment zones for ground seismic modules; Sect. 22.3 describes the experiments and results of testing the developed method.

22.2 Related Works There are a significant number of methods for planning the trajectories of a group of robotic agents that provide complete coverage of a given terrain [8, 9]. The vast majority of current trajectory planning strategies are based on the concept of cellular decomposition of the workspace. These strategies are based on dividing the workspace into square-shaped sectors. In each sector, the UAVs follow a pattern of reciprocating motion [10]. The centers of the sectors are represented as nodes in a contiguity graph, in which the edges connect the nodes if the sectors share a common boundary with each other. The problem of full coverage in this case is solved

22 Method for Searching Deployment Zones of Ground Seismic Sensors …

345

by performing a search over the graph to determine the order of sectors. The classic ways to optimize trajectories within these strategies are the choice of sector survey order [11–14], as well as the choice of sweep direction in each individual sector [15], which has a positive effect on the aggregate length of the routes, the duration of the flight missions, and the energy efficiency of the UAV [16]. The use of UAV groups in the full coverage strategies allows to obtain additional advantages expressed in the reduction of the total mission execution time due to the division of the workload, as well as in the increased fault tolerance of such solutions, provided by the possibility of reassigning sectors between agents in case of non-staff situations [17]. In the case of a group of robotic agents, the problem of planning full coverage trajectories is often formulated as the well-known vehicle route selection problem (VRP). In general, VRP is the problem of finding a set of routes to be assigned to a group of vehicles [18]. Converting the full coverage problem to VRP is usually done by constructing a contiguous graph such that coverage is achieved when robotic agents visit the full set of nodes or edges of the corresponding graph at least once [19]. Thus, in this work, the planning of trajectories of a group of UAVs is going to be implemented within the concept of cellular decomposition [15] in order to achieve complete coverage of the observed area by robotic agents in minimum time. In addition, in order to further increase the efficiency of trajectory construction, the strategy of selecting the optimal sweep direction will be additionally involved. Solving the task of building a local altitude map of the observed area by UAV means, it is necessary to dynamically estimate the spatial position of each section of the area relative to the current position of the agents. If the lenses of fixing video cameras are rigidly fixed on the UAV, the estimation of the spatial position of the area’s section can be performed relative to the camera. In general case, the estimation of spatial position of some part of three-dimensional scene relative to camera lens can be realized on the basis of depth map data of the corresponding scene. Well-known methods for obtaining depth map can be mainly referred to two main categories: hardware-based methods [20–22] and methods based on neural network models [23, 24]. The main disadvantage of the methods based on neural network models is a relatively low level of accuracy in the task of depth maps reconstruction in comparison with hardware methods. It should also be noted that methods of depth map creation based on neural network models do not allow to obtain distances from camera lens to scene sections in metric units, so they are not applicable for spatial position estimation of three-dimensional scene sections under study. The RealSense D435 depth camera is used in this study, which provides high accuracy in comparison with analogs [22], and also allows to estimate distances from the camera lens to the observed areas’ sections in metric units. Thus, within this study, the assessment of the spatial positions of the areas’ sections for the purpose of constructing a local altitude map of the observed territory is planned to be carried out using RGB-D images obtained by using UAVs equipped with RealSense D435 hardware depth sensors. Next, let us consider the proposed method of searching deployment zones for ground seismic sensors by means of a heterogeneous group of UAVs.

346

R. Iakovlev et al.

22.3 Method of Searching Deployment Zones for Ground Seismic Sensors by Means of a Heterogeneous Group of UAVs To solve the problem of searching deployment zones for ground seismic sensors by means of a heterogeneous group of UAVs, we propose a method consisting of the following stages: 1. Construction of an orthophotomap of the observed area using a group of fixedwing UAVs: (a) Area decomposition according to the strategy proposed in [15]; (b) Construction of the full coverage trajectories for a group of UAVs in terms of vehicle routing problem (VRP); (c) Construction of orthophotomap of the area on the basis of video data from robotic agents using SIFT method [25]. 2. Obtaining the target scheme of seismic sensors placement indicating the boundaries of potential regions of interest. 3. Constructing of the local altitude maps for each region of interest using a group of multirotor UAVs: (a) Decomposition of the observed area according to the strategy proposed in [15]; (b) Constructing of the full coverage trajectories for a group of UAVs in terms of vehicle routing problem (VRP); (c) Construction of the local altitude map of the region of interest based on RGB-D sensor data. 4. Searching for the zones to place seismic modules on each of the regions of interest. As shown above, the proposed method contains four key stages, each of which is aimed at solving a separate group of subtasks. Let us consider them in more details. The first stage is focused on the construction of a general orthophotomap of the observed terrain. At this stage, it is assumed that the group of fixed-wing UAVs M performs the coverage task on the polygonal convex area, represented by the set of vertices P in the space R2 . In the case of a nonconvex region, it is assumed that P represents a convex envelope of the area. The maximum flight time of each UAV is believed to be finite and known in advance. Each UAV is equipped with a fixed onboard camera pointing downward. In addition, it is assumed that all fixed-wing UAVs are equal in terms of hardware and power. The first step of this stage is devoted to searching the optimal coverage (sweep) direction perpendicular to the lowest height of the polygonal area, since in this case, the area can be covered with the least number of rows, which in turn contributes to the generation of trajectories that are optimal in terms of the route length, duration of flight, and energy expenditure. The optimal sweep direction can be found according

22 Method for Searching Deployment Zones of Ground Seismic Sensors …

347

to the algorithm from [16], after which the rows can be distributed over the terrain. The distance between two adjacent rows is chosen depending on the coverage area of the onboard cameras. Assuming that the fixing video camera is parallel to the ground plane (i.e., UAV is aligned), with the known image sensor’s width l, the focal length of the camera lens f, and the distance between the camera and the ground h (height of flight), the camera coverage width L can be determined as l L=h . f

(22.1)

The number of the coverage rows can be calculated as follows: Nl =

h min , L(1 − s)

(22.2)

where s ∈ (0, 1) represents the required overlap level between two adjacent images. The distance between two adjacent rows in this case is equal to d l : dl =

h min . Nl

(22.3)

Assuming that the polygon representing the workspace is rotated so that the optimal coverage direction is parallel to the OX axis of the global reference system, the coverage rows can be defined by two plane points (x, y) with equal y coordinates given as follows: yi = idl − 0.5dl , i = 1, . . . , n

(22.4)

The x coordinates in this case are determined by the points where the horizontal line with coordinate yi crosses the boundaries of the observed area. The extreme points of the coverage rows together with the coordinates of the UAV’s starting position are represented by the set of nodes V of the graph G = (V, E). Each node of the graph is numbered so that the starting position gets number 1, the nodes related to the first coverage row get numbers 2 and 3, those related to the second row are marked as 4 and 5, and so on. Each trajectory’s section is associated with subsequent even and odd nodes. The set of edges E consists of all the lines connecting the nodes of the graph, thus forming a complete graph. Mathematically, graph G can be represented by an n × n cost matrix C whose elements C ij are given by the Euclidean distance between spatial coordinates of nodes i and j. It is important to note that C is time-invariant and symmetric, i.e., C ij = C ji , ∀ (i, j) ∈ E, and its elements satisfy the triangular inequality, i.e., C ij + C jk ≥ C ik , ∀ (i, j), (j, k), (i, k) ∈ E. After constructing the graph of the investigated area, the coverage problem can be formulated as a vehicle routing problem (VRP). Before presenting the mathematical formulation of the routing problem, let us define the constants and variables necessary for the corresponding formulation. As stated above, the constant C ij represents the

348

R. Iakovlev et al.

cost of traversing edge (i, j). To note whether the route of the kth UAV runs from node i to node j, we use the binary variable X ijk ∈ {0, 1}. Let the constants Vijk ∈ R represent the flight speed of the kth UAV as it flies from vertex i to vertex j, the constant t s ∈ R represents the individual setup time, and L k represents the battery lifetime of the UAV with number k. Let also m ∈ N be the variable representing the number of UAVs intended for the mission, M ∈ N be the total number of available UAVs, O ∈ N be the constant number of UAV operators, and n ∈ N be the number of nodes in the graph. Finally, the variable d k represents the time required to launch UAV k. Based on the variables defined above, the route traversing time for the UAV k can be formally defined as follows: Tk =

n n ∑ ∑ Cij k X + dk V k ij i=1 j=1 ij

(22.5)

The key optimization direction in this case is the minimization of the total mission execution time, which, according to expression (22.5), can be achieved by minimizing the execution time of the longest route among all the UAVs involved. Thus, in a such minimum–maximum optimization problem, it is required to minimize the value of T k . To transform the minimum–maximum problem into a linear problem, we introduce an additional variable v, which represents the longest UAV route. In this case, the optimization problem can be defined as follows: ⎧ ⎪ ⎪ min(v) ⎪ n n ⎪ ⎨ ∑ ∑ Cij X k + d ≤ v, k = 1, . . . , M k V k ij i=1 j=1 ij ⎪ n ⎪ [ ] ∑ ⎪ ⎪ X 1k j = dk , k = 1, . . . , M ⎩ ts Ok

(22.6, 22.7 and 22.8)

j=1

The constraint in Eq. (22.7) considers the individual cost (time) of the UAV k (corresponding to T k ). By defining this constraint together with the target function in Eq. (22.6), we set a linear version of the minimum–maximum problem, where the maximum cost among all UAVs must be minimized. In inequality (22.7), the variable d k described in Eq. (22.8) accounts for the setup time t s , which represents the additional cost corresponding to the time it takes to prepare the UAV for the mission. Solving the problem represented by the target function in Eq. (22.6), we can obtain a solution where the minimum number of UAVs will follow the shortest possible paths, covering in minimum time the area described by graph G. It should be mentioned that although the cumulative mission execution time will be minimal, the target function does not explicitly account for the number of UAVs, which may cause that the suboptimal number of UAVs will be involved. The final step of the first stage, according to the proposed method, is aimed at constructing a general orthophotomap of the investigated area based on the video

22 Method for Searching Deployment Zones of Ground Seismic Sensors …

349

data obtained from the UAVs involved in the flight mission. As the main method of stitching frames from a set of video sequences in this work, the use of the widely spread SIFT method presented in [26] is considered. This method is used in a number of industrial projects and demonstrates a high level of reliability while working with image sets characterizing areas with homogeneous terrain. Thus, as the results of the first stage of the proposed method can be obtained a general orthophotomap of the observed area. The next stage of the proposed method is aimed at obtaining a target scheme of seismic sensors placement indicating the boundaries of potential regions of interest. The orthophotomap constructed before serves as the main operational data for the implementation of this stage. Nevertheless, it should be noted that determination of the placement scheme for the seismic modules is based not only on analysis of operational data, but also on a wide set of a priori information, including: results of preliminary calculations of seismic velocities for summation, results of time-variable filtering, and results of linking velocities and statics by area and intersecting profiles with already available seismic data for the territory [27]. Thus, the a priori and operational data are transferred to a qualified operator, who, in turn, carries out marking of target areas of seismic modules installation with indication of boundaries of potential regions of interest. Automation of this stage will be considered in subsequent studies. The third stage of the developed method is focused on the construction of a local altitude map for each of the regions of interest proposed by the operator for placing seismic modules. Input data for this stage are the characteristics of the given regions of interest, which include as follows: • Boundary of the region of interest is the region of interest in this case which is defined as a polygon of arbitrary shape described by an ordered set of global coordinates of the vertices of the given region of interest; • The target sector T b is the global coordinates of the center of the optimal sector for placing the seismic module in the given region of interest regb . Thus, at this stage, it is assumed that a group of multiprotor UAVs U should cover a polygonal region represented by a set Q of vertices in R2 space. In the case of a nonconvex region, Q represents the convex envelope of the region. Further, statement of the problem, as well as the working algorithms for the first two steps of current stage, is totally equal to steps 1.a and 1.b of the proposed method, respectively. Consider the third step of this stage, dedicated to the construction of a local altitude map for each of the given regions of interest based on RGB-D sensor data. Flying missions to each individual region are performed independently, the UAV moves along a horizontal trajectory at a low altitude, and RGB-D sensors are placed strictly in the direction to the ground in a fixed position. While the UAV is moving along the route, the following data are recorded: The position of the UAV (x k , yk , zk ) at the current time t obtained from the navigation module; the current depth map of the scene observed by the UAV H m*n . To build a local altitude map of the region of interest, first of all, it is necessary to determine the spatial location of scene areas H which correspond to pixels of the image received from the Intel RealSense D435 camera. A spatial coordinate system

350

R. Iakovlev et al.

connected with the lens of the camera should be defined as follows: Z axis is directed with the optical axis of fixing video camera, X and Y axes are collinear to x, y axes of image plane. In this case, the spatial sections associated with each pixel of the image are deflected from the normal to the XY plane by some flat angles in the XZ and YZ planes. Let the pixel size of the image obtained from the Intel RealSense D435 camera be equal to Imgpixx and Imgpix y in horizontal and vertical axes, respectively. For each pixel pi of this image with indexes in horizontal and vertical axes i pixx and i pix y , respectively, let us calculate the angular deviation: ) ( imgpixx Alpha ang_x pi = i pixx − ; 2 imgpixx ( ) imgpix y Betta , ang_y Pi = i pix y − 2 imgpix y where ang_x pi is the angular deviation for pixel pi along the x axis; ang_ypi is the angular deviation for pixel pi along the y axis; Alpha is the camera view angle along the horizontal axis of the image (x axis); Betta is the camera view angle along the vertical axis of the image (y axis). Since for each pixel pi using the RealSense D435 camera, it is possible to obtain estimates of the distances Di to the centers of the actual spatial sections associated with the pi pixels, and the position Ri of the spatial section covered by the pi pixel can be estimated relative to the reference system associated with the video camera lens as follows: ( Ri = |rrii | ∗ Di ( ) ( (, ) ) ri = tg ang_x pi ; tg ang_y pi ; 1 where r i is the directing vector characterizing the position of the spatial section of the scene, covered by pixel pi , relative to the reference system associated with the lens of the video camera. Accordingly, the spatial position Pi A of the scene sections associated with the pi pixels in the global coordinate system A can be determined according to the following expression: PiA = CtA Ri , where Ct A is the transformation matrix characterizing the spatial position of the capturing video camera of the UAV relative to the global reference system at time t, Ri is the vector characterizing the position of the spatial section of the scene, covered by pixel pi , in the reference system associated with the camera. Thus, it is possible to determine the spatial position of the scene sector observed by the UAV’s recording camera:

22 Method for Searching Deployment Zones of Ground Seismic Sensors …

351

( ) Surfkt = { PiA |i = 1, . . . N }, PiA = xiA , yiA , z iA , where N is the number of points of the three-dimensional space observed by the fixing camera of some UAV at a time t. According to the results of a mission of investigating some region of interest reg, all robotic agents consistently merge scene sectors Surfkt into a single altitude map. Merging is performed both in terms of time of observation t and in terms of robotic agents k involved in the mission. In this case, depending on the degree of approximation of the global coordinate grid, if the following expression is true for some sections P1 and P2 when merging two areas Surf1 and Surf2 : (

P1 = (x1 , y1 , z 1 ) ∈ Surf1 , P2 = (x2 , y2 , z 2 ) ∈ Surf2 , |x1 − x2 | < ε ∧ |y1 − y2 | < ε

(22.9)

then, the resulting value on the local altitude map in the given coordinates (x, y) will be defined as min (z1 , z2 ). Thus, according to the results of merging the scene sectors Surfkt , the final local altitude map is formed for the given region of interest. The final step of the proposed method is also implemented independently for each given region of interest and is aimed at searching the zones most suitable for the placement of seismic modules. On the first step on each of the regions of interest reg randomly selected a finite set of intersecting zones ai , the projection of which on the XY plane in a global coordinate system is a circle of radius r i (r l ≤ r i ≤ r h ), and the centers of such zones in the plane representation (XOY) are marked as ci . To carry out a comparative assessment of the obtained zones ai in terms of prospects of placing a seismic modules, it is required to determine a number of spatial characteristics of these zones, which include, in particular, the level of altitude differences in the zone, the position of the spatial center of the zone, as well as the effective angle of inclination of the zone relative to the horizontal plane XY. To estimate the values of the corresponding characteristics, first of all, it is necessary to determine the equation of the plane that approximates the considered zone ai . As such a plane, let us take the plane O, which is described by the minimum averaged distance between the points Pij belonging to the zone ai and this plane. The general equation of this plane may be specified as follows: Ax + By + Cz + D = 0; x, y, z ∈ R,

(22.10)

where A, B, C, and D are unknown parameters. The task of obtaining the values of these parameters can be represented as a parametric optimization problem with constraints: ∑ K |A∗xi +B∗yI +C∗zi +D| i=1

√

A2 +B 2 +C 2

K

−→

A,B,C,D∈R

min

(22.11)

352

R. Iakovlev et al.

Thus, it is required to determine such values of parameters A, B, C, D that while minimizing the expression (22.11), Eq. (22.10) will simultaneously remain valid. A wide range of well-known numerical optimization methods [28] as well as optimization methods based on genetic algorithms [29] can be used for solution of such an optimization problem. The resulting plane Eq. (22.10) with the known parameters will be the optimal version of the plane approximation of the spatial distribution of points belonging to the investigated zone ai . The effective inclination angle β i of a zone relative to the horizontal plane, thus, can be determined as follows: ) ( |n i Z | π , n i = (Ai , Bi , Ci ), βi = − arccos |n i | 2 where ni is the normal vector to the approximating plane of the zone ai . Z is the directional vector of the Z axis in the global coordinate system. The level of altitude difference within the zone ai is a characteristic of the average distance between the points belonging to the given zone and the approximating plane. This spatial characteristic of the zone with known parameters of Eq. (22.10) can be determined according to the following expression: | | ) ( ) ( K 1 ∑ | A ∗ x(Pij ) + B ∗ y Pij + C ∗ z Pij + D | Vi = , √ K j=1 A2 + B 2 + C 2 { Pij ∈ ai | j = 1, . . . , K } The final characteristic of the zone ai to be calculated at this step is the position of the spatial center C i of the zone, defined as K 1 ∑ Ci = Pij , { Pij ∈ ai | j = 1, . . . , K }. K j=1

Once all key spatial characteristics β i , V i , C i have been determined for zone ai , allocated on the investigated region of interest regb , the perspectives of seismic module placement in this zone can be assessed. A quantitative assessment of the prospects of placing the module in the zone ai can be obtained on the basis of the following empirical expression: E i = E(ai ) =

cos(βi ) . (Tb − Ci )2 Vi

(22.12)

It can be seen the maximization of expression (22.12) occurs when are minimal: The level of altitude differences in the zone, the distance from spatial center C i to the position of the target sector’s center T b, and the deviation of the approximating plane of the zone ai from the horizontal position is minimal. Thus, the most promising

22 Method for Searching Deployment Zones of Ground Seismic Sensors …

353

zone for placing a seismic module in a given region of interest regb will be the zone ab for which expression (22.12) will take the highest value: E(ab ) = max(E(ai )), i = 1, . . . , J. By performing the above procedure for each of the specified regions of interest, the proposed method allows to successfully implement the search for optimal zones for seismic sensor placement, represented by the corresponding sections of terrain ab . Next, let us proceed to the evaluation of the proposed method implementation.

22.4 Results Approbation of the developed method for searching deployment zones for seismic sensors by a heterogeneous group of UAVs was carried out in the Gazebo virtual simulation environment [30]. As part of the modeling experiment, the data of the general orthophotomap of the observed area were borrowed from the simulation environment. Construction of local altitude maps for the selected regions of interest in the following studies will be based on the usage of data from specialized depth sensors RealSense D435 installed on the UAV. The current experiment involved a full-featured depth sensor model presented in [31] which is provided by the selected simulation environment. Thus, in the current experiment, it was decided to simulate the output data of the RealSense D435 depth sensor by using a virtual model of a similar sensor, which allowed to fully exploit the algorithms proposed within the developed method. A virtual area with irregular terrain was deployed in the simulation environment, and the size of the virtual area is about 1 km2 . This area contains parts imitating the natural landscape, including rocky, hilly, flat terrain, as well as forest areas. On the deployed area, there are also many local obstacles of natural origin (massifs of trees and bushes, boulders, etc.). As part of the experiment with the involvement of a qualified expert, the formation of a target scheme of seismic modules placement on the investigated area was carried out. In total, 14 regions of interest were allocated with indication of corresponding boundaries as well as target installation sectors. To evaluate the results of the developed method, in each selected region, from 2 to 5 zones of interest were manually identified, which are the most optimal in terms of further placement of seismic modules. To perform each of the flight missions within the simulated environment, from 1 to 10 UAVs models of multirotor type were involved [32]. An independent flight mission was assigned to each of the specified regions of interest within the experiment. When performing, a flight mission in accordance with the proposed method sequentially was performed a following steps: the decomposition of the region of interest; the formation of full coverage collision-free trajectories for UAVs; the formation of a local altitude map of the region; the search for optimal zones for seismic module

354

R. Iakovlev et al.

placing. The IoU metric [33] was used to perform an assessment of the correspondence between the priory known optimal zones and the af zone selected based on the results of the method. If the value of the IoU metric for zone af and some given optimal zone exceeds 0.5, it is considered that an appropriate optimal zone has been selected. In case, if the values of the IoU metric between zone af and all given optimal placement zones are less than 0.5, then it is considered that the method has demonstrated an unsatisfactory result within the corresponding simulation and has not allowed to determine the optimal zone for module placement. A total of 90 independent flight mission simulations were conducted for each of the regions of interest, with the number of UAVs involved, the starting positions of the UAVs, and the working altitude of the UAV as the variable parameters. Based on the results of the corresponding group of simulations, for each of the regions of interest, the average percentage of successful searching results was determined, and the obtained results are shown in Table 22.1. According to the results presented in Table 22.1, we can conclude that, on average, the proposed method has demonstrated a sufficiently high quality of solving the problem of searching zones for placing seismic modules, and the averaged for regions of interest percentage of successful searching missions was 0.746. At the same time, we should note that there is quite a significant scatter between the averaged values obtained by regions of interest separately: The mean square deviation by regions of interest exceeds 0.08. Thus, within experimental evaluation of the proposed method, Table 22.1 Averaged proportion of successful searching results of optimal zones for seismic sensors placement by regions of interest Region No.

Estimated area of the region of interest (m2 )

Number of priori given optimal zones for placement

Average proportion of successful missions of searching optimal zones

1

23.800

3

0.655

2

16.000

4

0.853

3

10.400

2

0.744

4

3.600

4

0.891

5

8.200

3

0.677

6

14.200

2

0.624

7

14.900

5

0.767

8

7.900

4

0.731

9

7.900

3

0.695

10

7.300

4

0.812

11

5.900

2

0.667

12

8.700

4

0.753

13

26.600

5

0.728

14

2.300

3

0.843 0.746

22 Method for Searching Deployment Zones of Ground Seismic Sensors …

355

possible dependences between the shares of successful searching results and factors that could potentially have a significant impact on the quality of the developed method were further investigated, and the obtained results are presented in Fig. 22.1. The experiment investigated the dependences between the proportion of successful searching results and the following three parameters: the number of UAVs involved; the altitude of the UAVs during the flight; the weighed share of a priori given optimal zones in the region of interest. It can be seen, the graphs presented in Fig. 22.1a, c do not illustrate any explicit dependences between the corresponding values, thus it can be concluded that neither the number of UAVs involved, nor the weighed share of a priori given optimal zones has a significant impact on the final quality of the proposed method. At the same time, Fig. 22.1b allows us to conclude unambiguously that there is a stable decreasing dependence between the performance of the developed method and the altitude of the UAV flight during the mission. This dependence can be easily explained by the limited resolution of the depth sensors: the more distance between the UAV and the target, the lower the detalization level of the data from the depth sensors. Consequently, more-and-more information about the specifics of the investigated zone will be lost during subsequent analysis of the area. This will negatively affect both the quality of assessment of the spatial characteristics of zones, and the resulting assessment value of the prospects of placing seismological sensors in the given zones. Thus, within this work, it was proposed and successfully tested a method of automated searching for deployment zones of ground seismic sensors using UAV groups equipped with visible spectrum cameras as well as RGB-D sensors.

22.5 Conclusion In order to increase the level of automation of seismic works, an original method of aerial monitoring of the area by a heterogeneous group of UAVs was developed, aimed at solving the problem of searching for zones of deployment of seismic modules in the territory with a complex relief. The use of a heterogeneous group of fixed-wing and multirotor UAVs makes it possible to collect in a short time the necessary and reliable maps of the investigated areas for their further analysis on the presence of potential zones for seismic modules deployment. As part of the experimental evaluation of the proposed method, it was determined the averaged over the regions of interest proportion of successful missions of searching zones for placement the seismic modules, which reached 0.746. In addition, the work assessed the dependences between the proportion of successful searching results of optimal zones and the number of UAVs involved, the altitude of the UAVs during the flight and the weighed share of a priori given optimal zones in the region of interest. As a result, it was revealed that the quality of the proposed method is significantly affected exclusively by the altitude of the UAV flight during the flight missions. The detected dependence is of the inverse nature and is caused by the limited resolution of the depth sensors, which with increasing altitude of the UAV

356

R. Iakovlev et al.

Fig. 22.1 Dependences between the proportion of successful searching results of optimal zones and a the number of UAVs involved, b the altitude of the UAVs during the flight, c the weighed share of a priori given optimal zones in the region of interest

22 Method for Searching Deployment Zones of Ground Seismic Sensors …

357

flight begins to have a negative impact on the results of the spatial characteristics analysis of the investigated zone. Thus, in this work, it was presented and successfully tested a method of conducting aerial monitoring of a territory by a heterogeneous group of UAVs, aimed at solving the problem of searching for zones of deployment of seismic modules. The results obtained within the experiments confirm the high applied value of the proposed solution when performing of searching for deployment zones of seismic modules in the territory with a complex relief. Acknowledgements This research is supported by RSF project No. 22-69-00231, https://rscf.ru/ project/22-69-00231/.

References 1. Naugolnov, M., Bozic, M., Bogatyrev, I., Kultysheva, K.: Using of UAVs and computer vision for design and supervisory control of seismic survey works. Prog. 21 Euro. Assoc. Geosci. Eng. 2021(1), 1–5 (2021) 2. Grigoriev, G., Gulin, V., Nikitin, A., Sivoy, N., Bondarev, E., Islamuratov, M., Zakharova, O., Karpov, I., Liubimov, E., Votsalevskiy, V.: Integrated droneborne geophysics application as a tool for exploration optimization. In: Case Studies. SPE Annual Technical Conference and Exhibition. OnePetro (2021) 3. Ebadi, F., Norouzi, M.: Road Terrain detection and classification algorithm based on the color feature extraction. In: Artificial Intelligence and Robotics (IRANOPEN), pp. 139–146 (2017) 4. Bugaev, A.S., Antonov, A.N., Agafonov, B.M., Belotelov, K.S., Vergeles, S.S., Dudkin, P.V., Egorov, E.V., Egorov, I.V., Zhevnenko, D.A., Zhabin, S.N., Zaitsev, D.L.: Measuring devices based on molecular-electronic transducers. J. Commun. Technol. Electron. 63(12), 1339–1351 (2018). https://doi.org/10.1134/S1064226918110025 5. Egorov, I.V., Shabalina, A.S., Agafonov, V.M.: Design and self-noise of MET closed-loop seismic accelerometers. IEEE Sens. J. 17(7), 2008–2014 (2017). https://doi.org/10.1109/JSEN. 2017.2662207 6. Chikishev, D.A., Zaitsev, D.L., Belotelov, K.S., Egorov, I.V.: The temperature dependence of amplitude-frequency response of the MET sensor of linear motion in a broad frequency range. IEEE Sens. J. 19(21), 9653–9661 (2019). https://doi.org/10.1109/JSEN.2019.2927859 7. Zaitsev, D., Egorov, I., Agafonov, V.: A Comparative study of aqueous and non-aqueous solvents to be used in low-temperature serial molecular-electronic sensors. Chemosensors 10(3), 111 (2022). https://doi.org/10.3390/chemosensors10030111 8. Galceran, E., Carreras, M.: A survey on coverage path planning for robotics. Robot. Auton. Syst. 61(12), 1258–1276 (2013) 9. Darintsev, O., Migranov, A.: Analytical review of approaches to the distribution of tasks for mobile robot teams based on soft computing technologies. Inf. Autom. 21(4), 729–757 (2022). https://doi.org/10.15622/ia.21.4.4 10. Torres, M., Pelta, D.A., Verdegay, J.L., Torres, J.C.: Coverage path planning with unmanned aerial vehicles for 3D terrain reconstruction. Expert Syst. Appl. 55, 441–451 (2016) 11. Choset, H.: Coverage for robotics–a survey of recent results. Ann. Math. Artif. Intell. 31(1), 113–126 (2001) 12. Acevedo, J.J., Arrue, B.C., Maza, I., Ollero, A.: Distributed approach for coverage and patrolling missions with a team of heterogeneous aerial robots under communication constraints. Int. J. Adv. Rob. Syst. 10(1), 28 (2013)

358

R. Iakovlev et al.

13. Albani, D., Nardi, D., Trianni, V.: Field coverage and weed mapping by UAV swarms. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4319–4325 (2017) 14. Pshikhopov, V., Medvedev, M., Kostjukov, V., Houssein, F., Kadhim, A.: Trajectory planning algorithms in two-dimensional environment with obstacles. Inf. Autom. 21(3), 459–492 (2022). https://doi.org/10.15622/ia.21.3.1 15. Huang, W.H.: Optimal line-sweep-based decompositions for coverage algorithms. In: Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No. 01CH37164), vol. 1, pp. 27–32 (2001) 16. Li, Y., Chen, H., Er, M.J., Wang, X.: Coverage path planning for UAVs based on enhanced exact cellular decomposition method. Mechatronics 21(5), 876–885 (2011) 17. Xu, A., Viriyasuthee, C., Rekleitis, I.: Efficient complete coverage of a known arbitrary environment with applications to aerial operations. Auton. Robot. 36(4), 365–381 (2014) 18. Toth, P., Vigo, D.: The Vehicle Routing Problem. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA (2002) 19. Kivelevitch, E., Sharma, B., Ernest, N., Kumar, M., Cohen, K.: A hierarchical market solution to the min–max multiple depots vehicle routing problem. Unmanned Syst. 2(01), 87–100 (2014) 20. Zhang, Z.: Microsoft kinect sensor and its effect. IEEE Multimedia 19(2), 4–10 (2012) 21. Fernald, F.G.: Analysis of atmospheric lidar observations: some comments. Appl. Opt. 23(5), 652–653 (1984) 22. Keselman, L., Iselin Woodfill, J., Grunnet-Jepsen, A., Bhowmik, A.: Intel realsense stereoscopic depth cameras. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–10 (2017) 23. Eigen, D., Fergus, R.: Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2650–2658 (2015) 24. Laina, I., Rupprecht, C., Belagiannis, V., Tombari, F., Navab, N.: Deeper depth prediction with fully convolutional residual networks. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 239–248 (2016) 25. Yang Z.L., Guo B.L.: Image mosaic based on SIFT. In: 2008 International Conference on Intelligent Information Hiding and Multimedia Signal Processing, pp. 1422–1425 (2008) 26. Se S., Lowe D., Little J.: Vision-based mobile robot localization and mapping using scaleinvariant features. In: Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No. 01CH37164), vol. 2, pp. 2051–2058 (2001) 27. Kazanin, A.G., Kuoma, D.G., Bazilevich, S.O., Chizhikov, A.A., Prilipko, S.A., Lantsev, V.V., Demonov, A.P., Litvachuk, A.V., Lukovnikov, G.G., Ziborov, A.V., Dolotkazin, I.N., Koshelev, E.A., Petrov, B.E., Yerofeev, Yu.G., Agafonov, V.M.: Practical experience to operate molecular geophones in “crab” sea bottom recorders. Oil. Gas. Innov. 10(251), 23–26 (2021) 28. Izmailov, A.F., Solodov, M.V.: Numerical optimization methods (2003) 29. Gladkov, L.A., Kureychik, V.V., Kureychik, V.M.: Genetic algorithms (2010) 30. Gazebo. https://gazebosim.org/home. 10 Jan 2022 31. Intel RealSense Gazebo ROS plugin. https://github.com/pal-robotics/realsense_gazebo_ plugin. 10 Jan 2022 32. Using Gazebo Simulator with SITL. https://ardupilot.org/dev/docs/using-gazebo-simulatorwith-sitl.html. 10 Jan 2022 33. Shchelkunov, A.E., Kovalev, V.V., Morev, K.I., Sidko, I.V.: The metrics for tracking algorithms evaluation. Izvestiya SFedU. Eng. Sci. 1(211), 233–245 (2020)

Chapter 23

Method of Autonomous Survey of Power Lines Using a Multi-rotor UAV Anton Saveliev

and Igor Lebedev

Abstract Unmanned aerial vehicles (UAVs) are increasingly used to inspect power lines. In order to provide an autonomous examination of parts of the power lines line, in this work, key elements were identified that can be detected using technical vision algorithms and camera data. Such key elements include: support parts (upper and lower), wires, and traverses. For the purpose of autonomous examination of these elements, a new method of autonomous UAV movement along dynamically formed trajectories was developed. These trajectories are created based on the calculated positions of the UAV, depending on the camera parameters, reference points found in the database, and algorithms for examining individual elements using predefined trajectory patterns. This work is aimed at solving the problem of navigation in the immediate vicinity of the UAV to the power line support. The results of the initial testing of autonomous navigation and test flights in the simulator were also demonstrated. The developed method of global planning significantly accelerates the process of creating a mission by 1.5 times, and the flight time in offline mode along the calculated trajectory has decreased by 11% compared to the flight time in manual mode.

23.1 Introduction Worldwide, one of the main sources of energy transmission over long distances is overhead power transmission line (PTL). Timely monitoring of PTL allows you to determine the state of the elements of PTL and prevents emergency situations [1]. In order to reduce the risks to people’s lives and increase the efficiency of the entire procedure for inspecting infrastructure facilities, autonomous ground or air vehicles are used to perform the inspection [2]. Recently, unmanned aerial vehicles (UAVs) A. Saveliev · I. Lebedev (B) St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, 39. 14th V.O, 199178 St. Petersburg, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_23

359

360

A. Saveliev and I. Lebedev

have been increasingly used to monitor PTL. Therefore, an urgent task today is to develop a new method for autonomous movement of UAVs near PTL in order to examine the key elements of PTL. For autonomous planning of UAV movement, it is necessary to take into account the features of existing types of PTL structures [3, 4]. So, PTL tower come in several fundamentally different types. Based on the variety of supports and other parts of PTL, it is necessary to determine the key elements for the survey that combine all types of PTL [5]. Such key elements include: the support itself, traverses, wires, and the upper part of the support. Diagnostics of the support allows you to identify geometric changes in the support (slope), structural defects (cracks, fractures, rust), and foundation defects. Inspection of the traverses will help to identify defects or breakdowns of insulators, fasteners, vibration protection, if any, and also check the traverses for geometric deviations (slope, displacement). On the upper part of the support, there is a ground wire, which also needs to be examined to find defects in the cable itself and its fastening. In the field of aerial inspection [6], algorithms for time-optimal planning of the UAV trajectory for covering a three-dimensional urban structure were presented. With this approach, the covered structures (buildings) were first simplified to hemispheres and cylinders, and the construction of trajectories was implemented to cover these simple surfaces. In [7], the authors studied the problem of three-dimensional coverage of infrastructure facilities using an algorithm for calculating the object’s viewpoint and optimizing the route for aerial robots. In particular, they presented a method that supports the integration of several sensors with different fields of view and considered the limitations of UAV movement. In addition, in the field of covering several robots for aerial robotics in [8], a covering algorithm with several UAVs for remote sensing in agriculture was proposed, where the target area was divided into a number of disjoint subtasks and in order to avoid collisions. Different heights were assigned for each UAV; also, safety zones were defined. However, the complexity of the PTL design does not allow using all the algorithms described above for autonomous monitoring. In addition, the variety of types of supports and elements of PTL restricts the movement of the UAV along predetermined trajectories of movement, according to which it is possible to conduct a survey by means of the UAV. Therefore, it is necessary to develop a new method of autonomous planning of UAV movement for the inspection of poles and sections of PTL. The proposed method of PTL inspection is based on the process of detecting key elements. The developed method of UAV movement along dynamically formed trajectories provides a survey of parts of PTL based on the collected data on key elements that can be obtained using computer vision algorithms and camera data. Further, we will consider the developed method of autonomous UAV movement near PTL, including dynamic construction of trajectories for the examination of key elements of PTL.

23 Method of Autonomous Survey of Power Lines Using a Multi-rotor UAV

361

23.2 Description of the Developed Method of Autonomous Survey of PTL When examining elements and sections of PTL using UAVs, it is assumed that UAVs must ensure their precise positioning and positioning in open areas. The developed UAV motion planning method is necessary for conducting an autonomous survey of PTL. The first step in the UAV motion planning method is the creation of a flight mission by the operator. The mission consists of flight missions to survey key elements. All possible flight tasks are described in the state model in Fig. 23.1. After receiving a flight mission from the operator, the UAV starts moving. First, the UAV inspects PTL tower. The UAV moves along the original trajectory patterns (Figs. 23.2 and 23.3) and simultaneously updates the support data. The coordinate of the reference point where the key element of the PTL was detected is recorded in the UAV memory. To switch between flight tasks, the UAV needs to move from one reference point to another. Therefore, it is necessary to plan trajectories connecting these reference points. Next, the UAV moves from one reference point to another along the calculated trajectories in a given order of examination of the elements. All collected data are stored on board the UAV and, after an autonomous flight, is analyzed to determine the state of PTL elements by searching for defects in all images. Consider the actions of the UAV in the form of a state model for monitoring the elements of PTL (Fig. 23.1).

5

Inspection of power line traverses

4

Takeoff

4

1 0

Inspection of the power line tower

5

Land

2

3

5 Inspection of power line wires

4

Inspection of the upper part of the power line tower

5

Fig. 23.1 Model of UAV states during the examination of PTL

362

A. Saveliev and I. Lebedev Z

TOLP

TOLP

CIRP

Y

0

X

Fig. 23.2 Patterns of the UAV trajectory when examining a PTL tower Z 3

WIT 4

2

1

5

Y

0

X

Fig. 23.3 Pattern of wire inspection trajectory

23 Method of Autonomous Survey of Power Lines Using a Multi-rotor UAV

363

In Fig. 23.1, the numbers indicate the transitions between the states: 0—takeoff is completed; 1—receiving a command to examine the traverses; 2—receiving a command to inspect the upper part of the support; 3—receiving a command to examine the wires; 4—receiving a command for an additional examination; 5—examination completed. Each state is a specific flight task for the UAV. The set of flight tasks is called a flight mission. To ensure automated operation of the system, the operator sets the survey mission at the preliminary stage and cannot correct it during the operation of the UAV. The shortest survey mission consists of takeoff, survey of the PTL pylon and landing. The longest mission will consist of takeoff, pole survey, pole top survey, traverse survey, wire survey, and landing. The order of inspection of key objects may vary. However, the support survey is always present and always has the highest execution priority, i.e., executed first. The operator selects from the list of tasks the tasks that will comprise the survey mission. All tasks have their own execution priority. The job to survey the support has the highest priority, then, in order, the survey of the upper part, the survey of the traverses, and at the end the survey of the wires. For each flight task, separate algorithms for surveying the support, traverses, wires, and the upper part of the support are required. These algorithms are based on information about the UAV trajectory patterns depending on the key object being examined. In the context of this work, the trajectory pattern is understood as a set of logically acceptable ways to examine the key elements of PTL [9]. Consider two trajectory patterns, called “circumvolant path” (CIRP) and “take off and land vertically path” (TOLP). These two trajectories are sufficient for a complete survey of the support using UAVs [10]. In Fig. 23.2, these trajectories are marked with red and green lines. These routes allow avoiding collisions with PTL towers and simplify UAV visibility for most types of PTL. The TOLP trajectory mainly affects the movement of the UAV along the OZ axis, the CIRP trajectory–the movement along the OX and OY axes. In Fig. 23.2, the dotted line indicates the trajectory pattern “bypass”, the green line indicates the “vertical take-off and landing path”. To use these patterns of trajectories, it is necessary to have data on the safe distance from the PTL support, where the UAV will not be affected by the electromagnetic radiation of the PTL [11], at which it will be possible to carry out the unimpeded movement of the UAV relative to the elements of the PTL in strong wind. However, the TOLP and CIRP trajectory patterns are not enough to inspect the wires for such defects as: broken wires, defects in the carrier cable, displacement of vibration dampers, missing or incorrectly located connecting wires, breaks in the struts, and the presence of external threats (trees, debris and branches on the wires). In order to detect such faults, it is necessary to develop an additional trajectory pattern for the purpose of wire inspection. The wire inspection trajectory pattern is shown in Fig. 23.3 and is called WIT (“wire inspection trajectory”). Orient the PTL tower and the wires connected to it so that the direction of movement along the wires coincides with the direction of one of the axes (OX or OY). The beginning of the movement of the UAV for the inspection of wires begins at the reference point No. 1, where the upper part of the support was found. Then, to

364

A. Saveliev and I. Lebedev

examine the wires of the UAV, it is necessary to move on top of the wires from the reference point No. 1 along one of the axes (OX or OY), observing the minimum allowable distance. While moving from point No. 1 to point No. 3, the UAV takes pictures of the wires. When the second tower is detected, the UAV hangs in the air at point No. 3 and calculates the second reference point No. 2 to examine the wires. The location of reference point No. 2 directly depends on the length of the wires and the distance between the tower. After calculating point No. 2, the UAV moves to it and then continues to move along the proposed pattern of the wire inspection trajectory. During the movement, the UAV sends control signals to the stabilizer with the installed camera. This is necessary to constantly aim the camera at the object of examination. This wire inspection trajectory pattern provides data on power lines in the 0XY plane and an image of the power line section (two towers and wires between them) in one of the planes (0XZ or 0YZ). This is enough to be able to detect wire defects, analyze photographic materials for the presence of wire sag, and recreate a threedimensional visualization of the PTL section based on the obtained images. While moving along the trajectory patterns, the UAV takes photographs with a given parameter of frame overlap. This is necessary to create a 3D model of the examined section of the power transmission line. In order to calculate the location of the next waypoint where a photo will be taken, it is necessary to calculate the distance between the previous and current positions of the UAV, i.e., step size. The step size depends on the required sample distance to the object of study (GSD [cm/pixel]) and the overlap parameter (Fig. 23.4). The frame has a width ImW and a height ImL of the image in pixels. Knowing these values, the step length in meters is calculated using formulas (23.1) and (23.2) along the frame width p1 and along the frame height p2: ImW 1

ImL

Height overlap

3

Position UAV

1

p2

Position UAV

2

p1

3

Position UAV

Width overlap 2

Fig. 23.4 Demonstration of frame overlap in width and length

23 Method of Autonomous Survey of Power Lines Using a Multi-rotor UAV

365

p1 =

I mW × GSD × (1 − overlap), 100

(23.1)

p2 =

I m L × GSD × (1 − overlap), 100

(23.2)

where GSD is calculated by formula (23.3): GSD =

h × Sw × 100 , F × I mW

(23.3)

where h is the flight height of the UAV, Sw is the width of the matrix (camera sensor), F is the focal length. Longitudinal pitch (p1) and transverse pitch (p2) provide the specified overlap value. In Fig. 23.4 above, three areas are highlighted in green, yellow, and red, corresponding to the different acquired images and their overlap zone, determined by the frame length and width, and the specified overlap value. The colored dots represent the subsequent positions of the UAV from which the three images were taken. When the UAV moves along an elliptical trajectory describing the object of interest, the rotation of the gimbal ϑ is calculated using the following formula (23.4): ϑ = tan−1

ρ , h − h obj

(23.4)

where ϑ is the distance to the object, h obj is the height of the object of study. When the UAV moves along the OX, above the object of interest (Fig. 23.5), the rotation of the suspension ϑ around the OY axis is synchronized with the UAV movement at a distance x (m) along the OX axis and is calculated by formula (23.5): ϑ = tan−1

x − w2 , h − h obj

(23.5)

where x is the offset of the UAV and w is the width of the object. The novelty of the developed method lies in the use of special patterns of UAV movement when examining PTL elements. The calculation of each waypoint during the flight of the UAV along a given trajectory pattern occurs dynamically depending on the camera parameters and the specified frame overlap. During the execution of the flight task “examination of the PTL tower”, the UAV replenishes the database with the type of detected key elements and the coordinates of the control points that are necessary to link the flight tasks to each other.

366 Fig. 23.5 Illustration relating the parameters used in the formulas for calculating the rotation of the stabilizer

A. Saveliev and I. Lebedev

Z OY axis Directed "on us"

a a a

0

X

v h Object of interest

hob j

23.3 Description of the Developed Algorithms for UAV Movement According to Given Patterns of Trajectories Let’s consider in more detail the algorithms for examining the tower, traverses, wires, and the upper part of the support. To switch between the survey algorithms, the coordinates of the control points are required, which are determined in the flight task—the survey of the PTL tower (Fig. 23.6). The starting point of the UAV and the coordinate of the landing point, the minimum allowable distance to the support, the size of the camera frame, and the percentage of frame overlap to calculate the unit displacement p1 or p2 are used as input data for the algorithm for surveying the PTL. All data are set manually by the employees of the operational-field team or in advance in the system. The planning module has a TOLP survey trajectory pattern based on the data entered. The UAV takes off, starts moving along the TOLP upward to p2, and, using data from the camera installed on board, searches for a support. If the support was not found on the image, the operator will receive a notification so that he can make further decisions. During the movement of the UAV for a single movement p2 along TOLP, the support is photographed with a given percentage of overlap. After photographing the pole, the UAV needs to fly around the pole using the CIRP trajectory pattern. During the movement along the CIRP, the entire support is also constantly photographed. The unit displacement p1 is calculated from the given input data. Since the CIRP trajectory is a circular trajectory, the coordinate of the start point and the coordinate of the end point are the same. After flying along the CIRP, the UAV returns to the TOLP trajectory pattern, continues its upward movement for a distance p2, given the specified overlap percentage. Then, the CIRP pattern repeats again. And so on until the upper part of the support is found. If after surveying the support there is another task in the queue, then the UAV proceeds to its execution. If there are no more flight tasks, then the UAV moves to the landing point. An example of a survey trajectory is shown in Fig. 23.7.

23 Method of Autonomous Survey of Power Lines Using a Multi-rotor UAV

367

Begin

Takeoff

Movement on p2 "TOLP"

Start coordinate End coordinate Camera frame size (and % frame overlap) Minimum allowable distance to support

no

Has the top of the tower been found?

Tower in the frame? yes

no

yes

Frame capture (photography)

Tasks completed? Saving UAV location coordinates Movement to p1 on "CIRP" Capturing Frames with Specified Overlap

yes no

yes Go to the next task

no

Driving to the landing point

Finding Key Elements and Saving Control Points

Landing

End Is the current coordinate equal to the saved coordinate?

Fig. 23.6 Block diagram of the support inspection algorithm

The red dots mark the calculated positions of the UAV according to formulas (23.1) and (23.2), relative to the entered parameters: the frame width and length in pixels and the percentage of overlap. In each calculated position of the UAV, the angle of rotation of the stabilizer is calculated using formulas (23.4) and (23.5) to ensure frame overlap in all axes. If in the flight survey mission there is a task to survey the upper part of the support, then the UAV operates according to the algorithm in Fig. 23.8.

368

A. Saveliev and I. Lebedev Z

Fig. 23.7 Example of the inspection trajectory of a power transmission tower CIRP 3

CIRP 2 p2

CIRP 1

Y

p2 ...

...

...

p1 p1

p1

0 p2

*

Takeoff point

Fig. 23.8 Block diagram of the algorithm for examining the upper part of the support

X

Begin yes Tower survey

Have all the tasks been completed? no

Searching the top of the tower in the frame

Is the upper part detected? yes

Go to the next task no Movement to the landing point

Frame Capture

Landing

Saving the location coordinates of the UAV

End

The search for the upper part of the support in the frame is based on the NN for the search for key objects of power lines in the frame in real time. When the upper part is detected, the UAV fixes its location and takes a picture of the upper part of the tower on the camera. On the traverse, there are insulators, fittings, and other important components of the power line that need to be examined. The task for the inspection of traverses implements the algorithm for examining the traverses of the power transmission line support (Fig. 23.9). According to a predetermined priority, the task for surveying the traverses follows after the survey of the support and the survey of the upper part of the support. The

23 Method of Autonomous Survey of Power Lines Using a Multi-rotor UAV

369

Begin

yes Tower inspection

Have all the tasks been completed? no

Movement to the reference point for the survey of the traverse

Go to the next task Frame Capture Movement to the landing point Saving the location coordinates of the UAV Landing yes

no Have all traverses been examined?

End

Fig. 23.9 Block diagram of the algorithm for examining the traverses of the support

inspection process is similar to the process of inspection of the upper part of the support. From the base, the UAV receives the coordinate of the reference point for examining the traverses. The UAV moves to the sent coordinate along the shortest path, taking into account obstacles. With the help of a camera, the UAV detects a traverse. Traverse detection in the frame is based on the NN for the search for key objects of power lines in the frame in real time. When a traverse is detected, the UAV fixes its location and takes an additional photograph of the traverse using a conventional camera or special equipment. Figure 23.10 shows the algorithm for examining wires in order to detect defects. The task realizing inspection of wires follows after inspection of traverses. The reference coordinate from where the survey should be carried out is sent from the database to the UAV. As soon as the UAV has arrived at the designated coordinate, the wires are searched. The NS is responsible for searching for wires in the frame, detecting the key objects of the power lines in the frame. When the wires are detected, the UAV starts moving along the wires with a given step, fixes its location, and associates them with photographs of the wires [12]. The algorithms for examining the key elements of power transmission lines described above ensure the functioning of the UAV in the process of autonomous

370

A. Saveliev and I. Lebedev

Fig. 23.10 Block diagram of the wire inspection algorithm

Begin Movement to the calculated reference points

Tower inspection

Movement to the reference point for inspection of wires in the 0XY plane

yes

Have all the tasks been completed? no

Movement along the wires Go to the next task Capturing frames with a given overlap Saving the location coordinates of the UAV Calculation of the following reference points for surveying a section of a power line in the 0XZ Or 0YZ plane

Movement to the landing point Landing

End

operation. These algorithms go into the proposed method of autonomous survey of power lines.

23.4 Experimental Results For the initial testing of the developed method of UAV autonomous movement near power lines, ROS and the Gazebo simulator were chosen. ROS [13] is the most convenient platform, having various mathematical tools and software packages. In addition, many real navigation systems have their own package of versions in ROS and are very easy to integrate and use in the final product. Gazebo [14] Gazebo is an open-source simulator and is the most commonly used simulator for ROS. Also, in the work, a ready-made open-source UAV flight controller PX4 [15] was used. The developed simulation platform is based on the ROS-Gazebo-PX4 toolchain (Fig. 23.11). All components in the presented architecture are consistent across different topics in ROS. In particular, communication between the navigation system and the PX4 is via MAVROS. The PX4 Autopilot Simulator communicates with the Gazebo UAV to receive sensor data from the simulated world and send commands back to the motors and actuators. The PX4 has a state estimator and a motion control module based on the extended Kalman filter (EKF). Gazebo has an environment map called the world or workspace [16] and a UAV model (Fig. 23.12).

23 Method of Autonomous Survey of Power Lines Using a Multi-rotor UAV

371

Navigation block

UAV model in Gazebo Point cloud; data from INS

Dynamic Model

Local path planning block

Sensors

Control commands

global trajectory

Autopilot PX4

MAVROS Protocol

Global Planning Block

Input parameters

Local position

Fig. 23.11 General architecture of Gazebo-ROS-PX4

Fig. 23.12 Test environment in Gazebo

The UAV model supports dynamic simulation. The dynamic model in the simulator corresponds to the traditional quadcopter dynamic model, which can be found in general works on dynamics and control, such as [17]. The UAV is modeled as a

372

A. Saveliev and I. Lebedev

rigid body with 6 degrees of freedom (3 degrees of freedom in position, 3 degrees of freedom in rotation). The global planner works with the 3D workspace map to find the shortest path in 3D space and output the target point for the local planner. The local planner works directly with the point cloud to avoid potential collision with obstacles and plan a possible trajectory to the local target. The core component of the local scheduler is a sample-based method of finding a waypoint called the heuristic angular search (HAS) method. The presence of a global scheduler prevents collision with known obstacles. The global planner uses a survey trajectory algorithm to output a series of waypoints. Further, the found path is smoothed by the Bezier method. The goal of the local path planner is to find the tangent at the first waypoint of the Bezier curve [18]. The two schedulers work in parallel. Although the frequency of the outer loop of the global scheduler is low, the inner loop of the local scheduler maintains a high update rate and can continuously control the UAV [19, 20]. All navigation modules: global planning, local planning, are implemented as ROS packages and themes. Topics in ROS are special buses over which nodes (nodes) exchange messages (messages). Topics have anonymous publish/subscribe semantics that separate the producer of information from its consumer. In order to analyze the effectiveness of the proposed method, a global path planning block was implemented in the form of a ROS node. The input is a file with the parameters of the power line section. Based on these parameters, a global trajectory for the UAV is built. The calculated trajectory takes into account the allowable distance to the power line and the height of the power line towers. The constructed trajectory is sent to the flight controller for further movement of the UAV in the simulator. As experimental data, the time of calculating the trajectory and executing the mission in automatic mode and comparing time indicators with a manual mission were measured. In manual mode, the flight mission setup time was 120 s, and the flight time was 1627 s. The total setup and flight time in manual mode was 1747s. The data entry time for trajectory calculation is 78 s, while the offline global trajectory calculation time is 0.15 s. Flight along the calculated trajectory—1461 s. The total setup and flight time in autonomous mode was 1539.15 s. For each experiment, a series of 25 tests was run (Fig. 23.13), and average times were calculated as the sum of all times, separately in manual and offline modes, divided by the number of tests. Figure 23.14 shows the calculated survey trajectories of the power transmission line section, which consists of two towers and wires between them. To calculate the trajectory, the following parameters were set: the number of towers—2, the height of the towers—20 m, the coverage step (frame height), the safe distance to the tower is 3 m; the distance between the towers is 226 m. In Fig. 23.14, the red line shows the survey trajectories of the towers, the top of the tower, and the wire survey for top-view wire surveys. The green section of the trajectory is a route for examining the wires from the side, to calculate the sag of the wires. Since the inspection of the wires was the last task in the survey mission, the UAV lands after shooting the wires from the side. Figure 23.15 shows the trajectory of the UAV model in the simulator.

23 Method of Autonomous Survey of Power Lines Using a Multi-rotor UAV

373

UAV flight time

1700 1650 1600

Flight time in manua l mode Flight time in auto mode

1550 1500 1450 1400 1350 1300

1

3

5

7

9

11 13 15 17 19 21 23 25

Fig. 23.13 Measured UAV flight time in all experiments

Fig. 23.14 Calculated trajectories for two cases: a with a coverage step equal to 5 m; b coverage step equal to 3 m

Fig. 23.15 UAV movement trajectory in the simulator

374

A. Saveliev and I. Lebedev

The inspection trajectory of the power transmission line section in Fig. 23.15 is marked with a yellow line. This trajectory was taken from the flight log of the UAV model. From the results obtained, it can be concluded that automation of the process of surveying supports is a complex task, where it is necessary to solve issues related to UAV navigation near a complex infrastructure facility. In the work, it was proposed to divide the UAV navigation module into two blocks: local and global planning. The global survey trajectory planning was implemented as a ROS node. The global planner builds the trajectory of the transmission line survey according to the entered parameters: The distance between the towers is 226 m; the height of the towers is 20 m; the minimum allowable distance to the towers is 2 m. Trajectory decreased by 11%, compared with the flight time in manual mode.

23.5 Conclusion The proposed method for automated inspection of power lines is based on the original process of detecting key elements. The key elements are the pole itself, traverses, wires, and the top of the pole. To move in order to examine the key element of the power transmission line, determined by the flight task, the UAV uses a trajectory pattern. In the context of this work, the trajectory pattern is understood as a set of logically acceptable ways to examine the key elements of power transmission lines. In order to move based on these patterns, waypoints must be calculated. Each waypoint is calculated dynamically depending on the camera parameters and the specified frame overlap. During the execution of the flight task “examination of the power transmission line tower”, the UAV replenishes the database with the type of detected key elements and the coordinates of the reference points that are necessary to link the flight tasks to each other. The paper describes the UAV simulation platform for the study of the issue of navigation. The results of the initial testing of autonomous navigation and test flights in the simulator were also demonstrated. The developed method of global planning significantly speeds up the process of creating a mission by 1.5 times, and the flight time in autonomous mode along the calculated trajectory has decreased by 11% compared to the flight time in manual mode. Acknowledgements This work was supported by the RFBR Project No. 20-08-01056 A.

23 Method of Autonomous Survey of Power Lines Using a Multi-rotor UAV

375

References 1. Yang, L., Fan, J., Liu, Y., Li, E., Peng, J., Liang, Z.: A review on state-of-the-art power line inspection techniques. IEEE Trans. Instrum. Measur. 69(12), 9350–9365 (2020) 2. Deng, C., Wang, S., Huang, Z., Tan, Z., Liu, J.: Unmanned aerial vehicles for power line inspection: a cooperative way in platforms and communications. J. Commun. 9(9), 687–692 (2014) 3. Jenssen, R., Roverso, D.: Automatic autonomous vision-based power line inspection: a review of current status and the potential role of deep learning. Int. J. Electr. Power Energy Syst. 99, 107–120 (2018) 4. Zhang, Y., Yuan, X., Li, W., Chen, S.: Automatic power line inspection using UAV images. Remote Sens. 9(8), 824 (2017) 5. Sampedro, C., Martinez, C., Chauhan, A., Campoy, P.: A supervised approach to electric tower detection and classification for power line inspection. In: 2014 International Joint Conference on Neural Networks (IJCNN), pp. 1970–1977 (2014) 6. Cheng, P., Keller, J., Kumar, V.: Time-optimal UAV trajectory planning for 3D urban structure coverage. In: 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2750–2757 (2008) 7. Bircher, A., Kamel, M., Alexis, K., Burri, M., Oettershagen, P., Omari, S., Mantel, T., Siegwart, R.: Three-dimensional coverage path planning via viewpoint resampling and tour optimization for aerial robots. Auton. Robot. 40(6), 1059–1078 (2016) 8. Barrientos, A., Colorado, J., Cerro, J.D., Martinez, A., Rossi, C., Sanz, D., Valente, J.: Aerial remote sensing in agriculture: a practical approach to area coverage and path planning for fleets of mini aerial robots. J. Field Robot. 28(5), 667–689 (2011) 9. Krenev, A.A.: Some considerations for semantic segmentation on point clouds over time through trajectory pattern mining for lidar-registered object tracking. In: 23h International Conference Digital signal processing and its applications (“DSPA-2021”), pp. 198–203 (2021) 10. Bian, J., Hui, X., Zhao, X., Tan, M.: A point-line-based SLAM framework for UAV close proximity transmission tower inspection. In: 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1016–1021 (2018) 11. Gizatullin, R.M., Drozdikov, V.A., Konstantinov, E.S.: Simulation of functioning of elements of UAV electronic systems under influence of radiated electromagnetic interference of a highvoltage power line. Bull. Kazan State Power Eng. Univ. 4(44), 13–21 (2019). (In Russ.) 12. Shabanova, A.R., Tolstoy, I.M., Lebedev, I.V.: The mode of constructing safe trajectories of motion of the unmanned aerial vehicle while monitoring power lines considering the influence of their electromagnetic fields. Problemele Energeticii Regionale 3(44), 17–30 (2019) 13. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software, vol. 3(3.2), p. 5 (2009) 14. Koenig, N., Howard, A.: Design and use paradigms for gazebo, an open-source multi-robot simulator. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), vol. 3, pp. 2149–2154 (2004) 15. Meier, L., Honegger, D., Pollefeys, M.: PX4: a node-based multithreaded open source robotics framework for deeply embedded platforms. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 6235–6240 (2015) 16. Abbyasov, B., Lavrenov, R., Zakiev, A., Yakovlev, K., Svinin, M., Magid, E.: Automatic tool for gazebo world construction: from a grayscale image to a 3D solid model. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 7226–7232 (2020) 17. Mahony, R., Kumar, V., Corke, P.: Multirotor aerial vehicles: modeling, estimation, and control of quadrotor. IEEE Robot. Autom. Mag. 19(3), 20–32 (2012) 18. Choi, J.W., Curry, R., Elkaim, G.: Path planning based on bézier curve for autonomous ground vehicles. In: Advances in Electrical and Electronics Engineering-IAENG Special Edition of the World Congress on Engineering and Computer Science 2008, pp. 158–166 (2008)

376

A. Saveliev and I. Lebedev

19. Chen, S., Chen, H., Zhou, W., Wen, C.Y., Li, B.: End-to-end UAV simulation for visual SLAM and navigation. arXiv preprint arXiv:2012.00298 (2020) 20. Sevostyanova, N., Lebedev, I., Lebedeva, V., Vatamaniuk, I.: An innovative approach to automated photo-activation of crop acreage using UAVs to stimulate crop growth. Inform. Autom. 20(6), 1395–1417 (2021). https://doi.org/10.15622/ia.20.6.8

Chapter 24

Walking Robots for Agricultural Monitoring Dmitry Dobrynin

and Yulia Zhiteneva

Abstract The chapter discusses walking robots that allow monitoring of agricultural resources. Monitoring is understood as automatic collection of agricultural data: collection of soil and plant samples, assessment of soil cover characteristics, assessment of plant growth characteristics, etc. Such monitoring is necessary for the use of precision farming systems. Possible applications of walking robots for monitoring tasks are considered. The scenario of using walking robots as part of heterogeneous systems and features of interaction with flying robots are considered. It is shown that such a system scales well. The questions of the stability of the gates of walking robots for various schemes of constructing robot legs are considered. The speed characteristics of various schemes of construction of walking robots are considered. Algorithms of slow and fast gaits are considered. The requirements for drives to provide various types of gaits are given—the required drive power, torque, and speed characteristics are determined. The requirements for the technical characteristics of walking robots are formulated: drives, power systems, sensor systems, and control systems. Industrial walking robots are considered, and an analysis of their possible uses in agricultural monitoring is carried out.

24.1 Introduction The active introduction of robots into agricultural plant processing requires monitoring the state of the soil, plants, and the environment. Such monitoring is necessary for modern precision farming systems, which allow several times to increase yields from the same acreage. The basis of the scientific concept of precision agriculture is D. Dobrynin (B) Federal Research Center “Computer Science and Control” of the Russian Academy of Sciences, 44/2, Vavilova Str, 119333 Moscow, Russia e-mail: [email protected] Y. Zhiteneva State University of Humanities and Technology, 22, Zelenaya Str, 142611 Moscow Region, Orekhovo-Zuevo, Russia © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_24

377

378

D. Dobrynin and Y. Zhiteneva

the idea of the existence of heterogeneities within a single field [1, 2]. To assess and detect these in heterogeneities, the latest technologies are used, such as global positioning systems (GPS, GLONASS), special sensors, aerial photographs and satellite images, as well as special programs developed for agricultural management. The data obtained are used for sowing planning, calculation of fertilizer application rates and plant protection products, more accurate prediction of yield and financial planning. The use of robots for agricultural monitoring makes it possible to timely provide the planning system with the necessary information about the current state of plants, the qualitative parameters of their growth, the concentration of fertilizers in the soil, soil moisture, etc. Today, systems with AI elements can detect, diagnose, and outline solutions to plant problems before a person notices them. Monitoring should be carried out regularly on all controlled acreage. Unmanned aerial vehicles can perform some monitoring functions, for example, integral humidity measurement, assessment of the average condition of plants over a large area from photographs. However, spot monitoring—measuring the concentration of fertilizers in the soil and its humidity in certain areas of the field, sampling for laboratory studies, macrophotography of individual plants for pest analysis, etc., can only be carried out by ground robots. The use of small wheeled robots for ground monitoring tasks encounters a number of difficulties. Wheeled robots move well on flat surfaces and have a simple design and high load capacity. However, wheeled robots do not move well over rough terrain, they do not overcome obstacles well—pits, ditches, rocks, and they can roll over and lose control on large slopes. Wheeled robots also damage plants, as they leave a track, and cannot move in dense thickets. Walking robots, in comparison with wheeled robots, have lower movement speeds, a more complex design, and a high complexity of drives. However, walking robots have undeniable advantages—they move well over rough terrain, overcome obstacles well, and can get up when tipping over. Walking robots significantly less damage plants and can move in dense thickets. These properties of walking robots make their application promising for land-based agricultural monitoring tasks. Promising designs of walking robots for monitoring are six- and four-legged devices. Currently, mechanics and control systems are well-developed for them. The use of bipedal robots to perform monitoring tasks is currently not advisable, since such robots have not yet gone beyond the experimental samples.

24.2 Application Scenario Let’s consider a possible scenario of using walking robots for agricultural monitoring as part of heterogeneous systems (Fig. 24.1). Let a group of walking robots 2 monitor the field 1. Modern walking robots, as will be shown below, have a limited supply of energy. The average operating time of lithium-ion batteries is 1–2 h. This means that in order to effectively perform monitoring tasks during the working day, a mobile

24 Walking Robots for Agricultural Monitoring

379

Fig. 24.1 Scenario of application of walking robots for agricultural monitoring

control point 4 should be located near the robot group 2. It must recharge the robot batteries or replace them in the field. Ground walking robots have limited visibility, so the grouping of robots must be accompanied by one or more flying robots 3. The flying robot monitors from above. It can perform an integral assessment of the condition of plants in the field, as well as help ground walking robots to bind to the terrain. The entire group of robots can be controlled from a mobile control point 4, which stands on road 5 and does not enter the field. It can also transport walking and flying robots to the storage location. From the above scenario, it can be seen that the system of using walking robots scales well. In the minimal version, you can use one walking robot. With a large amount of work, it is necessary to use additional robots and a separate control point.

24.3 Types of Walking Robots and Their Gaits To date, a large number of walking robots have been studied theoretically and practically [3–8]. To be able to move in space, each robot leg must have at least three degrees of freedom. According to the position of the legs relative to the body of a walking robot, the following types of robots are usually distinguished (Fig. 24.2) (upper row—side view, lower row—front view): (a) Insect analogues are an insectomorphic construction. The parts of the leg have a long length, and the upper ends of the legs can be located above the robot’s body. This design of the legs allows you to get a large support base and provide a large margin of static stability. In addition, with small deflection angles of the drives, due to the large length of the legs, a high speed of movement can be obtained. In the early stages of the development of walking robots, such designs were very popular with researchers. The disadvantages of such a design

380

D. Dobrynin and Y. Zhiteneva

Fig. 24.2 Types of robots a analogues of insects; b analogues of reptiles; c analogues of animals

are large dimensions and increased requirements for drive moments, since the reaction forces of the support act on the robot body through long shoulders. (b) Analogues of reptiles—reptilian construction. The shins of the legs are positioned vertically, and the hips of the legs have a horizontal orientation. The support base of such structures is somewhat smaller than in the first variant, but it is possible to reduce the requirements for the lower leg drives, since it is located vertically and the reaction force of the support does not create a moment for it. The length of the legs of such robots is usually less than that of insectomorphic ones. The speed of movement is also usually less. By reducing the requirements for drive moments, reptilian robots have a higher load capacity. Options with six legs are possible, and then the middle legs are carried away from the body at a greater distance than the front and rear. This avoids the overlap of the legs when walking. (c) Animal analogues. All parts of the leg lie in a vertical plane, while the legs are always half-bent. The support base of this design is the smallest of the robots under consideration. But the dimensions of the robot for this design are minimal. The carrying capacity of such robots is usually less than that of reptilian robots. However, when walking, the legs move almost in the same plane, which allows you to increase speed and even use the running mode. Developments in the field of walking robots of the last decade have shown the practical possibility of building robots capable of walking quickly and even running. The gait of a walking robot is determined by the number of legs and the design of the leg [4]. There are the following types of robot gaits: (a) statically stable gates, when the robot can stop moving at any moment without loss of stability. The minimum speed of movement of the robot is not limited. (b) dynamically stable gates when the robot maintains stability in motion. The minimum speed of movement of the robot is limited, when the speed decreases below a certain threshold, the robot loses its balance. The robot can only stop at certain moments when three or more legs are standing on the support surface.

24 Walking Robots for Agricultural Monitoring

381

Fig. 24.3 Stability: a stability of hexapod robot; b stability of quadruped robot; c dynamic stability

(c) quasi-stable gates, when stability is provided by a special design of the robot leg. An example is bipedal robots with heavy legs, when moving which the swinging leg balances the tilt of the trunk, acting as a pendulum. Six-legged robots [9–11] (and robots with a large number of legs) may have static gait stability (Fig. 24.3a). When moving in threes, three of the six legs touch the support and form a support triangle, which is statically stable. The projection of the center of gravity Mg lies in the center of this triangle. As a result, equilibrium is provided automatically for such robots. This removes the restriction on the minimum speed of movement of the legs. It should be noted that the requirements for the drives in terms of travel speed are also reduced. Four-legged robots can have static and dynamic gaits. When carrying one leg, the support triangle is positioned in such a way that the center of gravity Mg is located near one of the sides of the triangle (Fig. 24.3b). At the same time, the margin of static stability (the distance from the projection of the center of gravity to the nearest side of the triangle) becomes small. When driving on an uneven surface, this can cause the work to tip over. An example of a static gait of a four-legged robot can be a gait with alternating transfer of one leg. At the same time, the other three legs rest on the surface and the contact points are arranged in such a way that the projection of the center of gravity always lies inside the reference triangle. It should be noted that such a static gait can be used mainly on a flat surface. Dynamic gaits require that during the leg transfer, the robot body maintains its position due to inertia. This imposes restrictions on the minimum speed of movement of the legs. Such gaits require fast drives, which is a difficult technical task. An example of the dynamic gait used in many developments of four-legged walking robots is the two-legged gait. With this gait, the robot simultaneously carries a pair of diagonal legs. The other pair of legs of the robot stands on the support surface. In this case, the reference polygon degenerates into a narrow strip (Fig. 24.3c). The projection of the robot’s center of gravity is located inside or near this strip. In order for the robot not to tip over, it is necessary to quickly transfer the legs. We will conduct a comparative analysis of the considered types of walking robots (Table 24.1), taking into account devices of small dimensions and weight. It should

382

D. Dobrynin and Y. Zhiteneva

Table 24.1 Comparative characteristics of robots Characteristic

A six-legged insect-like robot

A six-legged reptilian robot

Four-legged reptilian robot

A four-legged animal-type robot

Dimensions

Very large

Big

Medium

Medium

Movement speed

Average

Low

Average

High

Load capacity

High

High

Average

Average

Drive torques

High

Medium

Medium

Medium

Speed of drives

Low

Average

Medium, high

Medium, high

Types of gaits

Statically stable, dynamically stable

Statically stable, dynamically stable

Statically stable, dynamically stable

Dynamically stable

Stability

High

High

Average

Average

be noted that these criteria are not absolute, since within each class of robots these characteristics vary markedly. When choosing a device with a large load capacity, priority should be given to sixlegged reptilian-type devices. The ability to transport large loads with medium drive requirements and static stability makes them attractive. High demands on the drive torques and large dimensions of insect-like structures put them at a disadvantage. When choosing a device with a high speed of movement, preference should be given to a four-legged animal-type robot. They have the best speed characteristics. However, for fast movement, it is necessary to use fast drives and dynamic gaits.

24.4 Work Requirements To carry out agricultural monitoring, walking robots must perform the following functions: • • • • •

soil moisture measurement; measuring the concentration of fertilizers in the soil; macro photography of plants; collecting plant samples; storage of plant samples and other.

To do this, it is necessary to have appropriate equipment onboard sensors, highresolution cameras, a manipulator for collecting samples, containers for storing samples, etc. The control system of a walking robot has a complex structure (Fig. 24.4). The main part of such a system is the building management system, which must manage all global processes. It should include an action planning subsystem, route planning, and additional equipment management.

24 Walking Robots for Agricultural Monitoring

383

Fig. 24.4 Structure of robot’s control system

To move around the terrain, robots must have navigation systems (Fig. 24.4), allowing you to navigate the terrain and detect obstacles. Moving a walking robot over rough terrain at specified points requires two different navigation systems: local, with a range of about 10 m and global, which determines the robot’s route on the map. The local navigation system is guided by short-range sensors—stereo cameras, lidars, and sonars. It should detect obstacles, recognize surface types and, accordingly, set control modes for the motion control system. The Global Navigation system uses GPS or GLONASS to determine its location. It uses pre-built terrain maps to form a safe path for the robot to move. The motion control system should control the robot’s legs and provide a set speed of movement. Its functions also include ensuring the balance and the specified position of the robot body when performing service operations. Obviously, the control system of a walking robot should be built on a modular principle. All processes in the control system should proceed in parallel and in real time. For effective operation, the control system must contain several computing devices. For example, a local navigation system can use graphics accelerators to process large arrays of graphic data that come from video cameras. The remaining subsystems can be implemented on general-purpose processors. We will formulate the desired technical requirements for the work, based on practical considerations: • autonomy time—1 h or more (ideally—4 h or more); • travel speed—5 to 10 km/h (1.4–2.8 m/s);

384

D. Dobrynin and Y. Zhiteneva

• transported cargo—3 to 15 kg; • power sources—rechargeable Li-ion battery.

24.5 Industrial Analogues Currently, walking robots for industrial applications are being actively developed. The most famous of them are the Spot robot from Boston Dynamics [12], the B1 robot from Unitree Robotics [13], and the ANYmal C robot from the Swiss company ANYbotics [14] (Fig. 24.5). Table 24.2 shows some technical parameters of these robots. All these robots are built according to a scheme that copies a four-legged animal. Each robot leg has three degrees of freedom. As discussed earlier, such a robot construction scheme allows you to get a high speed of movement of the robot with small dimensions and medium (10–30 N*m) moments of the leg drives. Due to the

Fig. 24.5 Industrial walking robots: a Boston Dynamics Spot; b Unitree Robotics B1; c ANYbotics ANYmal C

Table 24.2 Parameters of industrial walking robots

Parameter

Boston Dynamics Spot

Unitree Robotics B1

ANYbotics ANYmal C

Length * width * height, cm

110*50*87

110*46*70

80*60*70

Weight, kg

72.6

40

50

Payload, kg

14

40

10

Protection

IP54

IP68

IP67

Speed, km/h

5.76

6.48

3.6

Speed, m/s

1.6

1.8

1.0

Working time, min

90

120

90

24 Walking Robots for Agricultural Monitoring

385

high location of the center of gravity, special measures have to be taken to maintain the robot’s balance. The presented works have a developed set of sensors, which include stereo cameras and laser rangefinders for local navigation. Robots allow the installation of additional equipment that is powered by the robot’s power supply. The robot control system contains one or two sufficiently productive onboard computers. Some of the robots presented, for example Spot and Unitree B1, can be equipped with manipulators for capturing objects. The manipulator expands the functionality of using a walking robot, allowing you to grab various objects. To work with the manipulator, additional modules have been developed that are installed on the robot chassis. All of these robots are built on a modular principle. The main functions of movement are performed by the robot itself. Thus, the walking robot is a platform with high cross-country ability and developed basic functions. Additional equipment installed on special mounts allows for additional operations—advanced laser scanning of space, high-resolution shooting, illumination of objects when shooting, etc. All the works in question move steadily on different surfaces, can overcome stairs and other obstacles. The software of these robots is actively developing. Currently, they are used in construction, monitoring of industrial enterprises, and other areas.

24.6 Conclusion The use of walking robots for agricultural monitoring is a promising direction that will allow us to qualitatively improve the collection of information about the state of plants and soil for precision farming. Thus, using precision farming technologies, it is possible to increase the yield of plants from the same areas, reduce fertilizer consumption, improve crop quality, and increase the profitability of agricultural work. The use of ground-based information collection systems allows you to get more detailed information about the state of plants and soil than satellite reconnaissance and aerial photography. This significantly improves the accuracy of forecasting. The analysis shows that today there are already technologies that allow you to create walking robots to perform agricultural monitoring tasks. Such systems have not yet been widely used, but they have good prospects for implementation in the process of growing plants. The main limitations today are the high price of such robots, the complexity of the robots themselves and control systems, but these obstacles can be successfully overcome with the further development of technology.

References 1. Auernhammer, H.: Precision farming—the environmental challenge. Comput. Electron. Agric. 30(1–3), 31–43 (2001). https://doi.org/10.1016/S0168-1699(00)00153-8

386

D. Dobrynin and Y. Zhiteneva

2. Beluhova-Uzunova, R., Dunchev, D.: Precision farming—concepts and perspectives. Problems Agric. Econ. 360, 142–155 (2019). https://doi.org/10.30858/zer/112132 3. Gonzalez-de-Santos, P., Garcia, E., Estremera, J.: Quadrupedal Locomotion: An Introduction to the Control of Four-Legged Robots. Springer-Verlag (2006). https://doi.org/10.1007/1-84628307-8 4. Hirose, S., Kikuchi, H., Umetani, Y.: Standard circular gait of a quadruped walking vehicle. Adv. Robot. 2, 143–164 (1986) 5. Jindrich, D.L., Full, R.J.: Many-legged maneuver-ability: Dynamics of turning in hexapods. J. Exp. Biol. 202, 1603–1623 (2005) 6. Beranek, R., Ahmadi, M.A.: Learning behavior based controller for maintaining balance in robotic locomotion. Intell. Robot. Syst. 82, 189–205 (2016) 7. Winterhalter, W., Fleckenstein, F., Dornhege, C., Burgard, W.: Localization for precision navigation in agricultural fields. Beyond crop row following. J. Field Robot. 38, 429–451 (2021). https://doi.org/10.1002/rob.21995 8. Ball, D., Upcroft, B., Wyeth, G., Corke, P., English, A., Ross, P., Patten, T., Fitch, R., Sukkarieh, S., Bate, A.: Vision-based obstacle detection and navigation for an agricultural robot. J. Field Robot. 33, 1107–1130 (2016). https://doi.org/10.1002/rob.21644 9. Tedeschi, F., Carbone, G.: Design of Hexapod Walking Robots: Backgroud and Challenges (2014). https://doi.org/10.4018/978-1-4666-7387-8.ch018 10. Vukobratovic, M., Borovac, B., Surla, D., Stokic, D.: Biped Locomotion: Dynamics, Stability. Control and Application (1990) 11. Golubev, Y.F., Korianov, V.V., Pavlovsky, V.E., Panchenko, A.V.: Motion control for the 6-legged robot in extreme conditions. In: Proceeding of the 16th International Conference CLAWAR, pp. 427–434 (2013) 12. Specification for robot SPOT from Boston Dynamics. https://www.bostondynamics.com/pro ducts/spot. Accessed 11 Dec 2020 13. Specification for robot Unitree B1 from Unitree. https://www.unitree.com/products/B1. Accessed 21 Jan 2021 14. Specification for robot ANYmal C from ANYBotics. https://www.anybotics.com/anymal-aut onomous-legged-robot/. Accessed 21 Dec 2021

Chapter 25

Computational Approach to Optimal Control in Applied Robotics Elizaveta Shmalko

Abstract The optimal control problem in its classical mathematical formulation is considered. The applied challenge is that the optimal control in the form of function of time cannot be directly applied to a real object because such control is open loop, and any disturbance of the object will lead to the fact that the goal will not be reached, and the value of the criterion will not be optimal. So, engineers stabilize the movement of the real object along the optimal trajectory. However, this approach does not ensure preservation of the optimality of the obtained controls. As a result, the optimal control problem in its classical formulation is less and less solved when creating new robotic devices, while the number of robots is increasing. This work expands the formulation of the optimal control problem, introducing initially the requirement to give the system describing the control object such properties that ensure the stability of solutions. Thus, the optimal control will be calculated already for the new model of the control object, which includes a stabilization system. The approach uses modern machine learning methods of symbolic regression and evolutionary computation. An example of the optimal control problem for a mecanum-wheeled mobile robot is demonstrated.

25.1 Introduction The main research question of this paper is the problem of optimal control in its applied aspect. When creating new autonomous robots and control systems for them, ideally, we still strive to develop optimal systems that are the best according to a given criterion. The optimal control problem is the most famous in the field of control theory. At one time, it attracted mathematicians to the field of control and made the control theory a part of mathematics. In the classical formulation of the optimal control E. Shmalko (B) Federal Research Center “Computer Science and Control” of the Russian Academy of Sciences, 44/2, Vavilova str., 119333 Moscow, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_25

387

388

E. Shmalko

problem, the control object is described by a system of ordinary differential equations, on the right side of which there is an unknown control vector. The initial and terminal conditions and the integral quality functional are given. It is necessary to find the control as a function of time. If we substitute this function into the right side of the differential equations, then we get a non-stationary system of differential equations with a known time function on the right side. A particular solution of this nonstationary system of differential equations from the given initial condition reaches the terminal condition, while the value of the quality functional is optimal. The optimal control problem belongs to the problems of infinite-dimensional optimization, since it is necessary to find a vector function of time, and not a constant vector in a real vector space of a certain dimension. Restrictions on control values and the possibility for the control function to have discontinuities of the first kind both in magnitude and in derivatives make it impossible to apply the methods of the calculus of variations to the problem of optimal control. Today a lot of approaches both direct and indirect to solving the problem have been developed. The direct approach [1] consists in reducing the optimal control problem to a nonlinear programming problem via a discretization of the control and the state functions on a time grid. This provides the transition from an optimization problem in an infinite-dimensional space to an optimization problem in a finite-dimensional space. But in this case, only approximate solution can be achieved due to control parameterization. Another approach is often called the indirect one, or Pontryagin maximum principle (PMP)-based approach. The Pontryagin maximum principle, formulated in the 1950s by Pontryagin [2], and then proved in the particular case by R. V. Gamkrelidze and in the general case by V. G. Boltyansky, is the most significant result in the field of optimal control. The application of this approach to the optimal control problem allows us to consider the optimal control problem as an optimization problem in a finite-dimensional space. According to the maximum principle, conjugate variables are introduced into the problem, which in this case play the role of Lagrange multipliers in the problem of conditional minimization. Conjugate variables are necessary in order to take into account the connections described by the mathematical model of the control object when minimizing the given functional. According to the maximum principle, a necessary condition for optimal control is to maximize the value of the Hamiltonian at each instant of time. To find the optimal control from the condition of the maximum of the Hamiltonian, it is necessary to know the value of the state vector of the control object and the vector of conjugate variables at each moment of time. The problem is that for the system of differential equations of the object model, the initial and terminal conditions are known, and for conjugate variables, there is a system of differential equations, but this system has neither initial nor terminal conditions. Therefore, there arises a boundary value problem of finding initial conditions for a system of conjugate variables such that when the control is calculated at each integration step from the conditions for the maximum of the Hamiltonian, the state vector ends up in the given terminal conditions. As a result of using this principle, the optimization problem in the infinite-dimensional space is transformed

25 Computational Approach to Optimal Control in Applied Robotics

389

into a boundary value problem, in which it is necessary to find the initial conditions for a system of differential equations for conjugate variables. Both of these known approaches provide the solution of the optimal control problem as a control function of time. And this is the key challenge from the application point of view. Such solution cannot be applied to the control object directly, since the obtained optimal control function is a function of time and its realization leads to an open-loop control system, so any discrepancy in time of object movement and control will lead to the fact that the control goal will not be achieved and the value of the quality criterion will differ from that obtained during mathematical calculations. In practice, real systems use state control systems, or feedback control, in view of the objectively existing differences of the model from the real control object and some other uncertainties. So, to use the received optimal open-loop solution in practice, it is necessary to construct a feedback stabilization system of movement along to the optimal trajectory in order to compensate the caused discrepancy between the real trajectory of the object and the obtained optimal one. But due to the introduction of the stabilization system, we may lose the optimality. Firstly, when calculating the optimal control, we use all available control resources, and it might not be enough resources of control for stabilization. Secondly, the movement in the neighborhood of the optimal trajectory can be often not optimal at all in terms of the functional value. Thirdly, a mathematical model of the control object with stabilization system doesn’t coincide with a mathematical model without stabilization system that was used for solving of the optimal control problem. This means, that the obtained optimal control will not be optimal for mathematical model of control object with the stabilization system. And fourthly, the problem of stabilization system synthesis for moving along the program trajectory is often more complex, than the initial problem of the optimal control. From the above said, it follows that the optimal control problem in its existing statement is not correct for a real control object, and some extended statement of the problem should be solved. The obvious way is to search for the optimal control function as a function of the state of the object, and this problem in control theory is called the control synthesis problem, firstly formulated in a general form by Boltyansky [3]. However, this problem is many times more difficult than the optimal control problem and still does not have a general solution method. Only approximate machine learning methods based on symbolic regression are currently able to solve the general synthesis problem [4]. However, in robotics it does not make much sense to solve the problem of general optimal control synthesis in view of the strong variability of the operating conditions and the high computational costs for recalculating the control function. In this regard, the new approach of synthesized optimal control [5, 6] seems promising. According to the approach, the optimal control problem is supposed to be solved after ensuring stability to the control object in the state space. Therefore, this approach is called synthesized optimal control. Its key idea is that a control function is found such that the system of differential equations will always have a stable equilibrium point in the state space. With that, the control system contains parameters that affect the position of the equilibrium point. Consequently, the object is controlled by changing the position of the equilibrium point.

390

E. Shmalko

Thus, we get a number of advantages. We implement a state control by introducing a stabilization system. There is now no need to solve the complex problem of general synthesis of control. It becomes possible to control the object optimally, while the optimal values of the control parameters can be calculated quite quickly using numerical optimization methods, including the possibility to recalculate these parameters even on board. And most importantly, all methods used for calculation are automatic numerical methods that do not require manual calculations, which means that they allow you to automate and universalize the process of developing control systems. In the following sections, we present the formulation of the synthesized optimal control problem and consider modern numerical methods suitable for solving it. In a computational example, the advantage of this approach will be demonstrated in comparison with the direct approach to optimal control for a mecanum-wheeled robot model under uncertainties.

25.2 Problem Statement of the Synthesized Optimal Control Consider the problem statement of the synthesized optimal control. Let the mathematical model of the control object be set as a system of ordinary differential equations: x˙ = f(x, u),

(25.1)

where x is a state space vector, x ∈ Rn , u is a vector of control. The control values are constrained: u ∈ U ⊆ Rm ,

(25.2)

where U is a compact set. For the system (25.1) initial conditions are set: x(0) = x0 ∈ Rn .

(25.3)

( ) x t f = x f ∈ Rn ,

(25.4)

Given a terminal condition:

where t f is a time of reaching the terminal condition (25.4), t f is limited and not set: ⎧ tf =

|| || t, if t < t + and||x f − x(t)|| ≤ ε , t + , otherwise;

(25.5)

25 Computational Approach to Optimal Control in Applied Robotics

391

where ε is a given small positive value, t + is a given limit time of control. Given a quality criterion in the form of integral functional: J˜ =

(t f f 0 (x, u)dt → min . u∈U

(25.6)

0

The task is to find the control function in the form: u = g(x, t) ∈ U,

(25.7)

such that if this control function g(x, t) is placed into the right side of the mathematical model (25.1) of the control object, then the partial solutions of the resulting system of differential equations: x˙ = f(x, g(x, t)),

(25.8)

) ( from the given initial condition x0 reaches the terminal condition x t f , x0 = x f , t f ≤ t + with the optimal value of the quality criterion (25.7) and for other partial solutions x(t, x∗ ), x∗ /= x0 , the following condition of δ-optimality is true: if for the given Δ > 0 and t ' < t + : | ( ' ∗) ( )| |x t , x − x t ' , x 0 | ≤ Δ,

(25.9)

then for ∀ε > 0 ∃δt, 0 < δt < ∞ such, that: | (' ) ( )| |x t + δt, x∗ − x t ' + δt, x0 | ≤ ε.

(25.10)

This additional condition of δ-optimality means that the optimal trajectory received as a result of the solution of the optimal control problem (25.1)–(25.10) must have an additional special property that it attracts all the nearest partial solutions located in some δ-tube. This requirement allows to obtain a control that will be insensitive to certain perturbations. The computational approach to find such function (25.7) that satisfied additional conditions (25.9)–(25.10) is called the synthesized optimal control [7]. Algorithmically, the solution of the synthesized optimal control problem and finding the control function (25.8) are assumed to be carried out as two sequential tasks. In the first stage, the stabilization control synthesis problem is solved and stability is provided for the object in some domain X 0 ⊆ R n . x˙ = f(x, u),

X 0 ⊆ Rn ,

( ) u = h x∗ , x ,

(25.11)

392

E. Shmalko

where control function h(x∗ , x) : R n × R n → R m has the following property: ( ) h x∗ , x ∈ U ⊆ R n , ( ) ∀x∗ ∈ X 0 , ∃˜x x∗ such that ( ))) ( ( ) ( = 0, f x˜ x∗ , h x∗ , x˜ x∗

(25.12)

det(A − λE) = (−1)n (λ − λ1 ) · . . . · (λ − λn ) =

n ⊓ (

) λ − λj = 0

j=1

where λ j = α j + iβ j , α j < 0, j = 1, . . . , n, i =

A=

∗

∗

∗

√

⎛

−1, ⎞

∂f(˜x(x ), h(x , x˜ (x ))) , E = diag⎝1, . . . , 1⎠. ⮩ ⮪⮫ ⮨ ∂x n

The function (25.11) is a stabilization system of the object (25.1). As a result, the object is stabilized relative to some point in the state space ∼ x (25.12). The position of this stabilization point depends on the parameters x∗ . The parameters x∗ can be directly the coordinates of stabilization point in the state space, or in general case x∗ can affect the position of a stabilization point ∼ x(x∗ ) in the state space. The property (25.10) says that the object with stabilization system x˙ = f(x, h(x∗ , x)) always has a stable equilibrium point ∼ x(x∗ ) ∈ R n . And the existence of the equilibrium point satisfies conditions (25.9)–(25.10) as far as near this point all solutions converges. In the second stage, the following optimal control problem is solved: )) ( ( x˙ = f x, h x∗ , x , x∗ ∈ X 0 ⊆ R n , x(0) = x0 , ( ) x t f = xf , (t f J=

( ( )) f 0 x, h x∗ , x dt → min . ∗ x ∈X0

(25.13)

0

Thus, in the problem (25.14) it is necessary to find a control function as a function of time: x∗ = v∗ (t), such that the system:

(25.14)

25 Computational Approach to Optimal Control in Applied Robotics

393

( ( )) x˙ = f x, h v∗ (t), x , ) ( 0 will have a partial solution x t,(x0 that ) from the given initial condition x will reach the given terminal condition x t f , x0 = x f , t f < t + , with the optimal value of the given quality criterion: ( ) J v∗ (t) =

(t f

( ( ) ( ))) ( f 0 x t, x0 , h v∗ (t), x t, x0 dt = min . ∗ x ∈X 0

0

25.3 Methods Overview Generally speaking, the presented two-stage or synthesized approach to solving the stated optimal control problem with the δ-optimality property (25.9)–(25.10) can be implemented by various well-known methods. Thus, the problem of stabilization system synthesis can be solved by constructing various optimal controllers in feedback, using any convenient methods like, for example, PID or PI controllers [8], analytical design of optimal controllers [9, 10], control Lyapunov functions [11], backstepping [12], neural network controllers [13] and others. Any method has its pros and cons. So, for example, the PID controller has a given structure and only the parameters can be optimally tuned, while the optimality of the structure itself for a specific task is not substantiated in any way. Another method of analytical design of optimal controllers works only for linear models, and, moreover, almost all known methods are very dependent on the type of control object model. But as already discussed in the introduction, we aim to generalize the methods and make them automatically implemented and suitable for a general class of tasks. In this regard, machine learning methods based on symbolic regression [4] seem to be very promising. Symbolic regression methods do an automated search for the mathematical expressions of the control functions. Using a given alphabet of the simple functions like sin x, x 2 , addition and many others, the methods of symbolic regression construct the optimal structure of the control function and simultaneously optimize the needed controller parameters. As optimization techniques, special forms of the genetic algorithm are used. The search is organized not directly on the functions but on their codes. Each symbolic regression method encodes a mathematical expression in its particular form, for example the genetic programming [14] uses a computational tree, the network operator [15]—an oriented graph. So, these methods vary according to the type of encoding of the mathematical expressions [4]. As a result, a researcher automatically receives the optimal structure of the controller with optimal parameters.

394

E. Shmalko

As soon as the additional requirement (25.9)–(25.10) is satisfied by the introduction of stabilization system, then the second task (25.14) of optimal control can be solved using either direct or indirect approaches.

25.4 Computational Example In order to demonstrate the advantage of the presented approach, we consider a specific example of solving the optimal control problem for an omnidirectional mecanum-wheeled mobile robot moving in a workspace with constraints. Let us solve this problem using different methods and check the performance of the obtained solutions by introducing a different level of uncertainty. The general layout of four mecanum-wheeled robot is shown in Fig. 25.1. The kinematic model of the robot is defined by the following equation [16]: ⎡

⎤

⎡

1 vx ⎣ v y ⎦ = 1 ⎣ −1 4 −1 ωz L+H

⎡

1 1

1 1

1 −1 L+H L+H

Fig. 25.1 General layout of four mecanum-wheeled robot

⎤ u1 1 ⎢ u2 ⎥ ⎥ −1 ⎦⎢ ⎣ u 3 ⎦, 1 L+H u4 ⎤

(25.15)

25 Computational Approach to Optimal Control in Applied Robotics

395

where vx and v y are components of the linear speed of mass center on the horizontal plane, ωz is an angle speed of the robot around vertical axis, u i = Rωi is a component of the control vector and a linear rim velocity of a wheel i, i = 1, 2, 3, 4; L , H are geometric parameters of the robot, as shown in Fig. 25.1. Let us convert the motion in the local robot reference frame to the motion in terms of the global reference frame: ⎡

⎤ ⎡ ⎤⎡ ⎤ x˙1 cos x3 − sin x3 0 vx ⎣ x˙2 ⎦ = ⎣ sin x3 cos x3 0 ⎦⎣ v y ⎦, x˙3 ωz 0 0 1

(25.16)

Thus, the mathematical model of the control object is: x˙1 = 0.25(u 1 (cos(x3 ) + sin(x3 )) + u 2 (cos(x3 ) − sin(x3 )) + u 3 (cos(x3 ) − sin(x3 )) + u 4 (cos(x3 ) − sin(x3 ))), x˙2 = 0.25(u 1 (− cos(x3 ) + sin(x3 )) + u 2 (cos(x3 ) + sin(x3 )) + u 3 (cos(x3 ) + sin(x3 )) + u 4 (− cos(x3 ) + sin(x3 ))), 0.25 x˙3 = (−u 1 + u 2 − u 3 + u 4 ). L+H

(25.17)

The control vector is limited: −10 = u i− ≤ u i ≤ u i+ = 10, i = 1, 2, 3, 4.

(25.18)

The initial state is given: x(0) = x0 = [0 0 0]T .

(25.19)

( ) x t f = xf = [1 0 1 0 0]T ,

(25.20)

The terminal state is given:

where t f is determined by (25.5) with ε0 = 0.01, t + = 2.2, | | f |x − x(t)| =

/ Σ3 ( i=1

)2 f xi − xi (t) .

Given the quality criterion: | | J2 = t f + p1 |xf − x(t)| + p2

(t f Σ 2 0

i=1

ϑ(φi (x(t)))dt → min, u

(25.21)

396

E. Shmalko

where p1 = 1, p2 = 3, φi (x(t)) = ri −

/ (

x1 (t) − x1,i

)2

)2 ( + x2 (t) − x2,i ,

(25.22)

i = 1, 2, x1,1 = 1.5, x2,1 = 2.5, r1 = 2.5, x1,2 = 8.5, x2,2 = 7.5, r2 = 2.5. According to the synthesized optimal control approach, firstly, the control function (25.11) of the stabilization system is searched. The initial domain as a set of initial positions was given by 18 points: { ]T } [ , X 0 = x0,N = −5 + id1 −5 + jd2 −5π/12 + kd3

(25.23)

where d1 = 5, d2 = 10, d3 = 5π/12, i = 0, 1, 2, j = 0, 1, k = 0, 1, 2, N = 0 . . . 18. The stabilization point was set in the origin: x∗ = [0 0 0]T .

(25.24)

The solution was obtained by the network operator method [4]. The trajectories of the robot on the plane {x1 , x2 } from eight initial states [0 −5 −5π/12]T , [0 −5 5π/12]T , [−5 5 −5π/12]T , [−5 5 5π/12]T , [5 −5 −5π/12]T , [5 −5 5π/12]T , [0 5 −5π/12]T , [0 5 5π/12]T are presented in Fig. 25.2. Secondly, the optimal control problem (25.14) was solved. The time limit was set t + = 2.2, and the time interval was Δt = 0.2. The time axis was divided into intervals: | | +| | 2.2 t = = 11. L= Δt 0.2 For each interval i, the optimal position of the stabilization point x∗,i was found, i = 1, . . . , 11. The optimal values of the vector x∗,i were searched by the particle swarm optimization (PSO) algorithm [17]. The following solution was obtained: Fig. 25.2 Stabilization of the robot from different points of the initial domain (25.23)

25 Computational Approach to Optimal Control in Applied Robotics

397

Fig. 25.3 Optimal trajectory of the robot with the synthesized optimal control

[ ]T [ ]T x ∗,1 = 2.5274 2.5017 −0.0826 , x ∗,2 = 3.9999 2.8955 −0.4245 , [ ]T [ ]T x ∗,3 = 0.3312 2.590 −0.6204 x ∗,4 = −0.7714 1.6551 −0.8193 , [ ]T [ ]T x ∗,5 = −3.9994 3.9732 −0.2602 , x ∗,6 = −0.6773 2.7642 −0.5739 , [ ]T [ ]T x ∗,7 = 0.1474 1.799 −0.9692 , x ∗,8 = 3.9825 3.9171 1.5708 , [ ]T [ ]T x ∗,9 = 4.0000 3.0982 −0.47155 , x ∗,10 = 4.0000 0.4174 −1.5708 , [ ]T x ∗,11 = 3.9691 4.0000 −1.5650 . (25.25) Optimal trajectory of the robot on the plane {x1 , x2 } with the obtained synthesized optimal control is shown in Fig. 25.3. The red circles are the phase constraints (25.22), and small black squares are projections of the points (25.25) on the plane {x1 , x2 }. In order to compare the received control, the same optimal control problem was solved by the direct method. The time axis was divided into intervals: L˜ =

|

t+ ∼t Δ

|

| =

2.2 0.2

| = 11.

In each interval, every component of the control vector was approximated by a piecewise linear function: ( ) t − j Δt ˜ + q j+(i−1) L˜ , i = 1, 2, 3, 4, u i = q j+1+(i−1) L˜ − q j+(i−1) L˜ − 1 ˜ Δt (25.26) ˜ ≤ t, j = 0, . . . , L˜ − 1. ˜ ≤ t < ( j + 1)Δt where j Δt Now ( the problem is transformed into optimization problem where it is necessary ) to find 4 L˜ + 1 = 48 optimal parameters: q = [q1 . . . q48 ]T .

(25.27)

398

E. Shmalko

The gray wolf optimizer (GWO) algorithm was applied in this example. The evolutionary method was chosen since according to [17] such optimization technique is more efficient in complex optimization problems. And in our example due to the presence of the phase constraints, the functional (25.21) has a non-convex form, which means that gradient methods may not find the global minimum. The evolutionary methods do not require the function to be differentiable, and they explore the search space widely and find a global minimum with high probability. In the result, the following parameters (25.27) were found: q=

[

17.7853 11.9510 11.1698 15.0867 17.2338

17.1191 18.9972 18.9902 0.8902 −13.0557 −8.9023 −2.4057 −3.5418 5.6931 12.4873 18.0487 18.4370 13.1061 12.7970 13.2688 14.9822 12.2821 18.4519 11.4719 10.5812 19.2237 1.5677 −2.2523 7.0795 9.9619 13.4154 11.6337 10.2377 16.6641 16.3856 15.1025 14.5589 16.0048 7.2910 5.4090 12.2878 14.5625 13.0459 −14.1582 −6.9339 −13.7779 −14.4530 −2.7509 −1.0726 15.7054 ]T 19.5760 10.1078 The optimal trajectory of the robot on the plane {x1 , x2 } with the obtained direct optimal control is presented in Fig. 25.4. Further, to test the sensitivity of the obtained controls to small uncertainties in terms of the functional values, perturbations were introduced into the object model: Fig. 25.4 Optimal trajectory of the robot with the direct optimal control

25 Computational Approach to Optimal Control in Applied Robotics

399

Table 25.1 Influence of perturbations on the obtained controls Synthesized

Level of perturbations

Direct

β0

β

J3

σ3

J4

σ4

0.1

0

2.9602

1.1451

5.1362

1.4206

0

0.1

3.8420

0.1033

3.8264

0.7024

0.1

0.1

3.5938

1.0386

4.5809

1.689

0.5

0

4.2634

1.2371

7.0330

2.4748

0

0.5

5.4189

2.1480

5.5249

1.1442

0.5

0.5

6.5218

2.2387

7.2133

2.4467

x˙1 = 0.25(u 1 (cos(x3 ) + sin(x3 )) + u 2 (cos(x3 ) − sin(x3 )) + u 3 (cos(x3 ) − sin(x3 )) + u 4 (cos(x3 ) − sin(x3 ))) + βξ (t), x˙2 = 0.25(u 1 (− cos(x3 ) + sin(x3 )) + u 2 (cos(x3 ) + sin(x3 )) + u 3 (cos(x3 ) + sin(x3 )) + u 4 (− cos(x3 ) + sin(x3 )) + βξ (t), 0.25 x˙3 = (−u 1 + u 2 − u 3 + u 4 ) + βξ (t). L+H and the initial conditions: xi (0) = xi0 + β0 ξ (t), i = 1, 2, 3, where ξ (t) is a random function that returns random value from −1 to 1. The control object with the synthesized optimal control and the direct optimal control was simulated for different levels of perturbations β and β0 . Results of the computational experiment are presented in Table 25.1, where J3 is an average value of the criterion (25.21) for the synthesized optimal control on 10 tests, σ3 is a standard deviation of these criterion values, J4 is an average value of the criterion (25.21) for the direct optimal control on 10 tests, σ4 is a standard deviation of these values. As seen from the results, the synthesized optimal control for all levels of perturbations is less sensitive than the direct optimal control. The direct optimal control is especially sensitive to perturbations of initial state.

25.5 Conclusions In this paper, we consider the question that in applied problems of robot control it is necessary not only to solve the problem of optimal control, but also to ensure the stability of the resulting solution to uncertainties. However, the classical approach of introducing a stabilization system after solving the optimal control problem does

400

E. Shmalko

not preserve the optimality of the obtained controls. In this regard, a computational approach of the synthesized optimal control is proposed and considered. It solves the problem of optimal control for an initially stabilized plant. Solutions are searched numerically using machine learning methods based on symbolic regression and evolutionary algorithms. As the experiments show, the approach is rather unsensitive to certain perturbations. The problem of the stabilization system synthesis is solved in advance, and already the optimal position of the equilibrium points, as parameters of the control system, can be calculated either in advance or in real time on board. An additional advantage of this approach is that control is implemented by moving equilibrium points, which can be optimally located in advance, or in real time, taking into account the emerging changes in the environment.

References 1. Böhme, T.J., Frank, B.: Direct methods for optimal control. In: Hybrid systems, optimal control and hybrid vehicles, pp. 233–273 (2017). https://doi.org/10.1007/978-3-319-51317-1_8 2. Pontryagin, L.S., Boltyanskii, V.G., Gamkrelidze, R.V., Mishchenko, E.F.: The mathematical theory of optimal process. In: Pontryagin, L.S. (ed.) Selected Works, vol. 4. New York, London, Paris, Montreux, Tokyo (1985) 3. Boltyanskii, V.G., Trirogoff, K.N., Tarnove, I., Leitmann, G.: Mathematical methods of optimal control. J. Dyn. Syst. Meas. Control. 93(4), 271–272 (1971) 4. Diveev, A., Shmalko, E.: Machine Learning Control by Symbolic Regression. Springer International Publishing (2021) 5. Shmalko, E.: Feasibility of synthesized optimal control approach on model of robotic system with uncertainties. In: Proceedings of 16th International Conference on Electromechanics and Robotics “Zavalishin’s Readings”, vol. 232, pp. 131–143 (2022) 6. Diveev, A.I., Shmalko, E.Yu., Serebrenny, V.V., Zentay, P.: Fundamentals of synthesized optimal control. Mathematics 9, 21 (2021) 7. Diveev, A., Shmalko, E.: Comparison of direct and indirect approaches for numerical solution of the optimal control problem by evolutionary methods. In: International Conference on Optimization and Applications, pp. 180–193 (2019). https://doi.org/10.1007/978-3-03038603-0_14 8. Åström, K.J., Hägglund, T.: The future of PID control. Control. Eng. Pract. 9(11), 1163–1175 (2001) 9. Smirnov, E.Ya.: Stabilization of Programmed Motion, 1st ed. CRC Press (2000) 10. Mizhidon, A.D.: On a problem of analytic design of an optimal controller. Autom. Rem. Control 72(11), 2315–2327 (2011). https://doi.org/10.1134/S0005117911110063 11. Raghunathan, A., Vaidya, U.: Optimal stabilization using Lyapunov measures. IEEE Trans. Autom. Control 59(5), 1316–1321 (2014). https://doi.org/10.1109/TAC.2013.2289707 12. Zidani, G., Drid, S., Chrifi-Alaoui, L., Benmakhlouf, A., Chaouch, S.: Backstepping controller for a wheeled mobile robot. In: 4th International Conference on Systems and Control (ICSC), pp. 443–448 (2015) 13. Velagic, J., Osmic, N., Lacevic B.: Design of neural network mobile robot motion controller. In: New Trends in Technologies (2010).https://doi.org/10.5772/7584 14. Marchetti, F., Minisci, E.: Genetic programming guidance control system for a reentry vehicle under uncertainties. Mathematics 9, 1868 (2021). https://doi.org/10.3390/math9161868 15. Diveev, A., Hussein, O.: Automatic solving of stabilization system synthesis problem by the network operator method. In: 15th IEEE Conference on Industrial Electronics and Applications (ICIEA), pp. 1165–1170 (2020). https://doi.org/10.1109/ICIEA48937.2020.9248195

25 Computational Approach to Optimal Control in Applied Robotics

401

16. Hsu-Chih, H., Chin-Wang, T., Chen-Chia, C., Jing-Jun, X.: FPGA-based mechatronic design and real-time fuzzy control with computational intelligence optimization for omni-Mecanumwheeled autonomous vehicles. Electronics 8, 1328 (2019). https://doi.org/10.3390/electroni cs8111328 17. Diveev, A., Konstantinov, S.: Research of practical convergence of evolutionary algorithms for optimal programmed control of a wheeled robot. Izvestiia Rossiiskoi akademii nauk. Teoriia i sistemy upravleniia 4, 75–98 (2018). https://doi.org/10.31857/S000233880002513-3

Chapter 26

Highly Maneuverable Small-Sized Wheeled Mobile Robotic Construction Platform Ekaterina Saveleva

and Evgeny Politov

Abstract Creation of robotic devices and their implementation in construction industry occurs to be an urgent research task. The market of ready-made robotic solutions for the construction industry is briefly considered. The shortcomings of existing solutions that hinder the pace of construction automation are given. The work aim is in developing of a highly maneuverable small-sized automotive robotic construction platform structural scheme that delivers concrete to specified points within the same level floor, as well as developing of a diagram of its control system. The proposed mobile robotic construction platform (MRCP) scheme, due to fourchannel control, makes it highly maneuverable and, with small dimensions, can reduce the complexity of construction processes when working on linearly extended and multi-floor facilities. The MRCP movement trajectory organization scheme as well as the structural scheme of the automatic control system (ACS) and the feedback in the process of human–machine interaction organization scheme are considered. A kinematic description of the MRCP movement is given. Further studies are planned to carry out a more detailed kinematic description of the MRCP working body movement. The results obtained will allow a comprehensive approach to the development of an automatic control system for the robot.

26.1 Introduction Construction industry automation and robotization are currently of particular importance within improving construction industry efficiency. Improving construction processes technology makes it possible to increase the competitiveness of the construction industry. It leads to acceleration of construction, reducing labor intensity and cost of work, improving of their quality, improving and facilitating working conditions, increasing the safety of work performed and, as a result, reducing worker injuries. E. Saveleva (B) · E. Politov Southwest State University, 94, 50 Let Oktyabrya, 305040 Kursk, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_26

403

404

E. Saveleva and E. Politov

Currently, the world market for ready-made technical solutions for the construction industry is represented by a wide range of automated devices, mainly on wheeled and caterpillar bases, equipped with manipulators (Hadrian, Hercules, Handle), lifting (JEKKO, MPK20), dismantling (BROKK, Atlant) and vibration equipment. Some platforms bases allow lifting loads up the stairs (Stair robot SR 1750). One of the topical areas of construction processes automation is development of mobile small-sized controlled and self-propelled devices for building materials delivery and solutions that have a low carrying capacity, but exceed person’s physical capabilities. The work aim is in developing of a structural scheme of a highly maneuverable small-sized automotive robotic construction platform that delivers concrete to specified points within the same level floor, as well as developing of a diagram of its control system. The existing solution’s disadvantages that hold back the construction automation pace are their high tonnage (over 8 tons) and large dimensions. That makes them insufficiently maneuverable and does not allow to be used out of the ground level or in cramped conditions. High tonnage equipment has traditionally high cost of standard workhours. The other feature of existing machines operation is also telecontrol, which implies full operator participation in process. At the moment, there are no automated self-propelled compact technical solutions that could be used for multi-story building’s construction and combine functions of manufacturing, transporting, and delivering concrete to the given points of grips within the one level floor. The necessity of designing such devices is conditioned to a significant amount of work and, as a result, the high cost of labor hours, high class of construction industry workers labor complexity, as well as a significant risk of injury at construction sites. Automation and intellectualization of such processes as production, transportation, and supply of concrete mixture to a given point or site are an urgent research task. Within the framework of this work, the rationale mobile robotic self-propelled platform with a load capacity of up to 400 kg for organizing local cement mortal supply is designed. The features of its constructive organization and control are discussed. Purpose of this work is in developing a structural scheme and a control model for a highly maneuverable small-sized self-propelled wheeled robotic construction platform (MRCP).

26.2 Proposed Structure and Description of the MRCP Figure 26.1 shows a self-propelled platform 1, equipped with an onboard control system (OCS) and laser sensors for detecting obstacles, which automatically transports the concrete mixture along given trajectory. A telescopic frame 3 is installed on the platform, the number of sections of which, as well as their height, can be selected depending on the planned relative elevation of the cement mortal supply. A bowl for

26 Highly Maneuverable Small-Sized Wheeled Mobile Robotic …

405

Fig. 26.1 Mobile robotic construction platform (MRCP) for supplying concrete: 1—supporting MRCP frame; 2—MRCP electric wheel drives; 3—system of the manipulator telescopic racks; 4—thicket for concrete mix; 5—mixer; 6—swivel mani–pulator frame; 7—pumping device; 8— sleeve for concrete mix; 9—manipulator boom; 10—concrete elevating device; 11—onboard control system, 12—LIDAR, 13—vision system (VS), 14—horizontal rangefinder (D1 -D8 ), 15—vertical rangefinder (D9 -D12 ), 16—contact interaction sensor (D13 )

cement mortal 4 is installed on telescopic frame, at the bottom of the frame there is a mixer 5, which allows mixing the ingredients of the prepared mixture. Swivel frame 6 rotates along concentric rails on the outer perimeter of the bowl, which allows the nozzle 10, located in the frame plane, supply concrete 360º around the MRCP main vertical axis. A controlled pump 7 is fixed on the frame 6, moving along the rail for the most efficient concrete in taking from the near-wall areas of the bowl. The hose for concrete supply 8 is fixed on a telescopic boom 9, which allows adjusting the radius of the mixture supplementation. The nozzle 10 rotates in the plane of the drawing. Thus, the proposed apparatus design has five degrees of freedom: It performs rectilinear motion and rotation in the overlap plane by 360º. The device is adjustable in height, the rotary modules of the device provide rotation of the supply nozzle along two coordinate axes. Such a MRCP constructive organization in combination with small dimensions (0.7 × 0.7 × 1.4 m) and the ability of loading up to 1.5 m3 of concrete provides the necessary and sufficient platform mobility. The stated aim of reducing the concrete work labor intensity is achieved by these conditions. To improve maneuverability, a four-channel robot wheel control scheme is used. Implementing this scheme in the context of selected three-wheeled model, two wheels are equipped with two engines each, rotating them about horizontal and vertical axes. The third wheel is movable relative to the horizontal and vertical axes. It involves movement without slippage and changing α angle in the movement process.

406

E. Saveleva and E. Politov

The type of motion organization, which involves two coaxial independent driving wheels and two motors that rotate them, is one of the most perspectives. A robot with such a motion organization, in comparison with other types of three-wheeled robots, is the simplest in terms of manufacturing and control. At the same time, its maneuverability is sufficient for solving many practical problems. Thus, the researching object of this work is a controlled dynamic system—a threewheeled robot with two independently motorized coaxial drive wheels. Conducting theoretical studies of the selected object movement involves the use of its mathematical model. This determines the need for an analytical review of existing approaches to the mathematical description of the considered wheel system movement.

26.3 Mathematical Model of MRCP Movement with Two Independent Driving Wheels Consider the robot as a system of five absolutely rigid bodies, one of which is a platform together with electric motors with gearboxes (electric drives), the other two are driving wheels, and two more are the rotary axes of the wheels [1–7]. The kinematic scheme corresponding to the MRCP straight-line movement is shown in Fig. 26.2. The three-wheeled robot with two driving wheels and one rear wheel movement organizing scheme for turning mode is given in Fig. 26.3. Position of these objects in coordinate system is determined by generalized coordinates vector q 1 = |x y ψ φ1 φ2 |T , where x, y are the coordinates of the point A (robot pole)—the middle of the segment connecting the centers C3 , C4 of the rear wheels 3, 4; ψ is the platform 1rotation angle around the vertical, counted from the axis O x; φ1 , φ2 —driving wheels rotation angles (with centers at the points C3 and C4 , respectively) relative to the body. Figure 26.4 illustrates the main kinematic parameters of the MRCP rear wheel motion. Fig. 26.2 Three-wheeled robot with two driving wheels and one rear wheel movement organizing scheme

26 Highly Maneuverable Small-Sized Wheeled Mobile Robotic …

407

Fig. 26.3 MRCP rotation kinematic scheme

Fig. 26.4 Robot rear wheel scheme: 1—wheel; 2—plug

The angular velocity of the platform is determined by formula (26.1) as follows: ⎛

⎞ 0 Ω = ⎝ 0 ⎠, ψ˙

(26.1)

where vector Ω is given by projections on the axes Ax yz. Driving wheels angular speeds are determined by the relations (26.2): ⎞ ⎛ ⎞ 0 0 ω1 = ⎝ φ˙ 1 ⎠, ω2 = ⎝ φ˙ 2 ⎠, ψ˙ ψ˙ ⎛

(26.2)

where ω1 , ω2 are given by projections on axis Ax1 y1 z 1 . Since the drive wheels move without slipping, projection of Eq. (26.2) onto the axes Ax1 y1 z 1 allows us to obtain three independent equations of non-integrable (non-holonomic) constraints (26.3):

408

E. Saveleva and E. Politov

⎧ ⎨ VC3 y1 = V P4 y1 = −x˙ sin ψ + y˙ cos ψ = 0 = x˙ cos ψ + y˙ sin ψ + l ψ˙ − r φ˙ 1 = 0 , V ⎩ C3 x1 VC4 x1 = x˙ cos ψ + y˙ sin ψ − l ψ˙ − r φ˙ 2 = 0

(26.3)

where l = AC3 = AC4 —half of the distance between driving wheels; r = C3 P3 = C4 P4 —driving wheel radius, P3 ,P4 —points of the drive wheels contact with surface. The dynamic equations of robot motion are considered for two characteristic stages: 1. Straight-line motion, in which ϕ1 = ϕ2 = 0 (4): ( ) ⎧ m V˙ = nc + i 2 ) − μn V + r1 ( M f r1 + M f r2) + maω2 ⎪ r (i 1 ⎪ ⎨ ˙ J Ω = ncl − i 2 ) − με ω + rl M f r1 − M f r2 − m aV ω r (i 1 . di 1 ⎪ + Ri 1 + nc + lω) = U1 L ⎪ r (V ⎩ i didt2 L i dt + Ri 2 + nc − lω) = U2 r (V

(26.4)

Here m is the total mass of the platform and stators of electric motors; J1 is the inertia moment of the robot about vertical axis passing through its center of mass C1 . This system (26.5) includes dynamic equations for four elements: point A speed V = x˙ cos ψ + y˙ sin ψ, platform angular velocity Ω = ψ˙ and currents in external circuits i 1 = e˙1 , i 2 = e˙2 of electric motors of steered wheels. Rotation in place around the instantaneous velocities center P ω1 = −ω2 . ( ) ⎧ ˙ = ncl (i 1 − i 2 ) − με ω + l M f r1 − M f r2 − m 1 aV ω ⎨ JΩ r r . + lω) = U1 L di1 + Ri 1 + nc r (V ⎩ i didt2 L i dt + Ri 2 + nc − lω) = U (V 2 r

(26.5)

In the given systems, R is the ohmic resistance of the rotor circuit, c is the coefficient of electromechanical interaction, m1 is the total mass of all elements of the MRCP, μn ,μβ are the viscous friction coefficients of for the straight-line and rotational motion of the robot, respectively, i1 , i2 are the electric motors external circuits currents, respectively, U1 , U2 are voltages in electric motor circuits. The position of the instantaneous center of rotation is described by formula (26.6): | | |X | P = || P ||. YP

(26.6)

For a platform with two driving wheels, control actions vector can be represented as (26.7): ) ( U = U1 , U2 , U 3, U4 ,

(26.7)

where U1 , U2 are steered wheel speeds; U 3 , U4 are angles of driving wheels rotation relative to vertical axes—φ1 (t),φ2 (t). The most maneuverability of the robot is achieved when PC → min.

26 Highly Maneuverable Small-Sized Wheeled Mobile Robotic …

409

Fig. 26.5 Block diagram of the MRCP control system

Thus, the problem of minimizing the rotation radius consists of reducing the RS distance. The variable parameter is the point P position (instantaneous velocities center), which is described by the function (26.8): P = f (φ1 , φ2 ),

(26.8)

The block diagram of the MRCP control system is shown in Fig. 26.5. The onboard control system (OCS) receives information about the site plan and the desired rC∗ and ψ ∗ values, as well as information from the MRCP monitoring module LIDAR. The data obtained make it possible to determine the MRCP position deviations-errors ΔxC , yC , Δψ. The error values are fed to the controller, which generates control voltages for four driving objects—robot wheels. The drives turn MRCP wheels axles to required angles φ1 , φ2 with the required angular velocity ω1 , ω2 . As a result of this, the robot center of mass C new coordinates is formed xC , yC , ψ. The control loop is closed through the use of the monitoring bodies of the MRCP, which transmit information to the onboard operating system. This cycle continues until the robot reaches final point.

26.4 MRCP Movement Control Motion control methods can be divided into explicit, that means that movement can be planned before the start, and implicit, when some system element’s state can be calculated directly in the movement. Implicit methods of motion planning use feedback, as a rule. In the simplest case, the difference between the system current state and the desired state is calculated and then used to move the system in the direction of decreasing this difference. There are many varieties of such methods [8– 10]. One of the most effective is the method of potentials. Explicit methods are alternative to implicit methods. Their essence is in necessity to know complete space configuration before motion start. As a rule, such methods use only system motion kinematics.

410

E. Saveleva and E. Politov

Fig. 26.6 Structural scheme of human–machine interaction in the conditions of construction manufacturing

To solve the task of trajectory constructing on an individual map, today use methods such as visibility graph (visibility), road map (road map), cell decomposition, and Voronoi diagram. In Fig. 26.6, the preferred scheme of human–machine interaction between the operator and the MRCP is shown. Figure 26.7 illustrates the MRCP control system structural scheme. The operator performs functions of setting the control points, and the MRCP is integrated into the technological process, receiving information from sensors 1–13, LIDAR and VS, excluding the functional operator participation in robot following to a given point process. At the same time, operator receives full information about the technological process from the MRCP OCS, and through the personal control of the manufacturing process. Depending on the information received, operator can be involved in technological process at any stage and give new commands to the robot. The presented model will be designed to solve the problem of transportation of building materials, which, in turn, imposes some boundary conditions and restrictions on the target site [11–18]. The characteristic features of the construction manufacturing territory are: • low-speed driving mode; • limited square and space; • the presence of static objects in the form of walls, scaffolding, columns, pallets, production equipment, vertical and horizontal apertures; • moving objects presence. Fig. 26.7 MRCP control system structural diagram

26 Highly Maneuverable Small-Sized Wheeled Mobile Robotic …

411

Fig. 26.8 Location scheme of MRCP control points of the beginning (A), bifurcation and rotation (Mi), and (Bi), and obstacles on the scheme of the construction site. 1—MRCP; 2—operator; 3—obstacle; 4—working area; 5—MRCP movement trajectory; 6—concrete station

Thus, this territory is described by a two-dimensional map of the floor with indication of permitted and prohibited driving zones (Fig. 26.8). The map is presented by a floor section plan with entered global coordinates of stationary objects and their boundaries. The peculiarity of internal premises maintenance, tunnels of facilities under construction and existing industrial zones, is associated with the impossibility of using GLONASS/GPS navigation and the insufficient accuracy of such systems in described conditions. These systems are designed to create and develop universal methods of control and global navigation of autonomous mobile robotic systems, including trajectory motion control, location determination in space and obstacle avoidance. To avoid the above problems with navigation, MRCP uses a local navigation system based on SLAM technology—simultaneous localization of the robot in the environment during movement and mapping of the surrounding space [19–31]. SLAM task is related to an unknown space map construction by a mobile robot while navigating a map being built. The implemented implicit control methods make it possible to work with non-stationary environmental conditions, as well as when it is necessary to act in an unknown environment, the presence of moving objects and the absence of external navigation. The SLAM task is in finding landmarks in space, search for their correspondences, calculate the location, clarify the location and positions of the landmarks. There are different ways to implement different subtasks. Thus, it becomes possible to combine different implementations of individual algorithms and improve them separately. For example, a replacement may be required to work in large spaces instead of working indoors. SLAM can be used for both 2D and 3D movements. To implement this method, the considered platform is equipped with a group of devices that receive information from the external environment. This group includes: • • • •

3D high-definition LIDAR sensor; vision system (VS); system of vertical and horizontal rangefinders; contact interaction sensor.

412

E. Saveleva and E. Politov

The required reference points initial set for achieving MRCP is setting by operator. The MRCP movement trajectory in the intermediate sections between the reference points is selected by the machine independently and is edited by the onboard control system (OCS) if it is necessary to bypass obstacles, which are detected using the VS and LIDAR. The VS function is recognition of reference benchmarks placed in predetermined areas of the premises. At the stage when operator maps the serviced area, information received from VS is necessary to initialize MRCP local position. As the device moves along the trajectory, LIDAR determines obstacles and their boundaries. Using information from LIDAR, the robot trajectory is independently edited by MRCP OCS, the object is being bypassed. At this moment, information entering OCS through the VS channel makes it possible to identify object that interferes with the direct passage along the original trajectory. If a human is identified on the MRCP path, platform stops for a while, then, if the obstacle disappears, it continues moving along the original trajectory. The initial movement trajectory is determined by MRCP independently. However, in the conditions of a construction site, there is a need for its editing by a robot, due to the presence of obstacles: stands and pallets with block elements, construction debris, and worker’s movement. Correction of the trajectory is carried out by the obstacle avoidance unit, taking into account information coming from range finders and the VS.

26.5 Planning of MRCP Trajectory Movement The MRCP navigation task is in determination of its characteristic points relative coordinates in a fixed coordinate system and the angular orientation of the hull in a fixed coordinate system. The results of solving this problem are used for creation motion control algorithms. The tasks of intelligent trajectory editing are performed by onboard control system and artificial intelligence mechanisms. At the moments of transition control from a human to a robot, the MRCP moves along a piecewise linear trajectory (Fig. 26.9). Thus, to perform a final linear movement, a transition is made from an absolute to a relative coordinate system. We will consider the MRCP movement along a conditional trajectory, consisting of straight-line segments shown in Fig. 26.7. The task of trajectory control is reduced to the fact that it is necessary to ensure the movement of the M point belonging to the MRSP along the straight-line segment M j−1 M j . The real trajectory differs from the given one M j−1 M ‘j . The radius—vector Δr , which determines position error, can be represented as (26.9): Δr = (Δx,Δy)T .

(26.9)

The error value is determined by formula (26.10): |Δr | =

√

Δx 2 + Δy 2 .

(26.10)

26 Highly Maneuverable Small-Sized Wheeled Mobile Robotic …

413

Fig. 26.9 Fragment of a piecewise linear trajectory of the MRCP in a—absolute; b—relative coordinate system

We define the equation of a straight line M j−1 M j . As it is known, the canonical equations of a straight line in space are equations that define a straight line passing through a given point collinearly to the direction vector. A straight line can be defined by two points lying on it M j−1 and M j . Then the canonical equations of the line will take the form (26.11): x − x j−1 y − y j−1 = . x j − x j−1 y j − y j−1

(26.11)

The above equations define a line M j−1 M j passing through two given points. Let’s introduce the concept of MRCP movement cycle, which consists of four stages: 1. 2. 3. 4.

rotation by angle φ j−1, j to move along a straight line M j−1 M j ; movement in a straight line M j−1 M j ; rotation by angle φ j, j+1 to move along a straight-line segment M j M j+1 ; movement in a straight line M j M j+1 .

Thus, the movement cycle consists of 2 turns and 2 straight-line stages. Further, the cycle designation is adopted: 2R2P. Using the concept of the 2R2P movement cycle (Fig. 26.10), it is possible to construct various trajectories of MRCP movement. We will assume that the total time of movement cycle consists of separate stages associated with the turn and movement in a straight line. For four stages, we have Fig. 26.10 Scheme of the planned trajectory section (cycle 2R2P)

414

E. Saveleva and E. Politov

T 1 = t1 + t2 + t3 + t4, where t 1 is the time of MRCP rotation from the initial / position to a given angle φ01 , at the point M1 ; t 2 is time in a straight line M1 M2 , (j = / 2); t 3 is MRCP rotation time at a given angle φ12 ; t 4 is straight-line time M2 M3 . The total time of MRCP movement along AB trajectory is (26.12): T AB =

N Σ

Ti ,

(26.12)

i=1

where N is the number of trajectory cycles. 1. Rotation On the time interval t0 ≤ t ≤ t1 rotation angle changes according to the law presented as a polynomial (26.13): φ ∗ (t) =

n Σ

ci · t i .

(26.13)

i=0

Boundary conditions in this section at the beginning and at the end of the trajectory, we will represent as follows: t = t1 , y = y1 , x = x1 ; y˙ = 0, x˙ = 0; φ = φ0 , φ˙ = 0 t = t2 , y = y1 , x = x1 ; y˙ = 0, x˙ = 0; φ = φ01 , φ˙ 01 = 0.

(26.14)

The coefficients are found from corresponding boundary conditions (26.14) as a solution of the following system of algebraic equations: A0 D = B0 . 2. Straight line On the time interval t1 ≤ t ≤ t2 MRCP moves in a straight line while the rotation ∗ . Coordinates determining MRCP center of mass angle does not change: φ ∗ = φ01 position changes according to the law (26.15): x ∗ (t) =

n Σ

di · t i , y ∗ (t) =

i=0

n Σ

ei · t i ,

(26.15)

i=0

where t1 is the start time and t2 is the end time of the interval. Coefficients ei are found from corresponding boundary conditions as a solution to the following system of algebraic Eq. (26.16): A1 E = B1 .

(26.16)

E = A−1 1 B1 .

(26.17)

From here:

26 Highly Maneuverable Small-Sized Wheeled Mobile Robotic …

415

Coefficients di are found from the corresponding boundary conditions as a solution of the following group of algebraic Eqs. (26.18, 26.19): A2 D = B2 .

(26.18)

D = A−1 2 B2 .

(26.19)

Boundary conditions at the second stage have the form (26.20): t = 0, y = y1 , x = x1 ; y˙ = 0, x˙ = 0; φ = φ1 , φ˙ = 0, t = t1 , y = y2 , x = x2 , y˙ = 0, x˙ = 0; φ = φ1 , φ˙ = 0.

(26.20)

In the same way, constants are determined at the third and fourth stages of the motion cycle.

26.6 Conclusions Creation and implementation of robotic devices in construction industry are an urgent research task. The MRCP usage for construction of linear-extended and multi-story facilities effectiveness in terms of automating the supply of solution to the working area and minimizing the physical costs of workers does not raise questions. An analysis of the available technical solutions showed the absence of mobile automated devices of small dimensions that could be used in work related to concreting. The proposed scheme of the alternative MRSP installation, due to the four-channel control, is highly mobile and, with small dimensions, makes it possible to reduce construction processes labor intensity when working on linearly extended and multistory facilities. The MRCP movement trajectory organization scheme, the operator introducing into the technological process scheme and the scheme of feedback in the human–machine interaction process organization are considered. A kinematic description of the MRCP platform movement is given. In further studies, it is planned to carry out more detailed kinematic description of the MRCP working module movement. Obtained results will allow a comprehensive approach to the robot automatic control system development. Acknowledgements The article was prepared with the Strategic Project “Priority-2030. Creation of robotic tools to expand human functionality” support.

416

E. Saveleva and E. Politov

References 1. Politov, E., Afonin, D., Bartenev, V.: Mathematical modeling of motion of a two-section wheeled robot. In: Proceedings of 14th International Conference on Electromechanics and Robotics “Zavalishin’s Readings” pp. 397–409 (2020) 2. Saranli, U., Avci, A., Öztürk, M.C.: A modular real-time fieldbus architecture for mobile robotic platforms. IEEE Trans. Instrum. Meas. 60(3), 916–927 (2010) 3. Sergiyenko, O.I., Ivanov, M.V., Tyrsa, V.V., Kartashov, V.M., Rivas Lopez, M., Hernandez Balbuena, D., Flores Fuentes, W., Rodriguez Quinonez, J.C., Nieto Hipolito, J.I., Hernandez, W., Tchernykh, A.: Data transferring model determination in robotic. Robot. Autonomus Syst. 83, 251–260 (2016) 4. Shin, W.Y., Shin, J.J., Kim, B.J., Jeong, K.R.: Line segment selection method for fast path planning. Int. J. Control Autom. Syst. 15(3), 1322 (2017) 5. Zashchelkin, K.V., Kalinichenko, V.V., Ulchenko, N.O.: Combined navigation method for an autonomous mobile robot. MNPK Modern Inform. Electron. Technol. 174–177 (2013) 6. Kartashov, V.M., Pososhenko, V.A., Tsekhmistro, R.I., Timoshenko, L.P., Kolendovskaya, M.M.: In: Methods of Orientation, Navigation and Control of Mobile Robotic Platforms (2019) 7. Nabiyullin, A.R., Kavalerov, M.V.: Multi-purpose wheel robotic platform with low cost. Extreme Robotics 1(1), 70–79 (2013) 8. Lesovik, V.S., Zagorodnyuk, L.Kh., Chernysheva, N.V., Glagolev, E.S., Kucherova A.S., Drebezgova M.Y., Kaneva E.V.: Modern three-dimensional technologies and factors constraining them. Bulletin of the Belgorod State Technological University 12 (2016) 9. Martynenko, Yu.G.: Motion control of mobile wheeled robots. Fundamental and Appl. Mathem. 8, 29–80 (2005) 10. Chikrin, D.E., Egorchev, A.A., Golousov, S.V., Savinkov, P.A., Tumakov, D.N.: Using dynamic reflexive graphs in solving problems of path planning and tactical control of a robotic wheeled platform. News of Higher Educational Institutions 4(48) (2018) 11. Arzumanov, A.A., Chasovskikh, I.A.: Use of a 3D printer in construction. Voronezh Scientif. Technical Bulletin 1(1), 119–121 (2019) 12. Bulgakov, A., Vorobyov, V.: In: Industrial Robots. Kinematics, Dynamics, Control and Management (2020) 13. Karlov, A.E., Postolny, A.A., Savelyeva, E.V.: Industrial exoskeleton for improving labor conditions. In: Science and Education: Domestic and Foreign Experience, pp. 47–51. (2019) 14. Kubalov, A.E., Glashev, A.K., Zairbekova, D.A., Aleksanyan, A.S., Chukhrov, N.M., Morozov, A.V., Sinenko, S.A.: The use of mobile robots in construction research in construction. Exact Sci. 4, 39–45 (2017) 15. Construction works and standard technological processes. Additive technologies. application of three-dimensional printing (3d-printing) in construction. In: General requirements, vol. 495 (2020) 16. Savelyeva, E.V.: Robotic systems for laying small-piece elements during the construction of walls of low-rise residential buildings. In: Science of the young-future of Russia, pp. 160–163. (2017) 17. Tsarichenko, S.G.: Extreme robotics in the Ministry of Emergency Situations of Russia—tasks and prospects. Safety Fire Technol 28 (2012) 18. Yatsun, S.F., Bartenev, V.V., Politov, E.N., Afonin, D.V.: Simulation of the movement of a tractor robot for transporting aircraft around the airfield. Bulletin of the Southwestern State University 22(2), 34–43 (2018) 19. Yousif, K., Bab-Hadiashar, A., Hoseinnezhad, R.: An overview to visual odometry and visual SLAM: applications to mobile robotics. Intell. Indust. Syst 1(4), 289–311 (2015) 20. Pavlovsky, V.E., Pavlovsky, V.V.: SLAM technologies for mobile robots: status and prospects. Mechatronics, Automation, Control 17(6), 384–394 (2016) 21. Asadi, K., Ramshankar, H., Pullagurla, H., Bhandare, A., Shanbhag, S., Mehta, P., Wu, T.: Vision-based integrated mobile robotic system for real-time applications in construction. Autom. Constr. 96, 470–482 (2018)

26 Highly Maneuverable Small-Sized Wheeled Mobile Robotic …

417

22. Bailey, T., Durrant-Whyte, H.: Simultaneous localization and mapping (SLAM): part II. IEEE Robot. Autom. Mag. 13(3), 108–117 (2006) 23. Cheein, F.A.A., et al.: SLAM algorithm applied to robotics assistance for navigation in unknown environments. J. Neuroeng. Rehabil. 7(1), 1–16 (2010) 24. D’Alfonso, L., Griffo, A., Muraca, P., Pugliese, P.: A SLAM algorithm for indoor mobile robot localization using an extended kalman filter and a segment based environment mapping. In: 16th International Conference on Advanced Robotics (ICAR), pp. 1–6. (2013) 25. Gawel, A., Blum, H., Pankert, J., Krämer, K., Bartolomei, L., Ercan, S., Sandy, T.: A fullyintegrated sensing and control system for high-accuracy mobile robotic building construction. In: International Conference on Intelligent Robots and Systems (IROS), pp. 2300–2307. (2019) 26. Luo, R.C., Lai, C.C.: Multisensor fusion-based concurrent environment mapping and moving object detection for intelligent service robotics. IEEE Trans. Industr. Electron. 61(8), 4043– 4051 (2013) 27. Luo, R.C., Chen, K.Y., Hsiao, M.: Visual simultaneous localization and mapping using stereo vision with human body elimination for service robotics. In: IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), pp. 244–249. (2012) 28. Ma, L., Falquez, J.M., McGuire, S., Sibley, G.: Large scale dense visual inertial SLAM. In: Field and Service Robotics, pp. 141–155. (2016) 29. Milford, M., George, A.: Featureless visual processing for SLAM in changing outdoor environments. In: Results of the 8th International Conference on Field and Service Robotics, pp. 569–583. (2014) 30. Nguyen, V., Harati, A., Siegwart, R.: A lightweight SLAM algorithm using orthogonal planes for indoor mobile robotics. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 658–663. (2007) 31. Nguyen, V., Gächter, S., Martinelli, A., Tomatis, N., Siegwart, R.: A comparison of line extraction algorithms using 2D range data for indoor mobile robotics. Auton. Robot. 23(2), 97–111 (2007)

Chapter 27

Control System for Robotic Towing Platform for Moving Aircraft Dmitry Afonin, Alexander Pechurin , and Sergey Jatsun

Abstract The paper considers the possibility of using a robotic mobile wheeled platform to solve the problems of moving aircraft along the airfield to the runway or hangar. As part of the work, a review of modern principles and methods used to control wheeled objects, including a system of wheeled objects, was carried out. A method for controlling the positioning of a wheeled platform using an optical sensor based on an optocoupler matrix has been developed. The design of the apparatus built according to the scheme of a robot with a differential drive is proposed, and a method for coupling the tug and the aircraft through the elastic element of the nose wheel as well as electronic and electrical equipment of the platform is proposed. The algorithm and logic of the control unit operation are given. Particular attention is paid to the synthesis of a multi-channel controller of the RBAS control system. A method is proposed that allows, based on the decomposition of system movements, to obtain the necessary controller parameters based on the qualitative indicators of the transient process. Mathematical modeling of the controlled motion of a wheeled platform has been carried out.

27.1 Introduction The development of aviation technology is associated not only with the creation and modernization of aircrafts, but also with the development of new means [1, 2], directly providing flights. The constant growth of air traffic as well as the increase in the number of aircraft types creates a problem for the logistics of modern airports. Therefore, to perform ground maneuvers with aircraft (AC), a towed aerodrome system (TAS) is used, which allows to significantly reduce noise and air pollution near the airport, as well as to reduce the inefficient consumption of the AC engines resource and provide significant savings of aviation fuel when moving the aircraft on the aerodrome on the thrust of its own engines. D. Afonin · A. Pechurin · S. Jatsun (B) Southwest State University, 94, 50 Let Oktyabrya, 305040 Kursk, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_27

419

420

D. Afonin et al.

27.2 Control Methods for Wheeled Robots The design and control system of a typical robot following a line is described in Jha and Dulal [3]. The principle of operation is based on the detection of a dark line on a contrasting surface using four pairs of IR LEDs and photodiodes arranged horizontally in a single line. Depending on the combination of the readings of the four analog signals from the sensors, the control module determines its state and decides which action to take to continue moving along the line. Increasing the number of light-sensitive elements allows calculating the center of the line relative to the center of mass of the robot by a special algorithm, so in Ryadchikov et al. [4] we used a line of sensors composed of five industrial-made “Eb-Linefinder” sensor boards with two infrared pairs mounted on each board, working on reflection. The essence is to assign a weight coefficient to each sensor, the farther the sensor is from the center, the greater the coefficient. Also presented and the method of finding the line in case of deviation from it, which is to move along the trajectory in the form of a figure of eight, until the detection of the line. To date, the widespread use of artificial intelligence for solving control problems. So, in Azarchenkov et al. [5] the application of a neural network recognizing road markings and returning the value of the angle of rotation of the steering wheel. The authors use end-to-end neural network, which inputs an unprocessed image from the camera with 66 × 200 resolution, and the feature is that no processing is required for the input data and at the output layer of the neural network returns the angle of rotation of the steering wheel in radians.

27.3 Design of a Robotic Aircraft Tug The aircraft tug is a wheeled platform [6, 7], built according to the scheme of a robot with a differential drive—the movement of which is based on two independent leading wheels located on either side of the robot body. Such a design eliminates additional mechanisms for changing the direction of motion, since the change of direction is carried out by changing the relative speed of rotation of its wheels. Figure 27.1a shows a scheme of coupling of the robotic mobile tractor (RMT) and the aircraft by holding the elastic part of the front landing gear strut (tire), which is the coupling element of the tractor and the aircraft. During the movement of the system, elastic deformation occurs in the elastic element of the nose wheel of the aircraft, which creates a force that drives the aircraft (Fig. 27.1b). The apparatus consists of a supporting body 1 with a compartment 2 for the front landing gear of the aircraft, drive wheels 3, and a freely rotating front wheel 4 (Fig. 27.2a). It is proposed to build the rear suspension of the wheel platform based on an independent scheme, including hubs 5, connected to the body by means of levers 8, hydraulic compensators 9, pneumohydraulic cylinders 10, as well as electric motors 18 and gears 19 (Fig. 27.2a, b). Moving the nose wheel of the aircraft in the compartment 2 is provided by moving the nose wheel of the aircraft on the ladder 6,

27 Control System for Robotic Towing Platform for Moving Aircraft

421

Fig. 27.1 Calculation scheme of the tugboat and aircraft coupling

Fig. 27.2 Apparatus design

the position of which is changed by the actuator 7, reliable retention of the wheel in the swing-out is provided by a magnetic lock 11, the latch is mounted between the loops 12 (Fig. 27.2a). In the forward part of the device, there are also electronic elements of the control system: an optoelectronic matrix 13 fixing a contrast line; ultrasonic rangefinders 14 warning of an obstacle presence; a first-person view camera 15 for visual positioning; a GPS module, an inertial measuring module and a single-board computer are located in the compartment 17 (Fig. 27.2a).

27.4 External Environment Perception System The autonomous operation of the tugboat is provided by a set of electronic modules [8–10]. If necessary, the positioning of the robot is carried out in the operator control

422

D. Afonin et al.

mode with the help of data obtained from the video camera mounted on the body. For remote control of the platform, devices for transmitting and receiving radio control signals are used. Determination of the vehicle’s own position and orientation in the autonomous control mode is performed with the help of an inertial measurement unit (IMU) and a GPS module, which transmits data to the onboard calculator. Contrast lines detection, applied all over the airfield, is performed by a module, consisting of a certain number of optical sensors, distributed on a grid. While navigating on the marked runway, collision avoidance is ensured by analyzing the data coming from ultrasonic sensors located on the perimeter of the hull. Processing of data from sensors, peripheral devices, following the control algorithm, as well as generation of control actions are performed by a single-board computer—a compact and powerful computing device, which ensures operation of the robot systems.

27.5 Description of the Positioning Control System Using an Optical Sensor Based on an Optronic Matrix The operation algorithm [11, 12] of the tow vehicle is shown in Fig. 27.3.

Fig. 27.3 General algorithm of the tugger operation

27 Control System for Robotic Towing Platform for Moving Aircraft

423

Fig. 27.4 Scheme of the tractor’s movement on the airfield

27.6 Description of the Work Area Based on the infrastructure and hangar locations, it is possible to make a scheme of the aircraft movement (Fig. 27.4), where A—conventional location of the tractor; B, C, D—hangars of towed aircraft; S 1 …S 6 —reference points of trajectory construction; V 1 …V 4 —decoupling zones; solid and dashed lines denote runway contours and trajectory of movement, respectively.

27.7 Construction of a Piecewise Linear Trajectory For a better quality of towing the aircraft over the airfield, it is advisable to divide the complex trajectory into a number of straight-line segments. Simplified laws of motion allow optimizing the performance of the main computing module and reducing the risk of critical drift from the desired trajectory. As soon as the robot receives the command to move the aircraft from hangar B to takeoff area V 1 , to perform the task the onboard evaluator builds a trajectory of motion, indicated by a solid line in Fig. 27.4, then divides it into straight-line segments (indicated by arrows). The desired position of the center of mass (CoM) of the tugger on each segment of the trajectory is defined by the point PD . The task of the tow vehicle control system is to minimize the deviation of the tow vehicle’s center of mass from the desired position.

424

D. Afonin et al.

Let us assume that the desired position defined by point PD occurs in the plane X0 OY0 according to the laws [13, 14] and is described by equations: (27.1)—for motion along axis M4 X0 and (27.2)—for motion along axis M4 Y0 : [

PD = X PD , Y PD

]T

[ =

n=5 Σ

]T a j · t , constant j

,

(27.1)

.

(27.2)

j=0

[

PD = X PD , Y PD

]T

[ =

constant,

n=5 Σ

]T aj · t

j

j=0

To determine the constants a j on the segment AA1 , write down the conditions determining the position of the point PD at the initial and final moments of time: ]T [ [ ]T ]T 0 [ [ ]T 0 0 = 00 ; t0 ≤ t ≤ tk , P D = X A Y A ; P D = X A Y A ; P˙ D = X˙ PD0 Y˙ PD0 ]T [ [ 0 ]T ]T k [ P¨ D = X¨ PD0 Y¨ PD0 = 0 0 ; P D = X A1 Y A1 ; ]T [ ]T [ [ [ k ]T k ]T P˙ D = X˙ PDk Y˙ PDk = 0 0 ; P¨ D = X¨ PDk Y¨ PDk = 00 . Applying the boundary conditions, we obtain a system of algebraic Eq. (27.3) allowing us to find the constants a j : ⎡

1 ⎢0 ⎢ ⎢ ⎢0 ⎢ ⎢1 ⎢ ⎣0 0

t0 1 0 tk 1 0

t02 2 · t0 2 tk2 2 · tk 2

t03 3 · t02 6 · t0 tk3 3 · tk2 6 · tk

t04 4 · t03 12 · t02 tk4 4 · tk3 12 · tk2

⎤−1 ⎡ ⎤ ⎡ ⎤ X PD0 a0 t05 ⎢ X˙ 0 ⎥ ⎢ a ⎥ 5 · t04 ⎥ ⎥ ⎢ PD ⎥ ⎢ 1 ⎥ ⎥ ⎢ X¨ 0 ⎥ ⎢ ⎥ 20 · t03 ⎥ ⎢ PD ⎥ ⎢ a2 ⎥ · ⎥ ⎥ = ⎢ ⎥. ⎢ ⎢ X PDk ⎥ ⎢ a3 ⎥ tk5 ⎥ ⎥ ⎥ ⎢ ⎥ ⎢ ⎣ X˙ P k ⎦ ⎣ a4 ⎦ 5 · tk4 ⎦ D 20 · tk3 a5 X¨ P k

(27.3)

D

A distinctive feature of this approach is zero initial and final velocities as well as accelerations, which will reduce shock loads on the system caused by abrupt start or stop.

27.8 Determination of Positioning Relative to the Line with the Help of an Optronic Matrix To control the position of the RAS relative to the set trajectory, an optronic matrix (Fig. 27.5) [15, 16] is used, which allows to estimate the deviation of the real position of the RAS from the set one with the accuracy, determined by the distance between the matrix LEDs. The RMB moves along a contrasting stripe plotted on a horizontal

27 Control System for Robotic Towing Platform for Moving Aircraft

425

plane. Each pair of light-sensitive elements is a matrix element. The output value of the sensors depends on the color of the surface reflecting the light. When the sensor detects a contrasting line the output value is “0”, if the sensor is not on the line it returns “1”. When positioning the matrix relative to the line in the position shown in Fig. 27.5a the matrix of output values from the sensors has the form -opt_mat = [101; 101; 101], and hence, the apparatus moves along the line. If the output matrix of the sensors has the form [110; 101; 011] and [011; 101; 110] (Fig. 27.5b, c), the ◦ ◦ apparatus is rotated by an angle of φ1 ≈ +45 and φ1 ≈ −45 relative to the line, indicating the need to adjust the position. Matrix values [111; 000; 111] (Fig. 27.5d) signal the perpendicular orientation of the apparatus relative to the line. Particular attention should be paid to the cases shown in Fig. 27.5e, f, because the ◦ ◦ angle of rotation relative to the line is equal to 90 ≤ φ1 ≤ 180 , they are due to the response to the perturbation or destabilizing effects from the BC. Since in cases (e) and (f) (Fig. 27.5) the output matrix values are the same as in cases (a), (c) (Fig. 27.5), it is necessary to describe a software tool that allows to correctly interpret the current state of the matrix. The comparison of matrix states in the current and previous iterations is proposed to be used as such a tool. For example, in the current iteration the state of the matrix is represented as opt_mati = [011; 101; 110], and in the

Fig. 27.5 Determining the orientation relative to the line using an optronic matrix

426

D. Afonin et al.

previous iteration, the matrix has the form opt_mati−1 = [101; 101; 101], then the ◦ apparatus is oriented to the angle of φ1 ≈ −45 , because the inertial characteristics, will not allow to turn to a larger angle in a short time interval Δt ≤ 0.001 c.

27.9 Synthesis of Multi-channel Controller for Robot Motion The structural diagram of the proposed automatic control system of the robot motion is shown in Fig. 27.6. The forces generated by the wheels are calculated based on the readings of the optocoupler matrix and the logical conditions presented below. It should be noted that the width of the contrast line must be equal to twice the distance between neighboring sensors wl = d. The output values of the sensors form a matrix: opt_mat = [C11 C12 C13 ; C21 C22 C23 ; C31 C32 C33 ]. The equations of the controller are based on the mismatch of the desired coordinate X C1 and angle of rotation φ1 with the given values X 1∗ and φ1∗ : ⎧

U0 = K PX · ex + K D X · U1 = K Pφ · eφ

dex dt

⎧ ;

Fig. 27.6 Block diagram of the automatic control system

ex = P | D − X C|1 . eφ = |φ1∗ − φ1 |

27 Control System for Robotic Towing Platform for Moving Aircraft

427

The control voltage is calculated from the current state of the matrix opt_mat. Comparison of the matrix states with the given ones is done using MATLAB software package function “is equal”, which returns “1” if they are the same and “0” if they are different. To create instructions loaded into the computational module, it is necessary to consider all possible combinations of optronic matrix sensors location relative to the contrast line, with matrix dimension 3 × 3 positions and the logic of influence are determined by the operator. The construction of the control voltage formation logic is based on minimizing the deviation of the sensor C22 from the contrasting line. The combinations of possible positions and the logic for calculating the control forces that counteract the deviation from the line are presented below: 1. 2.

2.1. 3. 3.1. 4.

5.

5.1. 6.

6.1. 7.

7.1. 7.2. 7.3. 7.4. 8. 8.1. 8.2.

◦

if opt_mat = [101; 101; 101] ⇒ U = [U0 , U0 ]T , φ1 ≈ 0 ; if opt_mat = [110; 101; 011] ⇒ U = [−U0 , U0 ]T + [U1 , U1 ]T , also in this case, it is possible to determine the approximate value angle φ1 from a ( ) of the ◦ right triangle C22 C12 C13 like: φ1 ≈ arccos d/2 · d 2 ≈ 45 ; if opt_mat = [011; 101; 110] ⇒ U = [U0 , U0 ]T + [U1 , 0]T ; if opt_mat = [110; 101; 101] ⇒ U = [−U0 , U0 ]T + [0, U1 ]T , φ1 ≈ 2 · √ 2 d/ (2 · d) + d 2 from a triangle C12 C32 C13 ; if opt_mat = [011; 101; 101] ⇒ U = [U0 , −U0 ]T + [U1 , 0]T ; ◦ if opt_mati = [101; 101; 101], φ1 ∈ [70◦ ; 110◦ ], φ1 ≈ 90 it is necessary to compare with the previous state of the matrix if opt_mati−1 = [110; 101; 011] ⇒ U = [−U1 , U1 ]T , if opt_mati−1 = [011; 101; 110] ⇒ U = [U1 , −U1 ]T ; if opt_mat = [001; 011; 111] ⇒ U = [U0 , U0 ]T + [−U1 , U1 ]T , until C22 /= 0, as soon as C22 = 0 the second condition is checked: if opt_mat = ◦ [100; 001; 011] ⇒ U = [U0 , U0 ]T + [0, U1 ]T , φ1 ≈ 45 ; T if opt_mat = [100; 110; 111] ⇒ U = [U0 , U0 ] , while C22 /= 0, as soon as C22 = 0, if opt_mat = [001; 100; 110] ⇒ U = [U0 , U0 ]T + [U1 , 0]T ; if opt_mat = [111; 110; 100] ⇒ U = [−U1 , U1 ]T , while opt_mat /= [100; 110; 111], as soon as opt_mat = [100; 110; 111] 5.1 is considered, ◦ φ1 ≈ 45 ; if opt_mat = [111; 011; 001] ⇒ U = [U1 , −U1 ]T , while opt_mat /= [001; 011; 111], as soon as opt_mat = [001; 011; 111] ⇒ considered in 5; if opt_mat = [110; 110; 110] ⇒ U = [−U0 , U0 ]T , while opt_mat /= [100; 110; 111] ∨ [110; 111; 111] ∨ [101; 110; 110] ∨ [110; 110; 111], φ1 ∈ [0◦ ; 20◦ ], based on the condition, there are 4 possible positions, the actions for each are discussed in the subparagraphs; if opt_mat = [100; 110; 111] ⇒ considered in 5.1; if opt_mat = [110; 111; 111] ⇒ considered in 8; if opt_mat = [101; 110; 110] ⇒ considered in 10; if opt_mat = [110; 110; 111] ⇒ considered in 9; if opt_mat = [110; 111; 111] ⇒ U = [U0 , U0 ]T , while C22 /= 0, as soon as C22 = 0 three states are possible: if opt_mat = [001; 100; 110] ⇒ considered in 5.1; ◦ if opt_mat = [101; 101; 110] ⇒ U = [U0 , U0 ]T + [U1 , 0]T , φ1 ≈ 26 ;

428

D. Afonin et al. ◦

if opt_mat = [011; 100; 111] ⇒ U = kc · [U0 , U0 ]T + [U1 , 0]T , φ1 ≈ 66 ; ◦ if opt_mat = [110; 110; 111] ⇒ U = [U0 , U0 ]T , φ1 ≈ 32 for now C22 /= 0, two states are possible: ◦ 9.1. if opt_mat = [101; 100; 110] ⇒ U = kc · [U0 , U0 ]T + [U1 , 0]T , φ1 ≈ 38 ; 9.2. if opt_mat = [101; 101; 110] ⇒ considered in 8.2; ◦ 10. if opt_mat = [101; 110; 110] ⇒ U = [U0 , U0 ]T , φ1 ≈ 32 while C22 /= 0; 10.1. if opt_mat = [101; 101; 110] ⇒ is discussed in 8.2; 11. if opt_mat = [111; 111; 110] ⇒ U = [−U0 , U0 ]T , until opt_mat /= [110; 111; 111], as soon as opt_mat = [110; 111; 111] ⇒ considered in 8. In the case of leaving the line the robot follows the logic of the last defined position. Thus, the work of the control unit consists in generating control voltages according to the comparison of the current state of the matrix with the predetermined ones. 8.3. 9.

27.10 Example of Simulation of RMB Motion Along a Contrasting Line Consider the mathematical model of the RBAS, which allows us to determine the) ( dependence of the coordinates of the center of mass of the aircraft C1 = X C1 , YC1 from governing forces F11 , F12 . The derivation of the equations of motion of the robot is based on the Langrage equations [17–22]. The system of differential equations of the SBS in the coordinate system (27.1) has the form (27.4): ⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ¨ ⎪ ⎪ ⎩ φ1 =

V˙ O(1)1 =

(1) F11 +F12 −P21 +m C1 ·|O1 C1 |·φ˙ 12 −Fr(1) −μVO ·VO(1) X X

1

1

m1+

2·JW 1 2 rW 1

(1) |A1 O1 |·(F11 −F12 )−P21 ·|O1 D1 |−m C1 ·|O1 C1 |·φ˙ 1 ·VO(1) −Fr(1) ·|O1 C1 |−μφ˙1 ·φ˙ 1 Y Y

J1 +

| A 1 O 1 |2 2 rW

(27.4)

1

·JW1

1

The angular velocity and linear velocity of the RMB are found using the numerical method of integration and are represented by Eqs. (27.5) and (27.6), respectively: VO(1)1 = VO(1)1 + V˙ O(1)1 · dt;

(27.5)

φ˙ 1i = φ˙ 1i−1 + φ¨1i · dt.

(27.6)

i

i−1

i

The transition from the local RMB coordinate system to the global one is described by Eqs. (27.7–27.9): ]T [ (0) V O1 = VO(1)1 · cos(φ1 ) VO(1)1 · sin(φ1 ) ; [ (0) r (0) D1 = S1 + T10 · |O1 D1 |

0

]T

;

(27.7) (27.8)

27 Control System for Robotic Towing Platform for Moving Aircraft

[ r C(0)1 = S1(0) + T10 · |O1 C1 | where T10

0

]T

,

429

(27.9)

| | | cos(φ1 ) − sin(φ1 ) | | is the matrix of the rotation of the RMB around | =| sin(φ1 ) cos(φ1 ) | (0)

the axis, and S1(0) = S1(0) + V O1 · dt. The following values are taken as parameters i i−1 i of the RBAS mathematical model: m C1 = 1kg—mass of the RMB without wheels, m W1 = 0.5 kg—the weight of the wheel of the RMB, m 1 = 1.5 ) mass2 of ( kg—total 2 kg m — RMB, r W1 = 0.3 m—the radius of the RMB wheel, JW1 = 0.5 · m W1 · r W 1 2 moment of inertia of the RBM wheel, J1 = JC1 + m C1 · |O1 C1 | + 2 · m W1 · JW1 = 0.12 kg m2 —the total moment of inertia of the RMB, t0 ≤ t ≤ tk , t0 = 0; tk = 10 s; Δt = 0.1 s.

27.10.1 Simulation Results Figures 27.7, 27.8 and 27.9 show the results of mathematical modeling.

Fig. 27.7 Graph of the change in the trajectory of movement

Fig. 27.8 Graphs of change in traction forces of wheels

430

D. Afonin et al.

Fig. 27.9 Graphs of changes: a the desired coordinate X C∗ and the real X C ; b error changes ΔX C ; c the desired coordinate YC∗ and the real YC ; d error changes ΔYC ; e desired speed VC∗ and actual VC ; f error plots ΔVC

27.11 Conclusion The method of controlling the positioning of an airfield tug using an optical sensor based on an optronic matrix has been developed; the design of the apparatus as well as the method of coupling the tugboat and aircraft through a coupling elastic element are proposed; the algorithm and logic of the control unit operation as well as the synthesis of a multi-channel regulator of the control system of the aircraft towing system are given. The method of trajectory control based on the decomposition of movements of the system is proposed. Acknowledgements The article was prepared with the support of the Strategic Project “Priority– 2030. Creation of robotic tools to expand the functionality of a person”.

References 1. Shi, A., Cheng, S., Sun, L., Liu, J.: Multi-robot task allocation for airfield pavement detection tasks. In: 2021 6th International Conference on Control, Robotics and Cybernetics (CRC), pp. 62–67 (2021) 2. Chernomorsky, A.I., Lelkov, K.S., Kuris, E.D.: About One way to increase the accuracy of navigation system for ground wheeled robot used in aircraft parking. Smart Sci. 8(4), 219–226 (2020) 3. Jha, S.K., Dulal, S.: Line Following Robot. Diss. Kathmandu University (2016) 4. Ryadchikov, I.V., et al.: Creating a robot of autonomous motion along the line. In: Technical Sciences: Problems and Prospects, pp. 19–25 (2015)

27 Control System for Robotic Towing Platform for Moving Aircraft

431

5. Azarchenkov, A.A., Lyubimov, M.S., Lushkov, V.I.: The algorithm of unmanned control of a wheeled robot using a neural network. In: Information Systems and Technologies (IST-2020), pp. 240–245 (2020) 6. Szcz˛es´niak, G., Nogowczyk, P., Burdzik, R., Konieczny, Ł: Analysis of the chassis design for a high mobility wheel platform. J. Meas. Eng. 4(2), 52–57 (2016) 7. Kundu, A.S., Mazumder, O., Lenka, P.K., Bhaumik, S.: Design and performance evaluation of 4 wheeled omni wheelchair with reduced slip and vibration. Proc. Comput. Sci. 105, 289–295 (2017) 8. Lösch, R., Grehl, S., Donner, M., Buhl, C., Jung, B.: Design of an autonomous robot for mapping, navigation, and manipulation in underground mines. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1407–1412 (2018) 9. Aldridge, R., Brandt, T., Parikh, C.: Autonomous robot design and build: novel hands-on experience for undergraduate students. In: Proceedings of the 2016 ASEE North Central Section Conference, pp. 18–19 (2016) 10. Ohi, N., Lassak, K., Watson, R., Strader, J., Du, Y., Yang, C., Gu, Y.: Design of an autonomous precision pollination robot. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7711–7718 (2018) 11. Iwendi, C., Alqarni, M.A., Anajemba, J.H., Alfakeeh, A.S., Zhang, Z., Bashir, A.K.: Robust navigational control of a two-wheeled self-balancing robot in a sensed environment. IEEE Access 7, 82337–82348 (2019) 12. Saenz, A., Santibañez, V., Bugarin, E., Dzul, A., Ríos, H., Villalobos-Chin, J.: Velocity control of an omnidirectional wheeled mobile robot using computed voltage control with visual feedback: experimental results. Int. J. Control Autom. Syst. 19(2), 1089–1102 (2021) 13. Richter, C., Bry, A., Roy, N.: Polynomial trajectory planning for aggressive quadrotor flight in dense indoor environments. In: 16th International Symposium of Robotics research, pp. 649– 666 (2016) 14. Wang, H., Wang, H., Huang, J., Zhao, B., Quan, L.: Smooth point-to-point trajectory planning for industrial robots with kinematical constraints based on high-order polynomial curve. Mech. Mach. Theor. 139, 284–293 (2019) 15. Bartenev, V.V.: Dynamics of Controlled Motion of a Mobile Robot with Two Independent Driving Wheels: Ph. Southwestern State University (2010) 16. Rafi, R.H., Das, S., Ahmed, N., Hossain, I., Reza, S.T.: Design & implementation of a line following robot for irrigation based application. In: 2016 19th International Conference on Computer and Information Technology (ICCIT), pp. 480–483 (2016) 17. Martynenko, Yu.G.: Motion control of mobile wheeled robots. Fundamental Appl. Math. 8, 29–80 (2005) 18. Dhaouadi, R., Hatab, A.A.: Dynamic modelling of differential-drive mobile robots using lagrange and newton-euler methodologies: a unified framework. Adv. Robot. Autom. 2(2), 1–7 (2013) 19. Liang, Z., Yuan, J.: Modeling and optimisation of high-efficiency differential-drive complementary metal-oxide-semiconductor rectifier for ultra-high-frequency radio-frequency energy harvesters. IET Power Electronics 12(3), 588–597 (2019) 20. Martins, F.N., Sarcinelli-Filho, M., Carelli, R.: A velocity-based dynamic model and its properties for differential drive mobile robots. J. Intell. Rob. Syst. 85(2), 277–292 (2017) 21. Seegmiller, N., Kelly, A.: High-fidelity yet fast dynamic models of wheeled mobile robots. IEEE Trans. Rob. 32(3), 614–625 (2016) 22. Hendzel, Z., Rykała, Ł.: Modelling of dynamics of a wheeled mobile robot with mecanum wheels with the use of Lagrange equations of the second kind. Int. J. Appl. Mech. Eng. 22(1) (2017)

Chapter 28

Simple Task Allocation Algorithm in a Collaborative Robotic System Rinat Galin , Roman Meshcheryakov , and Mark Mamchenko

Abstract The paper presents an algorithm of assigning a sequence of tasks in the collaborative robotic system (CRS). These tasks are represented both as a directed graph, and a timeline with the operations’ start events. The algorithm focuses on assigning the operations based on the availability of CRS members and their effectiveness. After the assignment, the execution time of the operation is recalculated according to the efficiency of the assigned humans and cobots, modifying the weight of the corresponding graph edge and rebuilding the time sequence after the current time event. If these conditions are not met, the assignment for the operation is postponed until the next event. The algorithm makes the transition from the current time event to the nearest one and traverses the graph from the first vertex to the last one. The analysis of modeling results shows that the use of the algorithm for a sequence of interdependent operations that requires human–robot collaboration allows reducing the time of execution of both the technological process and its individual operations. The algorithm can be used to test various hypotheses for the optimal number of participants to perform the operations by the CRS as a team of heterogeneous participants.

28.1 Introduction Within the collaboration, humans and cobots are required to perform a number of sequential tasks, when it is impossible to initiate the follow-up operation (or operations) only after the completion of the previous one (ones). Thus, any technological process, including the one requiring the collaboration of humans and robots, can be represented as a directed graph (digraph), where the edges represent the duration of the operations and the vertices are the completion events of the previous operation R. Galin (B) · R. Meshcheryakov · M. Mamchenko V.A. Trapeznikov Institute of Control Sciences of Russian Academy of Sciences, 65, Profsoyuznaya Str., 117997 Moscow, Russia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 A. Ronzhin and V. Pshikhopov (eds.), Frontiers in Robotics and Electromechanics, Smart Innovation, Systems and Technologies 329, https://doi.org/10.1007/978-981-19-7685-8_28

433

434

R. Galin et al.

(the first vertex in the digraph corresponds to the beginning of the technological process). Work time optimization methods and techniques using digraphs to represent the technological process with a limited number of participants engaged are known and consolidated into the NP-hard task of building the optimal schedule for the project (RCPSP, Resource-Constrained Project Scheduling Problem) [1–15]. For example, the work [16] presents a generalized approach to the solution of this class of tasks, taking into account the different types of competence of the participants, optimal tasks distribution, assigning a minimum number of workers with a limited budget, as well as the optimal scheduling model based on linear programming. There are also other approaches and ways of solving this problem. For example, in paper [17] proposes the solution based on the classification tree method for the CRS, taking into account the types of performers for the operations (human workers only, cobots only, human workers and cobots), duration and other characteristics of individual operations. And authors in article [18] propose to solve similar problems using the Markov decision process. However, despite the above methods and approaches to solving the RCPSP problems, they do not take into account the various number of both human workers and cobots that can be assigned to a specific operation (or specific operations that start simultaneously). The number of assigned human workers and cobots directly affects the duration of the operation, and it is possible to determine their minimum and maximum possible number (the introduction of the maximum possible number of assigned human workers and cobots is due to the fact that the larger number of actors does not contribute to the reduction of the operation time). However, the number of participants available for assigning at any given time moment may be less than the maximum possible for the corresponding operation. In addition, minimizing the time of each operation requires recalculating the number of participants that can be assigned to subsequent operations. The case of parallel tasks—when several operations begin simultaneously—requires separate consideration. Subsequent operations can be in the form of a series of operations, which «converge» in one vertex of the digraph (including in the one that corresponds to the end of the technological process). Thus, it is necessary to consider the task of optimizing the schedule of works and assigning the human workers and cobots for the above cases, including parallel operations (operation series) and various assignment scenarios.

28.2 Problem Statement and Algorithmic Solution n Introduce  mthe CRS as a set of human workers H = {h i }i=1 and collaborating robots B = b j j=1 , where n is the number of human workers, m is the number of cobots. Each cobot and person in the CRS has the following characteristics:

• unique ordinal number (separate for humans and cobots) as a positive natural number;

28 Simple Task Allocation Algorithm in a Collaborative Robotic System

435

• the number of the operation to which they are assigned (if the value is zero, then the CRS member is not assigned to any operation and is considered free); • performance indicator (rh i ∈ (0..1]—for i human worker and rb j ∈ (0..1] for j cobot). Suppose that the technological process is a sequence of atomic operations, i.e., operations that cannot be decomposed into simpler operations. Each operation has the following characteristics: • unique ordinal number as a positive natural number; • the minimum number of human workers required to perform the j operation— n jmin ; • the maximum number of human workers required to perform the j operation— n jmax ; • the minimum number of cobots required to perform the j operation—m jmin ; • the maximum number of cobots required to perform the j operation—m jmax ; • the number of human workers assigned to execute the j operation—n j ; • the number of cobots assigned to execute the j operation—m j ; • operation time (duration)—top.real j ; • standard operation time (standard duration) —top.st j (operation time with a minimum number of human workers and cobots with single efficiency). Present the technological process as a weighted digraph G = (V , E), where V = {v1 , v2 , . . . , vk } is a set of vertices corresponding to the events of the end of the previous operations, v1 is a source vertex, corresponds to the start time of the technological process, vk is a sink vertex, corresponds to the end time of the technological process. The weights of the edges correspond to the duration of the operations in seconds. To perform each operation, human workers and cobots are assigned from the free set of free workers, i.e., not performing any other operations. A person or a cobot can be assigned only to one operation and perform no more than one operation simultaneously. One or more operations are required before starting. After each operation is completed, human workers and cobots assigned to it are released. At the time of the start of each new operation, it is required to appoint a certain number of cobots and people, which should be no less than the minimum possible, but no more than the maximum possible number for a certain operation. Introduce the following assumption: human workers and cobots with the highest efficiency values among the available ones should be assigned to the operation. Let n jav and m jav be the numbers of people and cobots available at the beginning of the j operation respectively, H n jav B m jav —the sets of set of people and cobots available (not engaged in any operation) at the time of the start of the j operation respectively. Then four scenarios are possible: • if the number of human workers and (or) cobots available is greater than the maximum possible value for this operation (n jav > n jmax or m jav > m jmax or  n jav > n jmax ), then it is possible to assign any number of people and (or) m jav > m jmax

436

•

•

•

•

R. Galin et al.

cobots with the highest efficiency values, from the  minimum possible to the  maximum possible inclusive (n j ∈ n jmin ; n jmax or m j ∈ m jmin ; m jmax or    n j ∈  n jmin ; n jmax  ); m j ∈ m jmin ; m jmax if the number of human workers and (or) cobots available is less than the maximum possible, but more than the minimum possible value for this operation (n jmin <  n jmin < n jav < n jmax ), then it is n jav < n jmax or m jmin < m jav < m jmax or m jmin < m jav < m jmax possible to assign a number of people and (or) cobots with the highest efficiency values, from the minimum possible to the entire number people and    of available     n j ∈  n jmin ; n jav  ; (or) cobots n j ∈ n jmin ; n jav or m j ∈ m jmin ; m jav or m j ∈ m jmin ; m jav if the number of human workers and (or) cobots available isequal to the minimum n jav = n jmax )), then possible for this operation (n jav = n jmax or m jav = m jmax or m jav = m jmax  n j = n jav ); they are all assigned to the operation (n j = n jav or m j = m jav or m j = m jav if the number of human workers and (or) cobots available is less than the minimum possible for this operation, the assignment of the workers will be postponed until the number of human workers and (or) cobots available increases at least to the minimum possible value for this operation. In the first two scenarios, when cobots and human workers are assigned to an operation, its duration time may change. Since the standard duration of the operation is calculated at the minimum number of participants with single efficiency, the real duration top.real j of the j operation with a different set of human workers and cobots engaged (n j people and m j cobots) will be calculated as follows: top.real j

  n jmin + m jmin × top.st j = n j . m j s=1 r bs k=1 r h k +

(28.1)

After the calculation, this value should be assigned to the weight of the edge on the digraph that corresponds to the duration of this operation. It is possible that after some operation several other independent operations or operation series start and continue. Their completion may be a condition for starting some other operation or other sequence of operations. Each case of the beginning of several operations (or sequences of operations) on a digraph will be represented as «branching»—the outdegree of the vertex corresponding to the moment of completion of the previous operation will be the number of starting operations (or sequences of operations). Then for each «branching» case it is required to determine the composition of people and cobots to be assigned to each operation at once in the way indicated above. Separate consideration should be given to the case of assigning workers, when the number of people and (or) cobots available is less than the minimum required at the time of the start of one or more operations, and the assignment is postponed until

28 Simple Task Allocation Algorithm in a Collaborative Robotic System

437

the most temporally proximate operation starts (when the number of cobots and (or) human workers available may change). The complexity of the problem lies in finding the most temporally proximate operation, which may not be next in order. To solve it, introduce a function that converts the digraph into a time sequence. The essence of the function lies in finding all the paths on the digraph sequentially from the first to each other vertex, summing up the weights of all the edges for each of the found paths, and selecting a path to each vertex that has the maximum sum of the weights of the edges. The calculated maximum sum of the weights of the edges from the first to any other j vertex will correspond to the start time of the operation (or operations) that is (are) represented by this vertex of the digraph. In addition, the time values corresponding to the beginning of the operation(s) are mapped to the corresponding vertices of the digraph resulting in a vector of time values and vertex numbers. This function is presented as a pseudocode in Algorithm 1 “Converting the digraph of the technological process into a time sequence.” Input: G—digraph of the technological process. Output: Timeseq—vector with a set of time values of starting events of the operations and corresponding vertices numbers of the digraph (). 1:

function [Timeseq] = graph_to_timeseq(G)

2:

Timeseq(1,1) = 0;

3:

Timeseq(1,2) = 1;

4:

for each Node in G.Nodes from 2 to end

5:

[paths] = allpaths(G, from 1 to Node);

6:

for each path in paths

7:

time(path) = sum(G.Edges.Weight in path);

8:

end for

9:

Timeseq(Node, 1) = max(time);

10:

Timeseq(Node, 2) = Node;

11:

end for end function

It should be noted that, as a result of the assignment, the duration of the operation(s) may change, resulting in changes of the respective weights of the edges in the digraph. Therefore, it is also necessary to update the time sequence of the operations after each assignment, including if there is more than one subsequent vertex in the graph. To store information on operations on which the assignment has been postponed, introduce the array delayed, which is a set of pairs of vertex numbers on the digraph corresponding to the start and the end of each of the suspended operations. During each assignment to the operation(s), the possibility of using at least a minimum number of human workers and cobots is checked. If the number of available cobots and (or) people is less than the minimum required, the assignment to the operation is terminated, and the corresponding pair of vertex numbers is added to the delayed array.

438

R. Galin et al.

Make a generalized algorithm of assigning workers to an operation of the technological process as follows. In the beginning, a time sequence is formed from the digraph. Starting from the first-time event (the first start of the operation(s), the presence of delayed operations in the delayed array is first checked. It the array is not empty, then an attempt is made to assign workers to of the postponed operations. For each of the suspended operations, then the number of available cobots and human workers is checked. If it is less than the minimum required for a certain operation, the assignment process is terminated. When the above conditions are met, workers are assigned, and the corresponding edge weight of the digraph is modified. The time sequence is then rebuilt based on the new digraph. The algorithm of assigning workers to the postponed operations is given in Algorithm 2 “Assigning workers to a postponed operation.” Input: H, B—sets of human workers and cobots, op—set of operations, G— digraph, v_from, v_to—start and end vertices of the postponed operation on the digraph. Output: G—modified digraph, H, B—modified sets of human workers and cobots, delayed—set of pairs of vertex numbers on the digraph corresponding to the start and the end of each of the suspended operations, op—modified set of operations. function [G, H, B, delayed, op] = assign_to_ops_delayed (H, B, op, G, v_from, v_to) predecessors ← predecessors(G, v_from); if predecessors is notempty for each predecessor in predecessors for each h i in H if h i . current_op = op(predecessor) h i . current_op ← 0 end if end for for each b j in B if b j . current_op = op(predecessor) b j . current_op ← 0 end if end for end for end if num = findedge(G, v_from, v_to) B m jav ← B(B.current_op = 0) m jav ← size(B m jav ) H n jav ← H(H.current_op = 0) n jav ← size(H n jav ) n jmin ← op(from v_from to v_to).Min_human (continued)

28 Simple Task Allocation Algorithm in a Collaborative Robotic System

439

(continued) function [G, H, B, delayed, op] = assign_to_ops_delayed (H, B, op, G, v_from, v_to) n jmax ← op(from v_from to v_to).Max_human m jmin ← op(from v_from to v_to).Min_cobot m jmax ← op(from v_from to v_to).Max_cobot if n jav ≥n jmin if n jmin ≤ n jav < n jmax n jmax ←n jav end if else break end if if m jav ≥m jmin if m jmin ≤ m jav < m jmax m jmax ←m jav end if else break end if   for n j ∈ n jmin ..n jmax do: n j

H j ← h ji |rh ji = max(any h in H n jav )

i=n jmin

h ji . current_op ← op(from v_from to v_to); end for   for m j ∈ m jmin ..m jmax do: m j

B j ← b jk |rb jk = max(any b in B m jav )

k=m jmin

b jk . current_op ← op(from v_from to v_to) end for G.Edge(from v_from to v_to).weight

  n jmin +m jmin ×top.stop( f r om v_from t o v_to)

n j m j rh j + i=n rh j k=n jmin

k

jmin

i

op(from v_from to v_to).human = n j op(from v_from to v_to).cobot = m j delete row [v_from v_to] from delayed end function

Similarly, workers are assigned to operations (operations) starting from the first vertex, with the digraph and the time sequence also being modified after each successful assignment. The nearest time event (on the timeline) corresponding to the beginning of the subsequent operation (operations) is then selected. The above

440

R. Galin et al.

actions are repeated until the last vertex in the digraph, which corresponds to the end of the technological process. The algorithm of assigning workers to one operation is given in Algorithm 3 “Assigning workers to one operation.” Input: H, B—sets of human workers and cobots, op—set of operations, G— digraph, v j —vertex, corresponding to the start time of the operation, v j —vertex, corresponding to the start time of the operation, delayed—set of pairs of vertex numbers on the digraph corresponding to the start and the end of each of the suspended operations. Output: G—modified digraph, H, B—modified sets of human workers and cobots, delayed—modified set of pairs of vertex numbers on the digraph corresponding to the start and the end of each of the suspended operations, op—modified set of operations. function [G, H, B, delayed, op] = assign_to_ops(H, B, op, G, v j , delayed) predecessors ← predecessors(G, v j ) if predecessors is notempty for each predecessor in predecessors for each h i in H if h i . current_op = op(predecessor) h i . current_op ← 0 end if end for for each b j in B if b j . current_op = op(predecessor) b j . current_op ← 0 end if end for end for end if successors = successors(G, v j ) for each successor in successors B m jav ← B(B.current_op = 0) m jav ← size(B m jav ); H n jav ← H(H.current_op = 0) n jav ← size(H n jav ); n jmin ← op(from v j to successor).Min_human n jmax ← op(from v j to successor).Max_human m jmin ← op(from v j to successor).Min_cobot m jmax ← op(from v j to successor).Max_cobot if n jav ≥n jmin if n jmin ≤ n jav < n jmax (continued)

28 Simple Task Allocation Algorithm in a Collaborative Robotic System

441

(continued) function [G, H, B, delayed, op] = assign_to_ops(H, B, op, G, v j , delayed) n jmax ←n jav end if else delayed(end + 1) = [from, successor] break; end if if m jav ≥m jmin if m jmin ≤ m jav < m jmax m jmax ←m jav end if else delayed(end + 1) = [from, successor] break; end if   for n j ∈ n jmin ..n jmax do: n j

H j ← h ji |rh ji = max(any h in H n jav )

i=n jmin

h ji . current_op ← op(from v j to successor) end for   for m j ∈ m jmin ..m jmax do: m j

B j ← b jk |rb jk = max(any b in B m jav )

k=m jmin

bjk . current_op ← op(from v j to successor) end for G.Edge(from v j to successor).weight ←

  n jmin +m jmin ×top.st op(from v j to successor)

n j m j rh j + i=n rh j k=n jmin

k

jmin

i

op(from v j to successor).human = n j op(from v j to successor).cobot = m j end for end function

The generalized algorithm for calculating the minimum time of the technological process is presented in Algorithm 4 “Generalized algorithm for calculating the minimum time of the technological process.” As the time of the technological process to be minimized, use the maximum sum of the weights of the edges for all paths from the source to the sink vertices.

442

R. Galin et al. n←n

1: 2: 3:

m←m  n H ← h j ← |number ← i|rh i ← (0..1]|curr ent_op ← 0 i=1 m  B ← b j ← |number ← j|rb j ← (0..1]|curr ent_op ← 0 j=1

4:

G ← digraph(s ← s, t ← t, weights weights)

5:

Op {opw |number w |min_humanmin_human . . . . . . |max_humanmax_human|min_cobotmin_cobot . . . . . . | min_cobotmin_cobot|max_cobotmax_cobot . . . G.Edges

. . . |num_human[]|num_cobot[]|st_tst_t}w=1 6:

delayed ← []

7:

[Timeseq] ← graph_to_timeseq(G)

8:

t_time ← Timeseq(1).Time

9:

v_vert ← Timeseq(2).Node

10:

while v_vert ! = G.Node(end) do

11: 12: 13:

if delayed is notempty for each op_delayed in delayed do [G, H, B, delayed, op] = assign_to_ops_delayed(H, B, op, G, …

14:

… v_from ← op_delayed(1), v_to ← op_delayed(2))

15:

[Timeseq] ← graph_to_timeseq(G)

16: 17: 18: 19: 20: 21: 22: 23: 24:

end for else for each v_vert in [v_vert] [G, H, B, delayed, op] = assign_to_ops(H, B, op, G, v_vert, … … delayed) [Timeseq] ← graph_to_timeseq(G) end for end if [G, H, B, delayed, Timeseq, Nodetime] ← assign_to_ops(H, B, op, … … G, v_vert)

25:

t_time ← [min(Timeseq(:).Time > t_time)]

26:

v_vert ← Timeseq(:).Node where Timeseq(k).Time = t_time

27:

end while

28:

[paths] = allpaths(G, from 1 to G.Node(end))

29:

for each path in paths

30:

t1(path) = sum(G.Edges.Weight in path)

31:

end for

32:

t = max(t1);

28 Simple Task Allocation Algorithm in a Collaborative Robotic System

443

28.3 MATLAB Simulation To demonstrate the functionality of the algorithm we simulated a simple technological process, the process of assigning members of a CRS to certain operations, and compared the overall time the technological processes after the use of the proposed algorithm to the original time value in MATLAB environment. The following limitations and assumptions have been made for modeling: 1. A CRS has n = 25 human workers and m = 20 cobots. The efficiency for all n is the same and equals 0.5 (∀h human workers in the set H = {h i }i=1  i ∈m H : rh i = 0.5,i ∈ [1, n]), the efficiency for all cobots from the set B = b j j=1 is singular ∀b j ∈ B : rb j = 1, j ∈ [1, m]. 2. CRS members are required to perform 11 atomic operations. Each operation implies the mandatory collaboration of people and cobots for its execution. The specified parameters for the operations are presented in Table 28.1. 3. The technological process in the form of a digraph is presented in Fig. 28.1, and the start time of the operations and the corresponding vertices are shown in Table 28.2. Table 28.1 Parameter values of the operations Operation number

n min

n max

m min

m max

top.st

1

1

10

1

10

14

2

1

10

1

10

18

3

1

10

1

10

18

4

1

10

1

10

26

5

1

10

1

10

20

6

1

10

1

10

28

7

1

10

1

10

8

8

1

10

1

10

10

9

1

10

1

10

10

10

1

10

1

10

18

11

1

10

1

10

16

3; 32

1; 0

t = 14

t = 26

8; 88

4; 58

2; 14

10; 106

7; 78 5; 32

t = 28

6; 60

Fig. 28.1 Technological process as a weighted directed graph

9; 88

444 Table 28.2 Start time of the operations and the corresponding vertices

R. Galin et al. Digraph vertex number

Start time of the corresponding event

1

0

2

14

3

32

4

58

5

32

6

60

7

78

8

88

9

88

10

106

As a result of the algorithm, it was possible to assign to each operation the number of human workers and cobots that was significantly more than the minimum required. This allowed to reduce the duration time of the operations. The numbers of people and cobots assigned to operations (n j and m j ) as well as the duration time values of the operations (top.real ) are given in Table 28.3. Table 28.4 shows the start time of the operations on the digraph after the use of the proposed algorithm, the modified (time-reduced) technological process as a digraph is presented in Fig. 28.2. Tables 28.2 and 28.4 show that the total time of the technological process has been reduced by more than three times. Table 28.3 shows that the minimum number of people and cobots assigned to an operation was two human workers and two cobots, with a maximum of 10 people and 8 cobots. On average, 10 participants were assigned to an operation, with the minimum and maximum number of workers for an operation being 6 and 18 members of the CRS, respectively. Table 28.3 Parameter values of the operations (after the use of the proposed algorithm)

Operation number

nj

mj

top.real

1

7

4

3.7

2

7

8

3.1

3

3

3

8.0

4

4

4

8.7

5

2

4

8.0

6

5

6

6.6

7

9

2

2.5

8

10

8

1.5

9

4

7

2.2

10

4

3

7.2

11

7

2

5.8

28 Simple Task Allocation Algorithm in a Collaborative Robotic System Table 28.4 Start time of the operations and the corresponding vertices (after the use of the proposed algorithm)

3; 3.9

1; 0

t = 3.7

445

Digraph vertex number

Start time of the corresponding event

1

0

2

3.7

3

6.9

4

15.5

5

11.7

6

18.3

7

23.5

8

25.1

9

25.8

10

32.3

t = 8.7

8; 25.1

4; 15.5

2; 3.7

10; 32.3

7; 23.5

5; 11.7 t = 6.6

6; 18.3 9; 25.8

Fig. 28.2 Resulting technological process as a weighted digraph

The calculated values show that the time of both the technological process and its individual operations have been reduced (for specific input data of the simulation). The use of this algorithm allows not only to minimize the time of the technological processes that require the collaboration of people and robots, but also to test various hypotheses on the duration of the operations with different CRS composition and their performance values, as well as the number of operations, minimum and maximum possible numbers of CRS members to be assigned.

28.4 Conclusions and Future Work Thus, we propose a simple algorithm for tasks allocation in a collaborative robotic system, which provides time minimization of both the technological process and its individual operations. The conditions, dependencies, and durations of the operations are presented as a weighted directed graph whose vertices are temporal events of the start of one or more operations and the end of the previous ones, and edge weights are the durations of the corresponding operations. The resulting graph is converted into a time sequence of operations. The algorithm focuses on assigning workers to

446

R. Galin et al.

an operation (operations), the beginning of which corresponds to the current time event, selecting the number of cobots and people from the minimum and maximum possible range based on the available human workers and cobots at a certain time and their performance values. After the assignment, the duration of the operation is recalculated according to the efficiency values of the workers engaged, the weight of the corresponding edge on the digraph is modified, and the time sequence after the current time event is rebuilt. In case of the absence of the required number of available people and (or) cobots, the assignment of workers for the operation is postponed until the next time event in the sequence. The algorithm makes the transit from the current time event to the one, and tries to assign workers for the suspended operations (if there are any) and then—the current one(s). These actions repeat till the sink vertex of the digraph. As part of the experiment, a simple technological process was simulated to test the algorithm. The technological process as a weighted digraph of atomic operations implied mandatory collaboration of human workers and cobots for each operation. The analysis of the resulting modeling data showed that the proposed algorithm allows for minimizing the duration of certain operations and the overall technological process. It can also be used to test various hypotheses for the optimal number of participants to perform the collaborative operations, taking into account the performance of each CRS participant involved. Further work implies additional tests of the proposed algorithm to improve its efficiency and functionality.

References 1. Li, F., Li, X., Yang, Y., Xu, Y., Zhang, Y.: Collaborative production task decomposition and allocation among multiple manufacturing enterprises in a big data environment. Symmetry 13(2268), 1–22 (2021) 2. Khalid, S., Ullah, S., Alam, A.: Task distribution mechanism for effective collaboration in virtual environments. Proc. Pakistan Acad. Sci.: Pakistan Acad. Sci. A. Phys. Computat. Sci. 53(1), 49–59 (2016) 3. Khalid, S., Ullah, S., Ali, N.: The effect of task distribution model on student learning and performance in collaborative virtual environment. Sindh Univ. Res. J. (Sci. Ser.) 48(3), 521–526 (2016) 4. Armstrong, C.: Catalyzing collaborative learning: how automated task distribution may prompt students to collaborate. E-Learn. Digital Media 7(4), 407–415 (2010) 5. Malik, A.A., Bilberg, A.: Collaborative robots in assembly: a practical approach for tasks distribution. Proc. CIRP 81, 665–670 (2019) 6. Xiao, L., Pei, Y.: Collaborative distribution knowledge service system based on task context aware. In: 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, pp. 146–151 (2010) 7. Blankenburg, J., et al.: A distributed control architecture for collaborative multi-robot task allocation. In: 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), pp. 585–592 (2017) 8. Gu, B., Xu, M., Xu, L., Chen, L., Ke, Y., Wang, K., Ming, D.: Optimization of task allocation for collaborative brain-computer interface based on motor imagery. Front. Neurosci. 15(683784), 1–23 (2021)

28 Simple Task Allocation Algorithm in a Collaborative Robotic System

447

9. Yavuz, H.S., GöKtas, H., Çevıkalp, H., Sarıba¸s, H.: Optimal task allocation for multiple UAVs. In: 2020 28th Signal Processing and Communications Applications Conference (SIU), pp. 1–4 (2020) 10. Mazdin, P., Rinner, B.: Distributed and communication-aware coalition formation and task assignment in multi-robot systems. IEEE Access 9, 35088–35100 (2021) 11. Soleimanpour-Moghadam, M., Nezamabadi-Pour, H.: Discrete genetic algorithm for solving task allocation of multi-robot systems. In: 2020 4th Conference on Swarm Intelligence and Evolutionary Computation (CSIEC), pp. 006–009 (2020) 12. Zheng, T., Li, J.: Multi-robot task allocation and scheduling based on fish swarm algorithm. In: 2010 8th World Congress on Intelligent Control and Automation, pp. 6681–6685 (2010) 13. Bolu, A., Korçak, Ö.: Adaptive task planning for multi-robot smart warehouse. IEEE Access 9, 27346–27358 (2021) 14. Chen, J., Sun, D.: Coalition-based approach to task allocation of multiple robots with resource constraints. IEEE Trans. Autom. Sci. Eng. 9(3), 516–528 (2012) 15. Moradi, R., Nezamabadi-pour, H., Soleimanpour, M.: Modified distributed bee algorithm in task allocation of swarm robotic. In: 2019 5th Conference on Knowledge Based Engineering and Innovation (KBEI), pp. 755–758 (2019) 16. Kataev, A.V., Kataeva, T.M.: Dynamic Partner Network-based Project Management (Upravlenie proektami na baze dinamicheskoj seti partnerov). Southern Federal University Press, Rostov-on-Don (2017).(In Russ.) 17. Antonelli, D., Bruno, G.: Dynamic distribution of assembly tasks in a collaborative workcell of humans and robots. FME Trans. 47(4), 723–730 (2019) 18. Roncone, A., Mangin, O., Scassellati, B.: Transparent role assignment and task allocation in human robot collaboration. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 1014–1021 (2017)