Multi Agent Systems: Technologies and Applications towards Human-Centered (Springer Tracts in Human-Centered Computing) 9811904928, 9789811904929

Table of contents :
Preface
Contents
Editors and Contributors
1 Introduction to Multi-agent Systems
1 Introduction
2 Literature Review
3 Conclusions
References
2 Human–Computer Interactions Through Multi-agent Systems: Design and Implementations
1 Introduction
2 Previous Works
2.1 Collaborative Concepts of MAS and HCI
2.2 Self-Governing Versatile Automated Systems
2.3 Intelligent Air Traffic System
2.4 Home Automation System
2.5 Mixed Reality-Oriented Framework
2.6 Psychological Aspects of Human–Agent Conversation
2.7 Tele-Health-Oriented Automated Systems
2.8 Demonstration-Oriented Training
3 Musical Scale Predictor with HCI Using Multi-agent
3.1 Mathematical Theory of Mind (ToM) Replicas in Musical Scale Prediction
4 Conclusion
References
3 Social Behavior and Reasoning Through Multi-Agent Systems
1 Introduction
2 Literature Survey
3 Fuzzy Logic-Based Multi-agent System
3.1 Fuzzy Vaccination Drive
4 Conclusions and Future Scope
References
4 Prevalence of Multi-Agent System Consensus in Cloud Computing
1 Introduction
2 Agent
2.1 Autonomy
2.2 Pro-activity
2.3 Re-activity
2.4 Communication and Cooperation
2.5 Negotiation
2.6 Learning
3 Intelligent Agent (IA)
3.1 Simple Reflex Agent
3.2 Model-Based Reflex Agent
3.3 Goal-Based Agent
3.4 Utility-Based Agent
3.5 Learning Agent
4 Related Work
5 Applications and Objective Measures of Multi-agent Systems
6 Cloud Computing and Multi-Agent Systems
7 Consensus
7.1 Consensus in Distributed System
7.2 Consensus Protocol Properties
7.3 Consensus Subjected to Communication Restraint
7.4 Leader-Following Consensus
7.5 Group Consensus
7.6 Consensus Supporting Trigger Method
7.7 Time-Limited Consensus
7.8 Multi-Consensus and Multi-Tracking
8 Conclusion
References
5 The Role of Multi-Agent Systems in IoT
1 Introduction
2 What is IoT?
3 IoT Architecture
3.1 Hardware Layer
3.2 Communication Layer
3.3 Platform Layer
3.4 Applications Layer
3.5 Security Layer
4 Internet of Things Application Domains
5 Why Use the MAS Concept for IoT-Based Systems?
6 Types of Agents in a Multi-Agent System Based on IoT
7 Simulation Multi-Agent System for IoT
8 Security in Multi-Agent System Based on IoT
9 The Use of Multi-Agent Systems in IoT Applications
9.1 Multi-Agent Systems in IoT-Based Smart City
9.2 Smart Home Based on Multi-Agent System
9.3 A Multi-agent Framework for IIoT
9.4 Smart Healthcare Based on Multi-agent System
9.5 Smart Grid Based on Multi-agent System
10 Conclusion
References
6 Agent-Based Human-Supportive Applications
1 Introduction and Literature Review
1.1 Classification
1.2 Multi-Agent System
2 Supply Chain Management
2.1 Performatives and Protocols for Negotiation
2.2 Criteria for Performative Definition and Selection
3 Expert System Design
3.1 Structure
3.2 Advantages
3.3 Application
3.4 Problems
3.5 Latest Tools for ES
3.6 Research Aspects
4 Workflow and Business Process Management System
4.1 Benefits
4.2 Disadvantages
4.3 Use of Agents
4.4 Classification
4.5 Examples and Scope of Research
5 Process Control Management
5.1 Functionalities on Different Layers
5.2 Implementation of Agents
6 Production Systems
6.1 Approaches
6.2 Phases of the Approach
6.3 Example of a Distributed Manufacturing System
7 Conclusions
References
7 Multi-Agent System Applications in Health Care: A Survey
1 Introduction
2 Background
3 Literature Survey
3.1 Architecture of Multi-Agent System
3.2 Modeling of Multi-Agent System
3.3 Planning of Multi-Agent System
3.4 Review of Multi-Agent System Applied in Healthcare Domain
3.5 Multi-Agent System Applied in Health Care
4 Multi-Agent Systems Used in Health Care
4.1 Medical Data Management
4.2 Decision Support Systems
4.3 Planning and Resource Allocation
4.4 Remote Care
4.5 Composite Systems
5 A New Scheme for Medical Diagnosis Using Mas
5.1 Model 1 (Describes the Patient Treatment at Home)
5.2 Model 2 (Describes the Patient Treatment at Hospital)
6 Conclusion
References
8 Applications of Multi-agent Systems in Intelligent Health Care
1 Introduction
1.1 Artificial Intelligence
1.2 Distributed Artificial Intelligence
2 Multi-agent Systems in Distributed Artificial Intelligence
2.1 Concept
2.2 Multi-agent Systems (MAS)
2.3 Approaches of Multi-agent Systems’ Architecture
2.4 Usage of MAS in Recent Times
2.5 Fields of MAS Application
3 MAS in Health Care
3.1 Discussion on Expert Systems
3.2 Differences Between Expert Systems and Artificial Intelligence
3.3 How MAS is Useful in Healthcare Domain
3.4 Fields of Application in Health Care
3.5 MAS for E-Health/Telemedicine
3.6 Data to be Considered
3.7 Methodologies and Applications to Achieve Accurate Results
4 Recent Trends in Application of MAS in Health Care and Its Necessity
5 Conclusion
References
9 Multi-agent Reinforcement Learning for Stock Market Strategy Analysis
1 Introduction
2 Purpose of Our Study
3 Multi-agent Reinforcement Learning
3.1 Multi-agent Reinforcement Learning Applications
3.2 Simulation Analysis of Optimal Execution
4 Models and Methods
4.1 Impact on Optimal Liquidation by Parameter κ
4.2 Different Deep Reinforcement Learning Approaches
4.3 Deep Reinforcement Learning Used for Trading Agent
4.4 Multi-agent Approach for Stock Market Strategy Analysis
5 Implementation of Multi-agent Systems for Stock Market
6 Performance Analysis of Multi-agent Systems
7 Conclusion and Future Work
References
10 Multi-agent Systems: Future Initiatives
1 Introduction
2 Intelligent MAS
References
Index

Citation preview

Springer Tracts in Human-Centered Computing

Shibakali Gupta Indradip Banerjee Siddhartha Bhattacharyya Editors

Multi Agent Systems Technologies and Applications Towards Human-Centered

Springer Tracts in Human-Centered Computing Series Editors Siddhartha Bhattacharyya, Shree Tower, Opposite ISI Kolkata, Axis Bank, Dunlop, Bonhooghly, Kolkata, West Bengal, India Janusz Kacprzyk, Systems Research Institute, Polish Academy of Sciences, Warszawa, Poland Mario Koeppen, Network Design and Research Center, Kyushu Institute of Technology, Fukuoka, Japan Vaclav Snasel, Department of Computer Science, VŠB-TUO, Ostrava-Poruba, Czech Republic Rudolf Kruse, Faculty of Computer Science, Otto-von-Guericke University, Magdeburg, Sachsen-Anhalt, Germany

The book series is aimed at providing an exchange platform for researchers to summarize the latest research and developments related to human-centered computing. Human-centered computing is focused on the study of design, development, and deployment of human-computer systems based on mixed-initiatives. It may be visualized as a three-dimensional space comprising human, computer, and environment. This upcoming computing paradigm has emerged from the convergence of multiple disciplines concerned both with understanding human beings and with the design of computational artifacts. The field of human-centered computing is a multidisciplinary field encompassing disciplines such as computer science, human factors, sociology, psychology, cognitive science, anthropology, communication studies, graphic design and industrial design. The book series covers the topics and fields of distributed environments entailing Internet-based information systems, grids, sensor-based information networks, and mobile and wearable information appliances, multimedia and multi-modal human-computer interfaces, design of intelligent interfaces and information representation and visualization, multi-agent systems, effective and constrained computer-mediated human-human interaction, defining relevant semantic structures for multimedia information representation, community specific HCI solutions, collaborative and interactive systems, social interaction, social orthotics, affective computing, knowledge-driven human-computer interaction, human-centered semantic formulation, human-centered management science, and participatory action research. The series will include monographs, edited volumes, and selected proceedings.

More information about this series at https://link.springer.com/bookseries/16481

Shibakali Gupta · Indradip Banerjee · Siddhartha Bhattacharyya Editors

Multi Agent Systems Technologies and Applications Towards Human-Centered

Editors Shibakali Gupta Department of Computer Science and Engineering University Institute of Technology (UIT) Burdwan University Bardhaman, West Bengal, India

Indradip Banerjee Department of Computer Science and Engineering University Institute of Technology (UIT) Burdwan University Bardhaman, West Bengal, India

Siddhartha Bhattacharyya Rajnagar Mahavidyalaya Birbhum, West Bengal, India

ISSN 2662-6926 ISSN 2662-6934 (electronic) Springer Tracts in Human-Centered Computing ISBN 978-981-19-0492-9 ISBN 978-981-19-0493-6 (eBook) https://doi.org/10.1007/978-981-19-0493-6 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Shibakali Gupta would like to dedicate this book to his late father-in-law Ashok Kumar Chakraborty. Indradip Banerjee would like to dedicate this book to his youngest son Aishneek Banerjee, his beloved wife Puja Banerjee, his parents Ashis Banerjee and Tandra Banerjee, his parents-in-law Ujjal Chatterjee and Purnima Chatterjee. Siddhartha Bhattacharyya would like to dedicate this book to the late parents of his second-eldest brother-in-law, Late Nalini Bhuson Acharjee and Late Nirmala Acharjee.

Preface

Nowadays, the research in the field of multi-agent system (MAS) has gained widespread recognition due to its interdisciplinary nature and a vast versatile application domain including engineering, social science, economics, mathematics, operational research, etc. It has been proved that agents in MAS are the most appropriate technological paradigm for providing the most optimal solution for different kinds of complex real-world problems that may be industrial or it might be specific social problems too. Keeping these features in mind, we planned to tune the research of the latest multi-agent technologies and tried to compose its effect on humancentered computing (HCC) corridor. Actually, HCC primarily increases the human– computer interactions and with the establishment of human-centered systems (HCS) tries to relate human-like activities through different socio-perceptions like automation, reasoning, interaction, communication, cooperation, negotiation, agreement confirmation, team work, etc. The main objective of this volume is to amalgam the latest agent technologies in the HCC context to provide a new research direction to enrich the human socio-computations. The volume comprises eight contributory chapters apart from the Introductory and concluding chapters to report the latest developments in this direction. Chapter 1 provides a brief introduction of the subject matter with reference to the relevance and increasing importance of multi-agent systems (MAS) in different spheres of engineering and technology ranging from health care, planning, telecommunication, supply chain to name a few. It also sums up a brief review of the latest state of the art in this direction. In computing, multi-agent systems are a relatively new sub-field made up of several interacting computer elements known as agents. This interface is part of a global system representing software architecture for robot control to help the human operator. The implementation of human–computer interaction (HCI) using a multiagent system is evolving day by day. In Chap. 2, the authors examine different relevant research works which have been attempted to cover the various aspects of HCI. The chapter covers various technologies including tele-health, air traffic control, mixed reality, learning from demonstration, and psychological aspects of agents with HCI. vii

viii

Preface

Finally, the authors discuss how HCI using multi-agent systems can help the singers get the scale recommendation. Multi-agent systems are a relatively new sub-field composed of numerous interconnected computing elements known as agents. To satisfy their design intents, they can take decisions for themselves; i.e., they are capable to take autonomous decisions. Also, these agents are capable of communicating with other agents not only by commuting data but also by engaging in similar kinds of social activities like association, coordination, and the like. Social behavior exhibits the behavior among two or more organisms inside identical species and encompasses any conduct wherein one member affects the other, for example the interaction of two or more humans or organizations governed toward a common goal that is mutually beneficial, whereas social reasoning involves the propensity to illustrate inferences about others’ intentions, mentality, and actions or activities, in order to synchronize one’s own behavior. In Chap. 3, the authors develop a region-specific vaccination method using a multiagent framework. The Mamdani fuzzy inference system is used to build the proposed multi-agent framework. Cloud computing that follows service-oriented architecture is useful for intelligent agent or multi-agent system (MAS) communication. Their use in representation and construction and parallel and published applications is identified in Chap. 4 and shows similarities, contrasts, and potential combinations between cloud computing and multi-agent structures. Long execution complex structure with clever applications work with MAS to showcase cloud computing. The assembling of interfaces within MAS requires reliable scattering systems, and cloud computing systems require programs with clever, enthusiastic, versatile, and independent behavior. The engineering of a system consisting of MAS that primarily focuses on the materials of cost transactions between cloud users and providers is planned to mitigate the disadvantages of both cloud clients and cloud providers and exploit the full potential of cloud computing. As it turns out, as innovation develops and solves increasingly complex applications, the need for an integrated framework of multiple operators communicating in peer-to-peer mode is becoming clear. Central to the design and operation of such MAS is the focus of a problem and research question that has long been tested by all communities. Internet of Things and multi-agent systems are two concepts that have revolutionized the cyber and physical worlds, and putting these two concepts together in a single framework can increase the effectiveness of solutions. The main purpose of the IoT is to automate processes and create an intelligent environment that different elements can interact with each other. Systems with high dynamics and complexity can control by the Internet of Things architecture with the help of agent systems. Chapter 5 describes the role of multi-agent systems in IoT and application areas. To better understand this issue, it is essential to clarify what IoT is and which properties are used to define IoT agents in general. The main goal is to analyze other aspects of multi-agent systems as one of IoT’s key components. Furthermore, IoT agents should be distinguished from other agent-oriented technologies so that these two must briefly be described. In addition, the possible areas of application and properties of IoT agents are also elucidated in this chapter.

Preface

ix

One of the most intriguing breakthroughs in computer science is the use of agents. It has aided in increasing the automation quotient in the systems in which it has been implemented. Because of the broad extent of implementation and the impact on human civilization, research on human-supported applications that include automated and interactive agents has received increased attention. Various humansupportive applications, such as supply chain management, expert system design, workflow, and business process management, process control management, and production management, are extremely important in a business’s workflow. And, agents are demonstrating their incredible powers in various applications, which eliminate not only the complexity of manual control but also the ongoing human engagement. Chapter 6 discusses the applications that have been deployed with the assistance of agents, as well as the advances they have brought to the table. It also sheds insight on recent technological breakthroughs in this sector. Earlier medical systems suffered from many issues. The issues are mostly observed due to inadequate medical practitioners, poor medical infrastructure manually operated healthcare systems, and improper coordination between each and every aspects of the healthcare system. Most of the medical reports and records are based on paper and pen, which is difficult to manage and maintain and also raises a question of reliability. So, there is a need for a change in the existing system. Keeping this in mind, the concept of a multi-agent system has been introduced in the healthcare system. Multi-agent systems are considered as the best and most appropriate technology that can be applied to fulfill the need required in the healthcare systems. In multi-agent systems, the agents represent and act on behalf of users and owners with very difficult goals and motivations. The agents cooperate, coordinate, and negotiate with each other in the same way that human beings cooperate, coordinate, and negotiate with each other in day-to-day life and consequently can be used as the best technology to solve the problems encountered in the medical healthcare system. Chapter 7 focuses mainly on the architecture, planning, implementation, and application of multi-agent systems in the healthcare domain. Artificial intelligence has surged into human lives as an important requirement since the start of the twenty-first century. Artificial intelligence (AI) can be understood as replications of human intelligence in systems that can be devised to ‘think and act’ like humans. Machine learning refers to the concept of programs or applications that ‘learn’ automatically and adapt fresh data without human support and is a subset of AI. One of the imperative areas in AI is distributed artificial intelligence (DAI) where complex problems can be dealt with the concept of multi-agent systems (MAS). MAS has become a bridge to come over difficult tasks and to bring out the best possible solution(s). Healthcare data is a domain of data science and can be considered as one of the most dynamically generated data. Multi-agent systems in healthcare data are the best combination possible to utilize the advantages that are available in both the sub-fields. Different diseases can be addressed through the division of the task (s) as per the norms of MAS. The best results can be obtained here and it can be truly helpful for society and mankind. Chapter 8 focuses on the applications that are possible through MAS in health care. The fundamental concepts, important issues, and algorithms are dealt with in a comprehensible manner so that the contents

x

Preface

of this chapter can benefit the experts, researchers, data scientists, medicos, and, of course, society. The stock market has always been uncertain in terms of prediction and liquidation, which is a major problem in the stock market for any investor where a huge number of shares of a particular stock have been sold within a time period. Experts face issues in optimizing the liquidation which leads to obtaining a proper modeling system that can handle the varieties of challenges of the stock market and provide viable strategic solutions in the trading. Generally, the return on the invested amount depends on the purchase price, and there are two key factors responsible for making money on stocks; first is to buy a stock at the right time, and second is to sell it at the right time. So, to book a profit, both the decisions should be correct. In most cases, investors face problems in selling a stock, where the common problem lies in the human nature of running toward making more profit. The most important thing in the stock market is to make correct strategies and keep out all the human emotions in taking any decisions in terms of buying and selling or making strategies for buying or selling a stock. All the strategies should be based on the market conditions and stock news, not on human emotions and greediness. In Chap. 9, the authors use a multi-agent model by using deep reinforcement learning that allows capturing a high level of complexities compared to other machine learning models and strategies, so that the agents can be trained in a better way for making the decisions to sell a stock at right time. Chapter 10 draws a concluding note to the volume with future directions of research. The primary audience of this book includes the research students of computer science, information technology apart from the software professionals who can inherit the much-required developmental ideas to boost up their computing activities. Burdwan, India Burdwan, India Birbhum, India December 2021

Shibakali Gupta Indradip Banerjee Siddhartha Bhattacharyya

Contents

1

Introduction to Multi-agent Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . Indradip Banerjee and Siddhartha Bhattacharyya

2

Human–Computer Interactions Through Multi-agent Systems: Design and Implementations . . . . . . . . . . . . . . . . . . . . . . . . . . . Swarnavo Mondal, Indradeep Bhattacharya, and Shibakali Gupta

3

Social Behavior and Reasoning Through Multi-Agent Systems . . . . Indradeep Bhattacharya, Swarnavo Mondal, and Shibakali Gupta

4

Prevalence of Multi-Agent System Consensus in Cloud Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Santanu Koley and Pinaki Pratim Acharjya

1

5 33

55

5

The Role of Multi-Agent Systems in IoT . . . . . . . . . . . . . . . . . . . . . . . . . Mohmmad Gheysari and Mahsa Seyed Sadegh Tehrani

87

6

Agent-Based Human-Supportive Applications . . . . . . . . . . . . . . . . . . . 115 Abir Kumar Bardhan and Abeer Alsadoon

7

Multi-Agent System Applications in Health Care: A Survey . . . . . . . 139 Chandanita Thakur and Shibakali Gupta

8

Applications of Multi-agent Systems in Intelligent Health Care . . . . 173 M. Bhanu Sridhar

9

Multi-agent Reinforcement Learning for Stock Market Strategy Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 Akash Ranjan, Asim Kumar Mahadani, and Tarik A. Rashid

10 Multi-agent Systems: Future Initiatives . . . . . . . . . . . . . . . . . . . . . . . . . 221 Siddhartha Bhattacharyya and Indradip Banerjee Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225

xi

Editors and Contributors

About the Editors Dr. Shibakali Gupta is an assistant Professor of Computer Science and Engineering Department, University Institute of Technology, Burdwan University, West Bengal, India. He obtained his B.E. degree in Computer Science and Engineering from Burdwan University, West Bengal in 2005 and M.Tech. degree in the same subject in 2007 from KIIT University, Orissa. He received Ph.D. in the field of multi agent system, Computer Science. His research interests include multi agent system, intelligent healthcare system, robot systems and swarm robotics. He is the editorial board member of different International Journals. He is an author/co-author of 35 published articles in computer science and engineering field. He has published three book chapters and two edited books named Intelligent Multimedia Data Analysis and Big Data Security from De Gruyter Publication, Germany. He is the member of CSTA, IAENG. Dr. Indradip Banerjee received his Ph.D. in Engineering from NIT, Durgapur. He has completed MCA, PGDCA, Master of Marketing Management and BCA (Hons.). Currently he is belongs to the Computer Science and Engineering Department of University Institute of Technology, The University of Burdwan. His areas of interest are Network Security, Image Processing and MAS. He has published nearly 35 research papers in International and National Journals and Conferences. His research interests include cryptography, steganography, big data and image processing. He has published two book chapters in springer and one edited book Big Data Security in Frontiers in Computational Intelligence by De Gruyter Publisher, Germany. Dr. Siddhartha Bhattacharyya [FRSA (UK), FIET (UK), FIEI (I), FIETE, LFOSI, SMIEEE, SMACM, SMIETI] is currently serving as the Principal of Rajnagar Mahavidyalaya, Birbhum, India. Prior to this, he was a Professor with the Department of Computer Science and Engineering, Christ University, Bengaluru, India. He also served as the Principal of the RCC Institute of Information Technology, Kolkata. He served as a Senior Research Scientist with the Faculty of Electrical Engineering and

xiii

xiv

Editors and Contributors

Computer Science, VSB Technical University of Ostrava, Ostrava, Czech Republic. He is a full foreign member of the Russian Academy of Natural Sciences. He has co-authored six books, co-edited 86 books, and has authored or co-authored more than 300 research publications in international journals and conference proceedings. He holds 19 patents. His research interests include soft computing, pattern recognition, multimedia data processing, hybrid intelligence, social networks, and quantum computing.

Contributors Pinaki Pratim Acharjya Dept. of Computer Science & Engineering, Haldia Institute of Technology, Haldia, West Bengal, India Abeer Alsadoon School of Computing and Mathematics, Charles Sturt University, Sydney, Australia Indradip Banerjee Department of Computer Science, University Institute of Technology, The University of Burdwan, Burdwan, India Abir Kumar Bardhan Department of Computer Science and Engineering, UIT, Bardhaman, West Bengal, India M. Bhanu Sridhar GVP College of Engineering for Women, Visakhapatnam, India Indradeep Bhattacharya Department of Computer Science and Engineering, University Institute of Technology, Burdwan University, Burdwan, West Bengal, India Siddhartha Bhattacharyya Rajnagar Mahavidyalaya, Rajnagar, Birbhum, India Mohmmad Gheysari Doctor of Business Administration (Digital Transformation Field), Faculty of Management, Tehran University, Tehran, Iran Shibakali Gupta Department of Computer Science and Engineering, University Institute of Technology, Burdwan University, Bardhaman, West Bengal, India Santanu Koley Dept. of Computer Science & Engineering, Haldia Institute of Technology, Haldia, West Bengal, India Asim Kumar Mahadani Department of Computer Science and Engineering, Bankura Unnayani Institute of Engineering, Bankura, West Bengal, India Swarnavo Mondal Department of Computer Science and Engineering, University Institute of Technology, Burdwan University, Burdwan, West Bengal, India Akash Ranjan Department of Computer Science and Engineering, Bankura Unnayani Institute of Engineering, Bankura, West Bengal, India Tarik A. Rashid University of Kurdistan Hewler, Kurdistan, Iraq

Editors and Contributors

xv

Mahsa Seyed Sadegh Tehrani Master of Information Technology, Faculty of Management, Payam Noor University, Tehran, Iran Chandanita Thakur Department of Computer Science and Engineering, Vivekananda Institute of Science and Technology, MAKAUT University, Kolkata, West Bengal, India

Chapter 1

Introduction to Multi-agent Systems Indradip Banerjee and Siddhartha Bhattacharyya

Abstract Multi-agent systems (MASs) are rapidly increasing with the advancement of technologies, especially in the domains of e-commerce, health care, planning, telecommunication, supply chain, etc. Scholars across different disciplines have also become interested in it, such as computer science, electronics, engineering, and civil engineering. The arrangement in some environment that is equipped for an independent activity to complete its target is known as the agent. Decisions are made by the agents on a particular action to solve the task and complete its objective using various inputs like the past actions and interactions with it are surrounding agents. Human–computer interaction (HCI) implementation employing a multi-agent system is growing daily. Some contemporary applications in this sector covered by several technologies include telehealth, air traffic management, mixed reality, learning from demonstration, and psychological aspects of agents using HCI. Keywords Multi-agent systems · Human–computer interaction · Agents · Human-centered computing

1 Introduction Multi-agent systems (MASs) are a center place of studies of contemporary artificial intelligence. A multi-agent system includes a couple of decision-making agents which have interaction in shared surroundings to obtain common or conflicting desires. MAS studies span various technical problems, inclusive of the way to layout MAS [1] to incentivize certain behaviors in agents, the way to layout algorithms permitting one or more agents to achieve designated desires in a MAS, how records are communicated and propagated among agents, and the way norms, conventions and roles might also additionally emerge in MAS. An enormous array of programs I. Banerjee (B) Department of Computer Science, University Institute of Technology, The University of Burdwan, Burdwan, India S. Bhattacharyya Rajnagar Mahavidyalaya, Rajnagar, Birbhum 731 130, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 S. Gupta et al. (eds.), Multi Agent Systems, Springer Tracts in Human-Centered Computing, https://doi.org/10.1007/978-981-19-0493-6_1

1

2

I. Banerjee and S. Bhattacharyya

may address the use of MAS methodologies, consisting of self-sustaining driving, multi-robotic factories, automatic trading, business games, automatic tutoring, etc. Human-centered computing (HCC) [2] implies the study of the design, development, and formation of variegated–inventiveness of human–computer systems. It arises from the concurrence of many disciplines involved with each human comprehension and the creation of computing products. Interdisciplinary fields such as human– computer interaction and information science are fundamental to human-centered computing. Information science is concerned with procedures related to the collection, manipulation, and use of information, whereas human-centered computing deals with technology standards and processes. Ergonomics and usability are more important aspects of human–computer interaction, and information science deals with the collection, manipulation, and use of information. Human-centered computing practitioners and researchers come from a variety of disciplines, including computer science and human factors, psychology, cognitive science, anthropology, communication studies, graphic design, and industrial design. Some scholars examine how people embrace and adapt technology to their lives to understand humans, both as individuals and as social groups. Another trending topic is human–computer interaction (HCI) [3] which is gradually evolving year by year. Human–computer interaction (HCI) is the plan and execution of intelligent figuring frameworks that clients can connect with computers. The main ideas in HCI are usefulness and convenience. Administrations given normally by a framework are called capacities. Ease of use is the point at which a client uses the framework’s capacities effectively, appropriately, and obviously. Human–computer interaction based on multi-agent systems will incorporate HCI and MAS where the technologies are at the center of the proposition. Technologies communicate with the clients, understand their requirements, and then decide the plan of action according to the task assigned. All the technologies on this are discussed among various fields to gain knowledge and to innovate it more in near future.

2 Literature Review Several countries are experiencing an epidemic of COVID-19, which has put enormous strain on the ability of our country to treat patients, and the number of people who need to be tested is increasing. As more nurses succumb to the disease, the availability of competent experts to perform tasks like patient testing and providing ventilator-connected care decreases. In this pandemic phase, there is a need to explore digital technologies that can be used to train new nurses and other healthcare workers in safe and efficient patient-care protocols. In a study, Cecil et al. [4] proposed a virtual reality-based simulator depending upon the human-centered computing (HCC) principles. In their discussion about nursing’s understanding of scene scenes and safety procedures (e.g., the collection of nasal samples and use of ventilators on patients) as well as safety procedures and detail steps (e.g., using respirators on patients), nurse and nurse trainee participants discussed the role of factors associated with HCC.

1 Introduction to Multi-agent Systems

3

Deepfake (or digital persona) is mostly used in the production of synthetic media to replace someone who appears in an existing picture or video with someone who is absent in that media. Artificial intelligence (AI) is a powerful system of generating artificial images and sounds that create video and other digital representations that appear to be natural. Deepfakes have traditionally been employed to defame someone, with little regard for the user experience. However, in this context, Ramachandra et al. [5] introduced the concept of human-centered computing for the generation of digital persona of an eminent deceased artist. Their main intention was for people to converse with the persona projected on a holographic stage in a museum to have a better human experience. The digital persona replied to visitors’ queries about the departed artist’s creative journey and artwork in the voice of that artist. Using human-centered computing, they were able to ensure audience involvement in the product, also called the digital persona. A broad range of topics was covered in the research, including technology, cognitive psychology, ergonomics, social psychology, and organizational psychology. To develop the deepfakes for goods, they had shown the application, specifications, and consequences of the aforementioned priority areas. They had also shown the findings of a social experiment with youngsters that took place during their engagement with a digital persona. Canino-Rodríguez et al. [6] argued in 2015 that the limited efficacy of existing air traffic frameworks would necessitate an emerging age of smart air traffic system (SATS) that relied on recent robotic advances. A critical component of SATS is the new human–computer communication strategy for carrying out these activities. Regardless, efforts to develop such devices should be based on a realistic depiction of hypothetical air traffic circumstances that are comparable to present ones. This article was predicated on the onboard HCI to SATS signal, which was received from aircraft route frameworks, including traffic situation, regulators’ indications, and so on. As a result, the HCI was recommended to improve situational awareness and dynamic through the pilot cockpit. As a result, an approach based on multiagent frameworks is suited for illustrating such an environment. They demonstrated that existing strategies for designing multi-agent frameworks were a useful tool for depicting HCI. Song Zheng and colleagues published a paper [7] in 2017 on the widespread use of home control stage designs (IAPhome) that incorporated a multi-agent framework and correspondence adapter, which demonstrated critical flexibility and benefits from various perspectives, including machine connectivity, cooperative command, human–computer communication, and self-administration. The serviceoriented communication was an important foundation for planning and carrying out this engineering, which made it possible to coordinate many smart machines in an adaptive manner. In addition, a method was presented for addressing the integrated command issue of the exquisite house system using multi-agent programming. In a real-world smart home setting, the proposed platform architecture had been tested and the results proved its effectiveness in resolving community control issues for multiple smart devices.

4

I. Banerjee and S. Bhattacharyya

3 Conclusions Human-related computing is heavily focused on the primary objective of designing management systems with the human user as the key focus. Human-centered computing is made up of many computing features, the interconnections of which result in advancements in human activities. It is vital to highlight that, because of advances in human-centered computing, is less focused with computer science specializations. On the contrary, a shift in computing focus has resulted in social computing, accessibility, and organizational computing. Human-centered computing has achieved a wide range of human-related aims. Increasingly, multi-agent teams are becoming prevalent, and a uniform paradigm is essential to these teams. In addition to being able to function in dynamic environments, these teams must be able to communicate with any humans who may need assistance. Researchers can build multi-agent teams that are interdependent and still able to communicate with humans by leveraging existing human-agent research. It is more effective to build dynamic teams in place of dynamic agents to deal with the constantly changing nature of current workloads rather than constructing dynamic agents.

References 1. Albrecht SV, Stone P (2018) Autonomous agents modelling other agents: a comprehensive survey and open problems. Artif Intell 258:66–95. https://doi.org/10.1016/j.artint.2018.01.002 2. Alqahtani H, Liu C, Kavakli-Thorne M, Kang Y (2019) An agent-based intelligent hcl information system in mixed reality. In: 28th international conference on information systems development, Toulon, France 3. Billard AG, Calinon S, Dillmann R (2016) Learning from humans. In: Siciliano B., Khatib O (eds), Springer handbook of robotics, Springer handbooks. Springer, Cham, pp 1995–2014. https://doi.org/10.1007/978-3-319-32552-1_74 4. Cecil J, Kauffman S, Gupta A, McKinney V, Miguel Pirela-Cruz MD (2021) Design of a human centered computing (HCC) based virtual reality simulator to train first responders involved in the Covid-19 pandemic. In: IEEE international systems conference (SysCon), pp 1–7. https:// doi.org/10.1109/SysCon48628.2021.9447090 5. Ramachandra N, Ahuja M, Rao RM, Dubash N (2021) Human centered computing in digital persona generation. In: Fu W, Xu Y, Wang SH, Zhang Y (eds) Multimedia technology and enhanced learning. ICMTEL 2021. Lecture notes of the institute for computer sciences, social informatics and telecommunications engineering, vol 388. Springer, Cham. https://doi.org/10. 1007/978-3-030-82565-2_32 6. Canino-Rodríguez JM, García-Herrero J, Besada-Portas J, Ravelo-García AG, TraviesoGonzález C, Alonso-Hernández JB (2015) Human computer interactions in next-generation of aircraft smart navigation management systems: task analysis and architecture under an agent-oriented methodological approach. Sensors 15:5228–5250. https://doi.org/10.3390/s15 0305228 7. Zheng S, Zhang Q, Zheng R, Huang BQ, Song YL, Chen XC (2017) (2017) Combining a multi-agent system and communication middleware for smart home control: a universal control platform architecture. Sensors 17:2135. https://doi.org/10.3390/s17092135

Chapter 2

Human–Computer Interactions Through Multi-agent Systems: Design and Implementations Swarnavo Mondal , Indradeep Bhattacharya , and Shibakali Gupta

Abstract In computing, multi-agent systems are a relatively new subfield made up of several interacting computer elements known as agents. This interface is part of a global system representing software architecture for robot control to help the human operator. The implementation of human–computer interaction (HCI) using a multiagent system is evolving day by day. We examined different past research works and attempted to cover various works step by step and furthermore as per different various advancements and applications. Various technologies, which are covered here, are some recent applications in this domain like tele-health, air traffic control, mixed reality, learning from demonstration and psychological aspects of agents with HCI. Later, we introduced the musical scale recommendation and explained its importance in the musical world. Music is evergreen and popular worldwide. And in music, musical scale determination is one of the most complex things. We implement it in respect of this proposed topic. Here, we discussed how HCI using multi-agent systems can help the singers get the scale recommendation. Keywords HCI · Multi-agent system · Musical scale · ToM

1 Introduction This chapter enhances the idea of interaction between humans and computers using multi-agent systems, how they are implemented and are designed. The interesting way to start this chapter is by defining the term first agent. What is an agent? Well there is no proper definition but still, we can define it as. An agent is a PC framework that is arranged in some environment; furthermore, that is equipped for independent activity in this environment altogether to meet its plan targets [1]. The authors [2] expressed an agent is whatever can be observed readily to see its current circumstance through sensing elements and follow up on this environment S. Mondal (B) · I. Bhattacharya · S. Gupta Department of Computer Science and Engineering, University Institute of Technology, Burdwan University, West Bengal, Burdwan 713104, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 S. Gupta et al. (eds.), Multi Agent Systems, Springer Tracts in Human-Centered Computing, https://doi.org/10.1007/978-981-19-0493-6_2

5

6

S. Mondal et al.

Output

AGENT

Input

ENVIRONMENT

Fig. 1 An agent in an environment. It takes input from the environment which give rise to an output to the environment

through selectors. Macal expressed [3], an agent will have the accompanying qualities: (i) be recognizable—a distinct independent with a bunch of laws and regulations (numerical or rationale) that administer conduct and dynamic limit; (ii) be found— to become comfortable to an environment where it connects and cooperates with different agents; (iii) be objective-oriented; (iv) act naturally accommodated; and (v) adjustable, and can learn and adjust its conduct through time-sensitive encounters. Figure 1 gives a theoretical perspective on an agent. In this diagram, we can see the activity which is created by the agent to influence its current circumstance or environment. In most areas of sensible intricacy, an agent won’t have full oversight over its current circumstance. Mainly, an agent will have a collection of activities accessible to it. This arrangement of potential activities denotes that the capacity of the effectors of agents which means that the capacity of the agent to change its environment. It is to be noted that not all activities can be acted on in all circumstances, e.g., the action “get a few books” will fail if there are inadequate assets to purchase books. Smart (or objective) agents are utilized in a wide assortment of uses, generally from compact frameworks [4]. Agents are self-governing elements that are not exposed to the impact of outer course. They are evolved using a bottom-up approach and have the limitation of preparing data, while imparting it to different agents, through individual-based collaboration that doesn’t endure hierarchical control. In any case, when another agent undertakes the counterfeit, its activities can be casted by previous standards and which is founded before agent connections and endured via steps of time. These associations can be communicated by the exchange of information starting with single agent then onto the next. In such a manner, miniature and large-scale levels will regularly co-evolve without pre-characterized upper-level regulators. An agent might be objective-driven and can make free moves to arrive at its objectives. In this way, agents contrast conduct results with their objectives and adjust reactions later on. An agent’s conduct can be depicted by basic principles that are used to portray the hypothetical presumptions of agent, as computational methods that lead to the accomplishment of the objective. These methodologies comprise an arrangement for accomplishing agents’ destinations. A multi-agent system (MAS), planned and carried out through a few communicating agents, is broader and distinctly more eye opening than the singular (isolated incident) agent. In a genuine society, there are a

2 Human–Computer Interactions Through Multi-agent Systems …

7

different number of circumstances where the singular-agent case is appropriate. A genuine model is the master partner, where a representative behaves like an agent colleague to a client endeavoring to satisfy some assignment. MAS is a computerized environment of various communicating smart agents. MAS is ideally utilized in tackling issues that are troublesome (or unthinkable) for an independent agent. Likewise, with agents, there is no downright meaning of MAS so let us center around one definition that is generally consistent. MAS is characterized as ‘an approximately coupled organization of critical thinking elements (agents) that work together to discover answers to issues that are past the discrete abilities or information on every substance (agent)’ [5]. According to the pre-conditions actions are related, which characterize the potential circumstances in which they can be applied. The key issue confronting an agent is that of choosing its activities according to the plan to get the best possible action to reach its destination. In computer science (e.g., artificial intelligence), an agent-based model (ABM) for the most part expresses a computer-based strategy for considering the inner actions of a bunch of self-ruling elements. In non-processing-related logical spaces as in sociologies, ABM could allude to an entertainer in the social world and it is called agent-oriented communal reenactment. Davidsson [6], utilizing various mixes of center regions (e.g., agent-oriented computing, computer simulation, and sociologies), further partitions it is distributed into three classes: (i) societal parts of agent frameworks; (ii) multi-agent-oriented clone; and (iii) societal clone. So till now, the only agent is introduced here. To understand human–computer interactions using multi-agent, it is important to understand the agent first. We will move further to the next part gradually.

2 Previous Works Before discussing the related works, let us see the diagram of the agents. It is explained fully in the Introduction section. Here, we just go through the diagram to get a clear idea. Now, let us discuss various previous works related to various technologies or applications.

2.1 Collaborative Concepts of MAS and HCI In 1994, Thomas et al. [7] designed a methodology that is generic and also can be applied to any process control and application using a multi-agent framework. Their human–computer communication permitted to. (i) (ii)

The plan of agents classes with their specific graphical interfaces. The detail of agents from the simple interface.

8

(iii)

S. Mondal et al.

The control of a process, by picking the best representation of these agents.

In 1994, Hayes-Roth et al. [8] represented the guarantee of improvised coordinated creation as a worldview for multi-agent frameworks. They improvised some course of behavior that follows the direction where one or more humans can interact with one or more computers and do collaborative work. As a result, the performance directs the collaboration between human and computer agents.

2.2 Self-Governing Versatile Automated Systems In 1992, Nakauchi et al. [9] explain human collaboration is based on asynchronous communication. The creators had encouraged a package called Michele. They propose a multi-agent interface design called RT-Michele. A model of RT-Michele was executed, and self-governing versatile robots were created. RT-Michele was applied to robotize the office, where it provided not only asynchronous communication, yet additionally constant intelligent correspondence for helpful work among people and versatile robots. In 1995, Suzuki et al. [10] explain the connection between people administrator and the decentralization-oriented self-sufficient automated system. The administrator relates himself freely to the framework in the distributed self-sufficient system. The authors elected the administrator as a troubleshooter and a system monitor. The human administrator is viewed as an agent in the decentralized self-sufficient automated system. At that point, the human administrator and agents are talking about corresponding methodologies. The author recommends explicit and implicit methodologies to monitor the system, and a few checking techniques to carry out them: Time-sensitive and occasion-based techniques are used to express the corresponding environment and listening in messages technique is used to understand the expression. They also compared Solo improvisation and collaborative improvisation. The creators contrast the observing techniques all together by finding out how much data the human administrator can assemble in every strategy utilizing reproduction. At long last, the qualities of each checking strategy are investigated. In 2001, Laengle et al. in their paper [11] presented a new intelligent robot control scheme that empowers good effort of people through direct association of robots in a part known environment. Due to the great elasticity, the human–computer collaboration is relied to possess a wide scope of utilizations in worst conditions, not just in upcoming development and assembling enterprises yet in assistance branches. A multi-agent gives a suitable surrounding for the adaptability of the human–robot group. The robot system KAMRO (two-armed mobile robot system) was introduced which demonstrates the fundamentals between human and robot agents. KAMRO’s implementation was divided into controllers and transition mechanisms. Let’s, first discuss the controller. For the administrated control of the six levels of opportunity of the controller between the bot and the human, the authors utilized a half breed location-power regulator addressed in Fig. 2 [11, 12].

2 Human–Computer Interactions Through Multi-agent Systems …

9

Coordinate Transformation

[P,O]

Position control law

Arm

Constraints

[F,T] Force control law

Coordinate Transformation

Fig. 2 Mixture position-power regulator for a 6DOF

The internal state of the controller is denoted by [P, O], where (Px , Py , Pz ) denotes the position and (Ox , O y , P Oz ) denotes the direction of the sensor. [F, T ] denotes the outside condition of the controller. (Fx , Fy , Fz ) and (Tx , Ty , Tz ) are the power and forces, respectively, applied in torque/sensor which is joined in the sensor. The six-dimensional character grid is represented by I6 where S denotes the diagonal appointed matrix. If a part of S is ‘0’, the comparing level of opportunity is constrained by the ICS, though if the segment is ‘1’, the level of opportunity is constrained by the human (director law or agreeable movement) [11] (Fig. 3). Now, let us discuss transitions. There are two types of transitions: 1. 2.

Automatic transition Physical transition.

In automatic transition, after the reorganization of an inaccuracy the transformation in the middle of A and S happens naturally during the performance done by EO or when the boundaries are absent, the robot perceives its insufficiency to conduct the EO. In physical transition, when the administrator registers the relating powers on the end-effector an adaptation in the middle of level A and S can likewise be started forcefully. Every E Os have protected the movements so that if the applied powers crosses the given change edge, it starts the change. There are some rules which are explained through this example [11].

10

S. Mondal et al.

Human

Robot

,

,

Fig. 3 Controller administration for degree of freedom

if((working_loop_position = 6 ) and ( (working_position = inaccurate) or (working_position = insufficient) or ( > Transition Approach of x) or ( > Transition Approach of y))) then (transition = 1) [32] The authors presented a technique that coordinates the human–robot communication through the assignment’s plan and permits a dynamic combination of human and robot capacities as indicated by the condition of the undertaking execution. The blend of the abilities happens when it crosses over the position-power regulatorwhich upholds actual cooperation among humans and robots.

2 Human–Computer Interactions Through Multi-agent Systems …

11

2.3 Intelligent Air Traffic System In 2015, Canino-Rodríguez [13]explained restricted effectiveness of recent air traffic frameworks which will take part in near future as the smart air traffic system (SATS) that depends on recent robotic progress. The architecture of new human–computer communication for playing out these exercises is a vital component of SATS. This paper is based in the onboard HCI to SATS, the signal from the cockpit is received from airplane route frameworks, encompassing traffic circumstance, regulators’ signs, and so on. Thus, the HCI is proposed to upgrade the circumstance of mindfulness and dynamics through the pilot cockpit. Thus, a multi-agent framework-based methodology is appropriate for demonstrating such an environment. The authors exhibit that current techniques for planning multi-agent frameworks are helpful to portray HCI. The specific and compositional plan of SATS stages as a multi-agent framework that gives enough data to complete a definite plan of its comparing agents, i.e., airplanes, aviation authority, and so on. Subsequently, this data is utilized in this work to finish a more detailed configuration centered on the airplane agent. Inside this agent representative, basic numerical and calculations needed for performing explicit assignments are characterized as agent arrangements. Likewise, plans with comparable objectives are assembled into abilities that model explicit cockpit HCI parts (Fig. 4). There are six functionalities of the aircraft agent plan. (a) (b) (c) (d) (e) (f)

Aviation environmental information management. Onboard system alarm control. Resolve conflict detection. Emergency management. The guidance of the trajectory. Navigation procedure management.

Navigation Procedure Management is the center of cockpit HCI. This ability contains plans for overseeing direction arranging measures. The Navigation Procedure Management has some extra features also which can be added: (i) acquiring client-favored directions; (ii) performing robotized direction exchange measures; (iii) assessing 4D direction recommendations from different agents; (iv) producing new proposition for different agents; and (v) giving aviation direction along with arranged 4D directions. The current model can be implemented as the base for performing the circulated copy through discrete occasions where the agent’s messages trade characterizes occasions and incorporation instruments between frameworks. Consistent simulation is additionally conceivable when the airplane dynamics is carried out for quick time insightful simulation purpose.

12

S. Mondal et al.

Specification of the System Plot

Objective Shared Data

Functionalities

Engineering plan Protocols of Communication

Detailed Design Action of Agent

System Outline

Cockpit Architecture Agent Outline

Capability Outline

Data Descriptors

Fig. 4 Methodological approach

2.4 Home Automation System In 2017, Zheng et al. in their paper [14] proposed a widespread brilliant home control stage design (IAPhome) in context of a multi-agent framework and correspondence adapter, which shows critical flexibility and benefits in numerous angles, including miscellaneous machine connectivity, cooperative command, human– computer communication, and client self-administration. The service-oriented communication is a significant establishment to plan and carry out this engineering which makes it conceivable to coordinate miscellaneous smart machine in an adaptable manner. A substantial strategy of requesting the multi-agent programming strategy to tackle the incorporated command issue of the elegant house system is additionally introduced. The suggested platform framework has been tried in a genuine smart house environment, and the outcomes show the adequacy of our methodology for settling the community command issue of various keen devices. The technologies which are implemented were:

2 Human–Computer Interactions Through Multi-agent Systems …

(i)

(ii)

(iii)

13

Smart Home Communication Technology—Here, they explained and compared various communication protocols network communication principal and controller, configurable network communication modes, communication driver design, and development. Multi-Agent Collaborative Control Systems—Here, they explained various technologies of multi-agent and the smart house system where a device can exchange information with other smart devices that can access the home network. So, a collaborative control will be established. They designed an IAPhome platform multi-agent architecture where the HCI layer is implemented. An important agent called components is outlined at the HCI level, that is the smallest unit in a multi-agent framework. To build up a network association for various devices, a bunch of correspondence segment agents can be intended to construct correspondence boundaries for these smart tools expected to join the smart home organization. At that point, the control circle devices can be intended to handle the secondary activity of each control after the overall control errand of the framework is arranged and decayed. This cycle can be taken care of by the graphical setup dependent on numerous part devices. At that point, the information content conveyed unmanageable circle devices or gathered from sensors will be shipped off the general regulator through the correspondence system and handled by relating the database agents. At last, the storage agents will send their executing information to the relating intelligent devices through the correspondence system, and afterward drive the objective devices to finish the predefined control capacities, to accomplish the objective of community-oriented control. Human–Computer Multi-Agent Architecture—In contrast to different stages, IAPhome’s human–machine gives the design of the whole householdoriented environment, and smart devices’ implementation status and compatibility measures between these devices can be observed. The operation component agents, constrained by clients, can all the while access the relating smart devices and their encompassing signal which is in video format. All the information in the human–computer cooperation framework is gathered and put away in the regulator’s memory data set; furthermore, the human–computer cooperation final devices can exchange data with the information administration agents in the regulator layer to progressively acquire the control interaction information of smart devices continuously and show it in the HMI associate (Fig. 5).

2.5 Mixed Reality-Oriented Framework In 2019, Alqahtani et al. in their paper [15] introduced a plan of agent-based intelligent HCI (iHCI) framework utilizing shared data for mixed reality to get better client experience and data security dependent by setting up mindful registration. To

14

S. Mondal et al.

Human – Computer Interaction in MAS

Monitoring Agent

Operation Agent

Operation Agent

Operation Agent

Video Signal Agent

Controller Data Service Agent

Memory Database Activated Data

Activated Data

Activated Data

Activated Data

Engine Agent

Engine Agent

Engine Agent

Engine Agent

Controller Layer

Device 1

Shared Memory Data Service Agent

Device 2

Fig. 5 Human–machine communication in the multi-agent framework

execute an objective mindfulness framework, they proposed the utilization of nonboundary theoretical flexible learning and a piece learning methodology for getting better flexibility of the recognition. The suggested a configuration that includes the utilization of a setting mindful registering methodology to perceive designs for recreating human mindfulness and handling of sound system design investigation. It gives an adaptable customization technique to scene creation and control. It additionally empowers a few sorts of mindfulness identified with the intuitive objective, client experience, framework execution, classification, and agent recognizable proof by applying a few methodologies, for example, setting design investigation, adaptable learning, and information mindful private registering. (i)

System Design—The outline of the framework is separated into three principal system components: Mixed-Reality Fusion System Component, Client Awareness System Component, and Detecting Network System Component, as demonstrated in Fig. 6. Every system component is upheld by related sections. In the Mixed-Reality Fusion System Component, Improved Matting System, Changeable Scene Layering, and Setting Aware Computing support the framework to adjust to the surface, movement, and profundity information; In the Client Awareness System Component, Client Identification Component upholds the framework to perceive the client, giving the relating data to the association. In the Detecting Network System Component, Framework Security Module is considered for the data framework to ensure the data security

2 Human–Computer Interactions Through Multi-agent Systems …

15

Amplified Mixed Reality Framework Mixed Reality Merging Subsystem

Amplified Reality Merging

Client Awarness Subsystem

Detecting Network Subsystem

Client Identification

System Security

Subsystem Principal Component Analysis

PIC Sensor Network

Fuzzy Knowledge System Feature Extraction

Chaotic Confidential Tunnel

Adjustable Learning Personating Detection Knowledge System Correlation Analysis

Dynamic Point Layering Scalable Learning

Inputting Motion

Adjustable Learning Quality Of Experiencce and Service Management

Context Awarness

Sound system Pattern Analysis

Data Aware QoE Analysis Optimal QoS Management

Surface Depth Context Analysis Optimal QoE-QoS Management

Fig. 6 Component of amplified mixed-reality framework

in assortment, association, stockpiling, and correspondence of data during the HCI communication. “Quality of Experience (QoE) and Quality of Service (QoS) Administration (QoE, QoS, M) module fills in as a connecting module for both the Client Awareness System Component and Detecting Network System Component. QoE and QoS assume a significant part in both client mindfulness and the organization framework. With the help of the QoE, QoS, M, the harmony in the middle of the client experience and framework execution can be utilized to improve the convenience of the entire framework.” [15] Here, surface-based acknowledgment, movement-based acknowledgment, and setting mindful processing are examined and carried out to upgrade the separation. Standards of improvement to diminish the vulnerabilities are considered and examined. Relating improvement systems are presented to ensure the soundness of the arrangement. “QoE-QoS the executives is putted to evaluate the nature of the harmony between client experience and correspondence execution. This module is mostly answerable for the attention to client experience and framework execution. Carrying out an information mindful methodology for QoE-QoS the executives. Three fundamental sorts of QoE-QoS the executives (ideal QoE the board, ideal QoS the board, and QoE-QoS balance the board) are talked

16

(ii)

S. Mondal et al.

about. With the thought of client experience and framework execution, a bound together advancement technique with evaluation and standardization of information space has been presented to carry out QoE-QoS regulation.” [15]. After the evaluation, the information capacity is ciphered in a pressure structure. A convention with public key cryptography is intended to play out the encoding and decoding to ensure the classification of the information stream and verification, carrying out the attention to protection and agent recognizable proof (as far as hardware) to build up the security of the framework. With the thought of both proficiency and execution, a convention has been carried out to build up a turbulent passage for sensibly distributed correspondence. The framework can identify an exceptional node in the organization and secure both the data and correspondence in opposition to attacks. Associated techniques are planned with the thought of the basic space, affectability, connection, and constant appropriation to improve the mindfulness of the framework. Structural Framework—Figure 7 describes the framework design and the work process among the parts. Observation is utilized as the data source to conquer the imperatives of a detector-based information framework. In the

Thinking

OoE & OoS

System

Management

Encryption System

OoE & OoS

Thinking

Management

System

Facial Identification

Decryption System Knowledge System

Mask Mapping

Texture

PKC

ID

Client

Knowledge

Mapping

Recognition

System

Context Pattern Analysis

Presenting

TOI Pattern

Stereo Modeling

Scenario Merging

Stereo Database

Fig. 7 System architecture of mixed reality systems

2 Human–Computer Interactions Through Multi-agent Systems …

17

presented framework as portrayed in Fig. 7, the info is from visualizing frameworks that give visual information of the objective scene. Since high-goal information will be costly as far as a computational asset, before the communication, the information is prepared by QoE-QoS executive’s framework. This brings harmony between client experience and correspondence execution and gives the consciousness of client experience and framework execution. To form the awareness recurrent learning and fuzzy adaptable learning are implemented. Target recognition is initiated by recursive optimization. Comparing information is put away as a fluffy set with training boundaries. In the course of execution, modules are planned by the works [16–21], and comparing construction is blended as a coordinated framework. The framework is planned under CIS engineering, where in the framework the agent may be both a human client or apparatus (PC, camera, or different devices) or blend (client with a machine). Everything agents can gather information from and transfer to the framework relying upon their jobs. To execute target mindfulness, they applied a non-boundary theoretical versatile learning procedure [22]. According to the Gaussian Demonstration, a piece training procedure is intended to work on the flexibility of the acknowledgment. This empowers the framework to separate the objective by movement signals and play out a situation covering task without the help of earlier information on the scene. To execute framework mindfulness (SA), the authors applied information mindful figuring by utilizing the validation (Fig. 8).

Machinery Agent

Data & Filtering System

Agent A

Fig. 8 Structure of the cooperative information system

Agent B

18

S. Mondal et al.

To execute human framework mindful figuring, modeling the cognizance of client experience and framework execution dependent on subset training with the nature of involvement (QoE) and the nature of administration (QoS) and utilize the representation to assess client experience [16], by detecting the help. A camera framework is intended for catching visual information. Utilizing information mindful client experience displaying [16] to pack the information, they improved the proficiency of correspondence. Mean of score (MOS) is utilized to quantify the experience. Information of mindful classified data and correspondence methodology is utilized in the framework. The received information is encoded by applying an information mindful classified PKC methodology [20]. Since the information is circulated, the data can be gotten to and refreshed rapidly and an adaptable learning procedure [22] is utilized to prepare the information framework for distinguishing proof.

2.6 Psychological Aspects of Human–Agent Conversation In 2019 Çelikok et al. in their paper [23] contend for detailing human–computer association as a multi-agent issue, investing AI with a mathematical hypothesis of intelligence to understand and anticipate the client. To separate the methodology from past work, they presented a categorization of user modeling approaches given the degree of organization learned in the cooperation. They portray the ongoing work in utilizing settled multi-agent displaying to put together client representations for many arm formed raider that is constructed on interactive AI frameworks, comprising a proof-of-idea client study. Consider an easy variety of the twenty number of queries game in which the human player chooses an objective term also, the AI on the other side needs to figure the term by posing consecutive inquiries about the relevancy of various terms to the undisclosed objective. The person or client is told to assist the AI by discovering the objective term, as quickly as could be expected, by successively giving yes or no solutions to AI’s inquiries. The issue, from the side of AI perspective, is a many arm formed raider where the term with the greatest importance (among a predetermined set of terms) should established the least amount of communications. After executing two distinctive client representations for this undertaking, a passive client representation that presumes the client’s answers begins from a fixed relevancy outline, and an active client representation that expects that the client has a representation of the AI it is associating with. In 2021, Kopp and Krämer in their paper [24] contended that they should begin to re-consider the signs of agreeable communication and the center of abilities that have been produced for it, which the communicating agents should outfitted with the steady joint co-development. A huge group of task on human–human interaction and its mental cycles that is necessary to be applicable and important to consider when demonstrating the human–agent interaction. Standing out those from current concepts of human–agent connection and define ideas for the improvement of future frameworks.

2 Human–Computer Interactions Through Multi-agent Systems …

19

They suggested implementing an architecture that integrates, at least, three unique layers of participation and cooperation described by various types of informative efforts and socio-psychological procedures [25, 26]. (i)

(ii)

(iii)

Contact—This bears similarity to ideas of compatibility building [27]. Here, progressive co-construction and metalizing will lay on (quick) feeling handling and quick state changes of relevant occasions, for example, spatial direction, look, joint consideration, and appropriately coordinated and significant criticism signals. Perception—This layer is coordinated toward the approach of whether codevelopment of substance is effective and whether the open aim is met. Participation is acknowledged by steady variation as per theories of a shared arrangement is accomplished. Here, major human capacities, for example, expanding understanding and informative achievements utilizing the arrangements [28] assume a part. Ideas and Goals—Here, overall objectives as well as of the connection, accomplices are considered. This incorporates thinking about different arrangements, objectives, and wants, just as the coordination of reliant aims or scheme through methods for, e.g., suggesting, arranging, or embracing.

Figure 9 gives a symbolic representation of the fundamental standards as indicated by which we imagine fruitful models of HAI (human–agent interaction) to be organized. It represents the fundamental degrees between agent cooperation along with the relating angles being facilitated at each layer. Every agent shapes and keeps up applicable aims and convictions for three unique subjective mental viewpoints. These psychological states are prepared through the progressive forecast, control, and proof-based induction systems. Furthermore, every agent is anticipated to have the

Me You We

Goals and Ideas

Me You We

Perception

Me You We

Contact

Social Emotional Clarifying Hierarchy

Inter- Agent Communication

We You Me

We You Me

We You Me

Social Emotional Clarifying Hierarchy

Fig. 9 A multi-facet model of agreeable human–agent association based on metalizing and incremental, joint co-construction

20

S. Mondal et al.

option to plan from open conduct at the individual layer and utilizing suitable semiotic frameworks (e.g., through speech understanding and age). This model broadens a past proposition on exemplified cooperation [28] to a more thorough calculated system for building agents equipped for the cooperative mechanisms of discussion. The goal was to make the fundamental contentions for this sort of broad model and by the large idea, the detailed technique, and execution of which represent a continuing research program.

2.7 Tele-Health-Oriented Automated Systems In 2020, Lanza et al. in their paper [29] proposed to enhance tele-health frameworks’ features to incorporate the capacity to provide the patients with their requirements, working as human guardians. The goal is to assist the autonomous living of patients at their house without losing the opportunity to screen their well-being condition. Application situations are a few, and they expand from effortless clinical helping situations to a crisis one. For example, on account of a convalescent home, the framework would uphold in persistently checking the old patients. Conversely, on account of an epidemic diffusion, for example, the COVID-19 pandemic, the framework might help in all the early emergency stages, essentially lessening the danger of contagion. The paper suggests and depicts a multi-agent design for insightful clinical consideration. It was proposed to utilize the convictions agent design, a portion of it is contrived to be sent in a bot. The computational model which is carrying out the structure contain four components: B (convictions), I (goals), D (preference), and P (plan). Toward the start, convictions takes place for the agent, thus its information about the environment, and an arrangement library, a group of potential designs for arriving at the targets. An arrangement is chosen and afterward enacted only if a few pre-conditions are valid. The representation creates a sequence where an agent begins by refreshing the conviction, it’s like knowing the facts about the society and picking an expectation. At the time of iteration, the agent persistently gets reorganization from the environment, when important it upgrades the conviction, decides wants and expectations by registering recent convictions, and afterward produces an arrangement to arrive at the expectation. Initiating a proposal intends to choose the efficient proposal in the library for everyone the aim’s prerequisite, executing all the activities it contains. It stops and notices the environment following the accomplishment of each activity and has to refresh the conviction to reevaluate the goal. In the most pessimistic scenario, if neither any proposals nor any activities reside in the arrangement library for arriving at a goal the agent ought to be given different proposals; thus, its arrangement library must be advanced [29]. The system is carried out utilizing the Jason structure [30, 31], and the thinking ability of the agent is very much portrayed in [31–33]. Momentarily, thinking cycle of every agent includes four stages: (i) detecting stage; (ii) convictions amendment and refreshing stage; (iii) deliberative stage; (iv) acting stage. Each loop starts perceiving

2 Human–Computer Interactions Through Multi-agent Systems …

Remote Server

OWL Ontology

Physician Office

Memory Management

Patient Environment

Multi Agent System

Manager

21

Memory

Humanoid Robot Virtual Subordinate

Planner

Motion Sensor

Vision

Fig. 10 Tele-healthcare system

the environment and finishes the actions in the surroundings. When the agent has seen something from the environment, insider information controlling is initiated. The center of the logical ability is how the best fits plan of the agent chooses with the unique circumstance. The proposal is divided into three sections: activating event conditions: subplans; actions; communication. Let’s assume the operator scenario. Suppose a patient is influenced by lung infections, so the doctor embeds his finding and the connected treatment by utilizing his far-off regulator application. The model [29] addresses the port of the remote regulator application. Taking care of this port is part crafted by the virtual subordinate agent. According to the perspective of the agent, the finding is converted into a conviction or a group of convictions helpful for choosing a proposal. Alternatively, treatment is converted into suggestion. The framework is modified such that lets MyRob continually screen the patient, manage treatment, and update the doctor through the information administrative agent (Fig. 10).

2.8 Demonstration-Oriented Training In 2021, Papadopoulos et al. in their paper [34] introduced a new cognitive framework for multi-agent, Learning from Demonstration (LfD) machinery learning, focusing to empower the dependable distribution, adaptable, and expandable mechanical frameworks in enormous scope and complex conditions. Specifically, the planned

22

S. Mondal et al.

engineering benefits from the recent advancement in artificial intelligence held, by setting up a Federated Learning (FL)-based structure for manifesting a multi-human multi-robot shared training environment. The major formulation depends on utilizing different AI-enabled coherent processes (executing different automated tasks) that work at the edge hubs of an organization of mechanical stages, while worldwide AI framework (supporting the previously mentioned automated undertakings) are by and large made, and divided between the organization, by carefully joining data from an enormous number of human–robot collaboration cases. Concerning curiosities, the planned intellectual design (a) presents another FL-based formalism that expands the ordinary Learning from Demonstration learning worldview to help enormous scope multi-agent functional settings; (b) expounds past Federated Learning-based selfeducating automated plans to merge the human in the learning circle and (c) solidifies the major principles of federated learning with extra refined. Computer-based intelligence empowered learning approaches for modeling the multi-dimensional interrelatedness among the robotic tasks. Each mechanical stage comprises the accompanying principle layers: (a) the detector, (b) the automated machine, and (c) duty S = { si |i ∈ [1, I ], i ∈ N , I ∈ N } = {Vision, Light, Inversion, Chemical, Force Acoustic, Gasoline, Magnetic, Motion . . .}

(1)

The arrangement of possibly assistant types of detectors, which can be crucially wide and also relies upon the specific applying case are explained asT = { tk |k ∈ [1, K ], k ∈ N , K ∈ N } = {Sensing, Manipulation, Navigation, Control, Human - Robot interaction}

(2)

The set of accessible kinds of robots is signified: T = { r j | j ∈ [1, J ], j ∈ N , J ∈ N } = {Arm, AGV, Humanoid, UAV, Vehicle, Industrial, . . . }

(3)

Considering the previously mentioned formalisms, an automated stage Pl can be completely indicated as follows: Pl = {Sl , Rl , Tl |Sl ⊆ S, Rl ⊆ R, Tl ⊆ T } , l ∈ N

(4)

here L represents the complete number of automated stages in the reasonable environment studied.

2 Human–Computer Interactions Through Multi-agent Systems …

23

Within the designed collaborative learning environment with several agents, the human-based factor represents a significant building block to concurrently. (i) (ii)

transfer refined and demanding abilities to the bot and monitor/check the absorbing capability of the bot (learning process).

Having added the constituent elements of the proposed psychological AI engineering (specifically the robot stage, the human and the aggregate (FL) intellectual AI layer), their position furthermore, the relating formalisms, this subphase summarizes how the remaining intention of cooperative getting to know in a multi-agent human–robotic surroundings is fulfilled. Specifically, information/ability convey (from the human side toward the robotic one) is found out through the subsequent two pathways: (i)

(ii)

Straight Human–Computer Learning—A human instructor h m can give feedback to a utilized automated platform Pl concerning the working of AI model Wβ , coming about into the making of the neighborhood/altered model Wβl,m as per [35]; the assessed Wβl,m ; can likewise be kept up with and utilized as a ‘customized’ form of the worldwide model Wβ . Straight Robot–Robot Learning—Through the foundation aggregate learning component of FL [36, 37], information (Wβl,m ) from all robotic stages Pl is joined (to create a updated form Wβ of every individual Wβ ); the last is consequently shared/conveyed back to all the Pl in the network.

Repetitive execution of the above component prompts the manifestation of a collaborative human–robot LfD learning environment/plot. The learning capacities of the proposed cognitive AI architecture [explained in 34] are additionally built up by fusing: (a) implies for modeling, interpreting, and combining multi-user feedback information, specifically client weighting, parameter weighting, and client clustering [38–41] and (b) progressed AI learning procedures for assessing, modeling, and exploiting cross-task correlation information, namely transfer, multi-task, and metalearning [42–45] (Fig. 11).

3 Musical Scale Predictor with HCI Using Multi-agent Till now we understood various implementations of previous works for this proposed topic. In this section, we will discuss and demonstrate our plan regarding the proposed topic “’CI using multi-agent’. Before explaining our plan, let us discuss the importance of music in this world. Why our plan is related to music let us discuss. Music is one of the most popular subjects in this world. Music is a piece of the entirety of our lives, from when we’re growing up to when we’re old. We grow up to the accent of our mothers singing us youngsters’ tunes so that we’d rest. Music is quite possibly the most quieting and alleviating thing if you let it be. It comes from songs and tunes hung together by individuals who sing and play instruments. Music has for some time been a significant piece of human existence, and some

24

S. Mondal et al.

Cognitive AI Layer Transfer Learning

Node

Selection

Robotic Platform Task Layer

Multitasking

Meta Learning

Aggregator

Global AI model

Human Learning from Demonstration

Sensing, Navigation, Human-robot Interaction

Robot Layer

Tele operation

Arm, Humanoid, Vehicle, Industrial

Sensor Layer Vision, Light, Inversion, Chemical, Gasoline etc.

Fig. 11 Significant-level representative of the introduced multi-agent psychological AI cooperative learning [34]

specialists recommend that music is both an all-inclusive and animal category explicit characteristic of mankind [46]. Music-like language may work as a correspondence framework [47, 48] equipped for recognizing unobtrusive shades of involvement with exceptionally organized ways. Numerous ways of melodic construction and creations are discussed in these papers [49–52] with experiments. Till now we understood the importance of music. Now we will discuss how we can use it with our related topic. User modeling is utilized to customize user encounters and improve the convenience and execution of cooperative human–computer frameworks [53]. To work with proficient collaboration, the machine makes suppositions and perhaps finds out about the objectives, expectations, convictions, behavior, or different qualities of the client. Lately, the models of clients have joined statistical and AI strategies to naturally translate and sum up the information gathered in the communication. To separate the hypothesis of mind-based client demonstrating from past work, Çelikok et al. in their paper [23] present a categorization of users demonstrating ways to deal with four dimensions based on their suppositions about the behavior of the client during communication. Specifically, considering how versatile the user is thought to be concerning the framework, with more significant levels inferring more composite learning issues of the client model during the communication (Fig. 12). The categorization is divided into four parts [23]:

2 Human–Computer Interactions Through Multi-agent Systems …

L1

L3

25

L2

L4

Fig. 12 Categorization of user demonstrating ways to deal with four dimensions based on their suppositions about the behavior of the client during communication

(L1) fixed, prescriptive behavior, (L2) detached, receptive client behavior, (L3) effectively arranging client utilizing a detached, fixed framework, (L4) effectively arranging clients utilizing a versatile, non-fixed framework. The first level endorses a representation for the client. This can incorporate progressed and composite client representations. However, L1 user models don’t adjust or learn dependent on the connection information and have restricted methods for representing singular changeability. The second-level representations, on another way, get familiar with an outline (e.g., curiosity, inclinations, objectives, sort) of the client at the point of the collaboration, however, in any case model don’t has any strategic behavior. These representations incorporate collaborative filtering and multi-armed raider, which have been exceptionally fruitful in suggestion frameworks. The third level of representations recognizes the client as a functioning agent who proposes or possibly acts dependent on a stable or fixed representation of the environment. This third level incorporates converse support instructions and hypothesis of mental representations, for example, the Bayesian hypothesis of psyche [54] and machine hypothesis of psyche [55], and predicts the agent’s behavior by noticing its

26

S. Mondal et al.

activities. The fourth level further recognizes that the client will shape a representation of the intelligent framework which they are utilizing. Specifically, L4 contrasts from L3 when the framework is likewise versatile and finds out about the environment and the client dependent on perception during the communication. At that point, the framework of client’s representation contains a replica of the framework, together with the versatile behavior, on basis of which it gives suggestions to the client. This is guided by nested or recurrent mathematical multi-agent demonstration.

3.1 Mathematical Theory of Mind (ToM) Replicas in Musical Scale Prediction In the art of multi-agent, on the basis of ToM representations target to catch the tactical behavior and therefore the primary faiths of others for selecting ideal actions [56]. They are included under the recurrent reasoning class of opponent modeling [57–60]. Probably the most punctual example of such hypothesis of mind ways is that the intellectual chain of command hypothesis [61]. A lot of recently, the interactive partly evident Markoff’s resolution procedure (I-POMDPs) provides a resolutionbased hypothetical structure for algorithmic agents that react severally and solely expects to be halfway examination regarding the environment [58]. Back to the context, here we will discuss the Musical scale predictive framework. But, first, let us discuss it’s importance. Singers/musicians on stage get some requests for random songs. Now, those singers/musicians have to decide the musical scale on stage in front of a live audience which is time-consuming as well as sometimes not so accurate. Our representation will help to predict the scale of random songs concerning the singer’s vocal capacity. Till now, this type of representation is not designed. Not only in live concerts but also it will be useful during music compositions and musical reality shows too. The vocal range of humans changes rapidly during cough and cold or fever. Singers at that time can’t sing on their normal regular scale. This model will work at critical times too. So, we understood how effective this model can be in the world of music. Now, let’s see the model which predicts musical scale. The basic structure of this model is explained in this referenced paper [23]. Here we will use the basic structure to implement it to determine musical scales (Fig. 13). The rounds of communication comprise the framework proposing an element i(t) to the client and the client giving a response a(t). The framework attempts to help the client to get the most amount of the accumulated reward of the object options, where is the award for object i(t). We will understand this with an example to get a clear idea. We will understand this by an example. Let there are two singers ‘A’ and ‘B’. Singer A’s vocal range is from C3 to D6. B’s vocal range is from F2 to G5. That means A has the highest, and B has the lowest vocal range. Let, the original song ranges in between C3 to D4. Now as both the singers’ vocal ranges are matching

2 Human–Computer Interactions Through Multi-agent Systems …

27

Scale Suggestion i(t)

Action Chosen a(t)

Passive Client Representation

User’s Vocal Range ( )

Scale SugAnalyze the range

gestion i(t)

System Model

of User’s Vocal with respect of Original Range

Original Range of the song ( )

Action

Next-time

Chosen

Scale Sugges-

a(t)

tion i(t+1)

Active Client Representation for Predicting Musical Scales

Fig. 13 In passive client representation, the response on occasion t, a(t) depends on the object on occasion i(t). In active client representation, the client expects the arrangement’s next scale suggestion i(t + 1) representation and chooses the action a(t)

with the range of original songs. Now first understand what this musical note means (Fig. 14). Now assume singer A mostly sings high-pitched songs like Sufi while Singer B is comfortable with mid-range Bollywood songs. The system will recommend randomly a musical scale for the first time to the two singers. If a user can anticipate what the system will suggest, they can dynamically schedule their responses to direct the system toward applicable elements. The active client representation is based on the modeling of the client as a planner in a Markov decision-making process that settles a model of the framework bandit [54]. In music let suppose the system recommends the singer A to sing half a note higher than the original song, i.e., C#3 to D#4. Remember, the original song ranges between C3 to D4. But the system will randomly recommend the user. # means sharp in music, i.e., half a note higher

28

S. Mondal et al.

Fig. 14 Pictorial view of musical notes

than the original notes. As singer A is a Sufi singer so he/she will give feedback to the system and the system gradually will adapt Singer A’s favorable scale. The same method will be also used for B. As, B likes mid-range songs. The system will recommend randomly according to B’s vocal range. Maybe one or half note up or down than the original notes. B will give a reward for the recommendation. And later, the system will adapt and will recommend musical scales which will be favorable for Singer B to sing for some other songs. This will help the music industry and stage performers not to think much manually about their vocal ranges to particular songs. In recent times, human–computer interaction treats users as data sources and assumes that clients serve the system as a machine. If we want to understand the client in a better way, we have to put together human–computer communication with a multi-agent approach. This will increase richer interaction possibilities and also ways to communicate with ToM-oriented methods for modeling the client.

4 Conclusion With the agent-based approach, the systems are easier to use as they can learn from different users and adapt to the needs of different users. This is a very important and fundamental aspect of systems. The system’s quality directly depends on how it is represented and used by the user. We firmly believe that the idea of an agent as a rational decision-maker is important not only in AI but also in mainstream computing. Likewise, recent advances in the MAS investigation, allowing agents to collaborate and negotiate, will certainly be crucial meaning in the future. The understanding, and the concept of MAS applications will continue to grow as reported due to its

2 Human–Computer Interactions Through Multi-agent Systems …

29

rapid advancements. Now, we conclude what we discussed in this chapter and what we learned till now. At first, we discussed the agent. After understanding the agent we directly moved toward previous works which had been done on this proposed topic. In the beginning, we discussed some concepts of old works of the 90’s era, gradually year by year moving forward, and later discussed some recently proposed models in recent times after 2015 onward. This as a whole gave us an idea of how this topic is going popular year by year. We understood how HCI using multi-agent is implemented in psychology, aircraft models, healthcare systems, demonstrative learning models as well as the theory of mind. Later we discussed our idea to implement HCI using a multi-agent system in determining musical scales for the singers. This paper endeavored to give an outline of these issues and give a review of existing research through an extensive reference list.

References 1. Woolbridge M, An introduction to multi agent systems. Wiley (Book) 2. Russel S (2014) Norvig P artificial intelligence—a modern approach, 3rd edn. Pearson Education Limited, England, p 1079 3. Macal CM (2016) Everything you need to know about agent-based modelling and simulation. J Simul 10(2):144–156 4. Rocha J, Boavida-Portugal I, Gomes E (2017) Introductory chapter: multi-agent systems. https://doi.org/10.5772/intechopen.70241 5. Stone P, Veloso M (2000). Multiagent systems: a survey from a machine learning perspective. Auton Robot 8(3):345–383 6. Davidsson P (2002) Agent-based social simulation: a computer science view. J Artif Soc Soc Simul 5(1):1–7 7. Thomas MC, Gallesio S, Sanchez B, Tigli J (1994) Human computer interaction based on a multi-agent system. In: Proceedings of IEEE international conference on systems, man and cybernetics, vol 2, pp 1440–1445, https://doi.org/10.1109/ICSMC.1994.400048 8. Hayes-Roth B, Brownston L, Gent Rv (1995) Multi-agent collaboration in direct improvisation. In: Proceedings of the first international conference on multiagent systems 9. Nakauchi Y, Okada T, Yamasaki N, Anzai Y (1992) Multi-agent interface architecture for human-robot cooperation. In: Proceedings 1992 IEEE international conference on robotics and automation, vol 3, pp 2786–2788, https://doi.org/10.1109/ROBOT.1992.220012 10. Suzuki T, Yokota T, Asama T, Kaetsu H, Endo I (1995) Cooperation between the human operator and the multi-agent robotic system: evaluation of agent monitoring methods for the human interface system. In: Proceedings 1995 IEEE/RSJ international conference on intelligent robots and systems. Human robot interaction and cooperative robots, vol 1, pp 206–211. https:// doi.org/10.1109/IROS.1995.525798 11. Laengle T, Woern H (2001) Human-robot-cooperation using multi-agent-systems. https://doi. org/10.1023/A:1013901228979 12. Raibert MH, Craig JJ (1981) Hybrid position/force control of manipulators. ASME J Dynam Syst Meas Control 102:126–133 13. Canino-Rodríguez JM, García-Herrero J, Besada-Portas J, Ravelo-García AG, TraviesoGonzález C, Alonso-Hernández JB (2015) Human computer interactions in next-generation of aircraft smart navigation management systems: task analysis and architecture under an agent-oriented methodological approach. Sensors 15:5228–5250. https://doi.org/10.3390/s15 0305228

30

S. Mondal et al.

14. Zheng S, Zhang Q, Zheng R, Huang BQ, Song YL, Chen XC (2017) (2017) Combining a multi-agent system and communication middleware for smart home control: a universal control platform architecture. Sensors 17:2135. https://doi.org/10.3390/s17092135 15. Alqahtani H, Liu C, Kavakli-Thorne M, Kang Y (2019) An Agent-based intelligent hcl information system in mixed reality. In: 28th international conference on information systems development, Toulon, France 16. Li J (2015) The benefit of being physically present: a survey of experimental works comparing copresent robots, telepresent robots and virtual agents. Int J Hum Comput Stud 77:23–37. https://doi.org/10.1016/j.ijhcs.2015.01.001 17. Liu CZ, Kavakli M (2016) Fuzzy knowledge-based enhanced matting. In: 2016 IEEE 11th Conference on industrial electronics and applications (ICIEA). IEEE, pp 934–939. https://doi. org/10.1109/ICIEA.2016.7603716 18. Liu CZ, Kavakli M (2016) Data-aware QoE-QoS management. In: 2016 IEEE 11th conference on industrial electronics and applications (ICIEA). IEEE, pp 1818–1823. https://doi.org/10. 1109/ICIEA.2016.7603882 19. Liu CZ, Kavakli M (2016) Knowledge-based pattern-context-aware stereo analysis and its applications. In: 2016 international conference on digital image computing: techniques and applications (DICTA). IEEE, pp 1–8 20. Liu CZ, Kavakli M (2016) Mixed reality with a collaborative information system. In: Advances in services computing: 10th Asia-Pacific services computing conference, APSCC 2016, Zhangjiajie, China, November 16–18, Proceedings 10. Springer, pp 205–219 21. Liu CZ, Kavakli M (2018) An agent-based collaborative information processing system for mixed reality applications—part b: Agent-based collaborative information processing and coordination. In: 2018 IEEE conference on industrial electronics and applications (ICIEA), page submitted 22. Liu CZ, Kavakli M (2017) A data-aware confidential tunnel for wireless sensor media networks. Multimedia tools and applications, pp 1–23 23. Çelikok M, Peltola T, Daee P, Kaksi S (2019) Interactive AI with a theory of mind. https:// arxiv.org/abs/1912.05284 24. Kopp S, Krämer N (2021) Revisiting human-agent communication: the importance of joint coconstruction and understanding mental states. Front Psychol 23(12):580955. https://doi.org/ 10.3389/fpsyg.2021.580955.PMID:33833705;PMCID:PMC8021865 25. Allwood J, Nivre J, Ahlsén E (1992) On the semantics and pragmatics of linguistic feedback. J Semant 9:1–26. https://doi.org/10.1093/jos/9.1.1 26. Buschmeier H, Kopp S (2018) Communicative listener feedback in human-agent interaction: artificial speakers need to be attentive and adaptive. In: Proceedings of the 17th international conference on autonomous agents and multiagent systems; July 10–15; Stockholm, Sweden 27. Huang L, Morency LP, Gratch J (2011) “Virtual Rapport 2.0” in intelligent virtual agents. IVA 2011. In: Vilhjálmsson HH, Kopp S, Marsella KR (eds) Lecture notes in computer science, vol 6895. Springer, Berlin. https://doi.org/10.1007/978-3-642-23974-8_8 28. Kopp S (2010) Social resonance and embodied coordination in face-to-face conversation with artificial interlocutors. Speech Comm 52:587–597. https://doi.org/10.1016/j.specom.2010. 02.007 29. Lanza F, Seidita V, Chella A (2020) Agents and robots for collaborating and supporting physicians in healthcare scenarios. J Biomed Inform 108:103483. https://doi.org/10.1016/j.jbi.2020. 103483 30. Bordini RH, Hübner JF (2005) BDI agent programming in AgentSpeak using Jason. In: International workshop on computational logic in multi-agent systems, Springer, pp 143–164. https:// doi.org/10.1007/11750734_9 31. Bordini RH, Hübner JF, Wooldridge M (2007) Programming multi-agent systems in AgentSpeak using Jason (Wiley Series in Agent Technology). Wiley, USA 32. Chella A, Lanza F, Seidita V (2018) Human-agent interaction, the system level using JASON. In: Proceedings of the 6th international workshop on engineering multi-agent systems, Stockholm

2 Human–Computer Interactions Through Multi-agent Systems …

31

33. Chella A, Lanza F, Seidita V (2019) Decision process in human-agent interaction: extending JASON reasoning cycle. Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) 11375 LNAI, pp 320–339. https:// doi.org/10.1007/978-3-030-25693-7_17 34. Papadopoulos GT, Antona M, Stephanidis C (2021) Towards open and expandable cognitive AI architectures for large-scale multi-agent human-robot collaborative learning. In: IEEE Access, vol 9, pp 73890–73909. https://doi.org/10.1109/ACCESS.2021.3080517 35. Billard AG, Calinon S, Dillmann R (2016) Learning from humans. In: Siciliano B, Khatib O (eds) Springer handbook of robotics, Springer handbooks. Springer, Cham, pp 1995–2014. https://doi.org/10.1007/978-3-319-32552-1_74 36. Caccavale R, Saveriano M, Finzi A, Lee D (2019) Kinesthetic teaching and attentional supervision of structured tasks in human-robot interaction. Auton Robot 43(6):1291–1307. https:// doi.org/10.1007/s10514-018-9706-9 37. Zhu Z, Hu H (2018) Robot learning from demonstration in robotic assembly: a survey. Robotics 7(2):17 38. Capurso M, Ardakani MMG, Johansson R, Robertsson A, Rocco P (2017) Sensorless kinesthetic teaching of robotic manipulators assisted by observer-based force control. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pp 945–950. https:// doi.org/10.1109/ICRA.2017.7989115 39. Liu Y, Gupta A, Abbeel P, Levine S (2018) Imitation from observation: learning to imitate behaviors from raw video via context translation. In: Proceedings of IEEE international conference on robotics and automation (ICRA), pp 1118–1125 40. Xu Y, Yang C, Zhong J, Wang N, Zhao L (2018) Robot teaching by teleoperation based on visual interaction and extreme learning machine. Neurocomputing 275:2093–2103 41. Zhang T, McCarthy Z, Jow O, Lee D, Chen X, Goldberg K, Abbeel P (2018) Deep imitation learning for complex manipulation tasks from virtual reality teleoperation. In: Proceedings of IEEE international conference on robotics and automation (ICRA), May 2018, pp 1–8 42. Finn C, Yu T, Zhang T, Abbeel P, Levine S (2017) One-shot visual imitation learning via meta-learning. arXiv:1709.04905. [Online]. Available: http://arxiv.org/abs/1709.04905 43. Hussein A, Gaber MM, Elyan E, Jayne C (2017) Imitation learning: a survey of learning methods. ACM Comput Surv 50(2):1–35. https://doi.org/10.1145/3054912 44. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, Dieleman S, Grewe D, Nham J, Kalchbrenner N, Sutskever I, Lillicrap T, Leach M, Kavukcuoglu K, Graepel T, Hassabis D (2016) Mastering the game of go with deep neural networksand tree search. Nature 529(7587):484–489 45. Zhang M, McCarthy Z, Finn C, Levine S, Abbeel P (2016) Learning deep neural network policies with continuous memory states. In: Proceedings of IEEE international conference on robotics and automation (ICRA), May 2016, pp 520–527 46. Hodges DA (1996) Human musicality. In: Hodges DA (ed) Handbook of music psychology, 2nd edn. IMR Press, San Antonio, pp 29–68 47. Aiello R, Sloboda JA (1994) Musical perceptions. In: Aiello R, Sloboda JA (eds) Oxford University Press 48. Jones MR, Holleran S (1992) Cognitive bases of musical communication. American Psychological Association, Washinton, DC 49. Bharucha JJ (1984) Anchoring effects in music: the resolution of dissonance. Cogn Psychol 16:485–518. https://doi.org/10.1016/0010-0285(84)90018-5 50. Bharucha JJ (1994) Tonality and expectation. In: Aiello R, Sloboda JA (eds) Musical perceptions. Oxford University Press, New York, pp 213–239 51. Dowling WJ (1978) Scale and contour: two components of a theory of memory for melodies. Psychol Rev 85:341–354. https://doi.org/10.1037/0033-295X.85.4.341 52. Slodaba JA (1985) The musical mind: a cognitive psychology of music. Oxford University Press, New York 53. Fischer G (2001) User modeling in human–computer interaction. User Model User-Adap Inter 11(1–2):65–86

32

S. Mondal et al.

54. Baker CL, Jara-Ettinger J, Saxe R, Tenenbaum JB (2017) Rational quantitative attribution of beliefs, desires, and percepts in human mentalizing. Nat Hum Behav 1:1–10. https://doi.org/ 10.1038/s41562-017-0064 55. Rabinowitz NC, Perbet F, Song HF, Zhang C, Eslami SM A, Botvinick M (2018) Machine theory of mind. In: Proceedings of the 35th international conference on machine learning, ICML, pp 4218–4227 56. Hernandez-Leal P, Kaisers M, Baarslag T, de Cote EM (2017) A survey of learning in multiagent environments: dealing with non-stationarity. arXiv:1707.09183 57. Albrecht SV, Stone P (2018) Autonomous agents modelling other agents: a comprehensive survey and open problems. Artif Intell 258:66–95. https://doi.org/10.1016/j.artint.2018.01.002 58. Gmytrasiewicz PJ, Doshi P (2005) A framework for sequential planning in multi-agent settings. J Artif Intell Res 24:49–79. https://doi.org/10.1613/jair.1579 59. Oliehoek FA, Amato C (2016) A concise introduction to decentralized POMDPs. Springer 60. Pynadath DV, Tambe M (2002) The communicative multiagent team decision problem: analyzing teamwork theories and models. J Artif Intell Res 16(2002):389–423 61. Camerer CF, Ho T-H, Chong J-K (2004) A Cognitive hierarchy model of games. Q J Econ 119(3):861–898. https://doi.org/10.1162/0033553041502225

Chapter 3

Social Behavior and Reasoning Through Multi-Agent Systems Indradeep Bhattacharya , Swarnavo Mondal , and Shibakali Gupta

Abstract In computer science, multi-agent systems are a relatively new sub-field composed of numerous interconnected computing elements known as agents. To satisfy their design intents, they can take decisions for themselves, i.e., they are capable to take autonomous decisions. Also, these agents are capable of communicating with other agents not only by commuting data but also by engaging in similar the kind of social activities that we all engage in every day of our lives, like, association, coordination, and the like. Social behavior is behavior among two or more organisms inside the identical species and encompasses any conduct wherein one member affects the other. For example, the interaction of two or more humans or organizations governed toward a common goal that is mutually beneficial. Whereas, social reasoning involves the propensity to illustrate inferences about others’ intentions, mentality, and actions or activities, in order to synchronize one’s own behaviors. In this chapter, we have developed a region-specific vaccination method using multi-agent framework. We have taken the help of the Mamdani fuzzy inference system to build this multi-agent framework. Keywords FVMAS · Region specific vaccination · MFIS · Fuzzy vaccination drive

1 Introduction There exist numerous computational tools for the simulation and layout of emergency evacuation and egress. However, because of the shortage of human and social behavioral statistics, these computational tools depend upon assumptions that have been observed inconsistent or unrealistic. In a study, Pan et al. [12] proposed a multiagent-based framework to simulate the human and social behavior during emergency evacuation. They had developed a prototype system that was capable to illustrate some emergent behaviors, such as competitive, queuing, and herding behaviors. The I. Bhattacharya (B) · S. Mondal · S. Gupta Department of Computer Science and Engineering, UIT, Burdwan, West Bengal 713104, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 S. Gupta et al. (eds.), Multi Agent Systems, Springer Tracts in Human-Centered Computing, https://doi.org/10.1007/978-981-19-0493-6_3

33

34

I. Bhattacharya et al.

prototype system was primarily divided into two parts. Initially, the representation of the physical environment, and secondly, the human and social behaviors, which together made up the simulation scenarios. The fundamental step to construct the simulation system was to establish an appropriate representation for the physical environment (e.g., a building) consisting of relevant geometric information. Also, in this system, the autonomous agents were humans equipped with several sensors, rule base, and actuators. The actuators were only responsible for moving and controlling the system. Their purpose was to capture the human cognitive process (i.e., their conscious intellectual activity such as thinking, reasoning, remembering, etc.) by perceiving the environment, processing the information, and reacting to situations. Nowadays, the entire world is becoming spell-bound by witnessing the death-pool because of the novel coronavirus (Covid-19). In order to limit the spread of SARSCoV-2 [21], public health recommendations have encouraged social distancing and masks as economies across the USA. Repeated cross-sectional survey study, Sheehan et al. [16] had described changes in social behavior in Ohio (a state in the Midwestern region of the USA) during periods of decreasing and increasing cases. According to the authors, in Ohio, gatherings of more than 10 persons were strictly prohibited. Also, the adoption of social distancing and face masks was mandatory to prevent the rapid spread of the virus. Not only in Ohio, but we could also observe these kinds of behaviors almost all over the world. In this pandemic scenario, to evaluate its ubiquity, it is essential to have appropriate models that allow real-time structuring of the effect of various quarantine estimation by the state. In a study, Vyklyuk et al. [20] had improved the implementation of the SIR model [9]. The SIR model is nothing but a simple mathematical model of epidemics, where S, I, and R stand for, susceptible, infected or infectious, and recovered. Their proposed paper demonstrated definite methods to upgrade the rules of the interactivity and behavior of agents. Their developed model allowed us to evaluate the effect of quarantine, limitations on transport and communication between regions, to consider such factors like, mask wearing routine, prolonging social distancing, and so on. Several observations had been organized in their research, which made it feasible to evaluate both the effect of individual measures to prevent the pandemic and their extensive application. In the following section, we have discussed their work in detail. On the other hand, Saqqar et al. [1] proposed a paper to analyze and reason about group communicating social commitments in multi-agent Systems. In fact, their proposed research described the Computation Tree Logic Group Commitments (CTLGC ). It is nothing but a temporal logic of group commitments for agent communications which extends Computation Tree Logic (CTL) to reason about group social commitments and their fulfillments simultaneously. To do so, they classified groups of communicating agents into divisible and indivisible. Also, they divided group commitments into two categories, i.e., one-to-group and group-to-one commitments. They also supported the necessary social accessibility relations that were needed to capture the semantics of each type of group commitments. Thereafter, they used Benthem’s Correspondence Theory for modal logics to rationalize the soundness and completeness of the proposed CTLGC logic. Fundamentally, they defined a set of reasoning postulates in CTLGC and used the NetBill protocol, a concrete example from busi-

3 Social Behavior and Reasoning Through Multi-Agent Systems

35

ness domain to illustrate each reasoning postulate. Furthermore, they had adopted the interpreted systems as an underlying formalism over which they developed postulates were interpreted. This ensured that the CTLGC logic generated by any subset of the proposed postulates was sound and complete with respect to models that were built on the corresponding frames. By doing so, they provided a novel, consistent, formal, and computationally grounded semantic to reason about group communicating social commitments and their fulfillments in multi-agent systems and also proved the soundness as well as completeness of the proposed logic. Recent technologies and market solutions are unable to fulfill the Ambient Intelligence (AmI) vision of simplified people–environment interactions. The Internet of Things (IoT) technologies provide needed infrastructure for recent solutions. However, most approaches suffer from inadequate levels of intelligence and autonomy. In this context, Ruta et al. [15] proposed a novel semantic-based multi-agent system framework adhering to the emerging social Internet of Things paradigm. Their proposed framework was devoted to improve both automation and adaptivity, i.e., device agents self-organized in social relationships, interacting independently and sharing information, cooperating and organizing ambient resources. Benefits of their proposal were highlighted through an AmI case study in the field of Home and Building Automation (HBA). The authors primarily focused on agent coordination in purposely infrastructured environments and particularly in “domotics” scenarios through interaction paradigms inspired by social networks. The term “domotics” [18] originates from the Latin word “domus”, which means house. Domotics is the encounter of information technology, electrotechnics, and electronics that fabricates a home to become smart. The prime intention of the authors was to develop a semantic-based social multi-agent system. To do so, first, they had created a framework and architecture for the social agents. In order to build this framework and architecture, initially they made a mapping between technical feature and social environment. Let, denotes the ordered pairs representing the technical feature (TF) and social environment (SE). According to authors, these pairs were , , , , , , , , and . These pairs highlighted the basic correspondences of entities and features in a generic AmI domain to the proposed social multi-agent system environment. In their framework, every object acted as a social agent, and it represented its individual profile by describing its general attributes (e.g., device/object type, location, and hardware details) as well as the resources/services it could provide through possible configurations. An agent was also able to become a friend and/or follower of other agents. It could write posts on either its wall or friends’ walls when its capabilities changed or modified, and also when it produced new or updated information after a context analysis. Here, each post contained perceptions and events observed by the social agents. In their proposed SOA-based MAS (service oriented architecture-based multi-agent system), they classified agents into three categories, such as sensor agents, actuator agents, and smart agents. After defining the framework and architecture for social

36

I. Bhattacharya et al.

agents, they had defined the semantic-based social network interaction. Finally, they had selected HBA platforms as reference systems to assess both peculiarities and capabilities of the proposed semantic-based social multi-agent system. In this chapter, we have demonstrated a method to predict the importance of Covid-19 vaccination for any specific region using Mamdani fuzzy inference system and also depicted the Fuzzy Vaccination Multi-Agent System (FVMAS) framework to clarify our vision.

2 Literature Survey In this section, we have demonstrated several types of researches related to the multiagent system to portray social coordination, cooperation, behavior, reasoning, etc. • An unambiguous planning and distributed task allocation are contemplated to be demanding problems. To achieve planning and complete distributed tasks, there must be a need for the cooperation of independent agents called planning agents resided in a multi-agent framework. In this context, Gharbi et al. [7] proposed a method to solve the distributed task allocation problem where agents firmly allocated the jobs while they were constructing the plans. They had developed and verified certain characteristics using the computational tree logic (better known as, “CTL”) [11] with the model checker “its-ctl” [19]. They had also shown the functioning of a benchmark production system (BPS) to describe their contribution. Their results proved that the proposed system was well organized, as the time complexity was very less and it required a fewer number of message passing. To realize their propose system, first we need to understand their benchmark production system, “RARM”. It was comprised of two input conveyors (i.e., C1 and C2) and one output conveyor (i.e., C3), a robotic arm, and a processing unit. There were two types of workpieces: type A and type B. Conveyors C1 and C2 carried workpieces of type A and type B, respectively. At most one workpiece could be on the input conveyor. The robot r transferred workpieces sequentially to the processing element. The succeeding workpiece must be put on either C1 or C2 or both when it had been vacated by the robot (i.e., “r”). They had defined three production strategies, which have been referenced underneath. i In the initial production strategy, a type A workpiece had been inserted via the input conveyor C1 into the processing unit M to be treated, then it was shifted by r to the output conveyor C3. ii In the second production approach, a type B workpiece had been loaded via C2 into the processing element M to be processed, then it was moved out by r to the output conveyor C3. iii In the culminating production strategy, the type A workpiece had been placed via C1 into the processing unit M to be processed; afterward, the type B workpiece was appended into M, and they were finally congregated.

3 Social Behavior and Reasoning Through Multi-Agent Systems

37

The congregated outcome was taken by the robot r and put above the C3 conveyor of output. According to the benchmark production system, the outcome could be transformed on C3 only when it is empty and ready to receive the upcoming one produced. The architecture of their proposed production system (Fig. 1) has been referenced underneath. After describing their proposed production system (i.e., BPS), they had developed the conceptual planning model for distributed task allocation. As we know, planning agents can be categorized depending on their analyzing abilities, and they are: “reactive agents”, where sensing organs or inputs are responsible for decision-making capabilities; and “planning agents”, having enough intelligence to build decisions. They contemplate several parameters such as sensing inputs, aim or objective, purpose, and efficacy. In their work, the authors described an abstract model (conceptual framework) of an agent, which was comprised of several functionalities like, perception of external events, necessary decision taking ability, performing suitable actions to the environment. Therefore, every agent was independent during recognizing, functioning, and optimal decision taking. In general, planning agents determine a plan that is a sequence of activities to be carried out to achieve the target. It is required either to achieve an aim that convinces some conditions or to assess several assignments sequentially. However, the essential stride in their conceptual model was the decision-building capability. In this context, they proposed the desired environment as a transition system that was comprised of an instantaneous description of multiple states, the starting state or states, and the transition function that mapped one state to

Fig. 1 A well-defined architecture of the authors’ proposed production system (BPS, RARM) [7]

38

I. Bhattacharya et al.

another based on some predefined conditions. According to their model, the intelligent agent stared from an initial state, and tried to seek out the final state or desired state. In order to interpret the intramural behavior/mechanism regarding the conceptual model, they had also employed an algorithm named, “Generic-Behavior”. The architecture (Fig. 2) of the planning model (proposed by authors) has been referenced underneath to make our understanding clearer. In the above diagram, the dashed rectangle depicts the external environment from where the agent can percept an event. In their proposed algorithm (generic behavior of the conceptual model), they used a queue (Q) to store every arriving event. For simplicity, we have explained their proposed algorithm in layman terms. The events that are coming from the external environment should be stored in a queue (first come first serve basis). Let, Si, j be a two-dimensional array that can be used to save information about the current state. If a perceived event belongs to the immediate state Si, j then, for every next state Si,k , the agent need to search the plan list that satisfies Si,k , and also assess every possible plan’s prerequisite and preserve only satisfied prerequisites. After that, the agent must approximate its needy resources and arrange the remaining plans according to their importance. Like this way, the algorithm works. The prime objective of their paper was to solve the distributed task allocation problem. To fulfill this commitment, they had developed and tested the working of the planning model. Firstly, they talked about several sensors by which a planning agent could sense its domain in advance. The sensor S1 (sequentially, S2) was used to confirm whether there was a workpiece at location L1 (sequentially, L2) on the input conveyor C1 (see Fig. 1, and Fig. 2 respectively). In the following area, we have discussed the other sensing phenomena sequentially as the authors mentioned.

Fig. 2 Authors’ proposed conceptual planning model

3 Social Behavior and Reasoning Through Multi-Agent Systems

39

i The sensor S3 (sequentially, S4) had been employed to investigate the presence of a workpiece at the location L3 (sequentially, L4) on C2. ii The duty of the sensors S5 and S6 was to verify whether there was a workpiece at the location L5 and L6 respectively on C3. iii The sensor S7 inspected whether the type A workpiece at the processing unit (i.e., M) or not. iv Whereas, S8 authorized the verification of the type B workpiece at M. v If C1 (one of the input conveyors) was in its extreme right or left, S10 or S9 could perceive it respectively. vi Similarly, S11 or S12 determined whether C2 was in its utmost left or right respectively. vii On the other hand, sensors S13 and S14 detected whether the output conveyor (i.e., C3) was in its utmost left or right location. viii Finally, S15 (sequentially, S16) was employed to determine whether the robotic arm was in its lower or higher position. After that, they had defined the impact of the actuators on the environment. Here, the planning agent controlled their domain by using several actuators:i ii iii iv v vi

The actuator A1 guaranteed the motion of C1. Whereas, A2 shifted the input conveyor C2. A3 enabled the motion of the output conveyor (i.e., C3). The actuator A4 revolved a robotic agent. The actuator A5 uplifted the robotic agent arm vertically. Also, the actuator A6 picked up and dropped a workpiece using the robotic agent arm (i.e., “r” in Fig. 1). vii The actuator A7 managed the workpiece. viii In the end, actuator A8 assembled two pieces. After defining the functioning of numerous actuators, they had talked about the decision-making process where the planning agent determined the functions or actions to perform to achieve the goal. In this decision process, the authors mentioned a set of actions, and they were: {Conveyor1-left, Rotate1-left, Rotate1-right, Conveyor2-left, Rotate2-left, Rotate2-right, Conveyor3-left, Rotate3-left, Rotate3right, take1, take2, take3, load1, load2, load3, put1, put2, put3, process1, process2}. We have explained each action in the following area. i Conveyor1-left (sequentially, Conveyor2-left and Conveyor3-left) implies a workpiece of type A (sequentially, B and AB) is shifting to the left of C1 (sequentially, C2, C3) from location L1 (sequentially, L3 and L5) to the location L2 (sequentially, L4 and L6). ii Rotate1-left (sequentially, Rotate2-left and Rotate3-left) implies that the robotic arm (i.e., “r”) holding a type A workpiece (sequentially, B and AB) is shifting to the left from the L2 terminal (sequentially, L4 and L6) of C1 (sequentially, C2 and C3) to M (processing unit).

40

I. Bhattacharya et al.

iii Rotate1-right (sequentially, Rotate2-right and Rotate3-right) describes the reverse operation of Rotate1-left. Here, the arm shifts type A workpiece from the processing unit M to L2 terminal of C1 (similarly, for C2 and C3). iv take1, take2, and take3 imply the functioning of holding the respective workpieces (A, B, and AB, respectively). v load1, load2, and load3 indicate the loading mechanism. vi put1 (sequentially, put2 and put3) signifies the functioning to place a workpiece of type A (sequentially, type B and type AB). vii Finally, process1 and process2 imply the refinement or rectification of two workpieces (A & B), respectively. The processing operation can only be carried out by the processing element M. As we have discussed earlier that the prime intention of the authors was to seek out solutions for distributed task allocation. In general, the distributed task allocation is essential only when the amount of task is very large and cannot be executed by a single planning agent. In a multi-agent framework, agents can collaborate with each other to obtain a solution of a complex task. First, we have explained the actual definition of distributed task allocation then, discussed the authors’ principle regarding it. In general, the task allotment through a multi-agent framework consists of a group of planning agents, (a1 , . . . , ak ), where each agent ai owns a set of assets or resources, ri ={r1 , . . . , rk }. Also, there exists several tasks to be executed in the multi-planning agent (t1 , . . . , tm ) and each task tm may require a group of resources. In their proposed work, they had classified the planning agent, namely initiator agent and participant agent. The initiator agent requested assistance to complete its job. Whereas, the participant agent procuring the essential resources that obtained the announced task and forwarded an acknowledgment. To clinch the systematic manner of the participant agent, they proposed two feasible states, i.e., “busy” and “idle”. If an agent acted as an initiator agent, its state must be considered as busy. According to authors, if the planning agent had no task to perform then, its state should be considered as idle. In the following area we have discussed the distributed task allocation mechanism defined by the authors. i Fundamentally, the initiator agent (one of the planning agents) commenced the distributed task allocation process. It was almost impossible for an isolated planning agent to accomplish a complex job without having the required resources. This drawback droved their thought process toward distributed-multi-agent environment. In the distributed model, every initiator agent owned a list of participant agents. It dispatched resource-announce-message to the participant agents, holding the agent specification, the task specification, and the number of resources (i.e., ). ii On the other hand, the participant agent received that message delivered by the initiator agent. It examined the probability of applying the task allotment depending upon its immediate state. If the participant agent possessed the essential resources and was in an idle state, we could observe cooperation between the initiator and participant agent, or else it refused the request.

3 Social Behavior and Reasoning Through Multi-Agent Systems

41

If the participant agent was not in a busy state then, it provided the information regarding its specification, varieties of resources it retained, and the execution time. This information was nothing but a propose-message of the form: . Whereas, if it was in a busy state then, it provided a refuse message containing the agent specification (i.e., Agent-ID). iii After gathering the feedback from all the participant agents, the initiator agent then made a comparison between the available and required resources to complete its execution or task. After achieving all the required resources, it generated a notification to the selected participant agent. The authors had defined two cases that could be happened depending upon the state of the approved participant agent. iv If the elected participant agent was unconfined, it sent a confirmation bulletin to the respective initiator agent, its state transformed from idle to busy. They formed the confirmation message as: Confirm-Message=. Hence, the distribution task allocation protocol had been finished. v Else if the adopted participant agent was busy in processing another initiator agent’s request, it declined the request of the immediate initiator agent. The decline message comprised of the specification regarding the immediate initiator agent. Therefore, the it must repeat the identical procedure once again. In this way, they had developed a social multi-agent cooperation system for the distributed task allocation problem. In fact, their experimental results proved that their suggested methodology could give better accomplishment. The prime aims of their solutions were to guarantee robustness, reliability, scalability, openness, and structural flexibility. We have concluded our detailed review with the help of a sequence diagram (Fig. 3) that had been proposed by the authors to make our understanding clearer. In the above diagram, “nbr” denotes the number of participant agents (one of the planning agents) connected with the initiator agent. Also each diamond-shaped region implies the decision-building mechanism of the planning agents. • The potential to infer the internal states of others is a symbol of human social reasoning. It is far valuable to both interpersonal interactions and crew dynamics. This consists of reasoning about relationships between others, which includes cooperative and competitive relationships. In this context, Rabkina et al. [14] presented a multi-agent system framework that validated virtual agents to express the same social reasoning abilities. Specifically, they had shown that analogical reasoning was enough for simulating relationships between virtual agents in stag hunt where, stag hunt is a game which describes a conflict between safety and social cooperation. Also, they had shown that their analogical model was able to predict agents’ future functions based on their prior behavior. Initially, they had given the task description regarding their proposed work, from where we can learn the basic concept of the stag-hunt problem. They examined the intent recognition and action prediction capabilities of analogical reasoning using the spatial stag-hunt domain from Shum et al.’s proposed research [17]. In a

42

I. Bhattacharya et al.

Fig. 3 A sequence diagram for the distributed task allocation approach

classic stag-hunt framework, a map with hares and stags is produced. In this case, hares are low-value targets that can be represented by a solitary hunter without the cooperation of others. Whereas, stags are high-value targets that should be captured by a group of cooperating hunters. To capture a target, the hunters should hold the same space as the target at the same time. In their proposed work, they had employed Shum et al.’s version of the game where, three hunters, two hares, and two were set down on a 5 × 7 grid world. Some squares in it were not traversable. A stag was contemplated to be caught when two or more hunters were in the same square at the same time. Likewise, a hare was considered caught when there exists exactly one hunter in the same square. They simulated three time steps of nine different scenarios. At every time step, each hunter walked 0 or 1 squares left, right, up, or down. On the other hand, the hare was not able to move; however, stags may move to keep away from seizing. They had also depicted several scenarios (shown in Fig. 4) regarding the stag hunt taken from Sham et al.’s proposed research. Inside the following area, we have explained each scenario meaningfully. i In Fig. 4, we can observe the respective four scenarios (i.e., b, e, f, and h) depict a situation where only a hare is caught without any cooperation of the agents.

3 Social Behavior and Reasoning Through Multi-Agent Systems

43

ii In the case of a, c, and d, a pair of agents (respectively AC, BC, and AB) coordinates to capture a stag. iii In the rest of the scenarios, all three agents (i.e., hunters) cooperate themselves to capture a stag. In their research, they used analogical reasoning to deduce both agents’ intent and future behavior. As the Structure Mapping Engine (SME) was able to make all inferences simultaneously, so a single analogical comparison was sufficient for them to finish both tasks. Based on Fig. 4, they had performed several computations. At each time step, they computed: i whether each individual (i.e., hunter or target) was in motion, consistent with the Moving or Stationary (MOS) library. ii whether each moving agent (i.e., the hunters) had moved nearer to or farther from another agent, according to Qualitative Distance Calculus (QDC). iii whether a pair of agents had overall moved nearer to or farther from each other according to the Qualitative Trajectory Calculus (QTS [4]). iv whether two agents were approximately close, far, or placed on the same square at each time (i.e., before and after of step-1, after step-2, and step-3; based on QDC). They had also computed the causal relationships between relations defined above. As we have mentioned earlier that the prime goal of the authors was to develop predictions about agents’ coordination and future movements at each step. In this context, they had created a total of four structured cases for each scenario (i.e., out of them three for testing and one for training purposes respectively). According to their first case, the hunting agents moved toward each other. As a result, they reduced the distance between each other than in the previous time step. Whereas, in the second case, the hunting agent moved away from an immobile stag. The case three foretold a cooperation event between two hunter agents. It returned zero or one e true positive conclusions and up to two true negative conclusions. Finally, the case four foretold the coordination among all three agents and returned zero, one, or three true positive conclusions. • The social network complexity [6] appears from the mutual interactions between the individuals as well as from the inner complication of human behavior, which is already the end result of many anatomical and psychological processes. Still, social network architecture can serve to learn human as well as social behavior. In terms of mathematics, the entire social network can be represented by a graph (may be directed or undirected depending upon the type of relationship) or network, in which the individuals (humans) are vertices and the relationships are links or ties. The semantic behaviors of social networks can be defined as the suppositional behaviors of users in real life depending on the user behaviors of social networks, normally contents published by users of the social networks [10]. In this context, Ruta et al. [15] proposed their semantic-based social network interaction framework through multi-agent system. We have slightly discussed their works in the earlier section. Here, we have explained their findings elaborately. In their

44

I. Bhattacharya et al.

proposed framework, the social agents were framed in two possible categories: full ones and basic ones. These pair of agents engaged in two types of social relationships (i.e., following/friendship). According to the authors, the agents could share both information and services through a bidirectional friendship link. They could also read and write on each other’s wall, ask for the friend’s service descriptions, and turn on and off the friend’s services. The following/friendship norm was confirmed automatically by means of a match-making process involving the device profiles. Two agents were good individuals for friendship if one of the following conditions was satisfied. i Strong co-location: In this case, devices were located in the same room/area. ii Co-ownership: Here, the device agents were from the same manufacturer. iii Co-working: They were capable to pass comments referred to the same viewpoint and supplied functionalities related to the same activity (e.g., room lighting) or observed parameter (e.g., indoor temperature). Also, their proposed framework assisted both push and pull models. The working mechanisms of each model have been referenced underneath with the help of a diagram portrayed by the authors. i Push Model: If an agent ai desired to receive updated information from its immediate agent a j , it used to ask to become a follower. The follower ai started a distribution discovery conference after receiving an acknowledgment of a new post. ii Pull Model: If ai desired to access a j ’s wall, it must take permission to become a friend. After getting permission, it granted a j to access its own wall. Thus, a semantic matchmaking had been developed (Fig. 5).

3 Fuzzy Logic-Based Multi-agent System The conventional definition of reasoning is the action of thinking regarding something in a logical and realistic way. The social reasoning interprets the ability to draw conclusions about others’ motives, dispositions, and actions, in order to evaluate one’s own behavior. Nowadays, we are suffering from pandemic. Though we have vaccines, but not in sufficient amount. Moreover, in India, the vaccine distribution system cannot be implemented properly due to lack of planning and population. In this scenario, we propose a region-specific vaccination process through multi-agent system. In this system, the agents of different regions of a state analyze the condition/behavior of their respective regions and take decisions regarding the importance of vaccination. In this case, the condition/behavior includes the number of daily Covid-19 cases, severity of symptoms [5], and death rate of that particular region. Based on these, agents take decisions, i.e., whether the vaccination campaign should be started immediately or not. In this chapter, we have implemented Mamdani Fuzzy [13] Inference System (MFIS) by which the agents can take decisions. In Fig. 6, we

3 Social Behavior and Reasoning Through Multi-Agent Systems

45

Fig. 4 Representation of stag-hunt scenarios. Here each circle represents hunters’ (agents) current location in the grid world; also, the stags and hares are denoted with green and black, respectively [17]

have depicted the Fuzzy Vaccination Multi-Agent System (FVMAS) framework to clarify our vision. In the above diagram (Fig. 6), R1 , R2 , R3 denote specific regions from where the agents can sense several conditions regarding the pandemic as mentioned above. Our main intention is to create an agent-based vaccine distribution prediction system. To do so, we have developed one of the fuzzy inference systems, i.e., the Mamdani type.

46

I. Bhattacharya et al.

Fig. 5 A sequence diagram for semantic matchmaking [15]

3.1 Fuzzy Vaccination Drive In general, the fuzzy logic inference system [8] (Mamdani type) consists of a fuzzification inference unit, decision-making unit, knowledge base (integration of database and rules), and a defuzzification unit. There exists no need of cooperation between agents, as the individual agents are capable to sense the condition regarding specific regions and can take appropriate decisions according to it. In this section, we have demonstrated the core part of the FVMAS framework, i.e., the decision-making process. i In this case, the crisp inputs for the system are daily cases {VL, L, A, H, VH}, symptoms’ severity {m, M, S}, and the death rate {L, M, H}. Here, the abbreviations defined inside the curly braces of each crisp input are called descriptors.

3 Social Behavior and Reasoning Through Multi-Agent Systems

47

Fig. 6 FVMAS framework

The descriptors are used to rationalize the parameters (input and output parameters). Similarly, the crisp output parameter will be the urgency of vaccination {isig, M, sig}. In this case, we need to understand the meaning of each descriptor. ii Assume, representing an ordered pair implies the correspondence between each descriptor (Ti ) and its meaning (T j ). For example, , , , , , , , , , and . iii All of these parameters must be measured in percentage (“%”).

3.1.1

Construction of Membership Functions for Input and Output Parameters

In order to develop this model, we have used triangular membership function [2]. As we have mentioned already that our first input parameter consists of five descriptors so, we need to evaluate membership values for each descriptor. The figure (Fig. 7) referenced underneath helps us to derive the membership values for each descriptor. The membership values of each descriptor has been given below:

48

I. Bhattacharya et al.

Fig. 7 Membership function for daily cases

µVL (x) =



5−x ; 5



0 ≤ x ≤ 5. 5 ≤ x ≤ 10.

(2)

x−5 ; 5 35−x ; 25

5 ≤ x ≤ 10. 10 ≤ x ≤ 35.

(3)

x−10 ; 25 75−x ; 40

10 ≤ x ≤ 35. 35 ≤ x ≤ 75.

(4)

x−35 ; 40

35 ≤ x ≤ 75.

(5)

 µA (x) = 

µVH (x) =



(1)

x ; 5 10−x ; 5

µL (x) =

µH (x) =

0 ≤ x ≤ 5.

Figure 8 depicts the membership values for each descriptors (i.e., three descriptors). µm (y) =



0 ≤ y ≤ 35.

(6)

y ; 35 80−y ; 45

0 ≤ y ≤ 35. 35 ≤ y ≤ 80.

(7)

y−35 ; 45

35 ≤ y ≤ 80.

(8)



µM (y) = µS (y) =

35−y ; 35



In the case of death rate, also there exists a group of three descriptors whose membership values have been computed underneath (Fig. 9).

3 Social Behavior and Reasoning Through Multi-Agent Systems

49

Fig. 8 Membership function for symptoms’ severity

Fig. 9 Membership function for death rate

µL (z) =



µM (z) = µH (z) =

20−z ; 20

0 ≤ z ≤ 20.

z ; 20 45−z ; 25

0 ≤ z ≤ 20. 20 ≤ z ≤ 45.

(10)

z−20 ; 25

20 ≤ z ≤ 45.

(11)

 

(9)

Similarly, we also can derive the membership graph for the descriptors of the output parameter. The output parameter also has three descriptors to rationalize its importance. We have shown a critical region in the membership graph of the crisp output

50

I. Bhattacharya et al.

Fig. 10 Membership function for urgency of vaccination

parameter (i.e., urgency of vaccination). Depending on the rule base, the agents will examine whether the value (i.e., vaccination urgency in %) lies within the critical region or not. Its membership values have been given below: µisig (r ) =



3.1.2

0 ≤ r ≤ 25.

(12)

r ; 25 50−r ; 25

0 ≤ r ≤ 25. 25 ≤ r ≤ 50.

(13)

r −25 ; 25

25 ≤ r ≤ 50.

(14)



µM (r ) = µsig (r ) =

25−z ; 25



Creation of Rule Base

In this section, we have defined a rule base for the FVMAS framework. In computer science, a rule-based system is used to save and manipulate knowledge to interpret useful information in a meaningful way. Agents can decide their actions based on this previously defined rule base. A typical rule base consists of four fundamental components. They are as follows: i A set of rules which is specific type of the knowledge base. ii A semantic reasoner, which perceives information to take actions based on the coordination of inputs and rule base. In this case, agents will follow the same procedure. iii Temporary working memory. iv User interface.

3 Social Behavior and Reasoning Through Multi-Agent Systems Table 1 Rule base for the FVMAS framework Daily cases Symptoms’ severity VL VL VL VL L L L L L A A A A A H H H H H H H VH VH VH

m M S S m M S S S m M M S S m m m M M M S m M S

51

Death rate

Urgency of vaccination

L L L M L L L M H L L M M H L L M L M H H L L/M M/L/H

isig M M sig isig M sig sig sig isig M sig sig sig M M M sig sig sig sig M sig sig

According to this system, initially, agents must learn the rules (defined in the rule base) and store them into their respective knowledge bases. In the future, if the same scenario will occur again then, they can seek their knowledge bases.

3.1.3

Evaluation of Rule Base and Results

Assume, at a given instance of time, daily cases for a specific region R1 is 27%, symptoms’ severity and death rate are 22%, 17% respectively. At this point, the corresponding agent of that region will examine the respective membership functions. In this case, they are: = 0.32 (by Eq. (3)), µH (x) = x−10 = 0.68 (by Eq. (4)), µm (y) = µA (x) = 35−x 25 25 35−y y = 0.37 (by Eq. (6)), µM (y) = 35 = 0.63 (by Eq. (7)), µL (z) = 20−z = 0.15 (by 35 20

52

I. Bhattacharya et al.

Eq. (9)), and µM (z) = 20z = 0.85 (by Eq. (10)). Based on these membership values, a set of eight rules has been generated where, all the antecedents are connected by AND operator. i ii iii iv v vi vii viii

When cases are average and symptoms are mild and death rate is low. When cases are average and symptoms are mild and death rate is moderate. When cases are average and symptoms are moderate and death rate is low. When cases are average and symptoms are moderate and death rate is moderate. When cases are high and symptoms are mild and death rate is low. When cases are high and symptoms are mild and death rate is moderate. When cases are high and symptoms are moderate and death rate is low. When cases are high and symptoms are moderate and death rate is moderate.

After that, there is a need to compute the strength of each rule (i.e., from rule one to rule eight). As the antecedents are connected by AND, so min operator should be used to compute the strengths. Let, “Si ” denotes the strength of each rule where, i goes from rule one to rule eight. Therefore, S1 = min(0.32, 0.37, 0.15) = 0.15, S2 = min(0.32, 0.37, 0.85) = 0.32, S3 = min(0.32, 0.63, 0.15) = 0.15, S4 = min(0.32, 0.63, 0.85) = 0.32, S5 = min(0.68, 0.37, 0.15) = 0.15, S6 = min(0.68, 0.37, 0.85) = 0.37, S7 = min(0.68, 0.63, 0.15) = 0.15, and S8 = min(0.68, 0.63, 0.85) = 0.63. Now, there is a need to defuzzify the above fuzzy values to achieve a crisp output that has been given by the corresponding agent of region R1 . In this case, max operator should be used to compute the maximum strength among all the strengths. Hence, max(S1 , S2 , S3 , S4 , S5 , S6 , S7 , S8 ) = S8 = 0.63. It implies, when cases of any region (e.g., R1 ) are high, symptoms’ severity are moderate, and death rate is moderate; urgency of vaccination for that region will be significant (see Table 1). Now, r − 25 ; 25 ≤ r ≤ 50 25 r − 25 0.63 = 25 r = (0.63 × 25) + 25

µsig (r ) =

r = 40.75%; by Eq. (14). The value of r lies in the critical region (shown in Fig. 10). Therefore, the agent of R1 is capable to decide the necessary action. It must be stated by the agent that the vaccination campaign for region R1 should be started as soon as possible. Similarly, for the region R2 and R3 , the corresponding agents can compute the significance of vaccination.

3 Social Behavior and Reasoning Through Multi-Agent Systems

53

4 Conclusions and Future Scope In this chapter, we have mentioned several research papers based on social behavior and reasoning through multi-agent system. The prime goal of the multi-agent system to solve those problems that are very complex or time taking for an individual agent or monotonic system to figure out. In this context, we have developed a fuzzy logicbased multi-agent system to ease the planning process of vaccine distribution. We have also shown our experimental result that predicts accurately and gives satisfactory outcome. Nowadays, the entire social behavior has been transformed drastically due to this pandemic. That is why we have decided to extend this research to develop a generalized Covid-19 management system through a multi-agent system, which may include area-specific lock down prediction, food delivery management, etc. To develop the FVMAS framework, we mainly focused on the decision-making part of the agents. Since Mamdani systems includes more intuitive and uncomplicated to evaluate rule bases, they are compatible to expert system applications. Due to this reason, we have this model to rationalize our FVMAS architecture. There are various applications based on the Mamdani model, for example, weather forecasts system, advanced mechanics, choice examination, time arrangement forecast, etc. The World Health Organization (WHO) has proposed two stages for the distribution of vaccines. According to WHO, all the nations would receive vaccines proportional to their citizens in the first stage. If the frontline workers in health care, as well as social care, will be vaccinated with the first dose, the rate of success may reach approximately 3% of their population. Then, the excessive vaccine would be given until 20% of a country’s population is protected. In the next stage, the remaining vaccines should be delivered depending upon several parameters like; the rate of spread of the virus, the vulnerable health system of a country, etc. In this scenario, we hope our proposed methodology that is the agent-based vaccine distribution prediction system can help our society in this pandemic situation.

References 1. Al-Saqqar F, Al-Shatnawi AM (2020) Reasoning about group social commitments in multiagent systems. J Ambient Intell Human Comput. https://doi.org/10.1007/s12652-020-024987 2. Barua A, Mudunuri LS, Kosheleva O (2014) Why trapezoidal and triangular membership functions work so well: towards a theoretical explanation. J Uncert Syst 8:164–168 3. Conti M, Passarella A, Pezzoni F (2012) A model to represent human social relationships in social network graphs. In: Aberer K, Flache A, Jager W, Liu L, Tang J, Guéret C (eds) Social informatics. Lecture notes in computer science. Springer, Berlin. https://doi.org/10.1007/9783-642-35386-4_14 4. Delafontaine M, Cohn AG, Van de Weghe N (2011) Implementing a qualitative calculus to analyse moving point objects. Exp Syst Appl 38:5187–5196. https://doi.org/10.1016/j.eswa. 2010.10.042

54

I. Bhattacharya et al.

5. Dong X, Cao YY, Lu XX, Zhang JJ, Du H, Yan YQ, Akdis CA, Gao YD (2020) Eleven faces of coronavirus disease 2019. Allergy Eur J Allergy Clin Immunol 75. https://doi.org/10.1111/ all.14289 6. Dziura SL, Thompson JC (2014) Social-network complexity in humans is associated with the neural response to social information. Psychol Sci 25:2095–2101. https://doi.org/10.1177/ 0956797614549209 7. Gharbi A (2020) A social multi-agent cooperation system based on planning and distributed task allocation. Information 11:271. https://doi.org/10.3390/info11050271 8. Jamshidi A, Yazdani-Chamzini A, Yakhchali SH, Khaleghi S (2013) Developing a new fuzzy inference system for pipeline risk assessment. J Loss Prevent Process Ind 26:197–208. https:// doi.org/10.1016/j.jlp.2012.10.010 9. Ji C, Jiang D (2014) Threshold behaviour of a stochastic SIR model. Appl Math Model 38:5067– 5079. https://doi.org/10.1016/j.apm.2014.03.037 10. Li L (2017) Behavior analysis in social networks. In: Alhajj R, Rokne J (eds), Encyclopedia of social network analysis and mining. Springer, New York, NY. https://doi.org/10.1007/9781-4614-7163-9_110198-1 11. Li Y, Li Y, Ma Z (2015) Computation tree logic model checking based on possibility measures. Fuzzy Sets Syst 262:44–59. https://doi.org/10.1016/j.fss.2014.03.009 12. Pan X, Han CS, Dauber K, Law KH (2007) A multi-agent based framework for the simulation of human and social behaviors during emergency evacuations. AI Soc 22:113–132. https://doi. org/10.1007/s00146-007-0126-1 13. Pourjavad E, Mayorga RV (2019) A comparative study and measuring performance of manufacturing systems with Mamdani fuzzy inference system. J Intell Manuf 30:1085–1097. https:// doi.org/10.1007/s10845-017-1307-5 14. Rabkina I, Forbus KD (2019) Analogical reasoning for intent recognition and action prediction in multi-agent systems. In: Proceedings of the 7th annual conference on advances in cognitive systems. Cambridge, MA 15. Ruta M, Scioscia F, Loseto G, Gramegna F, Ieva S, Pinto A, Sciascio ED (2018) Semanticbased social intelligence through multi-agent systems. In: Cossentino M, Sabatucci L, Seidita V (eds) CEUR workshop proceedings, CEUR-WS, vol 2215, pp 96–102 16. Sheehan MM, Pfoh E, Speaker SL, Rothberg M (2020) Changes in social behavior over time during the covid-19 pandemic. Cureus 12. https://doi.org/10.7759/cureus.10754 17. Shum M, Kleiman-Weiner M, Littman ML, Tenenbaum JB (2019) Theory of minds: understanding behavior in groups through inverse planning. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 6163–6170. https://doi.org/10.1609/aaai.v33i01.33016163 18. Talgeri A, Kumar A, Adithya B (2014) Domotics—a cost effective smart home automation system using wifi as network infrastructure. Int J Eng Res Appl 4:52–55. ISSN: 2248-9622 19. Thierry-Mieg Y (2015) Symbolic model-checking using its-tools. In: Baier C, Tinelli C (eds), Tools and algorithms for the construction and analysis of systems. Lecture notes in computer science, vol 9035. Springer, Berlin, pp 231–237. https://doi.org/10.1007/978-3-662-466810_20 20. Vyklyuk Y, Manylich M, Škoda M, Radovanovi´c MM, Petrovi´c MD (2021) Modeling and analysis of different scenarios for the spread of covid-19 by using the modified multi-agent systems—evidence from the selected countries. Res Phys 20:103662. https://doi.org/10.1016/ j.rinp.2020.103662 21. Zangmeister Christopher D, Radney James G, Vicenzi Edward P, Weaver Jamie L (2020) Filtration efficiencies of nanoscale aerosol by cloth mask materials used to slow the spread of SARS CoV-2. ACS Nano 14:9188–9200. https://doi.org/10.1021/acsnano.0c05025

Chapter 4

Prevalence of Multi-Agent System Consensus in Cloud Computing Santanu Koley and Pinaki Pratim Acharjya

Abstract Cloud computing that follows service-oriented architecture is useful for intelligent agent or multi-agent system (MAS) communication. Their use in representation and construction, parallel, and published applications is identified here and shows similarities, contrasts, and potential combinations between cloud computing and multi-agent structures. Long execution complex structure with clever applications works with MAS to showcase cloud computing. The assembling of interfaces within MAS that requires reliable scattering systems and cloud computing systems that require programs with clever, enthusiastic, versatile, and independent behavior can be current systems and applications. The engineering of a system consisting of MAS that primarily focuses on the materials of cost transactions between cloud users and providers is planned to mitigate the disadvantages of both cloud clients and cloud providers and exploit the full potential of cloud computing. As it turns out, as innovation develops and solves increasingly complex applications, the need for an integrated framework of multiple operators communicating in peer-to-peer mode is becoming clear. Central to the design and operation of such MAS is the focus of a problem and research question that has long been tested by all communities. Arrange it like a cloud environment. Keywords Agent · Cloud computing · Consensus · Multi-agent system (MAS)

1 Introduction Distributed artificial intelligence (DAI) has established incredible considerations from the academic world due to its ability to deal with combined computing difficulties. These are classified into three types: parallel AI, distributed problem solving (DPS), and MASs [1]. MAS reflects cloud services control by service providers as a form of outsourcing, and any organization can implement cloud services by their own [2]. S. Koley (B) · P. P. Acharjya Dept. of Computer Science & Engineering, Haldia Institute of Technology, Haldia, West Bengal, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 S. Gupta et al. (eds.), Multi Agent Systems, Springer Tracts in Human-Centered Computing, https://doi.org/10.1007/978-981-19-0493-6_4

55

56

S. Koley and P. P. Acharjya

Cloud computing endows with easy scalability of data storage, no geographical restrictions, elastic services, affordable, highly reliable than efficiency, and easy to manage and adds a huge number of users day by day [3]. With the success of the Internet in previous years, the fashion for computing has changed. Computer resources are redistributed to remote locations as a replacement for PCs/laptops. Cloud computing has expanded the field of distributed computing systems, ending with high-speed Internet systems combined with a wide range of promotional computing services equipped with Web, grid computing, and peer-to-peer networks. Cloud properties are available as a service on the Internet. In fact, the cloud computing organism provides a significant framework for high-performance computing that dynamically familiarizes itself with the user and its application requirements. The pay-as-you-go-model is the main attraction of this type of computing; it is now considered the fifth utility followed by electricity, water, telephone, and gas. At present, cloud is primarily intended to handle highly portable computing workloads and to provide exceptional data storage amenities. In collaboration, these short targets have been united with the third objective of dropping the likelihood of administration and the cost of exercise. Together, MASs identify an additional distributed computing standard based on multiple interacting agents capable of intellectual behavior. MASs often practice scattering adversities through dispersion methods where many connect the path through collaborative methods. Intelligence is one of the most intriguing features of software agents. It seeks mutual assistance with a number of agents that can be run on parallel or distributed computers to realize higher effective performance that can reach the bottom of the combined major woes to maintain low implementation time. MASs are the result of research and study of biological behavior of nature, and it is the sophistication and improvement of the behavior prototypes of biological groups. A number of single agents come together to form a loose multiple structure, keeping in mind the ability to solve problems that are unable to work on their own due to problems of ability, knowledge or resources, or even low efficiency. The ability to perform additional compound and treacherous tasks, extraordinary functionality, excessive fault tolerant and strong, and MASs are cheaper and easier to build than single agents. MASs can better capture the properties and efficiency of manifold pain through asynchronous parallel activity among agents. Its loosely combined structure ensures the reusability and scalability of its process. MAS, where the agent is the basic component, is identified as the agent-centric multi-agent system (ACMAS). By specifying the organization, we get the mark identified as Organization Centered Multi-Agent System (OCMAS). Agencies consider what agents should do, not how they should do it, making it possible to structure the system without indicating any information in terms of possible acquisitions [4].

4 Prevalence of Multi-agent System Consensus in Cloud Computing

57

2 Agent An agent is a computational entity that acts in support of an alternative entity (or more than one entity) to accomplish a mission or recognize a specified goal. Agent systems are autonomous software programs incorporating realm acquaintance and devising competence to act with a precise amount of objectivity to finalize activities desirable to realize quantified objectives. They are set out to conduct in dynamically transforming surroundings. There are distant features of agents outlined as follows:

2.1 Autonomy The ability to act independently somewhat in the interest of clients or different projects likewise by adjusting the manner by which they accomplish their targets.

2.2 Pro-activity The ability to seek after their own separate fixed objectives, comprising by settling on choices as consequence of inner choices.

2.3 Re-activity The ability to respond to outside occasions and boosts and subsequently adjust their conduct and settle on choices to complete their errands.

2.4 Communication and Cooperation The ability to associate and interconnect with different agents (in numerous agent frameworks), to altercation information, get guidelines and bounce reactions, and collaborate to satisfy their own objectives.

58

S. Koley and P. P. Acharjya

2.5 Negotiation The capacity to complete composed discussions to accomplish a level of collaboration with different agents.

2.6 Learning The capacity to improve execution and dynamic after some time while associating with the outer condition. Albeit a solitary agent can perform and rush to play out guaranteed task, the agent architecture type was considered as a distributed computing model where a lot of operators collaborate each other by transferring information and collaborating to perform complex errands where cooperation, insight, adjustment, and dynamicity are key issues to be taken care of. This implies regardless of whether we can characterize a specialist in disengagement, the agent’s criterion can locate its total misuse on the off chance that we acknowledge agents as entities acting in an assortment of agents, along these lines actualizing the purported multi-agent system framework [5]. Truth be told, it is somewhat hard to envision that an agent will exist and work just as an independent substance and will never cooperate with different specialists (genuine or counterfeit) in its condition. Likewise, information agents, or individual agents, which are primarily expected to function as remaining solitary entities in taking care of issues, will positively progress their conduct and accomplished outcomes if it helps out different agents to exchange knowledge, to designate errand execution, or to trade information that improves the operator job and commitment. As indicated by these contemplations, the social element of agents is one of its fundamental highlights as shown in the figure below.

Precepts Sensors

Agent

Conditionaction rules (if-then rule)

Surroundings

What the world is like now What action I should do now Actions Actuators

Fig. 1 Reflex agent with diverse states

4 Prevalence of Multi-agent System Consensus in Cloud Computing

59

A robotic agent, for example, might use cameras and infrared range finders as sensors and various motors as actuators. There are software agents that can have keystrokes, file contents as sensory input, perform on those inputs, and display output on the screen. There are human agents too with eyes, ears, and other organs which work for sensors and hand, legs, vocal tract work for actuators, but human agent is not relevant to this work. There are some terms associated with agents such as sensors, effectors, and actuators. Sensor: A sensor is a device that identifies modification in the surroundings and transmits information to other electronic devices. An agent monitors its surroundings through sensors. Actuators: Actuators are parts of machines to switch over energy into motion. They are solely accountable for moving and controlling a system. Example of actuator can be an electric motor, gears, rails, etc. Effectors: Effectors are the tools that influence the surroundings. They may be legs, wheels, arms, fingers, wings, fins, display screens, etc.

3 Intelligent Agent (IA) An intelligent agent is a program with the intention to draw conclusions or perform a service based on its situation, user input, and experience. These programs may collect information in an unconventional way, in a standardized, automated plan or in real time as per the customer’s persuasion. These agents are sometimes called bots or robots. They take steps for the purpose of taking action through sensors and actuators based on the environment. An Intelligent Agent can acquire knowledge from the environment to achieve their goals. A thermostat can be an example of an Intelligent Agent. An Intelligent Agent can be a computer software system that performs autonomously to reach a promised goal and responds to the person or method that happens around it. It is programmed through a pitch of artificial intelligence (AI) known as machine learning (ML) and is enabled by sensors that differentiate it from detection and disposal in situations. IAs are exploited in the part that calls for individuals to work together with the purpose of being able to represent important social forces like Siri and Alexa. They can detect an application and proceed on their own to search for information. There are several types of intelligent agents.

60

S. Koley and P. P. Acharjya

3.1 Simple Reflex Agent This is usually the basic diversity. It takes action based on the existing condition of the machine as it moves a bit away from its environment as it tests its knowledge base for possible response to the case based on programming principles.

3.2 Model-Based Reflex Agent These types draw on the history and internal memory available in her N Suite to draw conclusions about the pre-built model of her environment. Its internal memory sets it apart for storing various steering histories to help it appreciate its charge, even if it cannot directly observe what action it wants to take.

3.3 Goal-Based Agent Each Intelligent Agent has a goal aimed at combating the desired with the situation. The IAs in this department use pre-programmed activities based on their potential products to consolidate their objectives. It can perform loneliness or lots of action depending on its solitude.

3.4 Utility-Based Agent This Intelligent Agent not only performs its objectives but also chooses the best way to accomplish its objectives, which sets it apart from other variations.

3.5 Learning Agent This agent can be taught on or after its practice by experiences. It is pre-built with basic knowledge but can take place and perform separately to further improve its functionality. Finally, a programmer does not want to provide all the information as he wishes; it works and is further improved individually. Characteristics of Intelligent Agent are as follows: (a) (b) (c)

Create opportunities to support the law innovation analytics set over a period of time. Get it done online and obviously in real time. Be able to analyze their performance, error, and success.

4 Prevalence of Multi-agent System Consensus in Cloud Computing Fig. 2 Intelligent Agent explanation

61

Intelligence Formalization Which concept method do we wish to form? Intelligence Possession How to carry out that concept method? Intelligence Evolution How to program the intelligence? Intelligence Consolidation How to connect intelligence with people or systems? Intelligence Implementation & Modification How do we steadily make it smarter?

(d) (e) (f) (g) (h)

Learn and develop in all ways through communication through their environment. Gain knowledge rapidly particular huge quantity of data. Acquiring knowledge from huge amount of data very quickly. Memory-based ideal model storage and retrieval capabilities. Utilize constraints to feature short- and long-term memory, age, and other properties along with other possessions.

An intelligent system can be divided into few parts as intelligence formalization, intelligence possession, intelligence evolution, intelligence consolidation, intelligence implementation, and modification where formalization deals with the concept method we wish to form, possession carrying out that concept method, evolution concerns how to program the intelligence, consolidation considers to connect intelligence with people or systems, and implementation and modification control how to steadily make it smarter and more sophisticated. This process is described in Fig. 2 layout.

4 Related Work The most perplexing checking arrangements are work in distributed computing stacks. Stacks give gatherings of apparatuses like resource management (counting virtualization), user management, APIs, and graphical user interfaces. The case of such stages is [6, 7] Open Stack, Open Nebula, Apache Cloud Stack, and Eucalyptus. Every one of them has a committed arrangement of observing apparatuses.

62

S. Koley and P. P. Acharjya

The fundamental component of complex checking is observing the security of the framework. Each effective arrangement needs to cover huge number of assortments of safety viewpoints: from observing of the information provenience to the checking of the privileges of end clients to recover the outcomes [8]. Furthermore, security is an important aspect of the cloud, and it must adhere to worldwide rules and regulations [9, 10]. Self-organization is a feature of cloud-based monitoring services, which could be used for execution monitoring, security monitoring, and client monitoring [11]. Fundamental apparatuses for cloud checking are observing interfaces, for example [12], checking administrations, for example [13], layouts, for example [14]. Likewise, structures were proposed for lessening the intricacy of overseeing mists frameworks. They give provisioning, deployment, observing, and variation of cloud frameworks without being language. Subordinate [15] cases of such are [16] OpenShif, Chef, and Puppet. The observing hubs were proposed, for example [17] libraries, Apache jclouds [16]. Ferry et al. [18] proposed the CloudML, a space explicit displaying language and a run-time climate that supports provisioning, deployment, observing, and transformation of multi-cloud frameworks. The framework is effective for utilization for demonstrating geographies, prerequisites, constraints, and conditions and a cloud merchant application which determines a set of extra prerequisites, imperatives, and conditions like financial constraints. Lopez, Tejera et al.[19] introduced a Savvy Cloud Learning Framework which is a blended technique that joins the cloud framework, learning the board framework and the specialists’ innovation. Multi-specialists framework strategy offers an ideal route for versatile frameworks where the design is changed constantly and required connection, joint effort, and exchange. At this point, instructive associations have started utilizing the cloud climate, consolidating it to their own framework settings, and using its enormous potential. Contingent upon the highlights of multi-specialists’ framework and distributed computing technique to help e-learning meeting, Mohammed Bousmah and others in [19] presented a plan of another line in distributed computing named Keen Cloud Learning Framework that ought to be rushed to help benefits wherever at whenever by utilizing any piece of gear, for all major parts in the learning meeting. Pireva, Kefalas, Dranidis, Hatziapostolou, Cowling et al. [20] discussed that the underlying model of a framework contains a multi-agent framework (MAS) which predominantly focuses on the value exchange strategy among cloud suppliers and cloud clients and is being developed to decrease the intricacy between the two of them. The instrument of finding the help is relying upon the best cost coordinating offered by the cloud supplier. Núñez, Andrés, Merayo et al. [21] have noticed the worry about a little notification that has been given to present a technique for designers and clients to ask for, find, build, and utilize distributed computing assets. Dynamic and versatile specialists are suitable instruments for discussing client induction, computerizing the administrations and assets disclosure, and exchanging and controlling the cloud assets. In his examination, he had presented a circumstance when cloud specialists work on the cloud working frameworks to offer savvy information use observing,

4 Prevalence of Multi-agent System Consensus in Cloud Computing

63

benefits and controlling administrations, and energy usage for the foundations of distributed computing, yet it has not been reproduced or tried. Dinesh Kumar and Ashwin [22] proposed that the cloud computing can be underlying with multi-agent frameworks to make a shrewd conduct in a cloud environment. The after effect of such a framework can get superior and make mists more autonomic and adaptable. Chen, Han, Jiang et al. [23] have presented another arrangement model dependent on multi-specialist framework which is utilized to adjust the progressions that can be occurred in cloud environment during the value exchange. Middle agent could cut down the exchange time and improve the rate achievement of arrangement. The model focuses on the opposition time history, which is influenced by numerous impacting factors all through exchange, and delivers reasonable proposition as per present market by blending every one of the elements. Monitoring administration for OpenStack is the Telemetry administration [24]. It collects estimations inside OpenStack framework and the warnings produced by other OpenStack administrations. Thusly, the estimations are refreshed if such a warning showed up. It gives just the metering. It does not give any investigation of the gathered information, and it does not impact the OpenStack climate. It is accessible simply by the order line customer. The booking administrations are the different parts in OpenStack. The libraries supporting security dangers of the framework are being worked on. They are isolated from the observing framework. The after effects of the estimations are not automatically examined neither by the security administration nor the booking administration. Scheduler offers the channels as the fundamental instrument; it does not give artificial intelligence-based arrangements. Moreover, observing administrations from OpenStack do not give similarity with different cloud APIs. Amazon CloudWatch is a checking administration intended for both metering also, naturally responding to changes [25]. This device can do metering just at five-minute recurrence or one-minute recurrence. CloudWatch occasions apparatus permits reacting by sending messages to the environment, initiating capacities or making changes. This is conceivable by characterizing the arrangement of rules. Occasions and rules of responding must be set up in the JavaScript Article Documentation design. Burden adjusting administration is isolated from checking administration also, and it is profoundly robotized. Application Burden Balancer is utilizing the Round Robin calculation. Old style Burden Balancer parts the solicitations uniformly between the accessible assets. Amazon CloudWatch is carefully committed to the AWS cloud, and some parts are confined topographically. The specialist-based, powerful help framework to screen, control, and improve disseminated frameworks was proposed by Legrand et al. in [26]. MonALISA framework comprises specialists that can team up and coordinate. It is committed for lattices, organizations, and applications running continuously. The framework is worked from dynamic administrations. It offers wide assortment of observing substances like, administrations, applications, and organizations. It might interface effectively with the elements or inactively record data. Moves can be made by the order or naturally. The framework is planned as the completely circulated pool of

64

S. Koley and P. P. Acharjya

specialists furnished with non-concurrent correspondence. The security framework is additionally carried out, and dispersed interruption identification framework is accessible. MonALISA can screen customers, occupations, and assets. Additionally, it offers the load balancing. The load adjusting depends on Space Name Framework convention. Specialists are not smart ones. Insightful specialists were utilized as the essential components for clever environment supporting robotizing of programming measures [27]. They successfully utilized a device for investigation of various conceivable equal procedures, load balancing, and execution advancement. They give design subordinate adjustment of the thought about environment.

5 Applications and Objective Measures of Multi-agent Systems As the quantity of sent multi-agent applications increments, further and better involvement in the innovation is acquired, empowering a solid assessment of the field from a more reasonable viewpoint. Specifically, questions identifying with how the hypothesis of multi-agent frameworks impacts on training, and how the down to earth improvement itself contrasts and different advances, can be replied in light of an uplifted degree of development [28]. Given the strains among theoreticians and professionals in registering all in all, not to mention their altercations in AI or multi-agent frameworks specifically, the conversation on specialist frameworks and applications was both fiery and eager. The conspicuous reaction to the topic of attributes of reasonable applications is that they merit thinking about when the application requires both appropriation and knowledge. Subsequently, a multi-agent approach would be reasonable for issues that are intrinsically, truly, or geologically conveyed where free cycles can be plainly recognized. Such issues incorporate, for instance, dispersed sensor organizations, choice emotionally supportive networks, airport regulation, or other arranged or disseminated control frameworks. A conveyed approach is not in itself enough, nonetheless, and there ought to likewise be necessities for insight or misleadingly in the sub-measures that include express thinking about conduct, for instance. In the event that the issue can be tackled by methods for a look-into table at every hub of an organization, a multi-agent framework would be inordinate. A scope of additional application zones that fit the bill for multi-agent arrangements can likewise be specified. These incorporate those requiring the interconnection and between activity of different self-sufficient, self-intrigued existing inheritance frameworks, master frameworks, and choice frameworks, or those requiring arrangements that draw from conveyed self-governing and narrow-minded data sources, like the Individual Travel Help demo from BT; those where the arrangements draw from various disseminated specialists, for example, medical services provisioning, in which some focal specialist cannot in any way, shape, or form play

4 Prevalence of Multi-agent System Consensus in Cloud Computing

65

out the undertaking without assistance from different specialists; issues that normally cross authoritative limits for which a comprehension of the collaborations among social orders and associations is required; and issues where no single specialist has an all-out see, however, a few specialists have neighborhood sees. The idea of responsibility for and techniques in the application is significant here and specifically when it is disseminated over various hierarchical elements so that no single element can or does approach all the data. Last specific models remember merchants for a commercial center and various elements in a business chipping away at others errands. In these circumstances, the issues to be handled do not have one in general objective, but instead comprise adjusting the (perhaps clashing) objectives of various substances. Similarly, as with some other piece of innovation, there are a lot of utilization that do not need a multi-specialist approach. Multi-specialist frameworks are not needed only to create seclusion; however, they diminish intricacy, additional speed; however, this might be an impact of their innate parallelism, dependability, adaptability, or re-ease of use. Similarly, they are not needed just on the grounds that an issue is excessively enormous for an incorporated single specialist because of asset impediments, nor in light of the sheer danger of a brought together framework, nor simply for reasons of effectiveness, heterogeneous thinking, and so forth. Several applications of MASs with its background are shown in Fig. 3. For instance, a finance framework may profit in a computer programming sense from an article situated methodology which gives modularization and reuse, yet such standard information handling issues do not actually require the correspondences overhead or usefulness of a multi-agent approach. Such applications generally require neither dispersion nor knowledge. Similarly, a little detached master

TV Applications

Storage Database H/W Embedded Agents

Mobile Users

Networks

Multi Agent System

Mobile Applications

Fig. 3 Applications of multi-agent systems with background

Main Server

Web Server

66

S. Koley and P. P. Acharjya

framework utilized in a solitary area requires the knowledge however not the appropriation, and the human interface is deficiently perplexing to merit considering it a specialist. At last, it is significant that a multi-agent system approach might be valuable, however excessive, while handling issues that are least demanding envisioned in a manner that seems to have the above-mentioned attributes, for example, battle reenactments. Expert’s frameworks are significantly more dangerous as far as improvement technique. All the more significantly, multi-agent applications require a participation information level, while master frameworks and others ordinarily work at the image and information levels. Notwithstanding, if utilizing a collaboration information level purchases nothing for the application, it ought to be stayed away from. For instance, Huberman’s warm economy multi-agent framework has been reprimanded, and in that it gives a second-rate answer for a solitary ‘specialist’ control arrangement. Ostensibly, this is an unmistakable disseminated critical thinking issue, yet assuming it did not include only one proprietor, a second-rate multi-specialist arrangement may be legitimate. The advantages of an explicitly multi-specialist procedure would be a decrease of the semantic hole between investigation from one viewpoint, and plan and execution on the other, prompting a decrease in an opportunity to plan and carry out, with the standard compromise between better extensibility and misfortunes in execution productivity and plan explicitness. The Web insurgency is bringing about expanded correspondence between unmistakable substances with various objectives and particular limits which should be secure. The new class of utilization, which administration this needs, will definitely utilize some type of multi-agent approach. Current techniques accentuate top-down plan, however, multi-specialist frameworks receive an alternate methodology—top-down inside the specialist—and base up in the specialist local area. The Web upheaval is bringing about expanded correspondence between particular elements with various objectives and unmistakable limits which should be secure. The new class of utilizations, which administration this needs, will unavoidably utilize some type of multi-agent approach. Current philosophies accentuate topdown plan, however, multi-agent frameworks embrace an alternate methodology— top-down inside the specialist—and base up in the specialist local area. In attempting to discover methods of surveying the commitment of multi-agent frameworks, one view contends that this is just truly with experience. Carefully talking, we need to know how positive or negative the multi-specialist approach is in correlation with the best elective arrangements. Almost consistently, the appropriate response is obscure in light of the fact that assets do not permit numerous ways to deal with and to be attempted. Besides, in reality there are normally quite a few unique methods of tackling a specific issue, and it is difficult to look at them in any goal design. By and by, this is the place where the ‘suck it and see’ applications referenced above come in. As a matter of fact of delivering little frameworks in an extensive variety of associations to tackle an extensive variety of issues, it gets conceivable to

4 Prevalence of Multi-agent System Consensus in Cloud Computing

67

make speculations regarding the advantages and disadvantages and to perceive what the proof is for what was professed to be acceptable and awful. Once more, the correlation with master frameworks or neural nets is enlightening. In the previous case, the advantages were required to be nearby proficiency due to less specialists required, yet ended up being nearby viability with better utilization of existing specialists and more broad conveyance of mastery. On account of neural nets, a few examinations have been made with identical numerical and measurable procedures, and the advantages are less in execution and more in the time and ability needed for improvement and augmentation. Without the involvement with both of these cases, we would not have been at all definite what to begin estimating. The elective view is that however thorough assessment is the scholarly response to this, and the market will ultimately be the most target evaluator. Organizations are getting tied up with disseminated article advances in light of the fact that there are clear necessities and advantages for them, and the equivalent is valid for multi-agent systems, which will not be taken up except if they show their need and potential. Temporarily, we need to characterize rules for judging multi-agent arrangements. Do they offer a worth added answer for other regular applications like control, master frameworks, conveyed critical thinking approaches, and board draws near? This basically requires the meaning of significant worth added which could go from magnificent where no arrangement was plausible without a multi-specialist approach, through negligible where a multi-specialist arrangement is insignificantly of worth to poor (where ordinary methodologies offer better arrangements). The elective view is that anyway intensive appraisal is the academic reaction to this, and the market will at last be the most objective evaluator. Associations are getting tied up with dispersed article progresses considering the way that there are clear necessities and benefits for them, and the comparable is substantial for multiagent systems, which will not be taken up aside from on the off chance that they show their need and potential. Briefly, we need to describe rules for judging multiagent courses of action. Do they offer a value-added answer for other customary applications like control, ace systems, passed on basic reasoning methodologies, and board moves close? This essentially requires the importance of critical worth added which could go from heavenly where no plan was conceivable without a multiexpert methodology, through immaterial where a multi-expert course of action is inconsequential of worth to poor where customary philosophies offer better courses of action.

6 Cloud Computing and Multi-Agent Systems As mentioned above, the cloud computing worldview has recently firmly taken the lead. Its enhancements have fueled progress in countless public and private settings. Its wide recognition in the business world as well as its simple and rapid coordination with traditional innovative design has promoted rapid development until now. In addition, the promotional model that pays only when the cost increases (such as

68

S. Koley and P. P. Acharjya

traditional practical items) is also an important factor in its rapid development. The definition given by NIST stands out among other definitions because it characterizes the worldview, in addition to its attributes, management, and ordering models. By and by, disregarding the quality and broadness of this definition, in excess of a couple of others, has noticed that the definition is not adequately aspiring. There is countless mechanical advancements that have arisen inside the extent of this worldview that are by the by not a piece of existing stages. The larger part just zeroes in their endeavors on giving equipment foundation administrations using the fundamental virtual innovation, without considering the abilities of the greater levels like stage and programming. Given these shortcomings and weaknesses, just as the impediments of cloud computing stages, and in a cloud computing or dance with the theories proposed in this investigation, a MASs structure dependent on VO has been chosen to manage these hindrances. Albeit one may at first think about these two disseminated frameworks (MASs and cloud computing) to be contrary, a point-by-point investigation exhibits that they are truth be told correlative, however divide extensive collaboration among them [29]. Above all else, cloud computing conditions be able to envelop the computational requirements for perseverance of data and the processing probable so as multi-agent systems need to be intended for various applications, for example, information mining, the board of complex administrations, and so on. Also MASs can be utilized to make a considerably more proficient, and versatile plan for the cloud computing climate than what is as of now a cloud computing. At last, the utilization of MASs in the structure of the plan for cloud computing frameworks gives this worldview new qualities like learning or knowledge, which makes it conceivable to grow significantly more progressed computational conditions altogether perspectives (canny administrations, interoperability among stages, proficient appropriation of assets, and so forth). The quantity of studies that can be found on the best in class relating cloud computing with specialist innovation is very low [30]. Nonetheless, this propensity be altering, getting progressively basic to discover research applications zeroed in countryside. Although a predetermined amount of research has been conducted on it, expert-based distributed computing or expert-based cloud scenarios are becoming a typical idea, which has recently been cited by different creators. Figure 4 depicts a practical block diagram of cloud service innovation.

Cloud Consumer Structure Distributed Database

Trust Assessor System

Broker System Cloud Source System (SaaS, PaaS, IaaS)

Fig. 4 Pragmatic block diagram for cloud service innovation

4 Prevalence of Multi-agent System Consensus in Cloud Computing

69

Primary subsystems are needed to find suitable cloud services such as cloud client structures, cloud source systems, trust assessment systems, broker systems, and distributed databases. In order to realize a reliable cloud service discovery system, different identified systems need to work together effectively.

7 Consensus The initial impetus of distributed computing and MASs is to obtain overall system reliability when there is a numerically flawed course of action. This habitually calls for synchronized methods to achieve compliance or at various stages of computation that comply with various data standards. The real-life example will describe the definition in a better way: Santa: Hey Pinu! Let’s have a dinner together. Pinu: What about Mutton? Santa: Right mutton would be best. Santa Claus and Pinu want to have dinner together. Pinu suggested bringing “mutton”. Santa Claus easily accepted the Lamb’s suggestion. They decide to cooperate to determine any value, one of which is their follower, and act according to that value. This can be called consensus. In short, consensus is the universal recognition of value.

There are several types of intelligent agents.

7.1 Consensus in Distributed System Both two sides had to have an opinion on what to eat in the given situation. Problems can arise if there are more people with different opinions and more votes on what to eat for dinner. The distribution of consensus is where more parties want to agree on something. To come to a conclusion, it is difficult. As the number of peoples involved in the contract increases, so does the complexity of implementing the concession (Fig. 5). As a simple definition, it can be said that a distributed system is a group of nodes (usually computers) that perform common tasks while communicating with each other through a network. With the idea of a distributed system, we can say that a node in a distributed system which agrees on a certain value is a distributed consensus. Here in the above picture, everybody in the meeting has agreed with their leader. This is the simplest form of distributed consensus. A distribution system is a collection of computer nodes that act as separate entities. In reality, these arrangements are never ideal; these tend to be hardware errors, packet drops, slow networks, clock skews, etc. Regardless of these failures, all systems of a distribution system should

70

S. Koley and P. P. Acharjya

Fig. 5 Consensus mechanism (simplest form)

agree on certain data values in order to function as their single entity. This adjustment of value by the distribution components is called consensus. This may seem like a simple task, but it is one of the most important and complex problems in the distribution system. Imagine a distributed database where data is distributed across all distributed regions, such as data nodes. Any such distribution system needs to focus on the next situation. Ordering of update or reliable multicast: In order to maintain the data consistency of such distributed databases, it is necessary to entertain in the same order. Exclusive access to a resource or mutual exclusion: If different data nodes try to meet a value at the equivalent time, the system must be able to give exclusive access to a source in one of the nodes in order for the data to be consistent, such as locking. Leader selection: If a leader has to be selected who receives a write request to simplify the process of updating the distributed database, the leader must be determined and approved by all components of the system. Detection of suspected failures or failure detection: If the leader containing the write request goes down and the other nodes of the system do not respond to this loss of data, the system will go into a dull state. It is therefore important for all components of the system to be aware of any errors, so that a slight revival or remedial action can be taken. The problem of consensus for view of the distributed system is directly linked as described here. In order to get to the bottom of this endeavor, we have to solve them one after another.

4 Prevalence of Multi-agent System Consensus in Cloud Computing

71

7.2 Consensus Protocol Properties There are a number of properties that must be adhered to by any consensus protocol as portrayed underneath: Agreement: Each component should be of equivalent value. Validity: If all the elements put forward the equivalent value ‘s’, then all the activities are required to settle on ‘s’. Termination: Each defective element comes to a conclusion with some value. If the protocol must not end, the process agrees to a similar item, which leads to no conclusion. Otherwise, the procedure will last forever, and the procedure will freeze one after another.

7.3 Consensus Subjected to Communication Restraint To achieve consensus and synchronization control of MAS, an important factor is the agent’s ability to exchange information over the network. Network interference not only breaks conventional ‘point-to-point’ signal control restrictions, but it also keeps control nodes from establishing connected dedicated lines, reduces the system cabling, and has many other benefits, for example low cost, trouble-free extensions, extended configuration, error detection, and system maintenance [31, 32]. Communication occurs in distributed networks, where cloud services play an important role. The objective of the consensus mechanism is described in Fig. 6. In contrast, network isolation from traditional control systems will produce various troublesome poles. The network contains a large number of data sources. The network’s communication channels are joined together once respective node sends data to the network’s end. However, network bandwidth is not complete, and network

Consensus Mechanism Objective

Unified Agreement

Prevent Double Spending

Fig. 6 Objectives of consensus mechanism

Align Economic Incentive

Fair & Equitable

Fault Tolerant

72

S. Koley and P. P. Acharjya

data traffic occasionally changes. As different nodes exchange data throughout the network, data conflicts, multipath communication, connection brokers, and network overflow often occur [33]. Therefore, packet loss will cause continuous delay, which will affect the performance of MAS and continue to cause its instability [34]. The necessity and appropriateness of the media consensus are identified on the basis of the tree conversion scheme [35]. For discrete time MAS by agent velocity in non-convex set, a distributed constraint protocol is designed to study the bounded delay consensus problem, using model transformation and bounded analysis methods. Use model conversion and boundary analysis methods to study the problem of finite delay detection, and implement the controversial MAS finite protocol for isolation time with agent speed in a non-convex array. The predictive power of organisms in nature, a small world prediction protocol, [36] was designed for A/R and Visek models [37]. For dynamic linear networks without leadership, a distributed predictive control protocol is expected, which proves that the predictive protocol can increase the speed of expression and reduce the occurrence of different types of events. The regression of MAS consensus is measured by saturated input and uses distributed predictive control methods and fixed control to achieve consensus and improve performance [38]. In the absence of a central leader, the consensus algorithm helps the distributed parties to come to a decision for reducing resource costs. These algorithms are typically used in distributed systems that are typically resource intensive, with nodes incorporating less bandwidth, power, or storage must call for integration with blockchain [39] and sensor networks [40]. Reducing resource overheads, as well as communication, computation, convergence time, and/or storage are some significant issues for consensus algorithms.

7.4 Leader-Following Consensus Recently, the issue of following the leadership of MAS has also been widely considered [41]. As indicated by the various properties of the leader, its following agreement issue is classified as a genuine leader case and a virtual leader case. To handle the challenge of consensus in a heterogeneous multi-agent scenario with time-varying communication and entry lag, a leader following the consensus protocol is deployed [42]. It has investigated the challenge of distributed tracking control of first-order agents with different unique leaders and coordinated Markov switching topologies [43]. The leader of the second-order MAS follows the problem of consensus concentrates [44]. After the dual integrator MAS meeting issue was hampered by external impairments and vulnerabilities, Su provided a revolutionary distributed internal model technique to handle the leader’s additional inquiry [45] (Fig. 7). Furthermore, after the distributed MAS [46] reached a consensus, network-based leaders have studied the full and half state input control of the transmission without speed estimation. The leader-following disposition consensus issue of different

4 Prevalence of Multi-agent System Consensus in Cloud Computing

73

Leader

Follower 1

Follower 3

Follower 2

Follower 4

Fig. 7 Example of leader–follower structure

unbending body frameworks subject to a mutually associated exchanging correspondence organization is examined next [47]. Combining the early control strategy with the general control method, another distributed loop regulator is proposed for many suspicious Euler–Lagrangian frameworks. This regulator can be adjusted to the non-uniform time limit and any variable corresponding to the delay and corresponding network for coordination exchange [48]. The control scheme that relies on the distributed dynamic adaptive neural network is designed to ensure that the unified performance after the error occurs between the follower and the distributed robust adaptive neural network is always semi-internationalized and ultimately limited [49], far away from the old style. The problem is normally retrospective to the ‘complexity explosion’ in the plan. For various inflexible shuttle frameworks, the way of thinking is driven by the unit quaternion, and appropriate nonlinear bystanders are set to obtain the leader after the transaction [50]. The two non-soft leaders follow Lipschitz’s indistinguishable nonlinear MAS [51] development convention. By proposing a neighborhood evaluator with reference direction and channel limiting, another soft-loop distributed control convention based on backtracking is proposed. The leader in nonlinear MAS order follows the control protocol [52].

74

S. Koley and P. P. Acharjya

Compared with the previous post-trade leader research, Caputo’s partial MAS covers finite and unlimited time subordinate Lipschitz coefficients [53]. A mandatory control convention is envisaged for nonlinear MAS with input commands [54]. We study a class of nonlinear stochastic MAS with mixed-fraction motivation, the significant problem of the leader following the protocol [55]. After the release of the MAS agreement, the world leader has concentrated limited control rights [56]. Under the fixed directed graph, a class of leader performance consensus problems after nonlinear MAS is studied [57]. For high-order nonlinear random MAS, a single boost in the regulator is used to compensate the time-varying coefficient of nonlinear capacity [58]. As we all know, the distributed adaptive state input control law will lead to a class of nonlinear suspicious MAS under the coordinated switching network to reach agreement [59]. Such type of consensus is very much common in cloud computing as distributed system today normally depends on cloud services.

7.5 Group Consensus In numerous viable circumstances, a bunch of agents should have the option to detect and react to sudden circumstances or any progressions when a helpful undertaking is carried out. In addition, dissimilar agents may distribute diverse tasks in cooperative control. In this manner, it is a significant issue that proper conventions are intended to make agents arrive at various consensus esteems. This problem is called the group consensus problem, which is more reasonable for managing community-oriented control problems [60]. The MAS group consensus problem, a fascinating subject in MAS distributed control, has a wide range of applications in multi-robot controllers, satellite clusters, and vehicle development. Recently, brilliant achievements of group consensus exploration have continued to emerge. For the first continuous or discretetime MAS with nonlinear information restrictions, [61] explored group consensus and used normal frames to investigate the group average consensus and group consensus formation issues of the first applied MAS [62]. The sloppy collection protocol problem of the second request for MAS under the coordinated organizational geography with acyclic segments [63] and then established some combination models for such algorithms. When the digital attack is recoverable, explore the group consensus of the leaderless MAS [64]; the appropriate state of the rally protocol of the oppressed MAS is given the digital attack. The issue of group consensus between the leader and the next of the secondary MAS has also been studied [65]. According to the group consensus of high-order direct dynamic MAS, a clustering method is proposed [66]. The consensus problem of group is checked by model changes of the auxiliary MAS with fixed topology and random switching topology [67]. The consensus of the heterogeneous group of MAS is concentrated by using direct small rate recommendations, performance guide recommendations, and small rate assumptions [68]. The group consensus theme of heterogeneous MAS [69], which proposes new conventions, uses state change strategies and obtains a comparable framework. Some relatively suitable conditions are realized to realize the

4 Prevalence of Multi-agent System Consensus in Cloud Computing

75

consensus group of heterogeneous MAS with fixed and interchangeable topologies [70]. The group consensus problem of nonlinear MAS [71] shows that consensus can be achieved in discrete time and continuous time. Examine the problem of reverse group consensus of powerful agents in the cooperative containment network [72], which can be divided into two subnets. According to tracking, if the mirror image is clearly related, the inverse group consensus problem can be realized. A distributed cooperative control of MAS is proposed for generating units distributed in a multi-micro network [73]. The proposed control technology preestablishes a fixed consensus estimation of different microgrids considering universal collaboration and includes a group consensus based on the compliance of distributed generators. In addition, there have been some interesting and brilliant achievements recently in handling the MAS group consensus problem, but recently there has been a time lag. The second-order group consensus delay of time-shifted MAS is dependent on the use of the second-order neighbor data [74]. Group consensus is characterised by the second-order MAS opposite beam consistency problem with delayed nonlinear elements and discontinuous correspondence in cooperatively competitive groups [75]. Focus on the time-lagged MAS group consensus problem and study the MAS weighted group consensus problem. A state-based prediction method is proposed for the group consensus supervisory authority of MAS, and the group consensus standard has been introduced. The upper limit of the longest delay time of the weighted group consensus can be expanded by modifying the range of the present state in addition to the hysteresis state in the control algorithm, allowing the MAS with dichotomous geography to achieve weighted group consensus. The group consensus problem of nonlinear MAS with delayed Lurie type elements is discussed, and the gaze control chart is planned under the undirected corresponding graph. The elements group the consensus topics of heterogeneous MAS with time delays, and the expert elements are displayed by a simple integrator and a double integrator.

7.6 Consensus Supporting Trigger Method In the hypothetical MAS investigation, it is generally expected that there will be sufficient energy, computing superpower, and continuous correspondence. However, in practical applications, the computing power and communication capabilities of the lone expert depend on the advanced microchip implanted, and the power comes from the inserted battery. MAS resources include agent processing capabilities, communication capabilities, and power reserves. Unreasonable estimation and correspondence will make the agent busy, unable to respond to other tasks, and even unable to work as expected, affecting typical frame-wide activities. In addition, the agent’s power is limited, and unnecessary calculations and communications consume a lot of power. Studies have shown that long-distance communication consumes most of the sensor’s energy [76]. Energy fatigue will cause experts

76

S. Koley and P. P. Acharjya

to neglect their work, affect the display of MAS, and even lead to frame collapse. To take advantage of the decentralization and enthusiasm of MAS, it is particularly important to cut down on communications and calculations, no matter how long you may wait. Therefore, when planning control technology, it is important to fully consider the energy use and organizational assets of the framework itself, which makes the acceptable control configuration of MAS really tested. How to reduce the use of MAS assets? The most straightforward technique is to reduce the amount of data transactions between experts by planning the transmission of the system. It is worth noting that the use of advanced signal control technology can save more data transactions and recorded assets than constant monitoring of signals. In fact, the high-level logo start-up strategy generally includes start-up time and start-up occasion. The old basically bypassed conventional test control, that is, the estimate update and frame control, was accidental, and the control data remained unchanged within the time frame of zero request holders. The latter decides whether to send a message by making a decision on a certain occasion. The benefit of occasional inspections is that they are easier to carry out in investigations and planning. In fact, ‘after a certain period of time’ can also be considered as an occasion, it can be said that starting at that time is an example of starting on a special occasion. Through event planning and trigger response, event-triggered instruments can outperform time-triggered components to save assets. Time migration technology generally refers to test control strategy, that is, data estimation and control task execution run intermittently. The consensus problem under the test control structure is called the sampling consensus. Since agent time is unlimited, the movement of data between agents is unimaginable. Thus, how to choose the inspection time frame to ensure consensus is the main research content of the sampling consensus. So far, there are many articles about MAS review consensus. At first, the research on consensus sampling was basically aimed at the main MAS integrator application [77]. Two sampling consensus algorithms are expected to be used to combine the second request MAS with coordinated geography. A distributed consensus protocol is planned based on the current and previous regional test data, and the essential then appropriate circumstances are in place to confirm the subsequent request for MAS’s consent. When the current area data is not free, only the position and speed test data are used to plan the agreement and to obtain important and suitable conditions for checking the agreement. An epic consensus agreement was proposed to reach national consensus during any huge review period. In addition, when the inspection leg is aperiodic, it is necessary to have a lower limit for the inspection leg. For the second request MAS with coordinated topology and nonlinear dynamics, the algorithm is executed to determine the most reasonable check interval [78]. All seats synchronize information updates at all times, and a clock synchronization method is required. In any case, it is difficult to guarantee the test time from time to time due to correspondence innovation, external interference, and other reasons. In this sense, it is important to plan an unconventional asynchronous sampling consensus algorithm, that is, every expert can update the information when they do. According to their trial period,

4 Prevalence of Multi-agent System Consensus in Cloud Computing

77

little attention is paid to the update season of the neighboring center. The unconventional check consistency analyse the Vicsek model, wherein every agent assesses the approaches of its neighbors in discrete situations. The second-order MAS asynchronous sampling coherence problem with time shift topology is considered, and the inspection cycle is planned [79]. By showing the exchange of network topology through Markov interactively and considering the influence of communication delay, another inspection data coherence control protocol with variable inspection period is predicted. From a resource utilization perspective, inspection control is moderate here and there. For example, when the status of two consecutive inspections is really very similar, if the control information source and commitment are still updated intermittently, will this lead to a waste of frame assets? In the final analysis, if there are few changes in the state, the past data can be used to replace the current value. In order to overcome the limitations of sample control and reduce the wasteful abuse of assets, the basic idea of occasion trigger control is used [80]. The correlation between time activation and event activation shows that the event activation process has more significant benefits in reducing data transmission. Under the event-triggered estimator, no-leader consensus and leader-following consensus can be achieved. Reckless event-based supervisors and initiation of subordinate states are designed for leaders following the MAS consensus, with a straightforward general model. A general event-based controller and trigger function are planned for each agent, which proves that the proposed general event-based technology can reduce the correspondence between neighboring agents. For MAS synchronization, a timing-based leader tracking program is envisaged, and a model-based strategy is used to predict overall node conditions. Issues of event-triggered group consensus and follow-up group consensus are studied. An occasional activation control arrangement is proposed to ensure consistency of the MAS number for the second application. A hybrid firing system is proposed for MAS with variable time delay, switching topology, and arbitrary organization attacks. As for subsequent breakthroughs, the MAS consensus was triggered.

7.7 Time-Limited Consensus An important consensus display file is a convergence speed. Convergence rate limit of the consensus algorithm calculation is to obtain a higher speed through an ideal vertex configuration, and many specialists choose topology. Until now, most of the consensus algorithm is an asymptotical consensus algorithm. In other words, the value of the ideal index of the congestion rate is an infinite time, everything is equal to the same situation, and everything is reliable within the limited time. As is possible, a large number of practical control frameworks require a harder assembly time and a single response and gradually move to framework marks, or zero within a limited time, the following errors can be achieved [81]. For example, the brake control frame

78

S. Koley and P. P. Acharjya

requires that the vehicle speed arrives or reaches a predetermined position within a limited time. Because limited time consensus algorithms benefit from a rapid binding benefit, solid enemies and unknown components were obstructed, and the finite time contract has a more fundamental design infrastructure. As is possible, the problem of finite time control is difficult for virtual inspection. This is not a sense of time invariance. Because there is no strong survey device, the planning and inspection of finite agreement algorithms are significantly problematic than symptoms agreed upon. In this way, proposed survey plans and technologies that can be performed on a subject of limited time agreement are an important design topic. So far, the main strategies for considering time-limited consensus can be divided into two types. One is the assumption of homogeneity. The homogeneous proposition consists of three stages: The first step is to show that the framework is globally asymptotically stable by developing useful Lyapunov and adding Barbalat’s lemma or Lasalle’s invariant rule under a given agreement; further progress is to use the homogeneous hypothesis which demonstrates that the framework is locally stable for a limited time; the third step is to speculate that the framework can contribute to global stability in a limited time by consolidating the first two stages. The other is the useful Lyapunov development, through which you can test the finite time consensus and also get the upper time limit. Time-limited consensus protocols can be divided into two categories: discontinuous protocols and continuous protocols [82]. Intermittent agreements mainly include swap agreements and extreme sliding mode agreements. For MAS with a dynamic leader, a propagation testimony is proposed that is based on a time-limited consensus tracking problem. Taking into account the strong leader of the dark speed promotion and the supporters with limited influence of the weird, the non-specific terminal sliding algorithm relies on the nonlinear consensus protocol to drive the follower condition to temporarily merge with the leader’s comparison condition. For MAS with Euler– Lagrangian elements [83], the planning is based on the convention of non-smooth sliding surfaces. Once the direction of the structure slides on the sliding surface, the condition of the structure reaches the predetermined situation of coordinated stress control. Taking into account the strong leader of the dark speed promotion and the supporters with limited influence of the weird, the non-specific terminal sliding algorithm relies on the nonlinear consensus protocol to drive the follower condition to temporarily merge with the leader’s comparison condition. For MAS with Euler– Lagrangian elements [83], the planning is based on the convention of non-smooth sliding surfaces. Once the direction of the structure slides on the sliding surface, the condition of the structure reaches the predetermined situation of coordinated stress control. Due to the great speed and immersion of information, by using homogeneous hypothesis to analyze the robustness of the closed circular frame, a plan was developed for first-time and high-demand limited time audiences so that for conditions, considering everything, they can join state leaders in a limited way weather. To ensure that all compliance criteria may be merged into a curved set of leader statuses, use

4 Prevalence of Multi-agent System Consensus in Cloud Computing

79

an appropriate limited time agreement with an obligatory coordination technique, while the algorithm is equally important for the situation of multiple static leaders. For the first time worldwide, MAS has requested a type of constant time constant protocol from integrators. Its ultimate goal is to plan and evaluate the merger time offline. Some consensus algorithms are based on neighbors’ performance information or total conditions, and there is also an algorithm that only requires relative state estimation, using binary consensus protocols and conspiracy fixed control. A nonlinear distributed protocol is proposed to realize the finite time consensus of heterogeneous frames [84]. The virtual rate is familiar with the protocol of the second request MAS and can follow the real rate in a limited time. The time consensus of heterogeneous secondary applications of MAS is concentrated at a huge and quantifiable speed. Consider the time-limited consensus problem of a second request for MAS with a different leader in a directed graph. Another non-singular finite time terminal sliding mode control technique is proposed for the second request MAS with deterioration. For the fixation error function vector, a robust finite time distributed consensus is given. A finite time control plan is proposed to perform the delivery and smoothness of the pioneering monitoring protocol in the geographic conditions, including the coordinated crosstree. By using a recursive strategy, the non-Lipschitz constant control law ensures finite time consistency of the demanding doubtful nonlinear MAS. The suspicious MAS for the second application has been issued with global consistency for a limited time and entered a decline concentration [85]. A distributed controller is proposed, which relies on the viewer in sliding mode to recognize global time-limited protocols after restricted control input. The nonlinear consistency convention of MAS with repeated Gaussian sounds characterizes the idea of finite time consistency probability [86]. The leader–follower finite time consistency problem of a type of variable nonlinear MAS is the most possible solution.

7.8 Multi-Consensus and Multi-Tracking Due to the association between the agent and its surrounding conditions, MAS creates complex grouping behavior. Experts have a keen interest on the clustering behavior of MAS. It has attained few important test outcomes and used them in the field of traffic light control systems, using adaptive components, insightful robots, and collective main frames. In a network composed of multiple agents, your center is an autonomous person with a certain amount of knowledge. When a group of agents comes together to execute a difficult assignment, due to various entrepreneurial tasks or changes in the environment in which agents are located, the development of MAS presents many states organized at certain stages. Multi-consensus means that MAS presents multiple protocol circumstances in which effective governance is in place agreements. Such conditions could determine in the MAS meeting, either or non-necessitate a MAS meeting. Multiple global positioning frameworks coordinate to track various wanted

80

S. Koley and P. P. Acharjya

circles (or virtual leaders) under appropriate distributed control conventions. MAS clustering practices, such as multiple consensus, multiple tracking, multiple accumulation, and other clustering practices, have important hypothetical importance for studying complexity assumptions. Recently, there have been many special screening results in multi-consensus and multi-follow-up. Compared to the group consensus problem, the multiple consensus problem has more substantial content and is closer to the actual design problem; therefore, the assembly agreement problem is a special example of the multiparty consensus problem. The MAS multiple consensus problem with nonlinear conventions is a convention proposed under the assumption that all subgroups meet the internal equilibrium conditions and focus on group consensus after issuing an uninterrupted time second request for MAS. A regulator convention that splits the entire frame into subsections through various leaders then studies second-request multi-agent packets in discrete time following the Markov exchange topology. Aiming at the internal model standard, the global performance change problem becomes MAS’s unique global reliability problem. The global problem of changing production of MAS enthusiastic collection is concentrated with nonlinear elements of uncertain secondary application. In wireless networks, tracking protocols work with source hubs to track the progress and ‘engagement’ of neighbors to recover their end-to-end postponement and complete organization routine. Through self-activated control [87], by displaying the multiple tracking address convention, remote self-organizing organizations with node failures follow the multiple subject tracking problem requesting the multi-agent network. The conditions of different agents in each second-order subnet are asymptotically bound to a similar desired direction. It familiarizes with the idea of insight by describing the degree of intermediate knowledge and, on the other hand, proposes a suitable interchange pacing convention using test position data, and then check speed data. The control protocol [88] is intended for complete multiple monitoring of limited factors, and the last error relates to the test time interval. The second request MAS multiple tracking problem is based on the location information from the test. Aiming at the fast terminal slider mode control strategy, a cyclic time-limited development was developed to study the MAS limited control problem tracking time scheduling. The problem of multiple consensus [89] becomes a problem of robustness of an inclusive error framework. Another nail-type consensus protocol with non-periodic discontinuous influence is designed for decision making by online leaders. In wireless networks, the multi-tracking protocol acts as a source hub to track the progress and ‘engagement’ of neighbors in order to progress their end-to-end delay and complete organization act. A minor self-coordinating remote organization that is disappointed with the hub, viewing multi-track targeting conventions, is useful. According to the conditions of the different agents in each second-order subnet, the problem of multiple tracking of the network of multiple master request agents through the automatic trigger control is gradually found in the similar required direction. The idea of insight is used to describe the intermediate degree of knowledge, on the other hand, it proposes the use of test position data and inspection speed information to

4 Prevalence of Multi-agent System Consensus in Cloud Computing

81

exchange cycle rhythm conventions. The control convention aims to achieve multiple tracking with limited factors, where the last tracking error corresponds to the test time range. Multiple tracking issue relies on a second MAS request to verify location data. Aiming for the time-limited development of fast terminal slider mode control technology in accordance with the convention, the limited time provision of MAS follows the control problem.

8 Conclusion The combination of clouds and agents involved in distributive consensus may be appropriate for the entire parties. In services where cloud data centers and cloud computing clouds play an important role, coordination and consensus mainly play a role in leadership selection, group membership, cluster management, service discovery, access/resource management, and continuous replication of master nodes. In this study, we considered how to do this, and what scientific fields and issues should be involved when conducting research that will create intelligent cloud services and high-performance multi-agent systems in the cloud. The integration benefit involved with multi-agent systems to require infrastructure and cloud computing systems to be distributed consistently with intelligent systems through dynamic, flexible, and autonomous behavior will have an impact on novel systems and relevance. All research communities should be aware of this possibility and do the cooperative research work necessary to reach that goal. In terms of response time and scalability of the MAS cloud service identification method, adding confidence to the process can improve service quality. A final memorandum regarding future line of work with MASs with cloud agrees to rely heavily on the technical part of the system and the surroundings. We hope that future versions of the cloud environment will include complex MAS concepts generated by the latest MAS method from MAS, such as environmental concepts or legal sets, to address the effectiveness of union roles. The use of such a method will result in the advancement of the platform and the necessary overall sovereignty of it from the necessary technical surroundings.

References 1. Nourah J, Iyad K, Aiiad A, Rashid M (2020) Distributed artificial intelligence-as-a-service (DAIaaS) for smarter IoE and 6G environments, sensors (MDPI), vol 20, p 5796. https://doi. org/10.3390/s20205796 2. Merizig A, Kazar O, López-Sánchez M (2019) A multi-agent system approach for service deployment in the cloud. Int J Commun Netw Distrib Syst 23:69. https://doi.org/10.1504/IJC NDS.2019.100642 3. Koley S, Ghosh S (2014) Cloud computing with CDroid OS based on Fujitsu server for mobile technology. Skit Res J 4(2): 1–6. ISSN 2278-2508

82

S. Koley and P. P. Acharjya

4. Abbas HA, Shaheen SI, Amin MH (2015) Organization of multi-agent systems: an overview. Int J Intell Inform Syst 4(3):46–57. https://doi.org/10.11648/j.ijiis.20150403.11 5. Pintea CM, Tripon AC, Avram A et al (2018) Multi-agents features on Android platforms. Complex Adapt Syst Model 6:10. https://doi.org/10.1186/s40294-018-0061-7 6. Grzonka D (2015) The analysis of openstack cloud computing platform: features and performance. J Telecommun Inform Technol 3:52–57 7. Kumar R, Gupta N, Charu S, Jain K, Jangir SK (2014) Open source solution for cloud computing platform using openstack. Int J Comput Sci Mob Comput 3(5):89–98 8. Jak´obik A (2016) Big data security. Springer International Publishing, Cham, pp 241–261. https://doi.org/10.1007/978-3-319-44881-7_12. 28 9. Cloud Controls Matrix ver. 3.0.1, Cloud security alliance. https://cloudsecurityalliance.org/ group/cloud-controls-matrix/. Last Accessed April, 2021 10. U. S. Department of Commerce (2013) NIST Cloud Computing Standards Roadmap, NIST Cloud Computing Standards-Roadmap Working Group, SP 500-291, ver. 2, Tech. rep 11. D. Petcu (2014), A taxonomy for sla-based monitoring of cloud security. In: Computer software and applications conference (COMPSAC), IEEE 38th annual, IEEE, pp 640–641 12. Yongdnog H, Jing W, Zhuofeng Z, Yanbo H (2013) A scalable and integrated cloud monitoring framework based on distributed storage. In: Web information system and application conference (WISA), 10th IEEE, pp 318–323 13. Trihinas D, Pallis G, Dikaiakos M (2018) Monitoring elastically adaptive multi-cloud services. In: IEEE transactions on cloud computing PP (99), vol 6, Issue 3, pp 800–814. https://doi.org/ 10.1109/TCC.2015.2511760 14. Nguyen TAB, Siebenhaar M, Hans R, Steinmetz R (2014) Role-based templates for cloud monitoring. In: IEEE/ACM 7th international conference on utility and cloud computing (UCC), pp 242–250. https://doi.org/10.1109/UCC.2014.33 15. de Carvalho MB, Esteves RP, da Cunha Rodrigues G, Granville LZ, Tarouco LMR (2013) A cloud monitoring framework for self configured monitoring slices based on multiple tools. In: Proceedings of the 9th international conference on network and service management, pp 180–184. https://doi.org/10.1109/CNSM.2013.6727833 16. Wettinger J, Andrikopoulos V, Leymann F, Strauch S (2015) Middleware-oriented deployment automation for cloud applications. IEEE transactions on cloud computing, pp (99). https://doi. org/10.1109/TCC.2016.2535325 17. Meng S, Iyengar AK, Rouvellou IM, Liu L, Lee K, Palanisamy B, Tang Y (2012) In: IEEE 5th international conference on reliable state monitoring in cloud data centres, cloud computing (CLOUD), pp 951–958. https://doi.org/10.1109/CLOUD.2012.10 18. Ferry N, Rossini A, Chauvel F, Morin B, Solberg A (2013) Towards model-driven provisioning, deployment, monitoring, and adaptation of multi-cloud systems. IEEE Sixth Int Conf Cloud Comput 2013:887–894. https://doi.org/10.1109/CLOUD.2013.133.29 19. Lopez-Rodriguez I, Hernandez-Tejera M (2011) Software agents as cloud computing services. In: Demazeau Y, Pˇechoucˇek M, Corchado JM, Pérez JB (eds) Advances on practical applications of agents and multiagent systems, advances in intelligent and soft computing, vol 88. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19875-5_35 20. Pireva K, Kefalas P, Dranidis D, Hatziapostolou T, Cowling A (2014) Cloud e-Learning: a new challenge for multi-agent systems. In: Jezic G, Kusek M, Lovrek I, Howlett R, Jain L (eds) Agent and multi-agent systems: technologies and applications, advances in intelligent systems and computing, vol 296. Springer, Cham. https://doi.org/10.1007/978-3-319-07650-8_28 21. Núñez A, Andrés C, Merayo MG (2012) MAScloud: a framework based on multi-agent systems for optimizing cost in cloud computing. In: Nguyen NT, Hoang K, J¸edrzejowicz P (eds) Computational collective intelligence, technologies and applications, ICCCI 2012. Lecture notes in computer science, vol 7653, Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-64234630-9_45 22. Kumar D, Ashwin R (2012) Multi-agent based cloud services, international conference on Egovernance & cloud computing services. Int J Comput Appl EGOV(1):7–10

4 Prevalence of Multi-agent System Consensus in Cloud Computing

83

23. Chen J, Han X, Jiang G (2014) A negotiation model based on multi-agent system under cloud computing. In: The ninth international multi-conference on computing in the global information technology, ICCGI 2014 24. https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/pdf/ser vice_telemetry_framework_1.1/Red_Hat_OpenStack_Platform-16.1-Service_Telemetry_Fra mework_1.1-en-US.pdf. Last Accessed April, 2021 25. Lee K, Murray D, Hughes D, Joosen W (2010) Extending sensor networks into the cloud using amazon web services. In: IEEE international conference on Networked embedded systems for enterprise applications (NESEA), pp 1–7 26. The Ultralight project: the network as an integrated and managed resource for data-intensive science. Comput Sci Eng 7(6):38–47, Nov-Dec 2005. https://doi.org/10.1109/MCSE.2005.127 27. Pllana S, Benkner S, Mehofer E, Natvig L, Xhafa F (2009) Towards an intelligent environment for programming multi-core computing systems. Springer, Berlin, pp 141–151. https://doi.org/ 10.1007/978-3-642-00955-6_19 28. Talia D (2011) Cloud computing and software agents: towards cloud intelligent services. In: Proceedings of the 12th workshop on objects and agents, Rende (CS), Italy 29. Bhargava R, Srivastva AK, Srivastava V (2015) A framework of multi agent system in cloud computing. Int J Sci Eng Technol Res (IJSETR) 4(6) 30. Bousmah M, Labouidya O, El Kamoun N (2015) Design of a cloud learning system based on multi-agents approach. Int J Adv Comput Sci Appl 6(3) 31. Hu J, Wang Z, Chen D, Alsaadi FE (2016) Estimation, filtering and fusion for networked systems with network induced phenomena: new progress and prospects. Inform Fusion 31:65– 75 32. Xia YQ, Gao YL, Yan LP, Fu MY (2015) Recent progress in networked control systems-a survey. Int J Autom Comput 12(4):343–367 33. Hu J, Wang Z, Liu S, Gao H (2016) A variance-constrained approach to recursive state estimation for time-varying complex networks with missing measurements. Automatica 64:155–162 34. Savino HJ, Souza FO, Pimenta LCA (2018) Consensus on intervals of communication delay. Int J Autom Comput 15(1):13–24 35. Sun YG, Wang L (2009) Consensus of multi-agent systems in directed networks with non uniform time-varying delay. IEEE Trans Autom Control 54(7):1607–1613 36. Killworth PD, Russell Bernard H (1979) “The reversal small-world experiment” Social networks, 1(1978/79), @Elsevier Sequoia S.A. Lausanne - Printed in the Netherlands 1:159–192 37. Vicsek T, Czirók A, Ben-Jacob E, Cohen I, Shochet O (1995) Novel type of phase transition in a system of self-driven particles. Phys Rev Lett 75(6):1226–1229. https://doi.org/10.1103/ PhysRevLett.75.1226,PMID10060237 38. Olfati-Saber R (2006) Flocking for multi-agent dynamic systems: algorithms and theory. IEEE Trans Autom Control 51(3):401–420 39. Zhao X, Lei Z, Zhang G, Zhang Y, Xing C (2020) Blockchain and distributed system. In: Wang G, Lin X, endler J, Song W, Xu Z, Liu G (eds) Web information systems and applications, WISA 2020. Lecture notes in computer science, vol 12432. Springer, Cham. https://doi.org/ 10.1007/978-3-030-60029-7_56 40. Kar S, Moura JMF (2009) Distributed consensus algorithms in sensor networks: quantized data and random link failures. arXiv:0712.1609v3, pp 1–54 41. Batres R, Braunschweig B (2002) Chapter 6.1—software agents, computer aided chemical engineering, vol 11. Elsevier, pp 455–483 42. De S, Sahoo SR, Wahi P (2018) Trajectory tracking control with heterogeneous input delay in multi-agent system. J Intell Robot Syst 92:521–544. https://doi.org/10.1007/s10846-0170715-2 43. Xie D, Shi L, Jiang F (2018) Group tracking control of second-order multi-agent systems with fixed and Markovian switching topologies. Neuro Comput 281:37–46. ISSN 0925-2312. https://doi.org/10.1016/j.neucom.2017.11.040

84

S. Koley and P. P. Acharjya

44. Ajwad SA, Moulay E, Defoort M, Ménard T, Coirault P (2021) Leader-following consensus of second-order multi-agent systems with switching topology and partial aperiodic sampled data. IEEE Control Syst Lett 5(5):1567–1572. https://doi.org/10.1109/LCSYS.2020.3041566 45. Matuš˚u RS, Cao C, Chengyu (2018) Nonlinear uncertainties cancelling in multi-agent systems enabled by cooperative adaptation. J Control Sci Eng 2018:1687–5249.https://doi.org/10.1155/ 2018/3927108 46. Cai H, Huang J (2017) Leader-following attitude consensus of multiple rigid body systems by an adaptive distributed observer approach. IFAC-PapersOnLine, 50(1):15446–15451. ISSN 2405-8963.https://doi.org/10.1016/j.ifacol.2017.08.1878 47. Rehák, Lynnyk B, Volodymyr (2021) Leader-following synchronization of a multi-agent system with heterogeneous delays. Front Inform Technol Electron Eng 22:97–106.https://doi.org/10. 1631/FITEE.2000207 48. Shahnazi R (2020) Cooperative neuro adaptive control of leader following uncertain multi-agent systems with unknown hysteresis and dead-zone. J Syst Sci Complex 33:312–332. https://doi. org/10.1007/s11424-020-8198-9 49. Rahimi N, Binazadeh T (2019) Distributed robust consensus control for nonlinear leader– follower multi- agent systems based on adaptive observer-based sliding mode. J Vib Control 25(1):109–121. https://doi.org/10.1177/1077546318772239 50. Ma T, Li K, Zhang Z, Cui B (2021) Impulsive consensus of one-sided Lipschitz nonlinear multi-agent systems with Semi-Markov switching topologies. Nonlinear Anal Hybrid Syst 40:101020. ISSN: 1751-570X.https://doi.org/10.1016/j.nahs.2021.101020 51. Song J, Li K, Hua C (2019) Output feedback consensus control of high-order nonlinear multiagent systems with full state constraints, Chinese control conference (CCC), Guangzhou, China, pp 6212–6217. https://doi.org/10.23919/ChiCC.2019.8865422 52. Almeida R, Girejko E, Hristova S, Malinowska A (2019) Leader-following consensus for fractional multi-agent systems. Adv Difference Equ. https://doi.org/10.1186/s13662-0192235-9 53. Anuradha M, Ganesan V, Oliver S et al (2020) Hybrid firefly with differential evolution algorithm for multi agent system using clustering based personalization. J Ambient Intell Human Comput. https://doi.org/10.1007/s12652-020-02120-w 54. Abdulghafor R, Turaev S (2017) Consensus of fractional nonlinear dynamics stochastic operators for multi-agent systems. Inform Fusion 44:1. https://doi.org/10.1016/j.inffus.2017. 11.003 55. Wang X, Su H, Wang H, Chen G (2016) An overview of coordinated control for multi-agent systems subject to input saturation. Perspect Sci 7:133–139. ISSN: 2213-0209. https://doi.org/ 10.1016/j.pisc.2015.11.022 56. Wang Q, Wang J (2018) Fully distributed fault-tolerant consensus protocols for Lipschitz nonlinear multi-agent systems, vol 6, issue 2018, pp 17313–17325 57. Cai Y, Zhang H, Zhang J, Wang W (2021) Fixed-time leader-following/containment consensus for a class of nonlinear multi-agent systems. Inform Sci 555:58–84. ISSN 0020-0255. https:// doi.org/10.1016/j.ins.2020.12.064 58. Jia Q, Tang WKs (2018) Consensus of multi-agents with event-based nonlinear coupling over time-varying digraphs. In: IEEE transactions on circuits and systems II: express briefs, pp 1–1. https://doi.org/10.1109/TCSII.2018.2790582 59. Liu, Jie JH (2018) Leader-following consensus of linear discrete-time multi-agent systems subject to jointly connected switching networks. Sci China Inform Sci 61:1869–1919.https:// doi.org/10.1007/s11432-018-9453-x 60. Chen TD, Fatemeh Bevrani H (2009) Multi-agent systems in control engineering: a survey. J Control Sci Eng 2009:1687–5249.https://doi.org/10.1155/2009/531080 61. Chen Y, Lu J (2013) Consensus of discrete-time multi-agent systems with transmission nonlinearity. Automatica 49:1768–1775. https://doi.org/10.1016/j.automatica.2013.02.021 62. Miao G, Ma Q (2015) Group consensus of the first-order multi-agent systems with nonlinear input constraints. Neurocomputing 161:113. https://doi.org/10.1016/j.neucom.2015.02.058

4 Prevalence of Multi-agent System Consensus in Cloud Computing

85

63. Shi L, Xie D (2020) Leader-following consensus of second-order multi-agent systems with time-varying delays and arbitrary weights. Trans Inst Meas Control 42(16):3156–3167. https:// doi.org/10.1177/0142331220942715 64. Park JH, Wang F, Yang Y (2018) On leaderless and leader-following consensus for heterogeneous nonlinear multiagent systems via discontinuous distributed control protocol. Mathematical problems in engineering, Hindawi, vol 2018.https://doi.org/10.1155/2018/2917954 65. Xu X, Chen S, Gao L (2014) Distributed leader-following finite-time consensus control for linear multiagent systems under switching topology. Sci World J 248041.https://doi.org/10. 1155/2014/248041 66. Cai N, Diao C, Khan M (2017) A novel clustering approach based on group quasi-consensus of unstable dynamic linear high-order multi-agent systems complexityhttps://doi.org/10.1155/ 2017/4978613 67. Qin J, Gao H, Zheng WX (2011) Second-order consensus for multi-agent systems with switching topology and communication delay. Syst Control Lett 60:390–397. https://doi.org/ 10.1016/j.sysconle.2011.03.004 68. Zhao Y, Wen G, Duan Z, Xu X (2011) A new observer-type consensus protocol for linear multiagent dynamical systems. In: Proceedings of the 30th Chinese control conference, Yantai, China, pp 5975–5980 69. Li X, Li C, Yang Y, Mo L (2020) Consensus for heterogeneous multi-agent systems with nonconvex input constraints and nonuniform time delays. J Franklin Inst 357(6):3622–3635. ISSN: 0016-0032.https://doi.org/10.1016/j.jfranklin.2019.12.035 70. Ni W, Cheng D (2010) Leader-following consensus of multi-agent systems under fixed and switching topologies. Syst Control Lett 59(3–4):209–217. ISSN: 0167-6911.https://doi.org/10. 1016/j.sysconle.2010.01.006 71. Miao G, Ma Q, Liu Q (2016) Consensus problems for multi-agent systems with nonlinear algorithms. Neural Comput Applic 27:1327–1336. https://doi.org/10.1007/s00521-015-1936-6 72. Hu H, Yu W, Wen G, Xuan Q, Cao J (2016) Reverse group consensus of multi-agent systems in the cooperation-competition network. IEEE Trans Circuits Syst I Regul Pap 63(11):2036–2047. https://doi.org/10.1109/TCSI.2016.2591264 73. Shen Y et al (2018) Distributed cluster control for multi-microgrids using pinning-based group consensus of multi-agent system. In: 5th IEEE international conference on cloud computing and intelligence systems (CCIS), Nanjing, China, pp 1077–1080. https://doi.org/10.1109/CCIS. 2018.8691332 74. Zhang X, Zhu Q, Liu X (2016) Consensus of second order multi-agent systems with exogenous disturbance generated by unknown exosystems. Entropy 18(12):423. https://doi.org/10.3390/ e18120423 75. Mondal S, Rong S, Xie L (2017) Heterogeneous consensus of higher-order multi-agent systems with mismatched uncertainties using sliding mode control. Int J Robust Nonlinear Control 27(13):2303–2320 76. Nada D, Bousbia-Salah M, Bettayeb M (2018) Multi-sensor Data Fusion for wheelchair position estimation with unscented Kalman Filter. Int J Autom Comput 15:207–217. https://doi. org/10.1007/s11633-017-1065-z 77. Xu X, Liu L, Feng G (2016) Consensus of single integrator multi-agent systems with directed topology and communication delays. Control Theory Technol 14:21–27. https://doi.org/10. 1007/s11768-016-5122-x 78. Song J (2018) Low-pass filter design and sampling theorem verification. AIP conference proceedings, vol 1971, Issue 1. https://aip.scitation.org/doi/abs/10.1063/1.5041159 79. Xiao F, Wang L (2008) Asynchronous consensus in continuous-time multi-agent systems with switching topology and time-varying delays. IEEE Trans Autom Control 53(8):1804–1816. https://doi.org/10.1109/TAC.2008.929381 80. Garcia E, Antsaklis PJ (2016) Chapter three—adaptive stabilization of uncertain systems with model-based control and event-triggered feedback updates. Control of complex systems, Butterworth-Heinemann, pp 67–92. ISBN: 9780128052464.https://doi.org/10.1016/B978-012-805246-4.00003-3

86

S. Koley and P. P. Acharjya

81. Mingyu F, Yujie X (2017) Finite-time tracking control for a class of MIMO nonlinear systems with unknown asymmetric saturations. Math Prob Eng 2017, Article ID 9452171, 1–10.https:// doi.org/10.1155/2017/9452171 82. Liu X, Lam J, Yu W, Chen G (2016) Finite-time consensus of multiagent systems with a switching protocol. IEEE Trans Neural Netw Learn Syst 27(4):853–862. https://doi.org/10. 1109/TNNLS.2015.2425933 83. Donea J, Giuliani S, Halleux JP (1982) An arbitrary Lagrangian-Eulerian finite element method for transient dynamic fluid-structure interactions. Comput Methods Appl Mech Eng 33(1– 3):689–723. ISSN: 0045-7825. https://www.sciencedirect.com/science/article/pii/004578258 2901281 84. Ling J, Wang R (2017) Finite-time consensus of heterogeneous multi-agent system with external disturbances. Chinese Automation Congress (CAC), Jinan, China, pp 1478–1483. https://doi.org/10.1109/CAC.2017.8243000 85. Zheng Y, Chen W, Wang L (2011) Finite-time consensus for stochastic multi-agent systems. Int J Control 84:1644–1652. https://doi.org/10.1080/00207179.2011.622792 86. Abdulghafor RA, Almohamedh S, Turaev H, Almutairi S, Badr (2019) Symmetry, nonlinear consensus protocol modified from doubly stochastic quadratic operators in networks of dynamic agents, vol 11, issue 12, pp 2073–8994. https://doi.org/10.3390/sym11121519, https://www. mdpi.com/2073-8994/11/12/1519 87. Lokhande SC, Xu H (2017) Optimal self-triggered control and network co-design for networked multi-agent system via adaptive dynamic programming, IEEE Symposium Series on Computational Intelligence (SSCI). Honolulu, HI, USA 2017:1–8. https://doi.org/10.1109/ SSCI.2 88. Huin L, Boulanger D, Disson E (2013) A MAS for access control management in cooperative information systems. In: Proceedings of the fifth international conference on management of emergent digital ecosystems (MEDES ‘13). Association for Computing Machinery, New York, NY, USA, pp 84–91. https://doi.org/10.1145/2536146.2536170 89. Wang J, Chen K, Zhang Y (2017) Consensus of high-order nonlinear multiagent systems with constrained switching topologies. Complexity 2017, Article ID 5340642, 11:2017. https://doi. org/10.1155/2017/5340642

Chapter 5

The Role of Multi-Agent Systems in IoT Mohmmad Gheysari

and Mahsa Seyed Sadegh Tehrani

Abstract Internet of Things and multi-agent systems are two concepts that have revolutionized the cyber and physical worlds and putting these two concepts together in a single framework can increase the effectiveness of solutions. The main purpose of the IoT is to automate processes and create an intelligent environment that different elements can interact with each other. Systems with high dynamics and complexity can control by the Internet of Things architecture with the help of agent systems. The present work describes role of multi-agent systems in IoT and application areas. To better understand this issue, it is essential to clarify what IoT is (part 1) and which properties are used to define IoT agents in general (part 2). This chapter’s main goal is to analyze other aspects of multi-agent systems as one of IoT’s key components. Furthermore, IoT agents should be distinguished from other agentoriented technologies so that these two must briefly be described. Also, possible areas of application and properties of IoT agents are shown in this article. Keywords Internet of Things · Multi-agent systems · Integrating technologies · Intelligent systems

1 Introduction Agent technology has been subjected to growing dynamism, especially in recent years. The amount of information that needs to be processed every day in the coming years is going to increase dramatically. A multi-agent system consists of a collection of long-lived agents. Each of these agents has a specific task that it performs independently. In order to meet higher-level goals that cannot be achieved by a single agent, several of them can communicate within the multi-agent system. To achieve this, the system must support some form of messaging, be it through the use of middleware M. Gheysari (B) Doctor of Business Administration (Digital Transformation Field), Faculty of Management, Tehran University, Tehran, Iran M. S. S. Tehrani Master of Information Technology, Faculty of Management, Payam Noor University, Tehran, Iran © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 S. Gupta et al. (eds.), Multi Agent Systems, Springer Tracts in Human-Centered Computing, https://doi.org/10.1007/978-981-19-0493-6_5

87

88

M. Gheysari and M. S. S. Tehrani

or through the provision of a discovery service as an aid to direct communication between the agents. The function of the agents depends on communication between them, so that there is already a large number of communication relationships for IoT systems. In addition, the decentralized self-organization of the dynamic processes in the Internet of Things and smart environments add a further level of complexity to the conventional control logic, since the distributed states and the control decisions based on them cannot be directly understood from the outside. A smart environment is an environment that is located in physical space and is composed of several intelligent systems. These systems can be visible and invisible and influence each other with the physical space and are implemented using a heterogeneous number of sensors and actuators that are connected in a network. In a study, author [1] proposed algorithms which describe intelligent behavior in physical layer for IoT solutions which integrates software agent for devices. In other research [2], a multi-agent algorithm for IoT-based home is presented. The algorithm can manage the “things” in a smart environment to increase quality and speed of operations. Smart things are linked to a cyber agent and all cyber agents cooperate with each other. So far, several articles have been presented on the application of multi-agent approach in IoT architecture. Authors [3] in an article describe a multi-agent platform as a software framework which can be used in IoT as agents. This platform presents appropriate method for agent developers to design and try different agents. In this research, all agents refer to platform and application layers and has limited the use of multi-agent systems in these two layers of IoT architecture. Same as this article, there are many works which present the use of MAS in IoT for these two level. IoT architecture is a multi-layer architecture and in fact, the main purpose of using MAS is to minimize the problems of controlling the huge volume of devices in a wide area network and be able to use all generated information. Therefore, consider agents only in the two platform and application layers cannot help us to achieve this goal. In this chapter, at first we review IoT in detail to understand how multi-agent systems and affect IoT-based solution in all horizontal and vertical layers. Multi-agent system in addition to improving the performance of IoT solutions can add to the complexity of the design phases. In the continuation of the chapter, we mentioned how to reduce design complexity of design and use multi-agent systems in different IoT solutions.

2 What is IoT? The Internet of Things (IoT) is an interdisciplinary framework, and there is no unified definition for it. However, it can generally be considered as a conceptual framework that integrates the physical and cyber worlds and has led to the convergence of information technology (IT) with operational technology (OT). As most users know, the classic Internet is called the social Internet, because it is shaped by interactions between people. But in the Internet of Things, devices are no longer just controlled by

5 The Role of Multi-agent Systems in IoT

89

Table 1 Definitions of IoT provided by different standard development organizations No. Organization

Definition

1

ISO/IEC

It is an infrastructure of interconnected objects, people, systems, and information resources together with intelligent services to allow them to process information of the physical and the virtual world and react

2

ITU-T Y.2060 A global infrastructure for the information society, enabling advanced services by interconnecting (physical and virtual) things based on existing and evolving interoperable information and communication technologies

3

IEEE

The Internet of Things (IoT) is a framework in which all things have a representation and a presence in the Internet. More specifically, the Internet of Things aims at offering new applications and services bridging the physical and virtual worlds, in which machine-to-machine (M2M) communications represents the baseline communication that enables the interactions between Things and applications in the cloud

users; they actually communicate directly with each other in the sense of machineto-machine communication and can therefore implement tasks and processes in a completely automated manner. Internet of Things merge “Internet” and “Thing” and define a new phrase in physical and digital world (Table 1). The Internet of Things is a critical enabling concept in the digital world, in other words, it is a framework that creates a vast network of humans, devices, machines, etc. This framework can provide innovative solutions using several technologies and infrastructures such as smart sensors, communications, cloud, artificial intelligence, and machine learning to monitor and manage different environments. The IoT can also be considered as a connected framework in which all elements are interconnected and can interact and collaborate with each other. In IoT, objects are given a unique identity and can communicate with each other or receive commands. With the Internet of Things, applications can be automated, and tasks can be carried out without external intervention. With the IoT, everything from a coffee maker in the kitchen to a gas turbine in the oil and gas industry can become a smart device that provides useful information. Devices are given a unique identity (address) in the network and are equipped with intelligence. This enables them to communicate over the Internet and perform tasks automatically. The Internet of Things helps people to live and work smarter and monitor all activities in their lives. In addition, to all the options that are provided to people to enhance their day-to-day activities, it becoming increasingly important to different businesses. In fact, the Internet of Things gives companies online insight into how their systems work and provide insight into everything from machine performance to supply chain and logistics operations. IoT enables companies to automate processes and reduce labor costs. Besides, it reduces waste and improves service delivery, makes the production and delivery of goods more cost-effective, and provides transparency in processes. Some of the general and key IoT characteristics are as follows:

90

M. Gheysari and M. S. S. Tehrani

• Data gathering and management: The IoT sensors distribute widely and collect data quickly and effectively to create a new way of collaboration among connected devices. • Interconnectivity: The IoT can interconnect anything (physical or virtual things) with the help of global information and communication infrastructure. • Things-related services: The IoT can provide things-related services within the constraints of things, such as privacy protection and semantic consistency between physical and their associated virtual objects. • Heterogeneity/diversity: The IoT devices should be heterogeneous based on different hardware • Platforms: They can interact with other devices or service platforms through different networks. • The potential high number of nodes: The number of devices that needs to be managed and that communicates with each other will be at least an order of magnitude larger than the number of devices connected to the current Internet.

3 IoT Architecture Without this, IoT technology stack, there would be no way to do anything with IoT devices and no reason to connect things to the Internet (i-SCOOP). Therefore, the IoT technology stack includes all the necessary technologies to get from the IoT device and data to an actual purpose and goal or a so-called IoT use case. The IoT architecture, similar to other interdisciplinary concepts, will promote every day. Various organizations and companies have presented different architectures for IoT, but the most common architecture proposed is a four-layer architecture which shown in Fig. 1 [4]. The four-layer architecture consists of some structural blocks and includes vertical and horizontal layers. In the first layer, data is generated, and in the second layer, the generated data is transferred. After that, the received data will analyze and store in the third layer, and in the last layer, the information will be presented to users. • Hardware or things layer: sensors, actuators, microcontrollers, microprocessors, and communication modules • Communication layer: gateway and telecommunication infrastructure • Platform layer: dashboard for data storage and analysis • Application layer.

3.1 Hardware Layer This layer is the basic layer in all IoT-based solutions. The hardware layer is more complex than other layers. This layer consists of sensors, actuators, communication

5 The Role of Multi-agent Systems in IoT

91

Fig. 1 IoT architecture

modules, microprocessors or microcontrollers, etc., which are all provided in the form of a node or device. IoT devices serve a wide variety of purposes, such as the quantitative determination of measured variables. Sensors are responsible for data acquisition, gateways for data preprocessing and transmission, and web/mobile application to analyze data’s output. No matter whether simple or intelligent, immobile or mobile IoT devices— there is a suitable infrastructure for every application scenario. An IoT device can take many colors and shapes. When you talk about an IoT device, you don’t think of just sensors or actuators or other related things, such as boards, processors/chips, transceivers, micro-electro-mechanical systems, or the like. A connected object can have a few or several thousand sensors and transducers. A telematics box like the one used in cars for car insurance typically has a few sensors, and an oil rig can have tens of thousands. • In the past, data was transmitted via wires and cables but today the increase of various standards and wireless communication networks, more data and information transmitted in real-time. Sensors are electronic and mechanical components that monitor changes in the target environment in a variety of ways. These sensors can include a very small sensor such as a heart rate monitor or a large sensor like a smart meter. Simple devices transfer the data directly to the cloud or a gateway, while smart devices first process the data with integrated processing technology. The use of a smart device as a gateway is referred as edge computing—devices on the edge of the solution local network. In all cases, sensors are part of the digital data backbone of networked and intelligent solutions. Everything that has to do with “Smart,” and IoT is based on sensors and other types of transducers. • Just like sensors, actuators are converters and perform some actions based on the sensor’s data and can control systems based on predefined conditions. While sensors record and transmit, actuators act and activate. In the IoT context, actuators

92

M. Gheysari and M. S. S. Tehrani

are in most cases about switching something on or off by applying force. But there are also many industries or robotics applications, such as actuators for grippers. • A microcontroller or microprocessor is the kernel of an IoT device. This part of the IoT device processes the data received from sensors and instruction. Microprocessor technology forms the technical basis of the Internet of Things [5]. Thanks to increasingly cheaper, smaller and more powerful microprocessors, objects can be equipped with electronic intelligence with relatively little effort.

3.2 Communication Layer The second layer is the communication, which is responsible for connect devices to cloud with a gateway and connectivity protocols. Due to the huge number of IoTcapable devices and the resulting excessive number of connections, it is becoming increasingly important to have a fully functional network as the basis for communication between machines, systems and people. It takes over the bridge between the individual communication partners in order to guarantee the best possible data flow. It is important to control all components of an IoT network, so that the most important criteria are met: • System availability • Data transfer • Network security In this layer, short-range connectivity protocols such as Wi-Fi, ZigBee, Bluetooth, etc., are mainly designed for short distance, low-to-medium data speeds and energy consumption and the cellular network is primarily intended for high data speeds and medium-to-high energy consumption [6]. The new technologies of the LPWAN (Low Power Wide Area Network) Group are designed for long ranges, small amounts of data and very low energy consumption, which mostly corresponds to requirements of the Internet of Things products various new radio technologies in LPWAN have been developed by multiple providers in recent years. These are radio networks with the following three main goals: • High radio range • Low energy consumption • Low cost This created ideal conditions for inexpensive, battery-operated IoT sensors and actuators, which can be connected to the Internet over large distances. With these technologies, the battery life should be up to 10 years. However, as a compromise, these radio technologies’ communication speed was reduced to below 100 bit/s up to a few kBit/s to achieve better receiver sensitivity and a greater radio range.

5 The Role of Multi-agent Systems in IoT

93

The best-known new representatives of LPWAN technology are LoRaWAN, Sigfox, and NB-IoT. Each of these technologies has its advantages and disadvantages, whereby the interpretation of these advantages and disadvantages depends heavily on the needs of the Internet of Things application. IoT gateways are the cornerstones of a convergent IoT architecture. They were specially developed to close the gap between the devices and platform. IoT gateways optimize a solution’s performance by collecting data from the operational processes at the place of their creation in real time and performing an initial preparation.

3.3 Platform Layer The platform layer is responsible for collecting, preparing, further processing, and storing data from the hardware and communication layer. The platform or cloud layer consists of computing and storage capacities. But it also includes integration into the existing IT infrastructure, into the ERP systems in use and other business-oriented company applications. This is where the link is made to the value chain of a company. An IoT platform enables the networking of more important devices and applications in the Internet of Things. The technology leads for the exchange of information, compensates them for the cross-system knowledge of their settings, and functions to control and analyze data. An IoT platform can be imagined as an interpreter and service center in the Internet of Things. It ensures that different types of devices and applications can exchange information via their own communication channels and understand each other in such a way that they automatically react to one another. Ideally, any type of device can be connected to an IoT platform via any available interface. Any type of data can be collected and evaluated to supply any desired application with the information. However, there are technical limitations to almost all IoT platforms. Depending on the provider, they support certain interfaces, data formats, communication protocols, and special services by monitoring and controlling devices, concentrating and evaluating data, initiating defined follow-up processes, or reacting to certain external events and commands. In addition to the horizontal platforms, vertical IoT platforms are developing that support industry-specific interfaces, protocols, data formats, applications, and processes. Selection of IoT platforms depends on many different factors and conditions; for example, it can be offered on a dedicated basis from the cloud. These platforms can also be offered and administered by service providers for several user companies. One of the most important properties of an IoT platform is openness, i.e., the support of as many standards, formats, and interfaces as possible, either related to the respective industry or to certain processes that are to be mapped in the IoT. In addition to this openness, security is also essential, in the sense of protection against possible attacks and unauthorized access to devices, connections, applications, and data. In general, IoT data can provide business-relevant insights that must remain confidential. These data are usually subject to data protection if they can be directly or

94

M. Gheysari and M. S. S. Tehrani

indirectly assigned to people. That is why security functions such as encryption and integrity control are just as important as compliance functions, ensuring transparency and auditability of services and processes. An IoT platform’s basic functions also include identification, connection, management of devices, collection, management and analysis of data, visualization of the analysis results, and reporting. How and where data streams are processed influences the choice between cloud and edge computing—or both. To save storage capacity and prevent latency, it may be advisable to pre-save the huge data streams generated by the IoT on the edge of the network and sort them according to relevance before they are saved. This results from the efficient use of the available bandwidth and performance.

3.4 Applications Layer The application layer uses the processed data for evaluations, decisions, and new business models. The application layer receives information from the layers below and processes it. It is divided into two further layers: the service layer, which contains the business logic and databases, and the interface layer, which provides various interfaces, for example, user interfaces (application frontend) or APIs. The application layer contains the data standards and protocols that are used by the application programs. These programs can be a browser, an e-mail program, or an MQTT broker. This is where the messages that represent the actual communication are processed. The control (setup/disconnection) of the connections with other applications is also controlled here, and the display of the information is processed.

3.5 Security Layer The Internet of Things creates completely new requirements in the area of cyber security, both in IT and in the OT area. In order to identify as many weak points as possible, it is necessary to illuminate not only individual areas of application and devices but also the entire IoT ecosystem. On one hand, this includes internal devices and applications (including operating systems, communication channels, connected devices), on the other hand, all external ones. The security aspect is one of the greatest concerns in the IoT environment. Data collected through smart devices can be personal and corporate in nature. They should be protected against theft and tampering both during transmission and for the entire duration of their storage [7]. An IoT application can, for example, store and link data about a person’s state of health, purchasing habits, whereabouts, and financial transactions carried out, property and business interactions over a long period in order to identify a person without specifying their name.

5 The Role of Multi-agent Systems in IoT

95

As IoT devices’ usage rate increases, more and more objects are connected to the Internet and collect more and more information about their users. Every day, intelligent things become the target of attacks, either for pure information acquisition or for the complete systems’ infiltration and takeover. IoT devices are exposed to these risks much more widely than previously used Internet-enabled components and services. Due to the increasing number of IoT devices, attackers can now generate massive DDoS attacks against companies and organizations. With an appropriate underwriting plan, it will help tackle the traffic before it reaches the organization. A proper and active risk response plan can combat DDoS traffic before it reaches the company. Also, IoT devices can use hardware-based trust anchors. These use a trustworthy boot process to ensure that devices work in a known, secure state and that content remains private. The best way to ensure data protection and combat botnets is devices that have to authenticate against other systems and are configured in such a way that secure unique IDs and passwords are used. In some cases, it may also be possible to implement encryption keys to protect the device’s identity. Special IoT devices with this capability are also so-called closed-circuit TVs or DVR devices. Other methods that can be used include the use of SSL certificates. Researching suitable products and implementing these skills will be a good starting point for improved IoT security. For improving security and lower the risk, the architecture of the Internet of Things and its components must take the following security aspects into account: • • • • •

Effective and reliable identity and access management, Encryption of all data transmitted over the public Internet, Protection of the individual systems by firewalls or firewall-like functions, Effective hardening of the systems to minimize the possibility of attack as well, Software and patch management over the systems’ entire operating period to remedy detected errors and security deficiencies.

4 Internet of Things Application Domains The IoT has huge potential for developing new intelligent applications in nearly every domain, such as personal, social, societal, medical, environmental, and logistics aspects. As shown in Fig. 2, the number of application domains has also been increasing due to its ability to perform contextual sensing. It allows, for instance, to collect information of the environment, natural phenomena, medical parameters, and user habits and then can offer tailored services based on information received. Such a phenomenon should enhance the quality of everyday life and should have a reflective impact on society and the economy irrespective of the application domain. A popular example is fitness wristbands and fitness trackers in the health industry, so-called e-health. They usually pass on the measured data, such as the number of steps, pulse rate, or blood pressure, to the user’s smartphone, so that it has an overview of user performance and health values. It works in a very similar way with medical

96

M. Gheysari and M. S. S. Tehrani

Fig. 2 IoT application domains

devices such as a networked pacemaker, which automatically notifies the emergency service in life-threatening situations. Or, the IoT can be cited as the foundation for Industry 4.0 and in production, Industry 4.0 specifically relies on IIoT technologies. Predictive maintenance is inseparably linked to IIoT and predictive maintenance is even considered “one of the key innovations in Industry 4.0”. For years, manufacturers in various industries have been using a time-based approach to maintaining their systems—preventive maintenance. This means that the maintenance work must take place at fixed intervals. Furthermore, preventative maintenance can be done too late or too early. This is proof that time-based maintenance is not cost effective a system is maintained regardless of its actual condition.

5 Why Use the MAS Concept for IoT-Based Systems? Since the emergence of the Internet, decentralization has been a fundamental principle for the design, organization, and operation of complex distributed systems and global network management. Similar challenges arise from technological developments such as the “Internet of Things” and communities and marketplaces of “Web 2.0”, which imply usage scenarios that would be difficult to handle with central architectures and design approaches. Multi-agent systems are one of the major concepts which have been used in the IoT framework and can be included devices, robots, networks, and software agents. In IoT-based systems, agents’ interaction is highly important and because of the framework’s nature, different types of agents must communicate with each other [2]. Multi-agent systems help IoT create intelligent systems and for enhancement, agents need to integrate and these systems make IoT solutions more efficient, reliable, and

5 The Role of Multi-agent Systems in IoT

97

secure. The IoT brings a change from fixed and static processes to self-controlled and dynamic processes. To achieve this, a timely, process-adaptive change will be required. This is accompanied by the trend toward higher transmission frequencies with decreasing volumes of individual programs and smaller commissions. The IoT systems can become very complex due to the simultaneous use of several concepts such as embedded systems, WSN, could and edge computing, robotics, etc. Also, because these systems are implemented and set up on a large scale, it makes more difficult to manage the whole system. Integration is another challenge without which no IoT system can work effectively. For this reason, various solutions for integration, including the use of a comprehensive, integrated platform and multiagent systems, have been proposed in various studies. One common solution, which proposed for IoT, is SOA1 Definitions for serviceoriented architectures address the most diverse computer science areas and range from a method to an architecture pattern or technology. But whichever point of view experts take, the advantages of an SOA mentioned in their definitions are similar. Typically, they highlight the following aspects: quick adaptability of processes, high future security, and simplified integration. Services offer functions that are required in a company and called up by different entities. The developers do not have to implement these functions multiple times but can call them remotely via services. On general, agents depending on where they are used have different meanings, which spread from “agents” to “intelligent objects.” The phrase “object-oriented is next to agent-oriented” has been said for a long time, but object-oriented and agent-oriented here must be guidelines for thinking and designing things. Multiagent concept is one of the solutions that can address IoT systems’ problems. MASs have been applied to various issues, including market simulation, monitoring, system diagnosis, and remedial actions [8]. Progress in AI, hardware, and sensor technologies has been achieved by the MAS community, resulting in agent technologies applied successfully to real-world problems. Multi-agent systems are a specific kind of distributed intelligent systems in which independent agents inhabit a world with no global control or globally consistent knowledge. The use of agents in network technology enables automatic and dynamic load distribution, high scalability, and fail-safe networks. With agent and multi-agent systems, the conditions can be mapped well than with an object-oriented approach. The individually specified behavior is used to demonstrate dynamic interactions, understand relationships, and explain phenomena. We can design an IoT framework based on a multi-agent system to manage the hardware and software parts. In multi-agent systems, each agents can include software and even robots and humans. In addition, in the IoT agents can consist of a variety of agents. Multi-agent systems architecture for IoT helps to enhance smart systems and make them more accurate and flexible. A multi-agent system has operative and non-operative features. The operative features are management, coordination, reasonableness, knowledge modeling and the non-operative features

1

Service-oriented architecture.

98

M. Gheysari and M. S. S. Tehrani

(a)

(b)

Fig. 3 Differences between a normal IoT architecture (a) and an IoT-based multi-agent (b)

are expandability, stability, and reliability. In order to be able to use such agents effectively, these agents have to coordinate themselves to solve tasks together. General benefits of multifactorial systems for the Internet of Things • Most of IoT solutions are physically distributed. For example in smart agriculture, several sensors installed on a yield to measure different parameters of soil, wheatear, and planet. Agents in physical level can reduce the problems associated with the multiplicity and scattering of hardware. • Using MAS approach increase entirely system efficiency, categorically the aspect of processing, dependability, and sanity. • IoT-based solutions are integrated and interconnected systems, and stopping one of the objects can severely affect the whole system. But with use of MAS approach in this architecture, we can manage resource restriction, efficiency bottlenecks, or serious failures. It has been shown that agent-based models are suitable for IoT systems with high dynamics and complexity. By decentralizing tasks, relatively simple heuristics are sufficient for robust and very flexible operations. The following steps have to be carried out by “agentification” of an IoT system: • Definition of suitable subprocesses (determination of competencies, black box consideration) • Software implementation (selection of an agent platform, interfaces to higherlevel and lower-level systems such as hardware and processing systems, time requirements) • Enabling agents to act in a process-adaptive manner (formulation of target criteria, recording of key figures, and evaluation during operation) • Implementation of a test environment (coupling of software agents and simulation model for time-gathered emulation of the real system) Figure 3 shows the difference between normal IoT architecture (a) and IoT

5 The Role of Multi-agent Systems in IoT

99

architecture based on multi-agent systems (b). The agents used in the IoT are not necessarily the same, and each can have a different nature and function. Agents can support smart systems and can evolve for a particular environment as it is used by users. In such a system, each agent can be assigned a task and purpose based on its nature. IoT agent is a component situated within the system and a part of an environment that senses that environment and acts on it over time to pursue its own agenda and affect what it feels in the future. Each IoT agent must have at least one of these specific properties: • • • •

Autonomy, “self-determined” behavior (not controlled by a central entity) Reactivity, ability to do an independent activity (not user-activated) Goal orientation, the ability for goal- and task-oriented coordination (cooperation) Ability to learn, ability to reason independently (dealing with uncertain knowledge) • Ability to communicate and cooperate (synchronization) • Mobility IoT agents need to behave according to the environment in which they are located. The extent to which each agent recognizes environmental information (whether it needs information about the entire environment, only the minimum amount of information around it, etc.) should be determined according to the problem to be addressed. Figure 4 shows different agents in IoT.

Fig. 4 Agents in IoT architecture

100

M. Gheysari and M. S. S. Tehrani

6 Types of Agents in a Multi-Agent System Based on IoT In general, IoT agents should be able to deal with the platform’s security aspects (authentication and authorization of the channel) and provide other common services to the device programmer. The fundamental difference between IoT agents and ordinary software agents is on their autonomy. From the user’s point of view, this increases the agent’s intelligence, which is expressed for the user. User initially defines the agent’s problem, possibly gives the agent special instructions and ideas, and the agent solves the problem independently. In many cases, the agent must communicate or cooperate with its environment in order to carry out its tasks. The environment can be composed of users or other agents. The most important point is that, an agent does not bring its user any advantages if the user cannot trust it, especially when it comes to sensitive areas such as air traffic control systems or trading systems in electronic markets. An IoT framework based on multi-agent systems is a complex goal system, and agents depend on how extensive and complex the respective target system is. For example, the purpose of purely collecting information on a particular topic would only require reactivity and no initiative from the agent, since in this case, it only has to monitor information sources and react to the arrival of new information. A more extensive target system consists of an overall goal and several quicker and easier subgoals to achieve. The implementation of MAS in IoT relies heavily on the use of MAS platforms and development tools. MAS platforms offer basic services for agents. This includes sending messages and searching for other agents or their services. The tools are intended, on the one hand, to facilitate the transition from the design to the implementation phase, and on the other hand, they support the application developer in troubleshooting. The agents can be classified in different ways. A simple classification is based on the function of a system. This leads to a breakdown with many different classes of agents. This general categorization also does not allow the complexity and characteristics of an agent to be identified. This environment can consist of other agents, human users, external sources of information, or physical objects, and the proposed architecture has three agents: (1) hardware agent, (2) software agent, and (3) human agent. • Hardware agents are those physical objects which were mentioned in the first section and use programming components. Hardware agents can be located in different environments while influencing them and influenced by them. Each hardware agent is programmed to communicate with other agents and to its computational environment. • Software agents are software components that can manage groups of processes and systems and most are in the third layer of IoT architecture, the platform. An interface agent can assist its user in a variety of ways: (1) training and instruction of the user, (2) support in the collaboration of different users, (3) performing tasks

5 The Role of Multi-agent Systems in IoT

101

on behalf of the user, (4) monitoring of events and processes, and (5) hiding the complexity of tasks. • Human agent is the main user of IoT systems, and the data must be analyzed and presented in a way that is understandable to this user. It is seldom that all the above attributes of an agent appear simultaneously in the same form. Each agent has some special characteristics based on its requirement and the environment it needs to operate. In this context, it is sensible and necessary to classify the agents more closely according to their functionality. The boundaries between the fields of activity are blurred due to the inconsistent assignment of properties. Nevertheless, the following three core types of an IoT agent can be identified: • Passive agents: These agents can understand the specific conditions of different environments and measure the changes. A passive agent that only reacts to changes in its environment is not sufficient in many cases. Instead, the agent must take the initiative itself in certain situations. For example, in a smart home, the thermostat can only measure temperature, and it is not necessary adjust the temperature. • Agile agents: These agents are not limited to understanding the environment and can also react with a little intelligence. Compared to the above example, a fire detection sensor should be able to generate an audible alarm or issue a command to activate sprinklers after detection and it has to be smarter. However, the agent must know in which direction he should take the initiative. • Interactive agents: These agents are able to communicate with other agents (and possibly humans) through communication protocols and languages and create a network of smart objects. In many cases, the agent must communicate or cooperate with its environment to carry out its tasks. Cooperation is one level above pure communication. A pure communication protocol is no longer sufficient. Agents must be able to exchange information about their goals and previous knowledge levels. The knowledge gained by this agent is then transmitted to the other agents. Tasks can be completed faster and more efficiently, inconsistencies are avoided, and the overall system’s performance increases. The focus of this group of agents is on the interaction between the individual agents. The characteristic of communication and cooperation is particularly pronounced here. The existence of a uniform agent language is a necessary prerequisite for efficient collaboration between the agents. The main characteristic of interactive agents is on the communication and cooperation level. The development of interactive agents’ operations is based exclusively on the communication and cooperation between the respective agents. Joint agents are mostly static, extensive, and imprecise. For this reason, the ability to learn is usually not well developed. Due to the size of the multi-agent system, the property of mobility is hard to be found either. However, this does not mean that an individual agent cannot be mobile within the overall system.

102

M. Gheysari and M. S. S. Tehrani

7 Simulation Multi-Agent System for IoT The simulation and the emulation are instruments of great importance in the context of the IoT. The simulation of IoT offers highly developed intelligence and further insights into product behavior. Thanks to simulation, companies better understand possible practical situations, as early as the design phase. They can, for example, identify product defects at an early stage. If access to comprehensive historical data is limited, companies can use the simulation models to generate different expected results or product performances by design. The simulation results serve as a source of data that can be used for monitored machine learning and predictive models. The permanent connection between the real world’s behavior and the simulation and machine learning results makes the data valuable. This gives companies predictive models and informative feedback loops that they can use to improve product design and modeling (Table 2). Table 2 IoT hardware and software simulators No. Name

Description

1

MATLAB

This powerful and well-known software has an Internet of Things module that allows people to create and test smart devices and collect and analyze Internet of Things data in the cloud

2

NS3

Another very popular emulator on the Internet of Things is NS3. This simulator is designed to support wireless sensor networks and the Internet of Things. One of the significant advantages of the mentioned simulator is a specific graphical and functional environment, making learning and simulation much easier

3

Contiki Cooja

This simulator is designed to support wireless sensor networks and the Internet of Things

4

Cup Carbon

This simulator has recently become very popular and offers significant features. This simulator is designed specifically for the Internet of Things and wireless sensor networks

5

Cisco packet tracer

Cisco packet tracer is a simulation tool that can simulate smart devices, components, sensors, drives

6

ANSYS IoT Simulator This IoT simulator helps ideators develop Internet of Things devices, networks, and test code

7

OMNeT++

OMNET simulator software is one of the most potent and popular computer network simulation tools

8

NetSim

NetSim has been one of the most popular simulators in computer network modeling and developers can simulate IoT systems. NetSim can be applied to predict how the respective network will perform

9

Fritzing

This software is not an IoT simulator, but considerable experience and knowledge of microcontrollers and electronic equipment can be achieved through it

5 The Role of Multi-agent Systems in IoT

103

8 Security in Multi-Agent System Based on IoT The main objectives IoT-security is confidentiality, integrity, and availability. When applying the security of such systems, the properties of communication between agents become a security-critical. Security issues have a key factor in the development of multi-agent systems based on IoT. The competition between the agents involved creates corresponding security needs for those involved concerning exchanging information between agents and between agents and the agent platform. Security requirements for the overall system must be improved, from one hand security requirements on the individual agents should be taken into account in addition, security requirements for the negotiation process, and communication between the agents should be considered. In a research [9], security issues including possible attacks on IoT agents and MAS were identified. The problem regarding agent authentication, authorization, and issues with the protection of agents from their hosts and vice versa was also alerted. In addition, there are out several security problems, such as verification of information that agents collect from the Internet, unauthorized access to agents, intrusions to MAS caused by agents, insecure communications among agents as well as between agents and humans, attacks from mobile agents to their hosts and attacks from hosts to the mobile agents, attacks on agents from malicious agents, attacks from users. For tackling security issues, security requirements to design all hardware and software agents should be considered in details. The general security requirements for the overall system should be derived from the system users’ various security interests and then reduce these to local security requirements for the individual agents. With the advancement of artificial intelligence technology and machine learning in event analysis and tracking, new solutions have been proposed to maintain the security of IoT-based systems. Cyber security systems that take artificial intelligence into their solution will help in the future to detect intelligent hackers and their attacks. With the help of AI, the detection rate of attacks in the networks, agents, and end devices in the IoT can be significantly increased. More powerful machine learning models learn from examples and can generalize them after the learning phase has ended. A statistical model based on training data is created for this purpose. That means: AI enables the early detection of potential cyber attacks. As multi-agent systems add intelligence to all sectors of IoT solutions, so using this method for the security of multi-agent systems based on the IoT can be very effective and useful. In a research [10] presented that the protected computing approach is another method for security issues which is based on the division of code into two or more parts. Some of these parts will be executed in a trusted processor, but the others will be implemented in a regular processor. These divisions are performed so that the execution of the application is not possible without the collaboration of the trusted processor. One of the most important aspects of this technique is how the code is divided.

104

M. Gheysari and M. S. S. Tehrani

9 The Use of Multi-Agent Systems in IoT Applications In this section, we address the use of multi-agent systems in IoT applications. For this purpose, we will describe the following solutions: • • • • •

Smart city solutions Smart home Smart healthcare Industrial Internet of Things Smart grids

9.1 Multi-Agent Systems in IoT-Based Smart City When it comes to smart cities, aspects such as sustainability, quality of life and climate protection are in the foreground. Many municipalities in the world have already adopted comprehensive digitization strategies. They are now striving for solutions that from one hand, accelerate internal administrative processes and on the other hand, offer long-term advantages and benefits to citizens, industry, and commerce. The IoT in particular plays an essential role in making smart cities a reality, in a smart city, cars communicate with houses, houses with digital devices, and these in turn communicate with the residents of the city. An essential part of the application of IoT in the smart city is sensors that collect data such as the current traffic volume, and digital platforms on which this data is collected and meaningfully linked and analyzed. For example, three typical services and processes within the urban infrastructure are: the regulation of road traffic, area-wide urban lighting, and smooth waste management. In all three cases, experts cooperate with city decision-makers to implement digital and IoT-based solutions that optimize the respective process and establish a soft, environmentally friendly, and safe process. (a)

An IoT-based Multi-agent Approach for Smart Parking

Parking is currently a real problem in major urban centers and drivers have to spend a long-time looking for a parking space almost everywhere. The cities’ parking guidance systems are not enough because most of them use date from the previous minutes. Smart parking is a system that helps the driver find the nearer empty parking space for their vehicles. The smart parking solution is not just for reducing traffic but making it easier for drivers to quickly locate existing ones such that time spent on the road is significantly reduced. Cellular connection, mobile, and web applications based on sensors have emerged for parking management. The IoT device, including sensors and microcontrollers, is located in each parking place and by using sensors

5 The Role of Multi-agent Systems in IoT

105

that detect whether space is empty or occupied. Data will transfer to servers and is displayed to the user using a web application. The key factor in smart parking is “sensor-based parking management.” Sensors on the floor or scaffolding recognize that an occupied parking space is free and transmit this information to software. The software evaluates the data and makes it available to drivers in real time via the Internet. A smart parking system with a combination of different technologies can include these components: • • • •

Showing empty parking spot Online navigation Payment system And even car identification

Designing an IoT-based multi-agent approach for smart parking requires the use of the following agents. These agents as shown in Fig. 5 must interact with each other to cover the three characteristics mentioned above: • Hardware agent: It is included mainly of connected devices such as ultrasonic sensors, implantable sensors in the ground, RFID tags, and cameras. These agents are responsible for checking if the parking is full and send the time of parking to the processing system. Ultrasonic sensors are mounted on the ceiling or wall to measure the distance by emitting ultrasonic sound waves and cameras to capture the parking environment. All these data send to the processing system by a communication agent to integrate into other systems and provide more services. • Processing agent: This agent consists of software and hardware agent. The hardware part includes a microcontroller or microprocessor and the software part consists of the programming language and user interface. By analyzing the

Fig. 5 An IoT-based multi-agent diagram for smart parking

106

M. Gheysari and M. S. S. Tehrani

measured distance or the images taken, this agent detects that the parking spot is empty or full and sends this information to the platform or web application. • Connectivity agent: This agent helps to the data transfer between sensors and the cloud, and it depends on smart parking system implemented on the side of the street or in a building. It can be short-range to long-range communication. • Platform agent: This agent must integrate all data in order to show the route to drivers for finding nearer parking space and also calculate the cost based on the time duration which the car has been parked. Electronic payment is also part of smart parking with network support. (b)

An IoT-based Multi-agent Approach for Smart Waste Management

Waste collection is an important part of waste management and an essential service for every city. The current facilities could be improved. Companies and municipalities are therefore developing more efficient ways of disposing of waste. Waste collection based on fixed schedules and routes proves to be a significant inefficiency in waste management today. With this static planning model, there is an appointed emptying day for each waste container and emptying takes place along a route that hardly changes from week to week. This means that containers will be emptied whether they are full or not. The Internet of Things can help to meet the sustainability goals of cities. At the same time, operating costs are reduced. With the smart waste management system, it is easier to manage and collect municipal waste in cities. By attaching low-energy, long-life sensors to traditional waste containers, it is possible to control the container’s condition precisely. Urban waste can be seen as one resource in the entire processing chain. A wireless IoT waste sensor that can install in the waste bins and continuously monitors them with ultrasonic sonar technology and transfers the data to the cloud will address this problem and the MAS approach helps to have a fully intelligent and integrated waste management system. This system is included three parts: • Waste container • Collector trucks • Operator For an IoT MAS-based waste management system, we have environment and agents: • Trash can agent which is a combination of hardware and processing agent. It is located on the top of the waste container and has specific characteristics such as capacity, size, state, and space position. • Location agent can show the position of the waste container and used it in a routing system for collector trucks. • The connectivity agent, in this case, should support GSMA or LPWA network because waste containers are scattered throughout the city. • By using platform agents, the integration of routing system, operators, and collector trucks will be easier.

5 The Role of Multi-agent Systems in IoT

107

In this system, each trash can agent wakes up every 30 min and measure distance by emitting ultrasonic sound waves. These data send by a connectivity agent (a standalone module like sim-card or an IoT gateway) to cloud server. Collector trucks move around the city, and it follows the path in a way that is showing by the routing system. When data received in the IoT platform, agent platforms’ task (decision and intelligence agent) begins. These agents must analyze data and prioritize trash cans on the threshold of filling or have been filled. The amount of trashes, trash can location, and collector trucks’ locations greatly impact this prioritization. These agents must record the location of each trash can in the GIS system based on these parameters. After that, the routing system must navigate the truck driver to consider trash cans to dispose of waste.

9.2 Smart Home Based on Multi-Agent System Smart devices in the house that communicate with each other make our everyday lives easier: When we leave a room, the lights, television, and heating switch off automatically. If it gets too cold, the heating switches on again. When we come home, the garage door opens. When we leave the house, the doors lock automatically and the alarm system is activated. This would enable us to save electricity, energy, and heating costs in everyday life. Besides, comfort and security are to be improved in the smart home. In a smart home, modern technology ensures that everything works the way you want. Smart applications automatically generate a personal feel-good atmosphere through light and temperature. An intelligent home pays attention to you and your loved ones through vigilant sensors and offers entertainment throughout the house. Because this smart technology is based on you, not the other way around. Some people want the greatest possible comfort when controlling lights, heating, shutters; others wish to more security and rely on motion and presence detectors or value windows that close automatically depending on the wind and weather. The desire to use energy as efficiently as possible or simply enjoying innovative technology can motivate people to inquire about smart home solutions. A smart home based on multi-agent system consists of three different parts: • House: which is known as environment • House owner: which is known as user • Smart home system which is a hardware and software system All these parts with several agents can interact, and at the top of layer, the user can control the house by using an integrated hardware and software system. Figure 6 shows the relation of agents in smart home scenario. • Hardware is a group of sensors, actuators, and even smart devices (built-in sensor, actuator, and microprocessor) installed in different parts of the house. Sensors

108

M. Gheysari and M. S. S. Tehrani

Fig. 6 Smart home based on multi-agent system

like motion detection, fire detection, temperature, camera, etc., monitor all events that happen in home. Sensors belong to monitoring part and actuators belong to the operational part. Actuators are activated based on data from sensors and commands to take action. • A smart home, until all hardware agents are on the same network user, can control the system by a web or mobile application, and this network is created just by a home gateway2 . Home gateway by using short-range communication protocols such as ZigBee, Z-wave, Wi-Fi or even Bluetooth creates a smart home network. Data is sent from sensor to gateway by mentioned connection protocol and sent from the gateway to cloud by Wi-Fi. • By placing a processing module in the IoT home gateway, the data’s sensor can be analyzed at the smart home network’s edge. In this scenario, this device can act as a communication agent, learning agent, and decision agent. After analyzing data, results are placed on the cloud and the necessary commands are sent automatically (by the decision support system) or manually (by the user and app) to the gateway. The gateway activates actuators, and then they remain in the same state until the sensors record a state change.

9.3 A Multi-agent Framework for IIoT The Industrial Internet of Things (IIoT) describes intelligent machines or systems which have been digitally networked, in an industrial context, especially in manufacturing. The goal is efficient, self-organized production in which machines, systems, products, and people communicate and cooperate with each other. This networking is intended to optimize the company’s entire value chain, which ranges from the idea 2

Smart hub.

5 The Role of Multi-agent Systems in IoT

109

of a product to development, and also covers production, marketing and sales, use, maintenance, and recycling. A multi-agent framework for IIoT consists of three major parts [11]: • measuring quality metrics of a machine by a different sensor and comparing them with minimum and maximum thresholds • using ML algorithm for detection of abnormal regression of the MAS quality and warning the maintainer • The intervention of the maintainer for preserving the application quality and thus avoid potential damage. In predictive maintenance, the actual condition of plants is the main factor. This approach is used to prevent downtime before it occurs. This helps to avoid inefficient maintenance routines and the associated costs and optimize the entire manufacturing process. The concept for predictive maintenance should always be planned long-term and comprehensively. In this approach: • The systems are equipped with sensors that record various parameters, such as temperature, vibration and noise, move, motor speed, liquid levels (oil, coolant, fuel, and so on), emissions, moisture and gas, concentration, radiation, position, amperage and charge, in real time. Various sensors on the machine monitor essential areas and the data recorded by them are filtered, prepared, and stored in the cloud. In this arrangement, the sensor information is sent to a central maintenance service in the cloud via a cellular or local network. However, in order to be able to reliably and seamlessly monitor the machines even when a constant real-time connection to the Internet is not guaranteed, part of the intelligence required for this has been shifted from the Internet directly to the communication gateway on the machine. This enables the data from the machine to be processed directly on site, depending on the application, and important partial analyzes to be carried out without transmission to the central IoT cloud. • This approach recognizes irregularities very early with the help of online analysis processes and sounds the alarm. The system can then recognize a downward trend in the future and warn employees, in some cases, even suggest an appointment for maintenance on its own. Therefore, in the ideal case, maintenance measures are initiated before there is a reduction in performance or deterioration of the devices and systems. • ML algorithms are able to uncover the behavioral patterns of systems that ultimately lead to failure in large amounts of data or even mountains of data. The patterns revealed are reflected in the predictive models. The models are trained based on the relevant new input data, using test data, and approved. Only with the approved models are predictions generated and recommendations for action given. These models will also be further improved and optimized. Figure 7 shows a multi-agent framework for IIoT.

110

M. Gheysari and M. S. S. Tehrani

Fig. 7 A multi-agent framework for IIoT

9.4 Smart Healthcare Based on Multi-agent System By intelligently collecting data on device performance and patient health, workflows can be optimized and the quality of care improved. The Internet of Medical Things (IoMT) can revolutionize health care in near future. One of the most popular systems in IoMT is Ambient Assisted Living. All is about monitoring and caring for the elderly and the sick in the home environment. The Internet of Things is also intended to help monitor patient compliance: 1. 2.

Control/operation of devices (controlling/operating) Monitoring (a) (b)

3.

with information, warning, proactive with alarm in the event of a fault, reactive

Location tracking (tracking)

5 The Role of Multi-agent Systems in IoT

111

The smart healthcare scenarios based on IoT and MAS create a body area network (BAN). Technological progress in the fields of wireless communication and the entire semiconductor industry has a significant influence on the field of sensor networks, which are used in a multitude of different applications. BAN stands for a new wireless transmission technology. This enables the connection of medical sensors and actuators worn on the body and transmitting the data to the medical staff. Since the human body is a highly sensitive area concerning radio radiation, the requirements for the radio sensors’ power consumption are particularly high. They must meet an average power consumption of less than 100 µW. The solution is ultra-broadband (UWB), and with slightly higher radiation, Bluetooth Low Energy (BLE) and ANT+. The data captured by sensors can help healthcare professionals gain a faster and more accurate overview of critical situations, and provide patients with more information about their condition and recovery. In IoMT, most of the sensors, especially in the non-clinical sector send the data directly to mobile phone applications by short-range connectivity agents. This optimizes the devices’ power consumption, and then the data is sent to the cloud by the mobile application. Due to the importance of time and the need for real-time processing, agents and cloud spaces cannot be effectively lonely and a real-time processing system is needed in health-oriented scenarios. IoT devices usually have a second “intelligent” part—namely the distributed “intelligence” that is located outside the device and normally in IoT gateways. This second part can analyze the information of the events monitored by IoT devices and has access to (analyzed) data. Edge computing and also mobile Edge computing are one of the interesting technologies for the health sector because it allows, among other things, the monitoring of patient information in real time. The task of edge computing in IoT smart environments is to do the calculations at the edge of the network, i.e., close to the IoT devices, for downstream data on behalf of cloud services and upstream data on behalf of IoT devices to be carried out. The resource-intensive training of algorithms can be carried out in the cloud layer and then distributed to the edge, where lighter functions are outsourced. Like other IoT-based solutions, MIoT based on multi-agent systems has similar hardware, software, and communication agent, but accuracy, coordination, and reliability for existing agent in this approach make it more important than other solutions. These agents must handle the relationship and information between patient, devices, doctors, nurses, and patient family. Therefore, their task is more sensitive than the tasks of agents in other solutions.

9.5 Smart Grid Based on Multi-agent System A smart grid is nothing more than a network of electricity suppliers which managed via a system of digitally controlled interfaces. In addition, the current flow and power supply are dynamically adjusted and thus react to required changes in the micro- and macrorange.

112

M. Gheysari and M. S. S. Tehrani

The architecture of this complex network is based on a layer model in blew: • The physical layer consists of the electricity generating systems, transmitters, substations, distributors, and energy consumers. • The network layer is on the physical layer and consists of a communication and network platform as well as a network gateway, head units, and smart meters. • An analysis layer is used for the subsequent value creation and uses the generated data at the application level. • The application layer is built above the network layer and automates core functions such as transmission, distribution, and network stability as such through various energy management systems. With the increase in grid-connected installations of solar modules, grid management has become even more complex. At the end of the line are the actual electricity and power generation companies. Traditionally, proprietary and vertically specific software from specialist providers is used here. The main component of a smart grid is: • Smart nodes with features such as: (1) hierarchical, decentralized structure, (2) direct communication between different actors, (3) collective decision making, (4) the power grid as a distributed system, (5) local sensors, (6) broadband connections to neighbors, (7) react independently to changes in their environment, (8) integrate independently into existing networks, and (9) act largely as autonomous units. • Microgrid with features such as: (10) dynamic cells from several intelligent nodes, (11) distributed electricity generators (e.g., solar cells, geothermal energy), (12) energy storage (e.g., heat storage) consumer, (13) dynamic regulation of electricity production and consumption, and (14) appear to the outside world as a single unit. • Smart consumer with features such as: (15) active participation in the power grid, (16) real-time information on consumption and network status, e.g., utilization, price, available providers, (17) smart decisions when it comes to electricity consumption, (18) postponing non-critical consumption to a later point in time, (19) preferred providers, and (20) feeding recovered energy back into the power grid. And based on characters that mentioned above, control of a microgrid by a multiagent system must have include these agents: • Control agent which – Monitors voltage and frequency – Can disconnect the micro grid from the main network in the event of a problem – Contact person for external communication

5 The Role of Multi-agent Systems in IoT

113

• Generator agent Represents a power source Controls the production depending on the energy demand in the micro grid • Platform agent Stores global system information Is considered a known point of contact for other agents • Consumer agent; smart meters Controls the consumption of various components depending on the network status Sets rules and priorities for the automatic control of consumers

10 Conclusion The development of intelligent agents in the Internet of Things is only just beginning today. The agent can continuously monitor their user, recognize changes in their choices and situations almost in real time, and orient himself accordingly. Should agent technology achieve broad acceptance and spread, a complex multiagent system on the Internet is achievable. Agents could communicate with one another and exchange information using a standardized agent language. There are many other IoT applications which can implement better by multi-agent concept, and in this chapter, some of them discussed. Future agent developments will have to combine several properties with meeting the high theoretical demands to a greater extent. We hope that this article can show researchers the IoT multi-agent-based system roadmap for future research.

References 1. Pico-Valencia1 P, Holgado-Terriza JA (2018) Int J Distrib Sensor Netw 14 2. Forestiero A (2017) Multi agent recommendation system in internet of things. In: IEEE/ACM international symposium on cluster, cloud and grid computing 3. Carabelea C, Boissier O, Ramparany F (2003) Springer, Berlin 4. Muccini H, Moghaddam MT (2018) IoT architectural styles. Software architecture book. Springer, pp 68–85 5. Adegbija T, Rogacs A, Patel C, Gordon-Ross A (2018) Microprocessor optimizations for the internet of things: a survey. IEEE transactions on computer-aided design of integrated circuits and systems, vol 37, Issue 1 6. Ray PP (2018) A survey on internet of things architectures. J King Saud Univ Comput Inform Sci 30(3) 7. Jurcut A, Niculcea T, Ranaweera P, Le-Khac N-A (2020) Security considerations for internet of things: a survey. SN Computer Science 8. Xie J, Liu C-C (2017) Multi agent systems and their applications. J Int Council Electr Eng

114

M. Gheysari and M. S. S. Tehrani

9. Hedin Y, Moradian E (2015) Security in multi agent systems. Procedia Comput Sci 60 10. Munoz A, Anton P, Mana A (2011) Advances in software engineering. Hindawi Publishing Corporation 11. Ghrieb N, Mokhati F, Ghorab MA, Guerram T (2019) Towards a preventive maintenance approach for multi agent applications. Multi Agent Grid Syst 16(1)

Chapter 6

Agent-Based Human-Supportive Applications Abir Kumar Bardhan and Abeer Alsadoon

Abstract One of the most intriguing breakthroughs in computer science is the use of agents. It has aided in increasing the automation quotient in the systems in which it has been implemented. Because of the broad extent of implementation and the impact on human civilization, research on human-supported applications that include automated and interactive agents has received increased attention. Various human-supportive applications, such as supply chain management, expert system design, workflow and business process management, process control management and production management are extremely important in a business’s workflow. And agents are demonstrating their incredible powers in various applications, which not only eliminate the complexity of manual control but also the ongoing human engagement. This chapter discusses the applications that have been deployed with the assistance of agents, as well as the advances they have brought to the table. It also sheds insight into recent technological breakthroughs in this sector. Keywords Agent · Interacting agent · Human-supportive application · Supply chain management

1 Introduction and Literature Review Technology based on software agents is amongst the most rapidly growing technologies. If we try to find the definition of agents, there are differences in pre- and post-AI era. At the earlier days of AI, the definition of agent was precisely defined in Carl Hewitt’s concurrent actor model. Here the agent was proposed as self-sufficient, correlative, and simultaneously executing object which Hewitt termed as “actor”. In addition to their own internal states, these objects can respond to messages sent by other objects of the same type. In the practical field, the agent is considered as a component of software or hardware that performs operation in order to accomplish A. K. Bardhan (B) Department of Computer Science and Engineering, UIT, Bardhaman, West Bengal 713104, India A. Alsadoon School of Computing and Mathematics, Charles Sturt University, Sydney, Australia © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 S. Gupta et al. (eds.), Multi Agent Systems, Springer Tracts in Human-Centered Computing, https://doi.org/10.1007/978-981-19-0493-6_6

115

116

A. K. Bardhan and A. Alsadoon

task on behalf of the user. There are several paradigms in which the existing agents can be classified.

1.1 Classification We can classify agents based on their mobility. Static agents and mobile agents both can be classified as agents. Second, they may be divided into two types: deliberate and reactive. To achieve their objectives, deliberate agents develop their own internal symbolic thinking model and plan and negotiate with other agents. Reactive agents, on the other hand, lack a symbolic model and operate in response to the current condition of the environment in which they are positioned. Third, agents can be classified based on a set of characteristics that an agent should have. Autonomy, learning and collaboration are the traits examined for categorization at BT laboratories. Autonomy refers to an agent’s ability to function independently without human intervention. Learning entails anticipating and reacting to changes in the environment. Co-operation entails communication among agents for them to reach an agreement in order to achieve their objectives. Agents are grouped into three types based on these three characteristics: collaborative agents, smart agents, and interface agents. Fourth, agents can be categorized based on their responsibilities, such as information collecting agents. These information agents can either be deliberative or reactive, static or mobile. Fifth, hybrid agents can be created by combining many types of agents outlined above. But in practical implementation, agents possess multi-dimensional features. So, combining all these dimensions, we can additively classify agents in seven types mentioned and described below.

1.1.1

Collaborative Agents

In a time-constrained open multi-agent environment, this type of agent stresses on autonomy and collaboration with other agents in order to complete the duties of related users. These agents may be able to learn from their surroundings; however, this feature is not critical for implementation. They also bargain in order to obtain an agreeable agreement. Most of the agents are static, massive, and coarse-grained components. These agents are classified. As a result, collaborative agents are used to build distributed systems that are far more competent than a single agent. Researches on collaborative agents aim to:

6 Agent-Based Human-Supportive Applications

(a) (b) (c)

117

Solve the problems which are too large to be handled by a single agent. Allow the communication and coordination with the existing legacy systems. Provide solution to problems which are inherently distributed.

Potential benefits from this research can induce modularity, reliability and speed. The Pleiades system is a nice example of prototype that implements the concept of collaborative agents.

1.1.2

Interface Agents

This sort of agent emphasizes autonomy and learning in order to complete the duties assigned to it by its masters. They assist users and those who are learning how to employ this application. It watches the user’s activity in the interface, learns new tricks and tactics, and recommends even better ways to complete tasks. In general, it may be accomplished in four ways: (i) by monitoring and copying the user, (ii) through feedback from the user, (iii) through explicit instructions supplied by users and (iv) by seeking recommendations from other agents in the environment. There are two types of learning modes: rote memory and parametric learning. The only way to communicate with agents is to ask for guidance. The advantages gained from the use of interface agents are highly tangible. It decreases the workload of the application developer while also making the user’s activity easier by responding to the user’s preferences and habits. Because of their simple structure and lack of collaboration requirements, interface agents are obvious workhorses in real-world applications.

1.1.3

Mobile Agents

Mobile agents are software components that can travel the network to connect with foreign hosts, do specified tasks, and then return to the base host. These agents are autonomous in nature and capable of cooperating, but they are not the same as collaborative agents. Although mobile agent applications are not extensively used today, their use is expected to skyrocket soon. Sony’s Magic Link PDA, or personal intelligent communicator, was the first commercial application to incorporate mobile agents. It was used to handle email, fax, phone, and pagers, as well as to connect the user to Tele Scriptenabled message and communication services such as America Online and AT&T persona Link Services.

1.1.4

Information/Internet Agents

Information agents oversee all information activities, whether managing or modifying it. It is not clear whether information agents should be mobile or static. The

118

A. K. Bardhan and A. Alsadoon

static ones are embedded in the online browser and gather information using various Internet management tools such as spiders and search engines. Since there is such a variety and volume of data available on the Internet, updating the indexes of static information agents is nearly impossible. As a result, most of them are turned into mobile agents.

1.1.5

Reactive Agents

Since reactive agents do not possess internal symbolic representations of their respective environments, they approach their behaviour by responding to the current state of their environment. Scientists identified three fundamental properties that distinguish reactive agents. First, consider emergent functionality: these agents are simple in structure, and their interaction with other agents is simple. The behaviour of the agents is not predetermined. The next approach is task decomposition: a reactive agent is a collection of modules that perform the tasks that are assigned to them autonomously. In nature, communication between modules is relatively limited and low levelled. Third, reactive agents like to deal with information in a form that is closer to raw sensor data, whereas other types of agents prefer to work with information that is represented in a high-level symbolic form. Additionally, conventional AI systems are rigid, slow responding, and fragile, whereas ML-based systems are flexible, quick responding, and customizable. There are few reactive software agent-based applications [1], owing to the lack of a uniform mode of operation and the agent architecture selected. The most fundamental architecture is one based on situated action principles. In this case, a situated action agent would respond in a way that seems “appropriate” in the given scenario. As a term, “situation” refers to a complex interplay of inner and outer circumstances.

1.1.6

Hybrid Agents

So far, we have spoken about five different sorts of agents: collaborative, interface, mobile, information, and reactive. Each of these agents has their own set of strengths and weaknesses. The simplest technique to achieve our aim of enhancing strengths and diminishing weaknesses is to mash up accessible agents. In such cases, the reactive half of the hybrid agent takes over from the deliberative half, which enhances robustness, reaction times, and adaptability. The deliberative half, on the other hand, deals with long-term goal-oriented difficulties. Layered topologies are common in hybrid agent designs. The InteRRaP architecture proposed by Muller et al. consists of three layers: BBL, LPL and CPL. The behaviour-based layer (BBL) comprises a collection of situation action rules and implements the framework’s reactive component. This layer oversees quick scenario identification in order to respond to time-critical circumstances. The layer that executes local goal-directed behaviour is the intermediate local planning layer

6 Agent-Based Human-Supportive Applications

119

(LPL). The top layer cooperative planning layer (CPL) coordinates collaboration with other agents to accomplish multi-agent architecture and resolves disputes among them. BBL operates with world, LPL operates with mental model, and CPL operates with social models of the knowledge base of the agent.

1.1.7

Heterogeneous Agents

Although this sort of agent includes various types of agents, it differs somewhat from hybrid agents. It is a combined configuration of two or more agents from distinct classes. A hybrid agent can be a component of a heterogeneous agent. The ability to communicate among agents of different classes is a critical need for this type of system. In order to serve this purpose, a language capable of agent-independent communication (ACL) is needed. Agents can select between two architectures: one in which agents manage their own coordination mechanism and one in which coordination is handled by a specific system application. The downside of the first architecture is that scalability in coordination has not been proven. As a result, the latter strategy is preferable. Communication takes place through a facilitator.

1.2 Multi-Agent System A multi-agent system is a collection of heterogeneous, computationally capable agents that can solve problems and interact with one another to reach the problem’s solution goal. Negotiation is quite vital in achieving its purpose. To carry out the negotiation [2], the agents interact with one another using Agent Communication Languages such as KQML, FIPA and others. All this technology is assisting many systems in improving their efficiency, productivity, and cost effectiveness while reducing human effort. Because of the incorporation of agent [3] technologies in this industry, this category of applications known as human-supported applications is acquiring a lot of benefits. In this section, we will go through in detail a few of the human-supportive applications where multi-agent systems [4] have had a significant influence.

2 Supply Chain Management Through a supply chain [5], raw materials are obtained and processed, and products are produced and delivered to customers through a network of suppliers, manufacturing plants [6], warehouses, distribution centres, and retailers. The supply chain

120

A. K. Bardhan and A. Alsadoon

management system (SCMS) executes the coordinations between the system components that control the involved processes at each step of supply chain [7]. In the computational world, each of these system components is implemented as a distinct agent. We can say a SCMS gets converted into a multi-agent system (MAS) where the agents cooperate with each other for uninterrupted activity of the system. But the agents must cooperate in a relatively dynamic fashion. That is because of two reasons. First, the entities involved in different stages of the supply chain have their own interests while interacting with others. Agents cannot frame this collaboration as purely scheduling because of their self-interest. Second, in a real business environment, entity can join or exit a supply chain [8] on their own decision, and it is impossible to predict the existence time period of that entity. To overcome these issues, a negotiation-based MAS framework is proposed. Here, the relationships among the agents are not predefined. Whenever a new order arrives, the virtual supply chain emerges automatically and executes the order. Through this model, we attack four general issues of supply chain: (a)

(b)

(c)

(d)

Communication: We need to have a common language among the negotiating entities [2] so that a meaningful communication happens among them. We also need to specify the protocols for negotiating. Representation: The objects on which negotiation occurs are mostly complex (abstract or physical). So, we need a distinguishable representation scheme for the objects. Problem solving: Many aspects of the negotiation are eligible to be represented as an exercise in distributed constraint solving. Various algorithms and techniques available for distributed constraint solving can be applied on negotiation problem. Human involvement: Negotiation must be carried in the context of human organization. The automated negotiation process must be coupled with human in both authorization and modification of supply chain and working as a part of larger workflow environment.

A software agent who can take communicating and problem-solving actions in specified domains is called a negotiating agent. Negotiating agents are of two types— functional agents and information agents. Functional agents are the ones who implement different functionalities required to be taken in that stage of supply chain. These agents are usually owned by the participating companies, and they can join or leave the SCMS at any time and can stay in the system if they want. But those actions should be informed to the information agents. Functional agents use Agent Communication Language (ACL) to communicate with each other and understand system ontology (knowledge about the product and the system is negotiating). They can be autonomous or semi-autonomous. The whole negotiation process is realized as cooperatively assigning values to a set of variables. Information agents are the ones who are predefined in the system, and they possess information which is helpful for the functional agents to find the negotiating partners. They can also interact with outside world. However, there is not any super-agent or distributed mediator to observe and

6 Agent-Based Human-Supportive Applications

121

handle the agent cooperation. A system is “dead” if either there are no agents registered in information agents or it is not possible to create a supply chain in high percentage when customer order arrives.

2.1 Performatives and Protocols for Negotiation Communication among independent software agents in a cooperative system is based on the Agent Communication Language (ACL), a language with well-defined syntax, semantics, and pragmatics. Functional agents use ACL to communicate in MAS for SCMS. One of the most widely used ACLs is Knowledge Query and Manipulation Language (KQML). The Foundation for Intelligent Physical Agents also defines an ACL. The reasons behind their vast popularity lie in the fact that they both have minimal set of performatives for defining agent actions and the ability to allow users to extend this set provided that the new performatives conform to ACL syntax and semantics. The KQML standard has no predefined actions for agent negotiations; the FIPA ACL standard has a few, but they are not sufficient for supply chain management. So, we design new negotiation performatives set specifically for the supply chain management.

2.2 Criteria for Performative Definition and Selection (a) (b)

Compatibility with existing performatives: In both KQML and FIPA ACL, the process of adding performatives to the existing set is almost similar. Interpreting new performatives based on negotiation protocol: It is a very difficult job to add performatives whose names can be derived from human languages. It is extremely hard to define agent behaviours without the knowledge of negotiation environment because of unique agent actions in each scenario.

A negotiation protocol is used to maintain the sequence of messages among the agents. In a MAS for supply chain management [9], very limited no. of negotiation protocols is allowed. That is why, only a specific negotiation protocol is used to describe the probable responses of the agents, and negotiation performatives are defined according to that. Use of formal tools like CPN for modelling the protocol makes it easy to verify the sufficiency of the performatives set. Even though agent technologies have done significant improvements in the supply chain management, the work boundaries of agents are still limited. The agents coordinate with each other, so that they meet all the constraints and reach their goal. But the real-life situation in a supply chain is too unpredictable to stick on a set of permanent rules. Human intervention is required to modify the constraints when it is needed. Also, research is required to understand the nature of convergence of the

122

A. K. Bardhan and A. Alsadoon

negotiating agents in order to minimize negotiation time and more fruitful decisions to solve a problem.

3 Expert System Design Expert system is a computer programme where the programme is capable of responding like a human or an organization to a problem. The programme simulates the judgements and behaviours of a human or an organization that possesses the knowledge, expertise, and experience in that field. It resembles the interactions between a user and a human expert for solving the problem the user is facing. Here, to find the solution of the problem, the end-user gives answer to a set of questions asked by the system, either by writing the answers or selecting from the list of answers provided by the system. The system takes those inputs and provides a solution based on its knowledge of task and set of procedures to use that knowledge for process scheduling. Both the knowledge and set of procedures are procured from professional experts in that genre. That is why, these expert systems [10] are typically very domain specific.

3.1 Structure An expert system is a problem-solving and decision-making system where decisionmaking is a complex process. These complex decisions are taken based on both heuristic and factual knowledge. This knowledge is stored in an easily understandable and accessible format so that the system can easily acquire and effectively use the heuristic knowledge to find solutions of the problems faced by the users. The format distinguishes among data, knowledge, and control structures. To achieve this, the systems are structured in three levels: (a)

(b) (c)

Knowledge base: consists of problem-solving rules, set of procedures to find solutions, and the data related to the problem domain. It is the nucleus of the expert system structure but it is not a regular database. It is created by the knowledge engineers. It is usually stored in terms of if-else structure and subject to change in case of different problem descriptions. Working memory: comprises the job-specific data for that problem considered. Inference engine: generic control mechanism that takes the knowledge from knowledge base and task-specific data as input and generates a solution or conclusion. It organizes the problem data and goes through the knowledge base to find the applicable rules. With increasing popularity, many commercial inference engines are getting launched in the market.

6 Agent-Based Human-Supportive Applications

123

3.2 Advantages (a) (b) (c) (d) (e) (f) (g) (h) (i)

Can always remain present irrespective of the presence of human experts. Consistency is better than human experts. Capable of explaining details in a simpler yet detailed manner. Cost-effective system. Often faster in giving response as compared to human experts. Can provide expertise on multiple problems. Capable of helping users at multiple places at the same instance of time. Can provide expensive, rare, and clever solutions. Easy to develop, modify, and maintain.

3.3 Application (a) (b) (c) (d) (e) (f)

PUFF: Medical system that analyses respiratory problems and suggests diagnosis for them. PROSPECTOR: Geologists use this tool to find sites for excavation by drilling or mining. MYCIN: Medical system capable of detecting any deformation or defect in blood elements. DESIGN ADVISOR: System that provides guidance to the designers to design processor chips of specification. DENDRAL: System that helps to identify the internal structures of chemical compounds. LITHIAN: System that provides advice to the archaeologists who examines stone tools.

3.4 Problems (a) (b) (c) (d) (e) (f) (g) (h)

Width of application domain is very limited. Systems do not learn; hence, they do not remain up to date always. All the rules and procedures need to be invoked in knowledge base. No scope of using “common sense”. Skilled people are required to set up and maintain the system. Chances of occurring error cannot be eliminated fully. Cannot be blindly trusted. Incapable of refining knowledge base. Might have a high development cost if the application domain is extremely new and unique.

124

A. K. Bardhan and A. Alsadoon

3.5 Latest Tools for ES ES/KERNEL2 is the latest Japanese tool that can build expert systems which are capable of functioning at a very large scale. It is the newest version of best-selling tool for ES. It provides the application developers a slew of choices for reasoning methods: rule-based reasoning, object-oriented reasoning, and assumption-based reasoning. And all of them can be used in a single expert system. But the thing that should be kept in mind is that the knowledge representation scheme should complement the opted reasoning method. This tool provides both graphic and multimedia objects for the business application [11]. It is extremely useable and easily configurable according to the demand of the application domain. It also assists the knowledge engineers in the design and construction of knowledge base. Despite being a fantastic tool already, research is still going on to make it even better, powerful, and efficient.

3.6 Research Aspects (a)

(b)

Distributed Robotics Recent application of distributed robotics is seen in China where they are used to transfer weather condition data from one region to other region of the country. As it is a distributed system, multiple robots are used, and all of them are trained by experts. One robot learns from environment, and then the second robot learns from the first one using ES techniques. The first robot takes input from the environment with the help of camera and sensors. Then it conveys the same to other robots and forecasts the weather. For each observation, the time spent to complete each objective is measured. It is observed that a simple robot takes about 8 minutes to perform this work while an expert system robot takes 6 minutes to execute the same. Learning Agents Building an agent for specific application purpose generally includes the process of construction of knowledge base and customization of agent shell specific to that application. This learning engine will provide the facility of constructing knowledge base by subject matter experts. Here the knowledge engineers will add capability to the engine that will eventually let the subject matter experts and typical computer users construct and maintain the knowledge base and agent in a fashion which is just as intuitive as using a simple PC for text processing or other basic works. This research aims for a revolution in the way a knowledge-based agent is built now. It aspires a transition in the building process, from being coded by knowledge engineers and computer scientists to being trained by subject matter experts and typical computer users.

6 Agent-Based Human-Supportive Applications

(c)

125

High-Performance Computing Research Centre This centre is currently engaged in the development of expert systems that can determine intramuscular fats in cows and pigs. The amount of fat is dependent on various characteristics and factors like breed, kind of food consumed, weather, location, and many more. Although animal scientists are aware of their goals, they cannot find the correlations among these many factors. So, the construction of neural network or fuzzy logic or expert system is planned, and the system is expected to provide guidance for better upbringing of the animal with lesser cost but without compromising on the quality. This institute has also initiated the development of a knowledge base that supports detection and diagnosis and executes research in mammography. This will be accomplished using expert system where the neural network will first execute the identification and modelling; then other factors will be considered along with the mammogram results to compute the accurate result and take the final decision.

The latest research is going on to build such expert systems [12] that are capable of understanding emotions of human, capable of judging the intelligence of a person, and creating new innovations for it. The system is also expected to maintain and update itself periodically. Today’s expert systems are serving in very narrow-spaced domains. The latest research also focuses on building larger knowledge bases as it will help the systems to put their footsteps in broader range of applications to make it faster, efficient with less human involvement.

4 Workflow and Business Process Management System Workflow management system is one of those important technologies in the large business environment. It provides an efficient and systematic way to model and control the complex business processes within and among organizations. This system allows explicit business process definition, responsiveness to sudden change in environment, and is capable of tracking regular operations. But to avail these benefits, the workflow management system is required to integrate with other technologies involved in the process, e.g. resource allocation and activity assignment. In contrary to the formal business management workflow paradigm, the agent-based technology provides flexible, intelligent, and distributed solutions for business process management [13]. The workflow management renovates the office automation area where different kinds of documents need to be digitized and revolved around among the corresponding employees. Before dwelling into benefits the workflow management system brings into the table, let us have a look into its definition. Workflow refers to a computerized process of facilitating or automating a business process.

126

A. K. Bardhan and A. Alsadoon

4.1 Benefits Now, the benefits availed due to incorporating workflow technology into business process management are as follows: (a) (b)

(c) (d) (e) (f)

Explicit definition of business processes allows determination of the responsibilities and coordinating relations. It organizes the business process into a set of modules that can be rearranged. That makes the system capable of reacting quickly to sudden changes in business policies and form a new business process if required. It can keep the track of day-to-day operations and is easy to optimize. It has integrated capability for combining applications across different platforms. It provides personal workplaces. It isolates business logics from the business tasks in a business process. It helps the user to focus on the task without dwelling into the complexity of business process.

4.2 Disadvantages But this generation of workflow management system has its own share of disadvantages: (a)

(b) (c)

(d) (e) (f)

(g)

Single Point of Control: In distributed enterprises, it is not possible to define and control business processes from a central point of control. Independent divisions take decisions for their part of business. No Automation: Although it determines business logics, still most part of functionalities is executed by human. Absence of Reactivity: Business workflow must be predefined, and all possible deviations from it must be taken into consideration. Therefore, the system cannot react to unexpected situations. Lack of Resource Management: It does not have the control of resource management. No Definite Semantics: Regardless of the type of information generated during the process, it is unable to take decisions. No Generic Interface: No generic interface is available in the WFMS for information exchange with other applications. Currently, it is done by API calls. No Scope of Interoperation: These WFMSs are developed by independent vendors, hence not much scope of interoperation.

To overcome all these shortcomings and make the WFMSs more efficient and effective, we incorporate the software agent technology [14] in business process management.

6 Agent-Based Human-Supportive Applications

127

4.3 Use of Agents A software agent is capable of reacting without direct involvement of human or any other agent and possesses control over its internal states and actions. When it comes to distributed business process management, the properties of being autonomous, collaborative, and intelligent are the most important characteristics of software agents. The benefits obtained from the amalgamation of agent technology with business process management are described below: • Distributed Architecture—Agent technology [15] provides the loosely coupled system architecture to integrate the involved distributed business process management systems [16]. • Automation—The feature of autonomous eliminates the human participation to a serious extent. Agents can start a workflow when triggered and can return complex responses to environment changes. • Interaction—The agents also incorporate semantic message exchanging so that the organizations can interact with each other. • Resource Management—Task assignment and resource allocation [17] are taken care of by the agents through the negotiation process. • Explanation of Heterogeneous Systems—Because the agents can be heterogeneous, interactions are carried out via semantic message passing. Using this method is more feasible than using API calls. • Developing Smart Decisions—Learning capabilities in an agent helps the system to take better decisions, though the learning techniques are not matured enough. But this agent-based business process management system is not free of niggles, such as: • Absence of coordination mechanism. • Difficulty in business process optimization. • Hard to keep a track of daily operations.

4.4 Classification The incorporation of agents in workflow management system for a business process management can be classified in two ways: (a)

Managing Workflows with Agents An agent can be incorporated in a workflow management based on this form [18]. All the functionalities are managed by a central workflow engine. An agent is invoked only when that work is being performed. The workflow system takes care of entry and exit of the agents. This form shows the use of concepts of agents done by current WFMSs. Let us have a look on what agents can do in this environment—

128

A. K. Bardhan and A. Alsadoon

• The agents can act as human assistant and provide an interface to the human users. • It gives interface to other applications as well eliminates the use of API calls. It also opens the possibility of designing a generic interface for all the applications. • It can execute scheduled tasks autonomously without human intervention. From a system point of view, the workflow engine controls the actions performed by the agents. Agents do not need to interact with each other unless there is a need to do so, and even if it needs, it is done through workflow engine. Also, the agents in this environment do not need to be “intelligent”. In most commercial products of this type, agent is nothing but a regular piece of software. (b)

Management of Agent-Based Workflows There are multiple agents in it, which makes it a distributed management system. These agents are not dependent on each other, and each of them must perform its own process execution. The process logics are infused in the agents instead of explicitly describing elsewhere. Here, the entire process is isolated between several units within an organization or even across multiple organizations. Therefore, it is not possible to control the business process using a single workflow engine. To overcome this issue, a simple solution is applied, plonking a workflow engine in each unit or organization. The whole business process is run through the interactions among these workflow engines. Here agents take on the responsibilities of inspecting, analysing, integrating, and automating the workflows in a business process. They also interact with each other. They also use their high-level abilities like learning, negotiation to make this system even better. But these abilities are not matured enough to rely on yet. So, all of this translates to the fact that agents in this scenario are much more complex as compared to the agents in the agent-enhanced workflow system. Now, let us have a look on the benefits it brings to the table: • Provides distributed system architecture. • Provides communication methods and languages like KQML and also allows interoperation among heterogeneous workflow systems. • Possesses decision powers to certain extent based on its goals and is capable of executing task without human participation. • Has ability to react to changes in the environment.

4.5 Examples and Scope of Research Although this type of system seems very promising and superior to the one mentioned earlier, there is no commercial system available in the market which implements this methodology. But there are some prototype systems built on this methodology too,

6 Agent-Based Human-Supportive Applications

129

e.g. ADEPT (Jennings et al. 1996), fire flow system (Yan 1999). This area is grabbing quite eyeballs for research. Some of the important sections that need to be explored further are as follows: (a)

(b)

(c) (d) (e)

System Architecture—There are various kinds of system architectures to explore. There are several types of agents, including application agents, supervisor agents, control agents, and admin agents. They carry out the actions of their corresponding users. Communication between the agents is handled by the routing agents. Another type of system architecture is based on mobile agents. These mobile agents represent the work items, and they decide the path an agent would choose. Server agents accept mobile agents at each node. These agents trigger function conditions for each agent and prevent one node from sitting idle. Negotiation—In agent-based business process management systems, the negotiation methods used in multi-agent systems can be applied. They can be classified in theoretical ones, e.g.—game theories, contract nets; and nontheoretical ones, e.g.—knowledge based, model based [19]. Communication language is also an important part of negotiation scheme. Resource Arrangement—Resource management is taken care of by agents by performing negotiation to accomplish the required tasks. Scheduling—It is also done by negotiation. Learning—Deciding about new processes according to changes in business environment. Communication among agents is indeed required for this.

So far, we have seen about the workflow and business management system, and we are quite sure that the dependency of the system on agents will continue to stay because of all the advantages it brings to the table. To make this system even more efficient, we think more research is required for development of a standard communication language. Also, as the agents must interact with each other to find solution of a problem, use of intelligent agents [20] is more pronounced. The learning methods should not only train the agents about its own functionalities but also the behaviour of the agents it is dealing with. That will help the agents to cut down the coordination cost as well. Researches on intelligent user interfaces can also contribute to this application.

5 Process Control Management The process control management system is a system that is concerned with continuous control of the running tasks. The process control systems can be represented in pyramid model. The number of layers and height of each layer can vary from model to model proposed by different authors. Here, we are using a customized model for demonstration purpose.

130

A. K. Bardhan and A. Alsadoon

5.1 Functionalities on Different Layers The following functions are available in each layer of continuous process control systems [21]: (a)

(b)

(c)

Description of Control Instrumentation Layer (Sensors, Actuators) A physical process within this layer is automated, along with the corresponding instrumentation that provides a user interface. Additionally, it is equipped with sensors, transmitters, actuators, etc. Mainly, this layer is concerned with how long the measurement process will last and how values will be estimated in case of failure. Time Determined Horizontal Communication In this layer, continuous signals are sampled and transmitted in a timely manner. Modern automation systems rely on complex computer networks to perform this task. Specialized industrial networks like fieldbus provide solutions to this task ranging from simple bus for sensor/actuator communication to complex networks created by DCS components (Fig. 1). Control Layer, Controllers, and Control Algorithms This layer comprises of time determined sampled control algorithm and programmable logic controllers (PLC). The complex nature of modern

Fig. 1 Structure of complex control systems based on the pyramid model

6 Agent-Based Human-Supportive Applications

(d)

131

processes does not allow simple old school control algorithms to control quality. MAS technology provides that complex approach. Asynchronous Vertical Communication Layer This layer provides communication between the direct control layer and supervisory control hardware. The communication process is asynchronous and does not require matching with real-time requirements. These characteristics allow the use of agents in this layer. Usual types of data transmitted are as follows: To administrate control layer • Records of sampled signal • Information about alarm events To govern control layer • Control algorithms • Control commands • New algorithms to be incorporate

(e)

Supervisory Control Layer This is the top most layer. It consists of the software and hardware which is required by a human to supervise the whole control system. The supervisor monitors the whole system and modifies the algorithms when required. Nowadays, complex control systems provide features like failure detection, remote expert communication link, external database connectivity and many more.

5.2 Implementation of Agents Now, let us have a look on the implementation of agents in various process domains: (a)

(b)

(c)

Separation, Sedimentation, and Transport: Although these processes are executed at different places, they all have one common feature that all of them deal with relocation of materials to some destination. The agent technology [22] provides solutions to these processes by attacking the issues at supervisory layer. Chemical Process and pH Control: These generally are the largest flock of continuous processes which needs automation. These processes are continuous with various degree of batching. As the batching is sequential, the MAS can be implemented for supervising these discrete steps of the complete process. It manages the switching among the batches, distribution of materials, and substrates between the continuous sub-processes. Control of the Thermal Process and Temperature: One of the processes that can be a continuous one is temperature control. Because the control task is executed through a time-determinate continuous control algorithm, the agent technology does the supporting tasks for the system.

132

A. K. Bardhan and A. Alsadoon

(d)

Flow and Pressure Control: Maintaining the flow level and pressure in hydraulics and pneumatics is the classic example of continuous processes. Agent-based control algorithms provide support to these processes by integrating MAS to the industrial grade hardware controllers. Combustion, Energy, and Heavy Industry: The energy industry and heavy industry have many isolated continuous processes which are ideal for application of agent technology. MAS applications are effective for supervision and management of those processes. The most popular approach is the implementation of agent-based supervisory layer. The solutions in this genre are mostly confined to supervisory and monitoring applications at the higher levels of the control pyramid. Environmental and Industrial Biotechnology Process: Both environmental and industrial applications of biotechnology consist of many continuous processes. Agent technology has provided solutions to them.

(e)

(f)

The control systems can be even more automated with the inclusion of reinforcement learning. That will make the agents capable of taking decision in case of sudden changes in the parameters which are continuously monitored for unhindered running of the continuous processes.

6 Production Systems Production systems are one of the most important systems of the business industry. The behaviour of the production system largely depends on the knowledge structure and interaction between resources instead of direct results of predefined and complex functions.

6.1 Approaches Let us have a look on the approaches for reasoning the system interactions: (a)

(b)

Static Approach. This is the conventional approach, and it uses static functions. The functions are arranged in a top-down manner [23]. To do that, the whole system is analysed, main tasks are identified, automation is incorporated as much as possible and tasks are scheduled for parallel execution to possible extent. Dynamic Approach: Agents are the key components in this approach. The system is built up in a bottom-up fashion. The auctions and negotiations among the agents directly schedule the tasks eliminating [24] the requirement of a separate scheduler. It also eliminates the need of production configuration engine for resource allocation [25] (Figs. 2 and 3).

6 Agent-Based Human-Supportive Applications

133

Fig. 2 Basic approaches involved in a production system

Fig. 3 Mechanisms involved in production system management

The process of configuration and then again reconfiguring it is the key step in setting up a production system [26]. Some approaches are proposed in the framework of plant automation based on distributed system (PABADIS). This approach insists dynamic allocation of resources and arranges production in a manner so that we get best flow of products.

134

A. K. Bardhan and A. Alsadoon

6.2 Phases of the Approach 6.2.1

Assessment of Problem

The older production models used to be extremely centralized. That is why they are not very easy to maintain and upgrade with time due to lack of flexibility. Thus, the corresponding system needs adaptable and autonomous control systems so that it supports readjustments of the operation parameters. Multi-agent systems provide solution to these requirements. The MAS for this problem should have adaptive algorithms and should have the ability to define operating nodes or agents.

6.2.2

Generalities in MAS

In a production system, an agent is considered as an intelligent autonomous planner or controller of a unit in that dynamic system [27]. Hence, the multi-agent systems are autonomous, intelligent, parallel, and distributed. It breaks down a complex problem into several sub-problems with relatively less complexity. These sub-problems are closely related. So, coordination among the agents is required so that the individual agents take such local decisions which can lead them to the best global result possible. To serve this purpose, reactive or cognitive agents are considered. These agents are capable of taking decisions for a particular task in the current situation from the very start of execution. That is why they can be considered as “situated” agents.

6.3 Example of a Distributed Manufacturing System Let us showcase a prototype model that is putting the above-mentioned concepts in use. This model is called virtual factory dynamics configuration system [28]. In this tool, the intelligence is distributed on process and product level. It helps software tools to improve the monitoring process and management efficiencies of the dynamic production system. (a)

Characteristics The components work with predetermined suppliers and do not use auctions. We incorporate auctions and coordination mechanism so that the agents can manage the distributed resource assignment issues. Different auction mechanisms are available for use. But a simple agent really struggles to evaluate bids. To solve that, a pair of buyer–seller agent that uses forward-chaining and backward-chaining algorithms for auctions is implemented. The contract protocol is used as the negotiation protocol.

6 Agent-Based Human-Supportive Applications

135

Fig. 4 Configuration of a virtual factory with VFDCS

(b)

Features • • • • •

(c)

Flexible and scalable in configuration Easy to maintain Supports E-commerce and Internet technologies Suits customer-oriented and market-driven paradigm Easy to integrate with existing system (Fig. 4)

Proposed Architecture Case (C): A cognitive agent processes information using mechanisms like case-based reasoning (CBR), knowledge-based system (KBS), game theory (GT), etc. Considering the type of knowledge and processing time, CBR is more effective than a KBS in modelling IADSS. In CBR, different cases are inputted, and the prototype base gets validated thoroughly. That in result makes the genetic or neural classifier tools get trained correctly.

Case (A) and Case (B): A reactive agent built with artificial neural network or genetic algorithm is used to model the functioning entity. They usually process information using mechanisms like stimulus response, self-reinforcement, etc. Based on the problem to be attacked, any of three concepts can be used to structure the agent (Fig. 5). Figure 6 depicts the solution we are using in PABADIS. Each product and process is an agent, and there are communication links among them for interaction.

136

A. K. Bardhan and A. Alsadoon

Fig. 5 Logic structure of three kinds of intelligent agents decision support system (LADSS)

Fig. 6 Dynamic approach based on interactions between cognitive/reactive agents

7 Conclusions So far, we have seen the implementation of agents in different human-supportive application. Agents have accelerated the improvement in human-supportive applications from different genres. Like in the supply chain management, the network of different entities is extremely unpredictable because it is not possible to predict which entity is going to stay in the network for how long. In case of sudden exit of an entity from network, old school centralized supply chain will not be able to decide how to make up for that absence. That is the place where agents have proven

6 Agent-Based Human-Supportive Applications

137

their mettle by intelligently coordinating those information and handling the sudden changes in the situation. The use of agents in expert system has reduced the need of human participation to serious margin. The intelligent agents in them with the help of pre-stored knowledge take decision and provide solutions to the people who really need it. They are fantastic tools as a saviour where human participation is required for continuous times. The incorporation of agents in the workflow management system [29] has made it so efficient and easy to systemize. The diversity and distributive nature of the system is so well organized because of this. The process control management system is also benefited from the use of agents as it increases the degree of automation and reduces human effort for sensitive continuous events. The production management system has also used agents for improved breaking down of a large complex process and solving interrelated sub-problems step by step. Although the application of agents has so many gains, the gains are not fully exploited yet. In most of the human-supportive application, the implementation is mostly distributed and unpredictable. The unavailability of worldwide standard communication language of agents has just notched up the limit. Also, the immaturity of training algorithms to make the agents self-sufficient has robbed away some expertise. Also, the hybrid characteristics of agent for equal participation of human and agent have also made the agents very application specific and put them in a narrow domain. Despite all those boundaries, it is sure that agents are going to stay in this infinite long journey of discovery and improvement of the current systems. With more researches on making the agent even more efficient, new research areas [19] are opening for applying the agents to solve new purposes. With this piece of chapter, we have tried to bring this whole picture as concisely yet in detailed fashion [30].

References 1. Rojek G Agent based system for assistance at industrial process control with experience modelling. AGH University of Science and Technology, Poland 2. Chen Y, Peng Y, Finin T, Labrou Y, Cost S Negotiation agents for supply chain management. University of Maryland Baltimore County 3. Oliveira E, Fischer K, Stepankova O (1999) Multi-agent systems: which research for which applications. Elsevier, Robotics and Autonomous Systems 4. Sycara KP 1998 Multiagent Systems. AI Magazine vol 19, no 2 5. Chu B, Yao J, Sun R, Wilhelm B A negotiation-based multi-agent system for supply chain management. University of North Carolina 6. Shen W, Norrie DH Agent-based systems for intelligent manufacturing: a state-of-the-art survey. Springer 7. NairPR E-supply chain management using software agents. Amrita Vishwa Vidyapeetham University 8. Fox MS, Barbuceanu M, Teigen R Agent-oriented supply-chain management. Enterprise Integration Laboratory, University of Toronto 9. Nissen ME Agent-based supply chain integration. Springer

138

A. K. Bardhan and A. Alsadoon

10. Kumar Y, Jain Y Research aspects of expert system. IET Bhaddal Ropar. Int J Comput Bus Res 11. Grandspenkis J, Pozdnyakov D (2006) An overview of the agent based systems for the business process management. In: International Conference on Computer Systems and Technology 12. Liao S-H Expert system methodologies and applications—a decade review from 1995–2004. Elsevier 13. Maamar Z, Shen W Integration of workflow and agent technology for business process management. Researchgate 14. Metzger M, Polakow G (2011) A survey on applications of agent technology in industrial process control. IEEE Trans Ind Inform 7(4) 15. Nwana HS, Ndumu DT A brief introduction to software agent technology, BT Research Labs 16. O’Brien PD, Wiegand WE Agent based process management: applying intelligent agents to workflow. Cambridge 17. Banerjee S, Hecker JP A multi-agent system approach to load-balancing and resource allocation for distributed computing. Springer 18. Gou H, Huang B, Liu W, Ren S, Li Y et al An agent-based approach for workflow management. IEEE 19. Jennings NR, Sycara K, Wooldridge M 1998 A roadmap of agent research and development. Kluwer Academic Publishers 20. Reaidy PJ, Massotte P Intelligent agents for production systems, in intelligent agent-based operations management. Researchgate 21. Jennings NR, Bussmann S Agent-based control systems. IEEE 22. Long Q, Zhang W An integrated framework for agent-based inventory-productiontransportation modelling and distributed simulation of supply chains. ScienceDirect 23. Frayret J–M, D’Amours S, Rousseau A, Harvey S, Gaudreault J Agent-based supply-chain planning in the products industry. Springer 24. Ahn HJ, Lee H An agent-based dynamic information network for supply chain management. Springer 25. Pu Z, Jiang Q, Yue H, Tsaptsinos M Agent-based supply chain allocation model and its application in smart manufacturing enterprises. Springer 26. Luder A, Cala A et al Design pattern for agent-based production system control—a survey. IEEE 27. Shen W, Norrie DH An agent-based approach for manufacturing enterprise integration and supply chain management. Springer 28. Um W, Lu H, Hall TJK A study of multi-agent based supply chain modelling and management. scirp.org 29. Ehrler L, Fleurke M, Purvis M, Savarimuthu BTR Agent-based workflow management systems (WfMSs). Springer 30. Kuo J-Y A document-driven agent-based approach for business processes management. ScienceDirect

Chapter 7

Multi-Agent System Applications in Health Care: A Survey Chandanita Thakur

and Shibakali Gupta

Abstract Earlier, medical systems were suffering from many issues. The issues are mostly observed due to inadequate medical practitioners, poor medical infrastructure, manually operated healthcare systems and improper coordination between each and every aspects of healthcare system. Most of the medical reports and records are based on paper and pen, which is difficult to manage and maintain and also raise a question of reliability. So, situation needs a change in the existing system. Keeping that need of change in mind, the concept of multi-agent system was introduced in healthcare system. Multi-agent systems are considered as the best and most appropriate technology that can be applied to fulfill the need required in healthcare system. In multi-agent system, the agents are representing and acting on behalf of users and owners with very difficult goals and motivations. The agents cooperate, coordinate and negotiate with each other in the same way that we cooperate, coordinate and negotiate with other people in day-to-day life and consequently can be used as the best technology to solve the problem arrived in medical healthcare system. This chapter focuses mainly about architecture, planning, implementation and application of multi-agent system in healthcare domain. Categories and Subject Descriptors General and reference: Surveys and overviews—multi-agent system (MAS), artificial intelligence, healthcare domain. Keywords Treatment as a Service (TaaS) · Patient Agent (PA) · Health at home semi-automated hospital

C. Thakur (B) Department of Computer Science and Engineering, Vivekananda Institute of Science and Technology, MAKAUT University, Kolkata, West Bengal 741249, India S. Gupta Department of Computer Science and Engineering, University Institute of Technology, Burdwan University, Bardhaman, West Bengal 713104, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 S. Gupta et al. (eds.), Multi Agent Systems, Springer Tracts in Human-Centered Computing, https://doi.org/10.1007/978-981-19-0493-6_7

139

140

C. Thakur and S. Gupta

1 Introduction There are many complicated and realistic problems in this world which are beyond the capability of human being to solve. But an efficient and logical way needs to be developed to solve those problems. These problems can be attempted to be solved by the agents because the agents are having some inbuilt capabilities like autonomy, reliability, reasoning, cooperation, coordination, etc. All the agents having those properties can interact with each other and can be used to solve the complicated problems which are beyond the range of human being. The system comprising of many agents, targeted to solve complex real-life problems, is called multi-agent system. The multi-agent system can be used in the following situations where (1) (2) (3)

The problem can be divided into interdependent sub-problems. The knowledge is administered in various places. The problem will be solved by combining the decision taken by each individual agent, keeping their autonomous nature intact.

Medical healthcare system is a system where many people are involved in many steps and their combined collaboration only can make the system to work successfully. So, keeping the above scenario in mind, we can apply the multi-agent system to solve the problem which arises in medical healthcare system where every person involved can be treated as agent. As we know agent has the capability to successfully interact, negotiate and coordinate with each other, approximately in a flawless manner, so they can be treated as more reliable than human being. Because of the above-mentioned reason. (1) (2) (3)

Problem can be divided into sub-problems. Knowledge can be administered in various levels. Their individual result can be combined to make a reliable decision which is very important, especially in medical domain (and that is only the aim of this paper).

This survey paper discusses the architecture of multi-agent system, then its planning, implementation and its application in healthcare system and some systems that were developed for facilitating healthcare system.

2 Background The progress of research of artificial intelligence (AI) can be listed as: (1) (2) (3)

Year late 1960: Progress of AI which leads to the invention of agent in future. Year 1966: The first software agent known as ELIZA (automated conversations and answering) [1] by Professor Joseph Weigenbaum [2]. Year early 1970: Evolution of agents.

7 Multi-Agent System Applications in Health Care: A Survey

(4) (5)

Year 1980: Researches on multi-agent system. Concept of organization and communication was understandable by the researchers. Year 1990: (a) (b)

(c) (6) (7) (8) (9)

141

“Agent-Based Modeling (ABM)got expanded within the social sciences.” “Large scale ABM sugars cape developed by Joshua M. Epsteil and Robert Axtell which is used to model the role of social and cultural development” [3]. “Carley ABM was developed by Carnegie Mellon University Kathleen to explore the core evaluation of social network and culture [4].”

Year 2000: “A survey over the evaluation of agent-based system [5] by Samuelson.” Year 2006: “Samuelson and Machel reached on the recent improvements in the multi-agent systems” [6, 7]. Year 2013 and 2014: “Method for basing agent-based simulation on models of human cognition which is known as cognitive social simulation by Ron Sun.” Year 2016: Practical application in distributed coordination of multi-agent system.

3 Literature Survey This section of the paper will first review the papers which are associated with multiagent system in various ways. We can categorize the papers in the following ways before we review: (1) (2) (3) (4) (5)

Architecture of multi-agent system. Modeling of multi-agent system. Planning of multi-agent system. Review of multi-agent system applied in Healthcare domain. Multi-agent system applied in health care.

The main idea about each individual paper is discussed below according to the above-mentioned category.

3.1 Architecture of Multi-Agent System “DECAF-A Flexible Multi Agent System Architecture, John R Graham, Keith S. Decker, Michael Mersic, September 2000” [8]. Distributed Environment Centered Agent Framework (“DECEF”) is an “agent toolkit” which helps us to build a MAS with an organized and well-defined software engineering approach. This toolkit provides a platform to “design, develop and

142

C. Thakur and S. Gupta

execute agent” (including large gained intelligent) and provide necessary support like “communication, planning, scheduling, execution monitoring, coordination, etc.” This can be treated as an “operating system of a software agent on which application programmer has a limited access.” DECEF is a highly threaded architecture. Different threads can run parallel for different modules. No specific domain knowledge is required for designing of DECEF. It is mostly used for biometrics, social simulation, teaching MAS concepts and protocols, chemical engineering, development of independent algorithm, etc. Designing of MAS with the help of DECEF does not need any specific domain knowledge. So, any beginners can take an attempt of doing it. Designing “new agent architecture” is an “important research work,” but it is dangerous for beginners, willing to work with the agent concept. “An Architecture for Multiagent Systems, Amit Chopra, Munindar P. Singh” [9]. This paper proposed a layered multi-agent system architecture based on commitments, as a notion of business-level interoperability supports. In the proposed architecture,” agents are components and the interconnection between the agents are specified in terms of commitment and abstracting away from low-level details.” A critical layer in this architecture is a “commitment-based middleware that plays an important role in ensuring interoperation and provides commitment-based abstractions to application programmer.” It is possible to “encode communication in terms of commitment operations and patterns.” It is possible to write agents in terms of BDI abstractions. Implementation can be done for the proposed architecture. So, powerful and less cost-effective language is needed for the implementation. “Architecture Properties of Multi-Agent Systems, Onn Shehory, Dec 1998” [10]. This paper is about solving a given computational problem—whether we can build a MAS to solve a problem or not. If yes, what type of MAS should be preferred? Author has discussed some architecture or infrastructure (ARCHON, OSACA, DIDE, ADEPT, MACRON, DECAF,) for MAS. Author analyzes different multi-agent systems and made MAS more accessible to system designer and one of the solutions of the computational problems. Some development tools and generic agent-oriented programming languages are needed to solve those MAS. A rigorous comparison between MAS and other software architecture styles can be sincerely examined. Interoperability between MAS and other architecture style can be investigated to provide appropriate mechanism for its implementation. “How to build a Multi Agent System, Tim Stockholm, Jens Nimis, Thorsten Schotz, Marcel Stehi” [11]. This paper presents a technical approach for combining the heterogeneous multiagent systems which cover services in the range of supply chain scheduling and production planning in shop floor. It also provides a proactive control tracking services. This large-scale adaptive MASs has an enormous applicability and reliability in overall supply chain process which will turn out to be a valuable addition to MAS, especially in supply chain management.

7 Multi-Agent System Applications in Health Care: A Survey

143

If it is possible for applying this system in other field of society other than supply chain management, then it will be good for human society.

3.2 Modeling of Multi-Agent System “Modelling and analysis of Multi Agent systems using Patient, Jose R. Celaya, Alan A Durocher, Robert J Graves, IEEE, 2007” [12]. This paper is about “design, modeling and analysis” of multi-agent system. Here, multi-agent system is treated as “discrete event dynamic system,” where patrinet is used as a” tool to access the structural properties of multi-agent system.” Deadlock avoidance is treated as a key property, and it is evaluated using “liveliness and boundedness properties” of the patrinet model. A simple example of multi-agent system with “two agents “was introduced, but the proposed model and methodology were applied to it with “multiple agents” to prove that the system is free from deadlock. Direct agent-to-agent communication with a large number of agents is yet to be implemented. “Modelling Secure Multi Agent Systems, Haralambos Mouratidis, Paolo Giorgini, Gordon Manson, July 2003” [13]. This paper introduces “the extensions to the “Tropos” methodology to enable it to model security requirements.” Security is achieved very easily in the development stages. The security is extended up to implementation phase. A clear distinction between function and security is clearly provided. A well-guided process of integrating security and functional needs through the software development process of agent-based system is needed. “Use and Modelling of Multi Agent System in Medicine, MazaHadzic, Darshan S Dillon, Tharam S Dillon, IEEE, 2009” [14]. This paper presents “an ontology-based multi-agent system.” This system helps us to fetch information about diseases of human being. This MAS consists of four agents: “interface, manager, information and smart agent.” The authors have used UML 2.1 for representing the MAS. Any MAS can be involved with many dynamic processes. UML 2.1 gives support for it. It can be implemented using Jade more efficiently.

3.3 Planning of Multi-Agent System “Multi Agent Planning—An introduction to planning and coordination, Mathijs De Weerdt, Adriaan Ter Mors, Cees Witteveen” [15]. This paper gives an idea about “planning technique for classical planning problem.” The authors have also provided the technique for extension of this problem. This work does not go into the details of the agreement between agents. But system throughput is dependent on the cooperation between agents. For “selling agent, the

144

C. Thakur and S. Gupta

resources are worthless if it does not sell them” and “buying agent will only buy resources if this leads to a reduction of cost.” Every time, there will be some possibility that the plan cab be deviated from its original. So, some preventive measure should be there to fight against the deviation and to make the system run properly. “Introduction to Planning in Multi Agent System, Mathijs De Weerdt, Brad Clement, Dec 2009” [16]. In this paper, the authors had tried to differentiate between “centrally creating a plan” and “constructing a plan.” They have proved that “constructing a plan” is easy to implement rather than “centrally creating a plan.” This paper gives a brief overview of multi-agent planning problems and multiagent planning techniques. It focuses on the ways to minimize communication or interaction for the most “efficient planning and execution.” Highly efficient planning algorithms are needed. “Planning for Large Scale Multi Agent Problems via Hierarchical Decomposition with Applications to UAV Health Management, Yu Fan hen, N. Keman, Ure Girish Choudhury, Jonathan P How, John Vian, AACC, June 2014” [17]. The authors have developed a “novel hierarchical decomposition approach” for giving solution to large-scale multi-agent “markov and decision process” (“MMDPs”) by exploiting coupling relationship in the “reward function.” MMDP is a natural frame work for solving “multistage decision making problems” such as “optimizing vision performance of unnamed aerial vehicles “(UAVs) with “stochastic health dynamics.” It is a “stochastic fuel consumption and health progression method.” It has achieved suitable performance (over fifty interactions) on the ten agents PST system, which has more than ten stages. MMDP needs a “hand-tuned approximate team transition method.” It is difficult to tune a model manually and get the same satisfied result for this domain.

3.4 Review of Multi-Agent System Applied in Healthcare Domain “Agent Applied in Healthcare: A Review, Devid Isern, Devid Sanchez, Antonio Moreno, ELSEVIER, 2010” [18]. In this paper, authors have discussed the role of Agents in healthcare system. To enrich the contents of the paper, authors have collected information from “medical databases” as well as “international conferences.” They have presented the internal architecture of an agent-based healthcare system and showed how communication can be coordinated to resolve healthcare problem. Multi-agent systems is not much implemented in health care yet. Only selected people have deployed it in clinical centers. Otherwise, it is with research work only. “Multi Agent Systems, Katia P. Sycara, 1998” [19].

7 Multi-Agent System Applications in Health Care: A Survey

145

In this paper, the author have presented some of the “indicative concepts” in multiagent system and the research work related to them. In her paper, the author have described multi-agent system as a “loosely coupled network of problem solvers” that interacts to solve problems which is beyond the limit of human being. “Rationality” was a global property of the multi-agent system which was measured by the “capability and consistency of a global solution.” But author has shown that multiagent system is lagging by “global awareness, global control or global data.” The author, in his own work “RETSINA,” (a multi-agent infrastructure) showed that every “RETSINA” agent is a “belief desire intention” (“BDI”)-type agent that integrates planning, scheduling, execution, data collecting, and coordination with other agents. In her paper, the author proposed the idea of building a special software that runs in “distributed and open conditions,” such as the Internet.

3.5 Multi-Agent System Applied in Health Care This category of paper can be divided into some sub-categories: (1) (2) (3)

Multi-agent system directly applied in health care. Security in multi-agent system applied in health care. Collecting requirements of hospital management system using multi-agent system. Multi-agent system directly applied in hospital management system. Multi-agent system applied in nurse scheduling in ambulatory system. Multi-agent system directly applied in patient scheduling.

(4) (5) (6)

All the categories are explained below.

3.5.1

Multi-Agent System Directly Applied in Health Care

“Multi Agent System Applications in Healthcare: Current Technology and Future Roadmap, Elahadi Shakshuki, Maleolm Reid, 2015” [20]. The authors have designed this paper for taking care of the patient “at home” for reducing the cost by using multi-agent-based approach. They have also forecast an idea about future healthcare services which can be provided. In this paper, authors have discussed implementation, drawback and results of Telecare of multi-agent distributed information platform (“MADIP”) and its prospective applications. This paper can be implemented, and cost factor can be explained clearly. “Distributed Hybrid Agent-based Discrete Event Emergency Medical Services Simulation, Anastasia Anagnostou, Athar Nouman, Simon J. E. Taylor, IEEE, 2013” [21]. In this paper, authors presents the development of a “distributed hybrid agentbased discrete event simulation (DES) model” from the knowledge of “emergency medical services (EMS).” The existing simulation models of EMS are considered as

146

C. Thakur and S. Gupta

either a “single model “or “several standalone models” that can describe different system elements individually. This work aims to provide “the feasibility of using distributed simulation technology” for implementation of “hybrid EMS simulation.” One good example of critical health services is response time for the ambulance arrival. The target is 75% of “category A calls” (“life-threatening emergencies”) are reached within eight minutes and ninety-five percent within 19 min. In 2011–2012, response time for 99% calls was within 19 min. But the results from this model for one-month simulation gives response time of 99% of incidents which was less than 19 min. Performance can be increased by disaggregating the process such as investigation, clinical examination and outdoor treatment, etc., and by increasing number of departments, number of ambulances, etc. “Hybrid Simulation for Health and Social Care: The way forward, or more trouble than its worth?, Sally C Brailsford, Joe Viana, Stuart Rossiter, Andrew Channon, Andrew J. Lotery, IEEE, 2013” [22]. In this paper, the authors have described the process of building a “hybrid simulation model” for a disease called “age-related macular degradation” (AMD). This disease can cause the “vision loss” problem for aged people. This model in implemented by using “discrete event” and “agent-based simulation.” In this model, each agent is also following some “discrete progression” method. In this paper, the intention of the authors is to establish a connection between health and social care system proprietary nature of any logic can sometimes work against, in creating a “reusable modular model.” The future model of AMD can be implemented by using hybrid approach. “Leveraging the learning Process in Health through Clinical Cases Simulator, Cecilia D Flores, Paulo Barros, Silvio Cazella, Marta Rosecler Bez, IEEE, 2013” [23]. This paper presents a multi-agent learning system for healthcare practitioners named as “Simulation for the decision making in the healthcare service” (SimDecs). The “SimDecs” was designed as a multi-agent system with three intelligent agents: domain agent, learning agent and mediator agent. This simulation model is close to the clinical experience, users can interact with virtual characters (avatars) representing real-life characteristics involved in a clinical history, and interactive clinical case simulation directly observes and reports clinical decision making in real time. “SimDecs” simulator tool is not implemented in teaching learning process, so that it can feel the gap between theory and practical, in training physicians. The results described are to be interpreted based on the sample applied, because it was not probabilistic. “Personal HealthCare Monitoring and Emergency Response Mechanism, Yong Lin, Xingjia Lu, Fang Fang, Jianbo Fan, IEEE, 2013” [24]. This paper analyzes the mechanism of multi-agent interactions in personal healthcare-based emergency response systems. It proposes an in-home pervasive interaction-based intelligent virtual agent as the center of multi-agent healthcare models. It also proposes the various approaches to analyze the interactions based

7 Multi-Agent System Applications in Health Care: A Survey

147

on sequential decision processes and game theory to facilitate home healthcare emergency. It can be implemented using Jade. “A Novel Architecture for Message Exchange in Pervasive Healthcare Based on the use of Intelligent Agents, Jao Luis Cardoso de Moraes, Wanderley Lopes de Souza, Luis Ferreira Pires, Luciana Tricai Cavalini and Antorio Fransisco do Prado, IEEE, 2013” [25]. This paper proposes an architecture for the exchange of context-aware messages in pervasive healthcare environments. This architecture is established, based on technologies from ubiquitous computing and intelligent agents and complies with the open HER dual model. The main work of this architecture is to support mobility and collaboration among medical professional when they perform clinical tasks. The authors have used the BDI model in the development of their architecture and used JADEX which supports the formal description of cognitive agents that is based on BDI model. The authors conducted a case study involving three cardiology departments. Cardiac surgery is implemented using agent technology with the help of agents like physician agent, resource agent, etc. To evaluate the acceptance of that case study, users were chosen in two groups: 57 caregivers (including physicians, nurses and medical students) and 122 patients. The caregivers were organized together because they work as a team in this scenario. The users utilized the system from Jan 1 to July 30, 2012, and given a ranking from 1 to 5. This technology acceptance model (TAM) can be extended to figure out the effect of other variables such as user understanding, skill and output quality. “Implementation Scheme for Online Medical Diagnosis system using Multi Agent System with Jade, Shiba Kali Gupta, Arindam Sarkar, Indrani Pramanik, Banani Mukherjee, 2012” [26]. The intention of this paper is to propose the idea of “OnlineDocs.com”. “OnlineDocs.com” is a system which provides the facility for the user to provide the information about the details of nearest best Healthcare system, diagnostic center, etc. The authors have used multi-agent system (MAS) to design “OnlineDocs.com”. This paper has been implemented using Jade, but there are some challenges in mainly five areas; (1) (2) (3) (4) (5)

Data security. Interconnection between user and agents. Synchronization between distributed services. Inter- and intra-communication between services. Transparency of environments among different agents.

In this paper, the authors have developed one project called “agent cities” which was designed to provide “foundation for intelligent physical agent” (FIPA)-based agent architecture for providing remote network facility. They have designed an advanced message system called “agent communication channel” (ACC). It is designed to control the communication within the local and the remote platform. The above-mentioned architecture was designed using JADE.

148

C. Thakur and S. Gupta

“Semantic Web Services for Medical Health Planning, Mohit Gangwar, R.S. Yadav, R.B. Mishara, IEEE 2012” [27]. Semantic web services (SWS), an agent-based composition method, was developed by authors using two sets of agents, i.e., “Service Requestor Agent” (SRA) and “Service Provider Agent” (SPA). These two agents have been designed to represent the “users side” and the “solution side,” respectively, for solving the problem of a medical health planning. SRA specifies the patient’s requirements for the various consulted doctors and the SPA provides the details of hospitals and doctors who are specialized in a particular disease. SPA is a software components. All the agents need to be invoked explicitly. Whenever the SPA is called, it will check which consulted doctor satisfy the query considering all medical centers considering the fact that, which hospital is near, how is the availability of doctor, etc. This proposed model is implemented using J2EE. The have used Tomcat webserver for database operation and Java APIs for manipulating, accessing and querying the “resource description framework” (RDF) files. The query language SPARQL is provided by the Java APIs which is used for querying processing. “An Agent Based decision support System for Hospital Emergency Departments, Manel Taboada, Fduardio Cabrena, Na Luysa Ig Lesias, Francisco Epelde, Emilio, ELSEVIER, 2011” [28]. A hospital emergency system based on multi-agent technology is presented in this paper. In this model, authors have made the agents to work based on the principle of state machine. Agents can interact with each other within the environment provided for them. Authors have validated the model using simulation. Authors have divided the agents in to two categories: “active” and “passive” agents. (1)

(2) (3) (4) (5)

The active agents represents human being and other entities that act with their own initiative, i.e., relatives of patients, doctors, “triage” and emergency nurses, etc. The passive agents represent solely reactive systems which are Loud speaker Patient information system Pneumatic pipes Central diagnostic services (radiology services and laboratories)

“Active agents” are described by “state machine” (Moore machine). Current state of an agent is represented by a collection of state variables known as state vector (P). Each time “state machine” moves to next state depending on its “current state” and “input vector.” “State variables” may be discrete or continuous. Initial states of the state variables are (1) (2) (3) (4)

Name/identifier (unique per agent). Personal details (unique per agent). Location (entrance, admission, waiting room). Action (what the individual is doing in a particular state, i.e., idle or giving information).

7 Multi-Agent System Applications in Health Care: A Survey

(5) (6) (7) (8)

149

Physical locations (Bartle index). Symptoms (healthy, vomiting, etc.). Communication skills (low or medium). Level of experience (low or medium).

For verifying system or model in the first cycle, an initial simulation is generated which is called Net Logo. The simulation has been executed many times for the equivalent of one-day activity. First, the total number of patient arrived to emergency department has been changed, maintaining the invariable mix of emergency department staff and the result has been shown in tabular form. Second, the throughput is tabulated with the change in level of experience of “Emergency Department” (ED) staff. The model can be upgraded by adding gradually the new agents and state variables. Parallel simulation with different parameters can be performed, and after comparing data from simulation and real system, adjustment can be done to increase the efficiency of the system. “A Multi Agent System Model for Evaluating Quality Service of Clinical Engineering Department, Laura Gaetano, Gabriella Balestra, August, 2011” [29]. In this paper, authors have built a model which uses a tool to access the quality of an existing clinical engineering department. The proposed model is able to find out the fault of the existing system and can also suggest the method to improve it, by using MAS. By using the inputs that characterized the situation they wanted to study, it was possible to analyze the real situation in order to highlight drawbacks and to suggest improvements. The authors can find out the drawbacks of the current situation by using Simulation through MAS approach. This is only the first step to guide clinical engineering to apply the basic quality principles. It can be extended more by implementing simulation software. “Remote Patient Monitoring and HealthCare Management Using Multi Agent based Architecture, Rizwan Muhammad Saleem Aslam Muhammad, Martiner Enriquez, IEEE, 2010”[30]. In this paper, remote patient monitoring provides low cost and reliable health care to elderly, chronically and acutely people either in outdoor or in indoor environment. This hypothesis is very effective to reduce healthcare resources and unnecessary hospitalizations. It minimizes the cost of treatment and monitoring. A more efficient testing environment can provided for evaluating characteristics and functionalities of remote patient monitoring system. “Multi Agent System Approach for Hospitals Drug Management Using Combinatorial Options, Ilaria Baffo, Gillscppe Stecca, Tashya Kaihava, IEEE, 2010” [31]. In this paper, reduction in cost is treated as a crucial point for existence of several medical systems. But a lot of hospital processes such as drugs and equipment purchasing and managing is not under control of hospital managers. This proposed work faced the cost and management criticalities in presenting the healthcare systems. The proposed model faces the challenges of

150

C. Thakur and S. Gupta

(1)

Collecting the benefits, deriving from successful collaborative models already used in industrial systems. The model exploits a technique based on combinatorial auction in order to effective assignment of drugs toward solving the limited resource assignment problem.

(2)

The proper guidelines can be provided to solve problems related to various parts of hospital. Redesigning of “logistics department” is needed for overall reduction of cost. The result of the test should be compared between previous and future in terms of time, cost, etc. “Cognitive Medical Multiagent Systems, Barna Iantovis, Jan 2010” [32]. The intention of the author in this paper is to design “artificial agents” that can solve problems by considering many conditions which can be overlooked by human beings which may result in elimination of some mistakes from the physician’s decisions. Example: Physician may forget to consider patient’s past history (i.e., allergy of a patient specified in medical history). The motivation of the paper involves designing a model based on multi-agent system for dealing with critical health problems which cannot be done by human effort efficiently. This paper mainly concentrates on “CMDS” (“Contract net-based medical diagnosis system”), which was proposed long back. Extended “CMDS” is cognitive, but it is difficult to differentiate between CMDS and cognitive. “ADE-Implemented Mobile Multiagent-Based, Distributed Information Platform for Pervasive Healthcare Monitoring, Chuan-Jun Su, Chia-Ying Wu, ELSEVIER, 2009” [33]. This paper presents design and development of a “mobile multi-agent-based distributed information platform” (“MADIP”) on top of JADE. According to this model, health can be continuously monitored by “lightweight portable monitoring devices” (example: “vital sign monitor”) without interfering patient’s real life. The usability of the system was tested in the “ERP/MC Laboratory” of Yuan Ze University. This system is evaluated based on certain criteria, i.e., (1) (2) (3) (4) (5)

Whether MADIP is easy to use. MADIP provides better service. Need help from technical people. Whether features offered by MADIP are sufficient. Whether it provides 24/7 service or not, etc.

Robust diagnostic algorithm can be explored to increase the work function of the “diagnostic agent.” Security issues should be taken care. In their research, they have adopted a JADE plug-in JADE-S that gives security characteristics so that it can be used in real environments. “A Unified Multiscale Field/Network/Agent-based Modeling Framework for Human and Ecological Health Risk Analysis, Panos G. Geogopoulos, Sastry S. Isukapalli, IEEE, 2009” [34].

7 Multi-Agent System Applications in Health Care: A Survey

151

In this paper, authors have tried to present “field/network/agent”-based model for supporting human and ecological health risk assessment. This modeling is about how network agent moves around different network fields and represents “spatiotemporal distributions” of physical properties, rule governing constraints, interactions and actors can make decision which affects the state of the system. Different “stochastic” and “deterministic” modeling case studies associated with risk are discussed here with proper examples. No model has been introduced, and the concept is not implemented. “Using Knowledge and Rule Induction methods for enhancing clinical diagnosis: Success Stories, Fariba Shadabi, Dharmendra Sharma, IEEE, 2009” [35]. In this paper, the authors have explained the method of combining “hybrid machine learning” and “data mining” techniques for producing classifiers in a “clinical setting” through some real-life example. If we want to improve the quality of healthcare system and need to incorporate advanced data mining approach in computerized patient records, it is a matter of long time and thousands of trials. “The CMDS Medical Diagnosis System, Barna Iantovies, IEEE, 2008” [36]. In this paper, authors have proposed an innovative “medical diagnosis multi-agent system” called “CMDS.” The proposed medical system consists of physicians and artificial agents which increases the advantages in the medical diagnosis elaborations by using efficiently the medical knowledge distributed between the members of the system. CMDS can solve a huge amount of health-related problems. The accuracy of elaborated diagnostic by artificial medical agents, members of the CMDS systems depends on the medical knowledge accuracy. Implementation can be done using Jade. “Mobile Multi agent Based Distributed Information Platform (MADIP) For Wide Area eHealth Monitoring, Chuan June Se, ELSEVIER, 17th September 2007” [37]. In this paper, the authors have presented “the design and architecture of a mobile multi-agent-based information platform” (MADIP) to support eHealth monitoring environment. An “eHealth monitoring environment” was developed on top of MADIP. An “agent software development kit” was adopted for “prototyping, concept proofing and evaluation.” The proposed platform is integrated with “lightweight” portable monitoring devices (i.e., “portable vital sign monitor”). Patient health can be monitored without interfering patient’s daily activity. Total system is working based on the cooperation of multiple agents, i.e., “resource agent, physician agent, diagnostic agent,” etc. The MADIP takes immense data from patient monitoring, monitor the patient’s status and alert its owner (relatives) about their health problems. But sometimes because of sports activity, drinking coffee, smoking, prescribed drugs made the system to send falls alarm to physician. So, a more practical and sophisticated diagnostic engine is needed to be developed. Security and privacy should be implemented to protect the personal health information from assault or corruption.

152

C. Thakur and S. Gupta

A basic security mechanism is introduced, called “capability-based mechanism,” that prohibits some special (aglets) link to any dynamic link library to provide some control over the security. By incorporating sophisticated technology, diagnostic method can be improvised. Robust diagnostic algorithm can be explored to get the diagnostic accuracy. “Multi Agent System based Efficient Healthcare Service, Byung-Mo Han, Seung-Jae Song, Kyu Min Lee, Kyung-Soo Jang, Dong-Rycol Shin, FEB 2006” [38]. In this paper, the authors have proposed a system which is MAS-based efficient healthcare system. This system is a combination of “medical sensor modules” and “wireless communication.” It provides a proper communication between the “Body Area Network” (“BAN”) and the subsystem in the hospital designed by using multiagent system. Jade or FIPA is used to develop the MAS. The aim of this paper is take care of the patient’s health-related information and to provide efficient medical service through medical staff cooperation. Performance evaluation and simulation can be done. “Mobile e-Health Monitoring: An Agent based Approach, V. Chan, P. Ray, N. Parameswaran, The Institute of Engineering & Technology (IET), 2008” [39]. The authors have presented a multi –agent-based architecture for mobile health monitoring which collects patient data, performing prescribed actions to patients and managing medical staffs in a mobile environment. Authors have implemented the architecture for a prototype using “Bluetooth supported eHealth sensors” for heart and weight monitoring and GSM mobile phones. This system has used Symbian OS. They have used J2EE for server-side agent programming and J2ME for mobile programming, to program their mobile agents. This paper has put light on some aspects for especially elderly people (who needs treatment from home most of the time): (1) (2) (3) (4) (5) (6)

Patient–doctor relationship is extended beyond the episode of visits. Proactive health care where doctor will be able to know about the patient before a fatal event occurs. Patient can be monitored at home or from anywhere. Patients have opportunity to avail high-quality information (from medical team at remote location). Agents can help identifying the patient’s need of medical attention. Better completion of treatment.

Ability/capability of mobile devices can be improved which in turn makes the agent to perform more efficiently in future. Security of mobile agents is an important factor which is needed to be incorporated. “Building Distributed e-Healthcare for Elders Using RFID and Multi Agent, Chuan-Jun Su, Stephen Chingyu Shih, International Journal of Engineering Business Management, 2011” [40]. In this paper, an intelligent homecare system is presented based on “radio frequency identification” (“RFID”) and multi-agent technology. The system enables

7 Multi-Agent System Applications in Health Care: A Survey

153

RFID homecare service and physiological rating based on the concept on “locationbased service” (“LBS”). To prevent the elderly from accidental fall at homecare system—a real-time safety measure is taken with the combination of two technologies; (1) (2)

Dynamic detection. Motion detection.

By using RFID technology, it can be decided that in real life the patient is taking the medicine on time or not. Complete tracing of patient is possible, i.e., dose of the medicine, mistake of taking medicine, fall control, etc. This system is designed using machine learning technique which made it update and output the information very fast. The system is helpful to detect accidental fall or any other critical circumstances. “Adaptive Medical Workflow Management for a Context—Dependent Home Healthcare Assistance Service, L. Ardissono, A. Di Leva, G. Petrone, M. Segnan, N. Sonessa, Elsevier, 2006” [41]. The authors have presented an architecture of a framework with the combination of “workflow management” and “context-aware action execution.” The framework is based on the “web services” and “autonomous agent technologies.” The purpose of designing framework is to support the home healthcare assistance services. The authors have described the “medical guideline,” specifying the “blood thinner management” using “BDI model.” The “intelligent component” may be employed as an “activity execution model.” It can select the best way to perform the guidelines, given the user interaction with the service and the patient state. This module can automatically collect user need efficiently. “Implementing requirement for Hospital Management System Using Multi Agent System, Anurag Tiwari, Praveen Kumar Patel, Vivek Kumar Singh, Anurag Shrivastava, International Journal of Engineering Business Management, 2013” [42]. In this proposed approach, authors have concentrated on the “data flow” in the “ehealthcare organization.” The data come from the outside world which is patient information. “patient agent” (PA) is responsible for providing service to the patient. If doctor wants to check up the patient, they can collect all the information about the patient and give the medicine using “doctor agent” (DA). All the internal activities are controlled by the “controller agent” (CA). This paper is implemented using JADE. Necessary data have stored in database and retrieved using JADE. “Health Delivery Systems—A Case for Multi Agent System, Jiang Tian, Haug Lorg, Tianfield, IEEE, 2013” [43]. In this paper, the authors have proposed the “synergizing” of multi-agent system to tackle the complexities of “health delivery system.” A nested architecture for “health delivery system” may be established where services and functional resource utilization, customer demands and other factors are taken into account in order to improve the efficiency of the system.

154

C. Thakur and S. Gupta

Some model can be provided for the health delivery system, and it can be implemented also. Health delivery and cooperation mechanism should be able to adapt itself with the change in environment. “Managing Healthcare work flows in a Multi Agent System Environment, Richardhill, Simon polovina, Marteen Beer” [44]. There are many complex health-related process where it is needed to apply multiagent system. But for applying multi-agent system, it is important to know the proper workflows of the system which can tell where and how to apply MAS. This paper uses a social care system as an example where the authors have described a technique called business healthcare process. This process is “captured,” “expressed,” “verified” and “specified” in a suitable format for a healthcare using multi-agent system. More research work can be done to use of this technique in other domain areas, in order to validate and enrich the approach in future. “Healthcare Expert System Based on Group Cooperation Model, Romeo Mark a Matco, Bobby D Gerardo, Jaewan Lee” [45]. This research implemented “Health Expert System” (HES) on the “group cooperation model.” This model consist of many health-related components. Communication can be carried out between the components using multi-agent system. There are some agent managers who are taking the request from client. There are some agents called “replica managers” who will create “replicating objects” to process the request accepted by agent managers from client. “replica manager” will serve the for the client in round robin fashion by using fuzzy least load algorithm. This experiment used Compliant Borland Visibroker 7.0 “COBRA” for implementation of the proposed system in group collaboration model. It can be implemented using some high-level language. “Medical Applications of Multi Agent Systems, Antonio Moreno” [46]. In this paper, the authors have tried to make a tool called medical applications of multi-agent system (“MMAS”) to deal with the health-related issues. By using some examples like taking care of palliative patients, management of organ transfer, etc., the authors have shown how this system can be utilized to solve those concrete problems. Security is a very important concern in this field because minute details of patient’s information is transmitted between agents. For anytime anywhere accessing information, intelligent agents should work in mobile devices such as mobile phones or PDAs.

3.5.2

Security in Multi-Agent System Applied in Health Care

“A Survey of Security in Multi Agent System, Rodolfo Carneiro Cavaleante, IgIbert Sittencourt, Alan Pedro da Silva, Marlos Silva, Evandro Costa, Roberio Santos Elsevier, 2012” [47]. In this paper, the authors have tried to find out some solutions of the security issues raised in the ground of multi-agent system. For making a secured multi-agent system, the authors have analyzed some issues that can hamper security. They have

7 Multi-Agent System Applications in Health Care: A Survey

155

also analyzed the characteristics of agents and find out the new threats and ways to attack. The authors examine the basic concepts and security in computing and some characteristics of agents and MAS that introduce new threats and ways to attack. In the later part of the paper, the authors have discussed some models and characteristics of multi-agent system described in other papers. Because of performance, scalability and fault tolerance, most of the computer software runs in the distributed system. If some machines fail, system can work properly. In earlier age, distributed system were used by just university researchers for emailing or to share printers on companies. At that time security was not a problem. But now several people are executing banking operation, shopping and accessing other services. So, the security of the system is a critical problem. The authors have discussed how security of computing differs from security of MAS. The design of a security infrastructure for MAS is entirely based on authentication. Always the main target of attack will be the backbone of the system. The backbone is nothing but the management agents (i.e., managing and directory services) as well as the communication channels. So, the kernel has to take care about the protection of the system. The person or organization who designed agents should have “public key certificates binding.” For management of certificates an Agent Certification Authority (ACA) is needed. The ACA is responsible to issue or deny certificates. ACA can verify the authenticity of the requested agents and their owners before issuing a certificate. This protocol also helps agent to identify whether it is communicating with authentic ACA or it is being hacked by malicious agent. We need to bring this framework to identify and make agents secure by just importing a library.

3.5.3

Collecting Requirements of Hospital Management System Using Multi-Agent System

“Gathering Requirements for Hospital Management System (HMS) Using Intelligent Agents, Nidhi Kushwaha, Shashank Sahu, P Ahmed, IJEIT, March 2012” [48]. This paper is designed to improve the performance of hospital management system (HMS). For improving the performance, the authors have designed a model that can sense and gather user’s requirement and send a report for HMS. In the time of development of HMS, intelligent agents (such as patient, doctor, nurse and environment agents) are to be installed to the user computer. With the help of self-learning capabilities (given to the agents), these agents interact separately with the respective user and automatically understands and gather user’s requirements, generate a report for user requirements and send that to the developer for the HMS enhancement. Once these kinds of agents are designed/developed, they reduce the cost at maintaining a hospital system and make the system easy to use.

156

3.5.4

C. Thakur and S. Gupta

Multi-Agent System Directly Applied in Hospital Management System

“Evolving Intelligent Agents for Hospital Management System, Nidhi Kushwaha, Shashank Sahu, Rajesh Kumar Tyagi, IEEE, 2015” [49]. This paper introduces software agents for hospital environment. The authors have suggested “four agents” for hospital management which are “patient agent,” “doctor agent,” “nurse agent” and “environment agent.” The patient agent helps patients to contact with the nearest doctor. “Doctor agent” works on the same way a doctor works and suggest the best possible treatment for a patient. “Nurse agent” helps to coordinate with the doctor agent. The “environment agent” is responsible for the management of the hospital, i.e., suggesting for the suitable room for the patient, etc. All these four agents work in a coordinated way with each other for collecting information from hospital management system. From patient entry to hospital, up to patient release—these two phases are described with the help of four algorithms which are implemented in java using eclipse IDE and apache tomcat server and oracle 10G as a back end. The four algorithms are “PATIENT_AGENT_ALGORITHM, DOCTOR_AGENT_ ALGORITHM, NURSE_AGENT_ALGORITHM and ENVIRONMENT_AGENT_ALGORITHM”. The proposed work can be extended in two ways. First way is to increase number of agents, and second way is to increase functionality of the agents. “Intelligent Agent Based System for Monitoring and Control of Hospital Management System, Rao Muzamal Liaquat, Ali Akthar, Nazar Abbas Saqib, IEEE, 2015” [50]. Aim of this paper is to gather information related to patient health problem and provide solution without direct intervention to doctor in a hospital using multi-agent system. Authors have explained the role of each agent in their intelligent agent-based monitoring and control system (IAEMC) in detail. Independent role of each agent is described in terms of six algorithms which are algorithm for; (1) (2) (3) (4) (5) (6)

Patient Agent. Expert Agent. Doctor Agent. Nurse Agent. Environment Agent. Reception Agent.

Communication diagram is provided which explains the interaction between all the agents. More number of intelligent agents can be introduced to increase the functionality and can be implemented using Jade or FIPA or any other suitable language. “A Multi Agent System for Hospital Organization, Hanen Jemal, Zeid Kechaon, Mounir Ben Ayed and Adel M. Alim, International Journal of Machine Learning & Computing, February 2015” [51]. This paper proposes a new complex medical system called “medical multi-agent system” (“MMAS”). To make an efficient healthcare, multi-agent system can be

7 Multi-Agent System Applications in Health Care: A Survey

157

incorporated with healthcare system which integrates multi-agent system in health care to make healthcare system as efficient as possible. According to hospital hierarchy, agents can be grouped into two layers. (1) (2)

“Multi-agents or superagents” (doctor, patient, nurses, etc.). “Swarm layer inspired from swarm Intelligence fields” (such as office, medical materials).

Knowledge distribution is done between the agents and the agents are assumed to have flexibility. Each hospital is divided into a set of departments based on resources interaction. Authors have assumed two types of interaction between agents: “informational and physical.” Resources can be of three types: (1) (2) (3)

“Human” (patients, nurses, agents, etc.). “Material” (carriage, dressing, chairs, etc.). “Informational” (patient records, schedule, cost, etc.).

In this paper, the author has explained the role and assigned task of all the agents (i.e., doctor, nurse, patient, access, discharge, service, medical center administration, calendar, emergency medical, security and safety, test result, etc.). The proposed MMAS is implemented using JADE. They have used JSP, Servlet, Jade and a fuzzy logic system to introduce a fuzzy learning algorithm for expert agents. Medical ontology and its cloud simulation can be implemented.

3.5.5

Multi-Agent System Applied in Nurse Scheduling in Ambulatory System

“Intelligent Patient and Nurse Scheduling in Ambulatory Healthcare Centers, Gregor Stiglic, Petel Kokol, IEEE, 2015” [52]. Ambulatory healthcare centers have lot of importance in the present society along with hospital or emergency healthcare system. It follows different scheduling algorithms to optimize performance. The aim of the ambulatory system is to reduce the waiting time of the patient and maintain high resource utilization. In this paper, authors present a long-term scheduling based on MMAS where agents use their time series forecasting and pattern realization abilities to predict possible patient flow peaks and inform those events to the main scheduling agent. To ease such peaks, they use adaptive nurse scheduling by adaptive nurse scheduler. This MMAS helps the personnel managers and other staffs in health centers to adapt the nurse and patient scheduling, independent of the current patient flow. It can be implemented using Jade.

158

3.5.6

C. Thakur and S. Gupta

Multi-Agent System Directly Applied in Patient Scheduling

“Finding Similar Patients in a Multi Agent Environment, Ayman Mansour, Hao Aying, Peter Dews, Yanqing Ji, John Yen, Richard E. Miller, R Michael Massanari, IEEE, 2011” [53]. This paper helps to find out the identical patients in a multi-agent environment. In multi-agent environment, “software agents” are located in different places but work collaboratively and productively to help one another for empowering their human usage. Though the agents are in different physical location, they need to achieve a common healthcare goal. They also showed how the agents equipped with fuzzy rules developed by the physicians in the team, collaborate and find similar patients in each agent’s patient database. They describe the architecture, design and implementation using popular agent language Jade. They have implemented a five-agent system. The number of agents can be increased. They have generated some preliminary simulation result. This result can be extended up to higher level. “Patient-Centered Multi Agent System for Healthcare, N. Benhajji, D. Roy, D. Anciaux, ELSEVIER, 2015” [54]. In this paper, authors have proposed a system for the management and the control of patient flow in a hospital system using multi-agent technology. In this system “patient agents” are represented by circles. The shared resources are represented by the eleven large boxes. All the resources are belonging to the same hierarchy, but patient agents are belonging to the upper hierarchy. Patient agents will move to resource agent for various needs (i.e., treatments, examinations or surgeries, etc.). There may be cases where either patient agents or both patients and resource agents are in conflict to resolve the conflict (conflict between “patient agents” and between “patient and resource agent”). First, negotiation will be done horizontally, if it fails it should be done vertically (higher level negotiation where patient agents start negotiating with each other). The above-mentioned model can be implemented to solve real-world problem arising in health sector. “Evaluation of A Multi Agent System for Hospital Patient Scheduling, Anja Zoller, Lass Braubach, Alexander Pokahr, Franz Rothlauf, Torsen O Paulussen, Winfried Lanersdorf, Armin Heinz” [55]. This paper evaluates the performance of a multi-agent system for patient scheduling for a real-life scenario. For the evaluation, authors perform a field study and made a realistic simulation model of a medium-sized hospital, based on the empirical data gathered from field study. In field, they have observed a classification of patients along with their health state and stability of the constitution. “1” represents a healthy state, health decrease rate of “0” is a stable constitution, whereas a health decrease rate of 0.0017 represents an unstable constitution which requires emergency treatment for a patient. For testing the hospital patient scheduling, they have used the characteristics for a medium-size German hospital with 500–600 (approx.) in-patient and 12 diagnostic resources in 5 different ancillary (support to primary activities or operation of an

7 Multi-Agent System Applications in Health Care: A Survey

159

organization), etc. For a standard working day, the average amount of patient arrival is 80–90 patients per day including at emergency rate of 5% of all arriving patients. The patient arrival is exponentially distributed over the day and starts doing many health activities where they get accumulated and needs the requirement of scheduling. The proposed simulation runs were setup for 22 days on a patient pre-filled hospital. Usually, patients do not get a prefixed time slot; rather, they wait in queue until they are requested by the ancillary unit. So, first come first serve (FCFS) only will describe the situation in real world. In all runs the multi-agent system (proposed by Paulussen et al.) showed suitable performance for real-time demand. It works well for a high load situation in a hospital with 40% decrement in the waiting time rather than the previous existing model. The concept of variable clinical pathway can be integrated directly into the patient agent to improve the schedule quality in uncertain environment. “Agent Based Patient Scheduling in Hospitals, Torsten O Paulussen, Anja Zoller, Franziska Rothlauf, Armin Heinzl, Lars Braubach, Alexander Pokahr, Winfried Lamersdorf” [56]. Patient’s priority based on health state not on money—keeping that fact in mind this paper was designed. To achieve this, a new health-dependent “utility function” is designed. Through this utility function, the patient agent can generate their “bids” for the “time slot auctions” at the resource agent. The suggested method is having four phases. (1) (2) (3) (4)

“The subscription phase” (“patient agents” subscribes the “resource agent” with their medical requirements). “The announcement phase” (“resource agents” initiates new resource and inform to the “subscribed patient agents”). “The bidding phase” (“patient agents” develop and submit their bids for the needed time slot). “The awarding phase” (the winner of the auction is determined).

The authors have investigated the performance of the proposed coordination mechanism in a dynamic environment using a “first come first serve” (FCFS) precedence rule in different circumstances like. (1) (2) (3)

“Short inter-arrival interval” with “few medical task for each patient.” “Short inter-arrival intervals” with “many tasks.” “Long inter-arrival interval” with “many tasks.”

The system is proved good in the case of resource conflict problem. The proposed system is tested for solving different emergency problems where each emergency patient receives only one task, but this task must be performed immediately. The proposed mechanism can be evaluated in a real hospital and can be integrated with the existing legacy system of the hospital. Agent-based system is suiting well suited with legacy system. So, it will encapsulate legacy system through agent.

160

C. Thakur and S. Gupta

“Multi Agent Based Information Systems for Patient Coordination in Hospitals, Torsten O. Paulussen, Armin Heinzl, Chriatian Becker, International Conference on Information Science, 2013” [57]. In this paper, authors proposed an agent-based coordination mechanism that overcomes the limitation observed in earlier patient coordination system. On the basis of distributed patient scheduling, authors have chosen a multi-agent system-based information system (MAIS) approach which initiates an “exchange-based coordination mechanism.” Patients are modeled as “patient agents.” “Patient agent” will be knowing about next step of the treatment process. They negotiate with other “patient agent” and “resource agents” to fulfill their resource requirements. The authors have shown that the new process results than FCFS as a common scheduling strategy in hospital. In the first scenario, they compare the “idle times” between “multi-agent-based approach” and “a FCFS strategy.” For each test, the inter-arrival time of the patients are uniformly distributed between 1 and 10 min, arriving until the 30th minute. Each patients receives up to five medical tasks with a duration between 10 and 16 min. These tasks are assigned with equal probabilities across ancillary units. The second scenario shows the difference between the two approaches. Except for one outlier, the auction-based approach performs better. In the third scenario, they have investigated emergencies. The difference between “FCFS” and “auction-based scheduling” is demonstrated, when the rate of 10% emergencies is introduced. Multi-agent system-based approach reduces the idle time compared to the FCFS approach. The paper needs to be implemented.

4 Multi-Agent Systems Used in Health Care The agents are taking a leading and innovative part in research work, especially in medical diagnosis and medical tasks. For health care, several applications of the agents were arranged in five fields. (1) (2) (3) (4) (5)

“Medical data management.” “Decision support systems.” “Planning and resource allocation.” “Remote care.” “Composite systems.” These five fields are discussed below.

7 Multi-Agent System Applications in Health Care: A Survey

161

4.1 Medical Data Management It presents all systems based on the “management,” “retrieval” and “processing” of medical data (e.g., “HER”—“Electronic Health Records”) such as: (1)

(2)

(3)

“The National Electronic library for Health” (“NeLH”): It is a governmental project of “National Health Service” (“NHS”) from UK. NeLH maintains a portal to collect medical information via Internet [58]. The “NeLH” system contains network agents for updating “information, management and automation” of the documents. “The Virtual Electronic Patient Record (VEPR)”: It represents a combination of agents which helps to merge the information from several systems [59]. Proactivity of agents is used to naturally discover, consult and retrieve patient’s data. Context-aware Hospital Information System (CHIS): This is a multi-agent system that provides “Intelligence and pro-activity” to healthcare environment [60].

4.2 Decision Support Systems This section represents some systems and approaches which supports some medical tasks such as: (1)

(2)

(3)

“Singh et al.” [61]: An intelligent “assistant agent” in health care is represented by “Singh et al.”. This assistant was designed to recover and use knowledge in order to solve medical problems. They use case reasoning in order to capture the experiential knowledge of health care [62]. “The health agents system”: It is the result of a funded research project [63]. The aim of this project is to create a network that can help in detection of brain tumorous. “Healthcare Services” (HeCaSe2): This is “a distributed system” to deliver healthcare services [64].

4.3 Planning and Resource Allocation It presents all systems based on physical and human resources like: (1)

(2)

The working groups “hospital logistics” including the “ADAPT-Project,” evolved a system called Agent Hospital [65]. It is tested for agent-based information systems in health care which supports both the “development” and the “valuation of modeling and implementation.” “CARREL”: This is an Agent-Mediated Electronic Institution for the allocation of organs and tissues for transplantation purposes [66]. “CARREL” assists

162

C. Thakur and S. Gupta

(3)

caregivers in the decision making during the sharing and allocation of pieces of tissues for transplantation. “The Medical Information Agents” (MIA): It represents a MAS to solve planning problems in health care [67]. The main aim of this project is to design MAS to achieve effective planning in the health care.

4.4 Remote Care It represents different methods for monitoring of patients such as: (1) (2)

“The Aingeru system”: It supports “intelligent, permanent and persistent” monitoring of elderly people through the “PDA” data [68]. “Koutkias et al.” [69]: It represents a system to ensure “monitoring, surveillance and educational services” for taking care of habitual diseases. The proposed MAS was designed to improve the performance by incorporating rule-based knowledge.

4.5 Composite Systems It presents all systems which offer entire solutions for health care. (1)

(2)

(3)

“Geriatric ambient intelligence” (GerAmI) is an “intelligent supervision system” delivering “physical and cognitive support” [70]. The main aim is to take care of elderly and Alzheimer patients in all aspects of daily life. Haeng-Kon Kim in 2013 presented a multi-agent system-based “proactive u-healthcare system” [71] which uses different methods to solve problems. The proposed system allows the system itself to identify u-healthcare domain problems. In 2013, Mutingi and Mbohwa proposed a “homecare multi-agent system” [72] architecture to make conclusion with multiple objectives. The system combines MAS based on “genetic algorithm” and “web services” that provide solutions in a “dynamic multiple-objective environment.” The proposed architecture consists of a number of agents that associated through efficient communication in homecare dynamic environments. Table 1 summarizes the existing works in MAS.

5 A New Scheme for Medical Diagnosis Using Mas In this section of proposed chapter, we attempt to propose an advance scheme of agent-based healthcare and medical diagnosis system using knowledge base and

7 Multi-Agent System Applications in Health Care: A Survey Table 1 Existing MAS woks in health care

163

Fields

Systems

Decision support systems

“Singh et al.” 2005 “Hecase2” 2008 “Health agents” 2008

Remote care

“Aingeru”2007 “Koutkias et al.” 2007

Planning and resources Allocation

“MIA” 2005 “CARREL”2008 “Agent Hospital” 2008

Planning and resources allocation “NeLH” 2005 “VEPR” 2006 “CHIS” 2008 Composite systems

“SHAREIT” 2008 “GerAmi” 2008 “K4Care” 2009 Mutingi and Mbohwa 2013 Haeng-Kon Kim 2013

intelligent agents residing on a multi-agent platform. In brief, we can say that there will be a number of agents like patient agent which is a group of individuals associated with sequential operational departments. Reception agent is responsible for the management of patient registration and admission in hospital. Hospital management agent is responsible to support in-patient till discharge. Medical insurance agent is responsible to take care of the medical cost of the patient. Nurse agent takes care of the admitted patient. Ward agent is responsible to keep records and facilitates patient treatment requirements. Ward agent is responsible to keep records and facilitates patient treatment requirements. Investigation agent is responsible for clinical report generation and helps in diagnosis methods and medical examinations for the patient. Shop agent supplies required medicines for the patient. Account agent is responsible for billing and maintaining financial transaction records. All the agents will have its own knowledge base associated with it; i.e., for doctor agent, it is KBda, nurse agent it is KBna, etc. (shown in Fig. 2). The total Health care of a patient from home to hospital is described with the help of two models.

5.1 Model 1 (Describes the Patient Treatment at Home) This describes the flow how initially a person (mostly will be referred as a patient in this whole paper) will initiate the treatment process and will undergo a systematic way to avail the treatment. The mobile device used by the person/patient is assumed

164

C. Thakur and S. Gupta

Fig. 1 Patient health record monitoring and capturing

Bluetooth enabled and connected to internet either through Wi-Fi or through data service of connection provider (Fig. 1). Patient will first install the mobile application available in the application store developed (need to be developed) by “patient agent” (PA) who deals with different general and specialized hospital managements, diagnostic centers, insurance agencies and ambulance agencies. PA agent is a combination of multiple sub domains which are like 24 × 7 × 365 hospital management support domain, insurance support domain, communication support domain and finance support domain. Payment to PA is online where PA is associated to a third-party payment gateway for secure payment information and tracking customer payment information data securely. Patient has to enroll to that “patient agent” with all the personal details which includes full name of patient, mailing address (physical and email), mobile number, age, insurance details, current health issue/s and alternative communication information. Patient will receive user credential to login to the PA portal and to access the cloud storage assigned to the patient for storing records. If the patient is already registered, then he will login to the portal and describe the current health problems. If the patient is newly registered and seeking health advice, then PA agent will start acting on the requirement basis. This will include: (1) (2)

If the patient can be taken care at home. Need an emergency transfer to hospital.

If the registered patient condition can be taken care of at home, PA will list the medical device requirements with costing, once the patient fills the required details after login to the PA application portal. If the patient can support costing and agrees to take health at home treatment, PA will send his local representative (nearer to

7 Multi-Agent System Applications in Health Care: A Survey

165

Fig. 2 Semi-automated multi-agent-based patient treatment system in hospital

patient address) to equip and configure the medical instruments which will monitor the patient health online and synchronize using the PA application with the patient mobile device. This mobile device will also be synchronized with the cloud storage (e.g., One drive, Google storage) provided by the PA at the time of user account creation. PA will gather the information from the cloud storage using the same PA application where the user registration number mapped to cloud storage patient profile path and will take necessary action which will include online doctor advisory or requirement of hospital transfer.

166

C. Thakur and S. Gupta

In case of hospital transfer, PA agent will contact Ambulance agent for sending patient to hospital. Ambulance agent is one of the sub-agents of PA’s application. Patient will pay for this first level of service through online PA payment gateway.

5.2 Model 2 (Describes the Patient Treatment at Hospital) Hospital healthcare management system of a serious patient coming from homecare system is presented here. If the record captured in “Model 1” seems critical, where a patient needs to be hospitalized, then this “Model 2” will guide the patient in sequential and automated way. This will take care of the transfer of a patient from home to hospital and arrange all the required facilities sequentially throughout the period of patient attachment to that hospital. PA is solely responsible to take care of the patient treatment, immediately after being transferred to hospital. The objective of this “Module 2” is to provide a guideline to a patient sequentially so that the treatment can be started as early as possible and to discourage hospital muddled system. After bringing patient to the hospital, PA will submit some of the major inputs regarding the present status of the patient (Pa) to reception agent (RA) with the request to admit Pa to the hospital under a most capable (PA maintains list of doctor in some specialized hospitals) doctor agent (DA). PA will request for a suitable bed (according to its economic condition and/or based on its medical insurance policy condition). RA will now search for bed and doctor agent (DA) through its knowledge base; if the bed is available RA will allocate the bed/room to the PA with the help of nurse agent (NA) through ward agent (WA). When the bed is not available according to PA’s desire, RA will search for higher/lower category rooms for PA after consulting with its own knowledge base (KBra) and the knowledge base of hospital management agent (KBhma). After search, if RA finds some higher or lower category rooms, it (RA) will intimate the same to PA. PA, in turn, can exercise its choice in choosing the bed according to its economic condition/medical insurance (if exists) policy condition. There can be a situation when more numbers of patients are waiting in the queue than the number of vacant bed/s. In that case RA will follow some priority scheduling algorithm on the available beds stored in its knowledge base, in consultation with HMA. RA will consider bounded waiting, mutual exclusion and progress to avoid critical section problem and deadlock. RA may take time for scheduling the admission for Pa and Pa has to wait for certain period of time. But if no room is available up to a certain permissible time, Pa can move to some other healthcare system. After the bed is allotted for Pa, RA will approach to the medical insurance agent (MA) for concurrence (only for medical insurance patient), then RA will check for room readiness from NA through WA and transfer the patient (under the most capable DA) and send the intimation to account agent (AA).

7 Multi-Agent System Applications in Health Care: A Survey

167

After preliminary checkups NA will send call book to DA. DA takes over the charges of Pa and checks its knowledge base KBda. If problem of Pa can be solved only with its own KBda, then DA will suggest for some medicine and investigation. NA will schedule investigation agent (IA) and shop agent (SA) accordingly. IA1 will try to investigate the problem through its knowledge base if possible, otherwise it will be sent to the next capable IA2, IA3,… IAn. On completion of investigation/s, IA/s will send reports back to DA through NA. Based on Investigation report/s, DA might refer the problem to some other DA for consultation. Based on suggestion from other DA/s and reports from IA/s, DA will generate most efficient solution for Pa and it will be continued till the recovery of Pa (Fig. 2). Everyday knowledge base of AA and MA need to be updated according to the information received from SA, IA, DA and WA. On recovery, AA will settle down all account related formalities with PA and MA (in case of medical insurance patient). NA will hand over discharge summary to PA before PA transfers Pa back to home. All related information about Pa need to be updated in knowledge base of HMA (KBhma).

6 Conclusion Health is wealth. So, health issues are important than any other issues in human life. So hospital, nurse, medical centers and person related to health care are very special for us. Whenever a person has critical health suffering, retrieval of patient’s medical history is important for saving patients life and a chain of process may be needed to support the patient, i.e., diagnosis, information retrieval, medication, performing tests, operations (if required), storing data and many more. If we analyze the knowledge required for solving a problem, it will be spatially distributed in different locations. Each specialist doctor located in different places uses their knowledge to solve the problem. Tests are carried out on different locations with the help of some different sets of knowledge. Giving solution to a particular case requires proper coordination between different individuals with their different skills and functionalities. As we know the agent in the multi-agent system acts like a human being and work more accurately and efficiently than human being. So, multi-agent system can be a better solution to this present approach, where patient and health agents are distributed over the cities and need proper and accurate communication between them. So, in this survey paper, we have tried to present the papers where MAS is applied in healthcare system. We know if we want to know about the tree, we should not only concentrate on the flower of the tree, we should equally know the root of the tree, how it grows to produce flowers. Keeping that fact in mind though the paper is based on MAS application in health care, but we started survey from the architecture of MAS then modeling and planning of MAS and then MAS applied in health care. Again the last category of papers is divided into sub-categories because health care is not only related to patients and doctors, it is influenced by many other categories. So, this

168

C. Thakur and S. Gupta

paper is further described the sub-category of papers related to health. As we know, for designing of any system, we need security. So, the first sub-category is the security in MAS applied in health care. Most of the time health issues are taking us to hospital. So, the next two sub-categories are gathering requirements in hospital management system using MAS and MAS applied in hospital management system. Next issue is nurse scheduling and ambulatory system, and last but not least how MAS is applied in patients scheduling. At last, we have tried to give some new system/idea which is named as Treatment as a Service (TaaS) about how healthcare system can be easily provided to the common people from home to hospital in 24 h basis. The future researchers in this field might get help from our survey paper as different varieties of papers are incorporated in this study. Acknowledgements We would like to express our gratitude to all those who gave us the possibility to complete this paper.

References 1. Weizenbaum J (1976) Computer power and human reason. W.H. Freeman and Company, Penguin Books Ltd, New York. ISBN 0-7167-0463-3 2. Chakraborty S, Gupta S (2014) Medical application using multi agent system—a literature survey. Int J Eng Res Appl 4(1):528–546 3. Epstein JM, Axtell R (1996) Growing artificial societies: social science from the bottom up. Brookings Institution Press, p 224. ISBN 978-0-262-55025-3 4. Carley KM (2002) Construct, computational analysis of social organizational systems. In: Proceedings of the national academy of sciences of the United States of America, vol 99 5. Samuelson DA (2000) Designing organizations, OR/MS Today, December 2000 6. Samuelson DA (2005) Agents of change. OR/MS Today, February 2005 7. Samuelson DA, Macal CM (2006) Agent-based modelling comes of age. OR/MS Today, August 2006 8. Graham JR, Decker KS, Mersic M (2003) DECAF-a flexible multi agent system architecture. Auton Agents Multi Agent Syst 7(1):7–27 9. Chopra A, Singh MP (2009) Architecture for multi agent systems: an approach based on commitments. In: Proceedings of the AAMS workshop on Pro-MAS, May 2009 10. Shehory O (1999) Architecture properties of multi agent systems. ResearchGate/Publication/2279275 11. Stockheim T, Nimis J, Schotz T, Stehi M (2004) How to build a multi agent system: the agent. Enterprise Approach, ICEIS 12. Celaya JR, Durocher’s AA, Graves RJ (2007) Modelling and analysis of multi agent systems using patrinet. In: Systems, man and cybernetics, ISIC, IEEE 13. Mouratidis H, Giorgini P, Manson G (2003) Modelling secure multi agent systems, ACM. In: Proceedings of the 2nd international joint conference on autonomous agents and multiagent systems, pp 859–866 14. Hadzic M, Dillon DS, Dillon TS (2009) Use and modelling of multi agent system in medicine. In: IEEE 20th international workshop, pp 303–307 15. De Weerdt M, Ter Mors A, Witteveen C (2005) An introduction to planning and coordination. In: Handouts of the European agent summer

7 Multi-Agent System Applications in Health Care: A Survey

169

16. De Weerdt M, Clement B (2009) Introduction to planning in multi agent systems. Multiagent Grid Syst 5(4):345–355 17. Chen YF, Keman N, Choudhury UG, How JP, Vian J (2014) Planning for Large scale multi agent problems via hierarchical decomposition with application to UAV health management, AACC, June 2014 18. Sanchez D, Moreno A (2010) Agent applied in health care: a review. Int J Med Inform 79(3):145–166 (Elsevier) 19. Sycara KP (1998) Multi agent systems. AI Mag 19(2):79–92 20. Shakshuki E, Reid M (2015) Multi agent system applications in healthcare: current technology and future roadmap, science direct. In: 6th International conference on ambient systems, networks and technologies, procedia computer science, vol 52, pp 252–261 21. Anagnostou A, Nouman A, Taylor SJE (2013) Distributed hybrid agent-based discrete event emergency medical services simulation. In: IEEE conference publication, pp 1625–1636 22. Brailsford SC, Viana J, Rossiter S, Channon A, Lotery AJ (2013) Hybrid simulation for health and social care: the way forward, or more trouble than its worth? In: IEEE WSC proceedings, pp 258–269 23. Flores CD, Barros P, Cazella S, Bez MR (2013) Leveraging the learning process in health through clinical cases simulator. In: IEEE conference paper, pp 1–6 24. Lin Y, Lu X, Fang F, Fan J (2013) Personal health care monitoring and emergency response mechanism. In: IEEE conference paper, pp 1–5 25. de Moraes JLC, de Souza WL, Pires LF, Cavalini LT, do Prado AF (2013) A Novel Architecture For Message Exchange In Pervasive Healthcare Based On The Use Of Intelligent Agents. In: IEEE ACS international conference, pp 1–8 26. Gupta ShibaKali, Sarkar A, Pramanik I, Mukherjee B (2012) Implementation scheme for online medical diagnosis system using multi agent system with JADE. Int J Sci Res Publ 2(6):1–6 27. Gangwar M, Yadav RS, Mishara RB (2012) Semantic web services for medical health planning. In: IEEE 1st international conference on RAIT, pp 614–618 28. Taboada M, Cabrena E, Iglesias ML, Epelde F, Luque E (2012) An Agent Based decision support System for Hospital Emergency Departments. In: Science direct, procedia computer science. ELSEVIER 29. Gaetano L, Balestra G (2011) A multi agent system model for evaluating quality service of clinical engineering department. In: Annual international conference of IEEE, pp 1209–1212 30. Saleem RM, Muhammad A, Martiner Enriquez AM (2010) Remote patient monitoring and health care management using multi agent based architecture. In: IEEE MICAI 9th Mexican conference, pp 118–123 31. Baffo I, Stecca G, Kaihava T (2010) Multi agent system approach for hospitals drug management using combinatorial options. In: IEEE 8th international conference, pp 945–949 32. Iantovis B (2010) Cognitive medical multi agent systems. Broad Res Artif Intell Neurosci 1(1):12–21 33. Su C-J, Wu C-Y (2011) JADE Implemented mobile multi agent based distributed information platform for pervasive healthcare monitoring. Appl Soft Comput 11(1):315–325. ELSEVIER 34. Geogopoulos PG, Isukapalli SS (2009) A unifed multiscale field/network/agent based modelling framework for human and ecological health risk analysis. In: Conference proceedings IEEE Engineering in Medicine and Biology Society 35. Shadabi F, Sharma D (2009) Using knowledge and rule induction methods for enhancing clinical diagnosis: success stories. In: IEEE conference ICECC, pp 540–542 36. Iantovies B (2007) The CMDS medical diagnosis. System. In: Symbolic and numeric algorithm for scientific computing, Synasc international seminar IEEE, pp 246–253 37. Se CJ (2007) Mobile multi agent based distributed information platform (MADIP) for wide area eHealth monitoring, Elsevier 38. Han BM, Song S-J, Lee KM, Jang K-S, Shin D-R (2006) Multi agent system based efficient health care service 39. Chan V, Ray P, Parameswaran N (2008) Mobile eHealth monitoring: an agent based approach, The Institute of Engineering & Technology (IET), vol 2, issue 2, pp 223–230

170

C. Thakur and S. Gupta

40. Su C-J, Shih SC (2011) Building distributed e-Healthcare for elders using RFID and multi agent. Int J Eng Bus Manag 3(1):16–26. https://doi.org/10.5772/45677 41. Ardissono L, Di Leva A, Petrone G, Segnan M, Sonessa N (2006) Adaptive medical workflow management for a context—dependent home healthcare assistance service, ENTCS, vol 146, issue 1 pp 59–68. https://doi.org/10.1016/j.entcs.2005.11.007 42. Tiwari A, Patel PK, Singh VK, Shrivastava A (2013) Implementing requirement for hospital management system using multi agent system. Int J Sci Eng Res 4(6): 2407–2411 43. Tian J, Huaglory, Tianfield (2009) Health delivery systems—a case for multi agent systems. In: IEEE international conference on system, man and cybernetics, pp 2818–2722 44. Hill R, Polovina S, Beer M (2005) Managing healthcare work flows in a multi agent system environment. In: 19th IJCAI 45. Mateo RMA, Gerardo BD, Lee J (2008) Health care expert system based on group cooperation model. Int J Softw Eng Appl 2(1):105–116 46. Moreno A (2003) Medical applications of multi agent systems. In: Conference proceeding intelligent and adaptive Systems in medicine, Prague, 15 pages 47. Cavaleante RC, Bittencourt II, da Silva AP, Silva M, Costa E, Santos R (2012) A survey of security in multi agent system. Expert Syst Appl 39(5):4835–4846. https://doi.org/10.1016/j. eswa.2011.09.130 48. Kushwaha N, Sahu S, Ahemd P (2012) Gathering requirements for hospital management system (HMS) using intelligent agents, IJEIT, vol 1, issue 3, pp 276–279 49. Kushwaha N, Sahu S, Tyagi RK (2013) Evolving intelligent agents for hospital management system. In: IEEE 3rd international conference IACC, pp 899–907 50. Liaqat RM, Athar A, Saqib NA (2015) Intelligent agent based system for monitoring and control of hospital management system. In: 2nd IEEE international conference on information science and security (ICISS), pp 1–5 51. Jemal H, Kechaon Z, Ayed MB, Alim AM (2015) A multi agent system for hospital organization. Int J Mach Learn Comput (IJMLC) 5(1):51–56. https://doi.org/10.7763/IJMLC.2015. v5.482 52. Stiglic G, Kokol P (2005) Intelligent patient and nurse scheduling in ambulatory healthcare centers, IEEE-EMBS, pp 5475–5478 53. Mansour A, Ying H, Dews P, Ji Y, Yen J, Miller RE, Massanari RM (2011) Finding similar patients in a multi agent environment. In: Proceedings of the 30th North American FUZZY information processing society conference, El Paso, TX, pp 18–20 54. Benhajji N, Roy D, Anciaux D (2015) Patient centered multi agent system for healthcare. Science Direct, IFAC Papers Online, vol 48, no 3, pp 710–714 55. Zoller A, Braubach L, Pokahr A, Rothlauf F, Paulussen TO, Lanersdorf W, Heinz A (2006) Evaluation of a multi agent system for hospital patient scheduling. Int Trans Syst Appl 1(4):375– 380 56. Paulussen TO, Zoller A, Rothlauf F, Heinzl A, Braubach L, Pokahr A, Lamersdorf W (2006) Agent based patient scheduling in hospitals. In: International handbook on information systems, Springer, Chapter 4, pp 255–275 57. Paulussen TO, Heinzl A, Becker C (2013) Multi agent based information systems for patient coordination in hospitals, ICIS 58. Tiwari A, Patel PK, Singh VK, Shrivastava A (2013) Implementing requirement for hospital management system using multi agent system. Int J Sci Eng Res (IJSER) 4(6):2407–2411 59. Kostkova P, Mani Saada J, Madle G, Weinberg JR (2003) Applications of software agent technology in the health care domain. In: Moreno A, Nealon JL (eds) Agent based up to date data management in national electronic library for communicable disease (Chap. 8), pp 105–124. Springer, Basel AG 60. Vieira-Marques PM, Robles S, Cucurull J, Cruz-Correia RJ, Navarro G, Marti R (2006) Secure integration of distributed medical data using mobile agents. IEEE Intell Syst 21(6):47–54 61. Tentori M, Favela J, Rodriguez M (2006) Privacy aware autonomous agents for pervasive healthcare. IEEE Intell Syst 21(6):55–62

7 Multi-Agent System Applications in Health Care: A Survey

171

62. Singh S, Ikhwan B, Haron F, Yong CH (2005) Architecture of agent based healthcare intelligent assistant on grid environment, vol 3320 of LNCS. Springer, Singapore, pp 58–61 63. GonzálezVélez H, Mier M, JuliàSapé M, Arvanitis TN, GarcíaGómez JM, Robles M, Lewis PH, Dasmahapatra S, Dupplaw D, Peet A, Arús C, Celda B, Huffe SV, LluchAriet M (2009) Health agents: distributed multi agent brain tumour diagnosis and prognosis. Appl Intell 30(3):191–202 64. Isern D, Moreno A (2008) Computer based execution of clinical guidelines: a review. Int J Med Inform 77(12):787–808 65. Kirn S, Anhalt C, Krcmar H, Schweiger A (2006) Multi agent engineering: theory and applications in enterprises. In: Kiran S, Herzog O, Loakemam P, Ch OS (eds) International handbook on information systems. Springer, Berlin, Heidelberged. Hospital—Health Care Applications of Intelligent Agents, pp 199–220 66. Cortés U, VázquezSalceda J, LópezNavidad A, Caballero F (2004) UCTx: a multi agent approach to model a transplant coordination unit. Appl Intell 20(1):59–70 67. Braun L, Wiesman F, van der Herik J, Hasman A (2005) Agent support in medical information retrieval. In: Cortés U, Fox J, Moreno A, Nealon J (eds) Proceedings of the 3rd workshop on agents applied in health care, AAAI, Edinburgh, Scotland, pp 16–25 68. Bagüés MI, Bermúdez J, Burgos A, Illarramendi A, Tablado A, Rodriguez J, Goni A (2006) An innovative system that runs on a PDA for a continuous monitoring of people, IEEE, pp 151–156 69. Koutkias V, Chouvarda I, Maglaveras N (2005) A multi agent system enhancing homecare health services for chronic disease management. IEEE Trans Inf Technol Biomed 9(4):528–537 70. Isern D, Moreno A, Sánchez D, Hajnal A, Pedone G, Varga LZ (2011) Agent based execution of personalized home care treatments. Appl Intell 34(2):155–180 71. Corchado JM, Bajo J, Abraham A (2008) GerAmI.2008. Improving healthcare delivery in geriatric residences. IEEE Intell Syst 23(2):19–25 72. Kim HK (2014) Convergence agent model for developing u-healthcare systems. Future Gen Comput Syst 35:39–48

Chapter 8

Applications of Multi-agent Systems in Intelligent Health Care M. Bhanu Sridhar

Abstract Artificial intelligence has surged into human lives as an important requirement since the start of twenty-first century. Artificial intelligence (AI) can be understood as replications of human intelligence in systems that can be devised to ‘think and act’ like humans. Machine learning refers to the concept of programs or applications that ‘learn’ automatically and adapt fresh data without human support and is a subset of AI. One of the imperative areas in AI is distributed artificial intelligence (DAI) where complex problems can be dealt with the concept of multi-agent systems (MAS). MAS have become a bridge to come over difficult tasks and to bring out the best possible solution(s). Healthcare data is a domain of data science and can be considered as one of the most dynamically generated data. Multi-agent systems in healthcare data are the best combination possible to utilize the advantages that are available in both the sub-fields. Different diseases can be addressed through division of the task(s) as per the norms of MAS. Best results can be obtained here, and it can be truly helpful for the society and mankind. This chapter focuses on the applications that are possible through MAS in health care. Basics, important issues and algorithms are dealt with here in a comprehensible manner, and it is believed that the contents of this chapter can be immensely useful to the experts, researchers, data scientists, medicos and of course, to the society. Keywords Healthcare data · Data science · Telemedicine · E-Health systems · Societal needs

1 Introduction 1.1 Artificial Intelligence Throughout the world, the continuously evolving technologies and trends in the IT field have become an imperative part in the human lives. Recent generations have M. Bhanu Sridhar (B) GVP College of Engineering for Women, Visakhapatnam, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 S. Gupta et al. (eds.), Multi Agent Systems, Springer Tracts in Human-Centered Computing, https://doi.org/10.1007/978-981-19-0493-6_8

173

174

M. Bhanu Sridhar

turned to the concept of artificial intelligence (AI) to enhance their lives and improve the facilities that are both being offered and utilized to produce fruitful results. Though being an old concept [1], modern AI is being viewed as the best possible solution for many problems that keep on surfacing in the IT field. After being hit hard by recent COVID-19, the importance of the domain of health care has been really realized by the society and industry, and a rush can now be observed for development of applications that are more useful for the societal needs in the health domain. This area is the most sought after by scientists, researchers, and practitioners to bring out new concepts and enhance the existing ideas in healthcare domain. This idea is among the greatest challenges of AI [1] and lately has become an important part of medical systems and devices [2]. Usage of different algorithms, methodologies, and theorems so as to obtain the best possible solution to different problems in human lives is the new trend of AI usage. The applicability of these concepts to healthcare domain has increased and in an accurate manner, recently. Users of healthcare applications have the necessity of data appropriate to their needs and less error-prone, so that endorsements made available by the systems are secured and satisfying [3]. Thus, time is apt for the concept(s) of AI to take over extra areas in health domain so that the results are best possible both for medical practitioners and patients. At this juncture, the AI field should research a bit deep to achieve the above said goals. Hence, the concept of distributed artificial intelligence has come about as the savior of time and energy. Previously, research in artificial intelligence was mostly oriented to single agent systems. In that environment, an agent has many activities including gathering data, planning, designing, and execution. But this approach hindered the growth of performance and accuracy due to the presence of multiple agents in the real world. Keeping this short come in mind, the idea of allocation of short tasks to multiple agents surfaced in the research community. All the agents coexist and cooperate with different possible collaborations and short goals in DAI but work toward the same goal.

1.2 Distributed Artificial Intelligence Distributed artificial intelligence has existed as a sub-field of artificial intelligence since many years [4] (Fig. 1). Distributed artificial intelligence (DAI) can be classified as a sub-domain of AI which ranges from basics to MAS. It generally deals with the bringing up solutions that range over different areas like health care, cloud computing, and so on [5]. It can primarily deal with learning, reasoning, and planning, and is an important subarea of AI. Here, replication receives more attention rather than likelihood. Here, sovereign agents concerned with learning reach conclusions through interaction and communication. According to [6] and [7], a DAI can be quoted as an area concerned with:

8 Applications of Multi-agent Systems in Intelligent Health Care

175

Fig. 1 Distributed artificial intelligence (Image Credit aurielaki/Shutterstock)

• Distribution of tasks between agents, • Distribution of powers, • Interactions between agents. Lately, DAI is being looked at as a different field with its own sub-fields.

2 Multi-agent Systems in Distributed Artificial Intelligence To build a system concerned with the above said area, it is important to concentrate on its method of dealing with the work and its architecture [8]. The agents involved like humans or systems in between also have to be dealt with a careful ideology (Fig. 2). In distributed problem solving (DPS), several agents work with intercommunications to crack an explicit task [9]. The main point here is that communication and cooperation are very much needed since an agent does not possess data, knowledge, or abilities to resolve the complete task. The challenge for any researcher is to make sure that information and abilities are appropriately assigned so that agents do not conflict with each other.

176

M. Bhanu Sridhar

Fig. 2 Dispersed problem solving

2.1 Concept Centralized control systems place a no. of constraints on the systems that are making them as a base and trying to bring out solutions. In such situations, multi-agent systems offer amicable solutions through distribution of data by communication between the agents. Though not every MAS takes full advantage of this potential, they can be quoted as offering production systems that are decentralized, emergent, and concurrent [8]. This methodology of concentration on ‘independency’ is overriding the concept of a consolidated databank and a governing system with interconnected agents, where an agent has a rational interpretation of its environment and can respond internally to any queries or tasks assigned. The performance of the concerned software is not planned initially but materializes through the dynamic communication between agents. The schedule is put in place through simultaneous autonomous decisions of the local agents (Table 1). The self-governing agent approach can meet with different consequences. Systems of autonomous agents, if not administered properly, might lead to instability if any mishap occurs or generally. The level of significance of these tasks needs to be evaluated carefully since ideals figured by conservative software may not be applicable in real time. Detailed predictions that are permitted are often shelved by the real world. An autonomous approach offers some substantial benefits over conservative systems. This happens since every agent can be the place of contact and the computational part takes note of this, that too without any requirement of a consolidated database. Since the system performance surfaces from local decisions, the system

8 Applications of Multi-agent Systems in Intelligent Health Care Table 1 Autonomous agent compared with conventional ideas

177

Issue

Autonomous agents

Conventional

Model

Economics, biology

Military

Issues favoring conventional systems Theoretical optima?

No

Yes

Level of prediction

Aggregate

Individual

Computational stability

Low

High

Issues favoring autonomous agents Match to reality

High

Low

Requires central data?

No

Yes

Response to change

Robust

Fragile

System reconfigurability

Easy

Hard

Nature of software

Short, simple

Lengthy, complex

Time required to schedule

Real time

Slow

realigns itself habitually to different situations like exclusion or inclusion of agents. The software of each agent is modest than that for a centralized methodology. As such, it is easier to prepare, test, correct, and maintain. As the system schedules itself dynamically, no separate scheduling phase is needed, reducing the time to schedule and complete the assigned task.

2.2 Multi-agent Systems (MAS) MAS is a pool of independent agents that converse among themselves to organise activities, through which, a problem can be solved collectively and concurrently [3]. An agent can be considered as a part of the system that utilizes AI concepts to select the suited activities that are to be performed so as to reach a specified goal (Fig. 3). It is assumed that multi-agent systems possess the under listed properties: • They can respond sensibly and compliantly to the dynamic variations in their situation. • They have an independent behavior and are not controlled by any exterior object. • They can initiate and accomplish proactive activities that provide assistance in reaching their aims.

178

M. Bhanu Sridhar

Fig. 3 Agent-based modeling: https://bit.ly/2FcXOB5 [10]

• They can connect with users or other agents. By this concept, they can interchange data and coordinate their tasks to bring out a combined solution of any problem. • They possess the capabilities of analysis, development, and knowledge capabilities that allow them to exhibit an intellectual and timely behavior. Agents are given a chance to introduce novelties in the ways of their responses to the users. As an example, we can say they recognize the user by behavior or voice; acclimatize the UI to their way of functioning, and document analyzes, etc. (Fig. 4).

2.3 Approaches of Multi-agent Systems’ Architecture The main approaches of multi-agent architectures are being discussed here, as per their harmonization. The organization categories are: (a) Centralized multi-agent coordination, where an agent assumes the work of collection of fractional tactics from agents, integrates them, and resolves conflicts (if

8 Applications of Multi-agent Systems in Intelligent Health Care

179

Fig. 4 MAS for decision support [15]

any), and (b) dispersed multi-agent coordination where agent-control is not centralized but can possess intercommunication among themselves for creating their plans and potential solutions [11] (Fig. 5). Although there are several aspects that may be considered to classify multi-agent systems, a classification scheme is given below based on interaction characteristics. Centralized Architectures: In this case, an ‘agent controller’ keeps an eye on other agents. A controller/coordinator is a specified agent who can manage other agents and can direct the flow of information among them. Each agent has to approach the coordinator intercommunication among them. We can also visualize an architecture

Fig. 5 Autonomous agent system architecture [12]

180

M. Bhanu Sridhar

Fig. 6 Classification of multi-agent systems [6, 7]

where central coordinator exists to handle coordinators’ communication process. The key representatives of coordinator are the mediator and the facilitator. Decentralized architectures: According to [12], an independent agent is the one that concentrates whether: (a) (b) (c) (d)

If it is controlled by an agent or human, If it can interconnect instantly with other agents in the current system or other systems, If it has familiarity of further agents and their background, and If it dedicates its own tactics for its own intent (Fig. 6).

2.4 Usage of MAS in Recent Times How can we interpret the word ‘agent’ in modern and distributed artificial intelligence? Many versions have been shaped as of now [6] (Fig. 7).

Fig. 7 Potential explanation of an ‘artificial agent’ [6]

8 Applications of Multi-agent Systems in Intelligent Health Care

181

The idea of personal assistants existed in many software products previously. Microsoft has built Wizards and System Agent in Windows 95 and paperclip-assistant Clippit has appeared in MS-Office. Mac OS contains a learning agent Open Sesame!, Lotus Notes V4 also has built-in agents. Among the modern personal assistants a voice helper Siri, existing in Apple’s products can be distinguished [7]. Consequently, agents can be quoted to be independent software modules that can work together with the user. If we say ‘semi-autonomous,’ we mean the level up to which an agent can depend on the user, i.e., the user’s ability to change the level of sovereignty of the agent. This ensures a friendly character of the UI. These concepts relate to the theory of ‘human–computer interaction (HCI)’ and are the roots of development of intellectual interfaces [6]. The demand for software agents was initiated with the development of the Internet and arrival of chatbots. Information agents, like PointCast, can convey updates to users and report variations in particular sites. Shopping agents and crawlers roam in the links and index information for the search engines. Some others can supervise a transaction with the users to interchange crypto-currency and so on. The extreme projections for further applications of modified ‘agents—user assistants’ are related with exact information search in Internet, as well as the support of multi-criteria decision(s). It can be anticipated that the concept of ‘agent ware’ will become extensive soon [7]. Recently, the application of software engineering techniques to agents, known as Agent-Oriented Software Engineering (AOSE) [13], has produced several related research studies. These works have been focused on searching for methods which are able to model complex systems with distributed characteristics.

2.5 Fields of MAS Application Thus, a multi-agent approach would be practical for tasks that are essentially distributed where autonomous procedures can be clearly distinguished [14]. These include distributed sensor networks, hospital management systems, air traffic control, or others. At the same time, requirements for adaptivity in other capabilities that include cognitive capability of the agent should also be put in place. Many reasons can exist for taking up a distributed approach to the software systems that are in the pipeline of reputed companies. Minsky’s Society of Mind paradigm [15] suggests the use of MAS if a range of self-sufficient sub-parts of functionality are in existence which obviously requires the use of AI in all senses. A multi-agent approach can also be useful for a single robot manipulator that considers every joint as a potential agent, where response to a stimulus can be obtained. Multi-agent systems are effectively utilized in various domains like power engineering, logistics, food delivery, and most recently in health care extensively. After COVID-19, companies and employees have got used to the concept of ‘work from home (WFH).’ At this juncture, MAS concept comes handy in all the above said domains and many other emerging potential areas.

182

M. Bhanu Sridhar

It is being agreed that more research has to be done in MAS and its accuracy levels in different domains. Though it provides at acceptable levels of accuracy, more ideas and algorithms have to be brought up the researchers. Since the world is moving at an entirely different angle after the pandemic, every field can exploit the application of MAS in its domain, that too at a higher level of exploitation of resources and providing services at higher performance and accuracy.

3 MAS in Health Care At all possible levels, health care is an enormous open environment characterized by distributed decision making and care management, which requires the communication of complex forms of information between a variety of clinical and other settings. Coordination between groups of healthcare professionals with very different skills and roles is also considered to be important [3]. The aim of such systems is to operate effectively in the above said environment, so as to meet the needs of patients and healthcare providers. Practitioners in healthcare environments require that the information is appropriate and noise-free to make recommendations offered by the software systems secure and reliable [16]. The multi-agent system approach offers an authoritative base for exhibiting and resolving problems. This approach can be effectively prolonged to plan the subdomain of health data. This has been proved by the concept of telemedicine, after people are now days confined to their houses. Before considering what can be achieved by MAS in health care, a brief definition of what is a multi-agent system must be understood and applied [17]. Healthcare-based MAS applications are designed to reduce stress on the system concerned with medical data; it can reduce the charge of service-providing. As advancements in biomedical sensor nodes and power harvesting procedures are coming up, healthcare-based multi-agent system applications are now easier to utilize with and more capability [17]. In the current healthcare scenario, patients have multiple health records in different healthcare information systems, which mean patient-related data being split into different systems. The need to access patient data across these systems and manage the information flow between various systems has increased the complexity of healthcare information systems [18]. Therefore, interoperability in healthcare information systems has become an important requirement—not a feature. This is where the MAS concept comes in to picture and more explicitly, for the healthcare domain, where its services are most needed to be utilized for these days. It can be deduced from this brief discussion that multi-agent systems save time both for patients and medical practitioners and hence the precious lives.

8 Applications of Multi-agent Systems in Intelligent Health Care

183

3.1 Discussion on Expert Systems Expert systems (ESs) are a type of practical artificial intelligence (AI) and are used for specific purposes, like predicting the disease of a patient from the answers to some queries that are posed. But it should be noted that this idea has never considered the point of cases where the patient might be at another location. Then, questions posed by ES are not answered, and the prediction or output completely follows the currently available data; the results from the tests conducted on the patient are considered. It should be noted that ESs are generally used for predicting the disease of a patient or the side-effects of medicines but not for prescribing medicines. The data collection used by expert systems is classically error-prone. Much more, expert systems use computational engines that are not capable of reasoning in the required angle of approach. The logic that is used is based upon facts that may undergo changes, which further degrades the level of the expert systems in health care. The world has been hit hard by the current going COVID-19 pandemic. With no medicine available, no previous experience, and inappropriate and scattered data, the reality has been realized—new approaches should be considered, to be ‘really’ helpful to the patients. Proceeding from this perception, statistical models or machine learning algorithms can be used in such situations. They can now be applied in healthcare applications to suggest suitable medications. Concrete simulations such as the Gaussian mixture model and the Pearson family of equations are the best ones possible. The model that is suited for the considered situation or data must be decided depending on specific factors like type of data, time available, etc. [19].

3.2 Differences Between Expert Systems and Artificial Intelligence It should be understood primarily that expert systems are an applied area of AI. An expert system (ES) employs knowledge about its application domain and uses a process to resolve problems that need human capability. The power of ES stems primarily from the specific knowledge about an arrow domain stored in the expert system’s knowledgebase [20]. It is important to persist that expert systems are subordinates to decision makers and not alternatives for them. Expert systems do not have human competencies. They use a knowledgebase of a domain and depend on the facts of the current situation. The knowledge base of an ES also contains empirical knowledge—rules of thumb used by professionals of that domain. The strength of an ES derives from an organized collection of facts and heuristics about the system’s domain. It is based on available data concerning the domain, which is acquired from human experts and other sources. The accumulation of knowledge in knowledge bases, from which conclusions are to be drawn by the inference engine, is the hall mark of an expert system.

184

M. Bhanu Sridhar

Artificial intelligence (AI) is concerned with development of systems that possess features of human intelligence. They are designed to emulate the human skills of thinking and intuiting. Artificial intelligence is a technology based on a combination multiple disciplines like computer science, biology, mathematics, and engineering. The goal here is to bring up computers that can think, see, hear, walk, talk, and feel [21]. A sub-domain of AI is the expansion of computer functions concerned with human intelligence, like reasoning, learning, and problem solving. An ES is surely not an alternative for the overall performance of a human expert in problem solving. But they can reduce the work the individual has to do, to solve a problem. They can also lay the red carpet for creative and innovative angles of problem solving. Note that technologies do not offer an easy or total solution to a problem. Large systems are costly and require significant development time and computer resources. Expert systems surely have their own limitations and drawbacks.

3.3 How MAS is Useful in Healthcare Domain In the different domains that can be deduced on the available data, healthcare domain occupies a prominent place. Since it is concerned with the question of life or death of patients or the health situation of the society itself, healthcare domain is being given utmost importance in modern fields of artificial intelligence, data science, and machine learning. Whatever may be the data that is being analyzed, the process has to be carried out with paramount care. As discussed previously, existing methodologies like normal data entry, giving appointments to the patient and reports to be typed by the doctors is not as effective as they seem. In modern times, the concept of artificial intelligence has been look upon with more eager and enthusiasm. In recent years, agent-based systems have developed as an effective methodology to bridge the gaps in the healthcare data. This domain is an enormous area framed by its distributed decisional features and its organization of care, which requires a communication and a complex management between the various medical areas. The beginning of MAS using in health care eased the stress for the concerned people to take decisions and ensured the communication by reducing errors of analysis and treatment. All this process reduces the time needed to look for the medicinal resources [22]. The aim of agents is to observe information base in the situation context and to support the decision making process concerning an action by professionals in. Lastly, it should also execute background actions with their own communication processes [23]. Note that expert systems utilize logic in every condition, whereas agents act more like individuals [24].

8 Applications of Multi-agent Systems in Intelligent Health Care

185

There are a variety of areas in healthcare systems that could reap the benefits from multi-agent system technology. (1) (2) (3) (4)

Disease detecting systems, Treatment endorsing systems, Systems that inspect and use the patient history, Other units like comforting units and so on.

It is remarkable that many problems that appear in health care have many similarities. Observing and zeroing them is the best way possible to unearth a structure that can address the challenges. The following points can be quoted as the most important ones [25]: • It is usual that the knowledge required to solve a problem is distributed in different locations. Consider the problem of granting appointments to outpatients or the tasks to be finished when a doctor is on rounds in a hospital [26]. It can consist of scheduling different tasks to be performed on a hospitalized patient. Generally, each department of a hospital retains its own data about the patients hospitalized in that unit and about the schedule of some other activities. There are units that provide services to all the other units of the same hospital, such as radiology or sample collection/testing. If any constraints or restrictions are in place due to some implicit reasons, multi-agent systems are very helpful to control the situation and apply some reasoning to produce reasonable results. • The solution of a problem involves the coordination of the effort of different individuals with different skills and functions. Typically, the provision of care to hospitalized patients involves a number of staff members of medical center (e.g., receptionist, nurses, in first phase, specialists in a second stage, and others like technicians, management people, etc.). All these should synchronize their actions to deliver the best treatment possible to the approached. • Medical care problems are complex, and software engineering solutions for them are not as easy as they might look. Using outdated models and methodologies will surely not suffice for the current day scenario of endemics and pandemics. If ventilators’ data or available beds’ data is scattered, as it is, no patient can be allotted a bed when it is deduced that he has been infected with COVID-19. Still yet, a normal patient wishing to consult a physician cannot be granted an appointment without integrating, and organizing the data. Data does exist but is not understandable, fully centralized at each levels and no coordination exists. This situation results only in an utter chaos. MAS can be very helpful in the above said situation(s) to produce satisfactory results. • There exists much medical knowledge in the Internet, lately. It is essential to offer best ways of retrieving the most applicable data easily and compliantly.

186

M. Bhanu Sridhar

Access to medical data is essential for doctors and patients, in their own angles and views. Doctors can compare the deduced results with previously available data or results, check out the medicines available and prescribe the best ones. Patients may also claim an active role in controlling data manipulation and security and are required to find appropriate data [25].

3.4 Fields of Application in Health Care The fields of application themselves occupy a vast place and position in the domain of health care. Intelligent agents have been suggested to deal with diverse problems in the healthcare field. Some of the fields in which they are already being applied are those given below [25]: • Planning: Planning the activities to be carried on an in-patient or out-patient [26]. • Organ transplant: Synchronizing the supervision of organ and tissue transplants among hospitals [6, 27]. • Community care: Synchronizing all the activities in an integrated manner to provide an efficient health care to the patients [28]. • Information access: Deploying information agents that gather data from Internet [1, 29] and provide users with information about the hospitals or doctors around [2]. • Internal hospital tasks: Monitoring the application of medical protocols [30] or controlling the usage of antibiotics [31]. Agent-based technology is in use in many healthcare system applications [32]. These systems include patient data management and monitoring [33, 34] but none addresses the issue of hospital search and appointment scheduling via intelligent agents on mobile devices. An intelligent agent-based hospital search and appointment system can allow the users to search for the best available hospital and also to fix an appointment. In the existing system, scheduling the appointments is not done in an organized manner. It can be deduced obviously that an ‘appointment agent’ here only appeals for an appointment date. No replies are received concerning what can be the real date of availability of the doctor. The agent cannot take up negotiation and scheduling hospital appointment considering emergencies or priorities. A substantial research is being conducted here where different types of MASs are being proposed for solving a variety of problems. Still, in comparison with other tasks, limited work has been done in applying this technology in emerging sub-fields and domains. The details of such system—telemedicine—will be discussed in the next section.

8 Applications of Multi-agent Systems in Intelligent Health Care

187

3.5 MAS for E-Health/Telemedicine In previous decades, telemedicine or e-health was a concept known to be developed for/used by old age people or people who are living in distanced places and soon. The current scenario has entirely changed the approach of telemedicine or its need for the society. Everyone is confined to their homes, and the need for telemedicine was realized immediately. Many hospitals, care taking centers, medical practitioners, etc. have realized their own applications or have become parts of the above said. The real importance of MAS is apparent now: without MAS in e-health, the healthcare sector might come to a standstill. The autonomous agents, with their own mini-goals but all aiming for the same conclusion, utilize a mutually accepted process for the management of the healthcare data of their potential patients. This is precisely one of the situations where multi-agent systems can be applied more competently [31]. According to different and autonomous goals of the involved agents, negotiation can be competitive or cooperative. In this ideology, agents can take on two roles, i.e., manager and contractor, in a decentralized market structure; a manager has some task to assign, while a contractor can propose itself as the most appropriate executor [35]. Communication among all the agents or persons is the most encountered problem in health and social services. Typically, agents have different requirements like a variety of healthcare problems, security problems, and so on. Many specialists might be involved in this total process like doctors, caretakers, and registered medical practitioners who are from diverse organizations [35]. A group should be in communication with other groups/teams to share the acquired data deduced results or predicated future predicaments. This process gives the healthcare industry a solid Launchpad that can supervise the current health conditions of the patients and be ready for any calamities that might surface in future, like different variants of Coronavirus (Fig. 8). Online healthcare services like telemedicine have proved to be very supportive not only for the patients but also for the specialists, particularly after the pandemic, these days. The data gathered can be very handy for further studies and deductions about what might happen in the future. As weather data is being monitored consistently, there is a need for monitoring the health data too in a concurrent and consistent way. It seems to be the only chance that the humanity has in its hand to save itself from unexpected health calamities. Robotic surgery is very common these days and tele-assistance and is an important ingredient of e-health services. Especially, the communication among the agents and exchange of data seems to be the real basis for the improvement in the usage of multi-agent system in e-health services. Agent technology provides a means through which active communication among the agents can work out without conceding the security of the concerned agents or persons. In the mean time, we should recollect that every agent is totally autonomous; it can respond according to their own rules. This process provides a surety that an agent is under the control of the institute or company to which it belongs to. If the total ideology follows this process carefully, the data is safe, can be exchanged, and can be worked upon to understand and discover pattern (if any).

188

M. Bhanu Sridhar

Fig. 8 Broad representation of telemedicine [35]

As an easier usage of a platform for communication forms the base for e-health systems, the MAS can be considered as ultimate goal for e-healthcare services, apps, and other processes and be a basis for next generations. Likewise, the central venue that security and privacy-awareness have depicts their importance with respect to e-health. In the agent world, the issues of privacy-awareness are dependent on the concept of trust. This indirectly showcases that more work has to be done on the aspect of security. Usage of social services highlights the importance of maintaining confidentiality and security for patients’ details. Technological changes are general and are also to be considered, with extensive implementation of various types of computing devices, accompanying users through different locations. Accordingly, e-health deals with mobile users in tele-assistance scenarios and should also accommodate full details with security and privacy for the users. In smart devices, various methods are already in place to gather, communicate and work upon healthcare information, like pulse rate, BP, sweating rate, cholesterol, etc. [36]. These apps or devices are more significant to monitor patients with chronic diseases at remote places. The above said systems typically possess an architecture through which data can be accessed through an app. This chance to take a look at the information may be for normal or daily data of the patient or for acquiring the data generated or transmitted by the patients themselves so as to keep an eye on their health situations. But this process should exist and proceed in such way that a patient can interact with the monitoring

8 Applications of Multi-agent Systems in Intelligent Health Care

189

Fig. 9 Procedure of diagnosis of diseases

doctors with total availability of all quality attributes like usability, security, privacy, and fault-tolerance. Integration of the concepts of the above said quality attributes should produce an identifiable data. This data plays a key role in locating the patient and informing him about his current situation so that he would be ready for any emergencies[37]. These apps should take advantage of mobile devices to gather, access, and transmit the data of a patient without bothering the patient and inform him if need exists. Data can also be gathered using sensor technology from different locations and can be integrated and analyzed without interfering anywhere in the daily chores of the concerned persons (Fig. 9). All mentioned requirements depict the distinctive types of multi-agent systems, and many ubiquitous online services are put in place via multi-agent perceptions and skills [35]. Specially, the JADE framework (or JADE-LEAP) does the work of allotting the agents, locating them, taking a note of their locations without being bothered about their network(s) or the types of devices that are being used by them. As noted previously, gathering the data from concerned persons at remote locations and integrating the same have an important role to play in this concept of e-health monitoring [35]. This concept brings the concept of MAS into the light once again for being the backbone of telemedicine services. Due to the MAS concept of identifying the agents, noting their locations and not being bothered about their network or devices, the users are able to utilize the services independently. Note that the communication among the agents and exchange of data also has its role for making conclusions or coming up with a solution for a problem that has surfaced suddenly. The above said details are the points which are to be realized as the important reasons for utilizing the multi-agent systems concept, particularly in healthcare domain, to obtain results with good performance and accuracy so that patients, doctors, and service providers are also satisfied with the offered unhindered service.

190

M. Bhanu Sridhar

3.6 Data to be Considered A study [38] deduced that nearly 37% of the allocated time was spent by physicians in administrative work. For example, if a patient decides to consult an ophthalmologist, he/she has to first ring the hospital, ask about the availability of the doctor, fix a date and time, and register with an identified patient number. After all this procedure, in which a relative or caretaker plays the major role, the patient has to physically visit the hospital and wait for the doctor. Many times, the author noticed that even after the fixed time, medical consultant never becomes available due to the pending or on-going work of previous appointments. The test is to be done, reports are to be waited to be obtained, then examined, and after coming on a conclusion, it has to be explained to the patient, medicines are to be suggested, and the reports are to be typed by the doctors themselves. All this procedure truly hinders the ‘patience’ of the patient and the medical caretaker. Due to the above said administrative work, a doctor loses his interest or ‘zeal’ to deal with the patients, and in the second half of the concerned day, the work becomes mechanical, not interesting. If all these works were distributed among other people (agents), much of this time could have been spent on examining patients. It has been noted by everyone that telemedicine is the best option for the current times and of course the future. After this discussion, the data to be considered will obviously be the details of patient, doctor, medical testers, previous visits’ reports, previous consultancy reports (if any), the scattered data of the patient in another hospital, etc. This time-taking process surely hampers the real goal: cure the patient and make him/her healthy and happy. Many patients, particularly in remote areas, do not have much time or knowledge in approaching the hospitals, which are generally located in large cities. To bridge this gap, it is better to utilize the autonomous agent-based telemedicine apps, in remote areas with the assistance of some health workers. This makes the distant patients to consult with a doctor in alive session and obtain a potential solution to the problem. This angle of MAS application in health care apparently is a boon to the society and more research, a bit deeply, has to be carried out in different angles like algorithms to be used, methodologies to be applied, and so on to obtain satisfactory results.

3.7 Methodologies and Applications to Achieve Accurate Results Much research work has been done recently for increasing the use of multi-agent systems for the awareness of applications and services, with specific consideration to communication among agent platforms and with other systems. Several technological specifications are the outcomes of such hard work that is being carried out unknowingly on the back of the screen. The two main results as of now are: (i) FIPA

8 Applications of Multi-agent Systems in Intelligent Health Care

191

Fig. 10 Generic system architecture for multi-agent systems [29]

specifications [39], a set of specifications proposed to support the interoperability between varied agent-based systems; and (ii) an agent development framework, called JADE [40], that implements Foundation for Intelligent Physical Agents (FIPA) specifications. This methodology also approves and supports interaction between agents/systems through interconnected and depending technologies like Java and CORBA. Recently, after the deadly pandemic, every hospital or consortiums have released their own mobile apps for the purpose, each designed on the same basics and similar outputs. Python also has been brought into this picture recently and the concept of application of multi-agent systems application in health care keeps on changing, from time to time (Fig. 10). Multi-agent systems have already been the foundation to the existing online medical services and applications that are being planned for future. These applications are being used for various purposes like taking a note of daily data, monitoring the health situation, telemedicine, smart healthcare services, and diagnosing the samples from remote areas [35]. They can also be used for some other purposes like hospital management without physical presence of persons or even doctors. Currently, the integration of smart-home automation for assisting chronically ill patients, elderly, impaired, psychologically disabled people, etc. are in existence [41, 42]. All research works in this sub-area of artificial intelligence are targeting the newly surfaced requirements and ideas in addition to the existing ones. Telemonitoring is not only needed for patients suffering with Alzheimer’s or cancer but also for those who need assistance for monitoring endemics or pandemics. Even assistance for basic needs might also find its place in future. Still yet, another context like information from a smart-home environment gives the monitoring people a better idea on what is happening as of now and what might happen in future. Best examples are the current condition of the person, his/her BP level, pulse rate, etc. and provide suggestions for a healthy life.

192

M. Bhanu Sridhar

On the other hand, every patient, these days, intends to consult doctors online and obtaining the medicines too, online through different applications available. Not only for lonely or elder persons, online consultation is a part of our lives after pandemic situation and will remain in place for the future too. Overall, the usage of multi-agent systems is the real need for all the above said reasons and is an important factor for the comprehensible and effective development of smart living apps. Practo, Lybrate, DocsApp, MFine, IOnline Doctor, 1mg, etc. are some of these, to list a few [43].

4 Recent Trends in Application of MAS in Health Care and Its Necessity Monitoring the health of the users or patients online through different devices or apps has become a common aspect in the existing situation(s) and can be quoted as a ‘societal need.’ Through such apps, hospitals, clinics, or doctors themselves can observe the progress of a patient online and suggest proper medication. Periodical exchange of such data between different health workers can be very useful for researchers to predict the current and future situations of the patients. As it can be seen, communication and integration of health data of one or many patients among them will further help the patients. Any emergencies can or might be predicted before or the institutions can be kept on alert all the time so that they can be attended as soon as possible. Many studies, research works can be carried out on such data to understand the current trend or pattern. In the middle of all this, we should notice the thin connection(s) underlying in between: all are agents and the multi-agent system concept is extensively being used with positive and consistent results. These apps can offer other services like comfort services (smart homes), assistive services (virtual assistants), information services, and communication services. This new trend has paved a new way for the society and a new ideology that has become a part of the society. Many hospitals provide their own websites as applications with chatbots to provide the same assistance as some mobile app to listen, consider, suggest, and satisfy the needs of the concerned patient(s). Shifting to the other angle, the apps can operate through different algorithms or procedure that can be considered like famous machine learning algorithms, deep learning algorithms, and RFID technology. The data considered can be studied upon, classified or clustered as per requirements, come to a conclusion and suggest a solution for the patient. But this angle, where a doctor might or might not be present every time, hinders the confidence of the patient, and surely, less satisfaction is obtained. This is one reason why the above said angle was not studied in deep. Telemedicine or e-health was considered more deeply since the doctor or a medical consultant is somewhere involved in a face to face talk with the patient. This makes both the parties to understand the other person’s thoughts, expressions and come to a conclusion. The

8 Applications of Multi-agent Systems in Intelligent Health Care

193

ailing patient needs mental satisfaction and surety, not anything what a chatbot prints on the screen or messages. Still yet, the current pandemic situation has made all the prediction techniques go wild. What all had been predicted in last one year or so has never come true due to lack of data, lack of experience or inapplicability for unexpected changes in situations. Since the results in this type of approach lead to nowhere, it is important that we should rethink the angle of dealing the situations concerning the healthcare angle and pursue a different one immediately. If online healthcare applications are located with the administrators of a clinic or hospital, they can be quoted as smart healthcare applications. Through these applications, the works of doctors and other health workers can be made more interconnected and easier. This goal is reached by the utilization of mobile devices where data can be exchanged and updated. Recently, RFID technology is being used for further improving these applications [44]. If an emergency case surfaces, concerned doctors can access the existing data of the patient(s) while they are still in the ambulance or waiting for it. Such applications bridge the gap between the hospital and patient. Examples are: ERMA [45], Akogrimo [46], and CASCOM [47].

5 Conclusion Though the multi-agent approach to various types of health services and applications is on the rise, it cannot be confidently stated that all the problems have been solved effectively. Nonetheless, many of those problems are inherent to the situations that the applications face. Hence, MAS has a better chance to recognize the problems and suggest a solution. Programs put up on the multi-agent model have yet not reached the required levels of accuracy and not met the societal requirements or those of the IT industry yet. But they are surely on the path toward a complete maturity. The diversity and complexity of the e-health scenario make it one of most interesting field of research and application, which is able of verifying the advantages of their use and of conditioning their evolution. Thus, multi-agent systems can make us benefit in all presumable ways. Most of the emerging technologies and ideologies can be applied and mixed with the concept of multi-agent systems to become a big boon for healthcare services and societal needs [48]. The chapter has touched all the angles of artificial intelligence, distributed artificial intelligence, multi-agent systems and their application in healthcare domain equivocally, and it is believed that it will pave the way for further improvement of the above said topics and bring a positive end to the ailing of patients and meet all the needs of the medical practitioners.

194

M. Bhanu Sridhar

References 1. Holzinger A et al (2019) Causability and explainability of artificial intelligence in medicine. Wiley Interdisc Rev: Data Mining Knowl Discovery 9(4):e1312 2. Nilsson NJ (2009) The quest for artificial intelligence. Cambridge University Press 3. Nealon J, Moreno A (2003) Agent-based applications in healthcare. In: Applications of software agent technology in the health care domain. Birkhäuser, Basel, pp 3–18 4. Stone P, Veloso M (2000) Multi agent systems: a survey from a machine learning perspective. Autonom Robots 8(3):345–383 5. https://francesco-ai.medium.com/distributed-artificial-intelligence-3e3491e0771c 6. Ponomarev S, Voronkov AE (2017) Multi-agent systems and decentralized artificial super intelligence. arXiv pre printer Xiv: 1702.08529 7. Ponomarev S, Voronkov AE (2017) Multi-agent systems and decentralized artificial super intelligence. arXiv preprint arXiv:1702.08529(2017) 8. Parunak HVD (1996) Applications of distributed artificial intelligence in industry. In O’Hare GMP, Jennings NR Foundations of distributed artificial intelligence. Wiley Interscience, pp 139–164 9. Durfee EH, Lesser VR, Corkill DD (1989) Trends in cooperative distributed problem solving. IEEE Trans Knowl Data Eng 10. https://www.bankofengland.co.uk/quarterly-bulletin/2016/q4/agent-based-models-understan ding-the-economy-from-the-bottom-up 11. Andreadis G et al (2014) Classification and review of multi-agents systems in the manufacturing section. Proc Eng 69:282–290 12. Shen W, Norrie D (1997) Facilitators, mediators or autonomous agents. In: Proceedings of the second international workshop on CSCW in design, Bangkok, Thailand 13. Jennings NR (2000) On agent-based software engineering. Artif Intell 117(2):277–296 14. Aylett R et al (1998) Agent systems and applications. Knowl Eng Rev 13(3):303–308 15. Reeke Jr GN (1994) Marvin Minsky, the society of mind. Contemplating minds: a forum for artificial intelligence, vol 48. MIT Press 16. Liao S-H (2004) Expert system methodologies and applications—a decade review from 1995 to 2004. Elsevier 17. Shakshuki EM, Malcolm R (2015) Multi-agent system applications in healthcare: current technology and future roadmap. In: ANT/SEIT, Elsevier, pp 252–261. https://core.ac.uk/download/ pdf/82700092.pdf 18. Moreno A (2002) Medical applications of multi-agent systems [online]. Available: http://cyber. felk.cvut.cz/, EUNITE03-BIO/pdf/Moreno.pdf 19. Sridhar MB, Srinivas Y, Krishna Prasad MHM (2014) Software reuse in a paralysis data set based on categorical clustering and the Pearson distribution. J King Saud Univ-Comput Inf Sci 26(3):347–354 20. https://www.umsl.edu/~joshik/msis480/chapt11.htm 21. Iantovics BL (2012) Agent-based medical diagnosis systems. Comput Inf 27(4):593–625 22. Chakraborty S, Gupta S (2014) Medical application using multi agent system-a literature survey. Int J Eng Res Appl 4(2):528–546 23. Cortés U, Annicchiarico R, Urdiales C (2007) Agents and healthcare: usability and acceptance. In: Agent technology and e-health. Birkhäuser Basel, pp 1–4 24. Henderson H (2007) Artificial intelligence: mirrors for the mind. Infobase Publishing 25. Moreno A (2003)Medical applications of multi-agent systems. Computer Science and Mathematics Department, University of Rovira, Spain 26. Li Y et al (2016) Peer reviewed: agent-based modeling of chronic diseases: a narrative review and future research directions. Prev Chronic Dis 13 27. Muaz N, Hussain A (2011) Agent-based computing from multi-agent systems to agent-based models: a visual survey. Springer Scientometrics 89(2):479–499. https://doi.org/10.1007/s11 192-011-0468-9, http://www.springerlink.com/content/c3g15318738w0710/

8 Applications of Multi-agent Systems in Intelligent Health Care

195

28. Khan F, Reyad O (2019) Multi agent based systems for e-healthcare security. Inf Sci Lett 8(2):67–72. https://doi.org/10.18576/isl/080204 29. Iqbal Z et al (2018) A comparative study of machine learning techniques used in non-clinical systems for continuous healthcare of independent livings. In: 2018 IEEE symposium on computer applications & industrial electronics (ISCAIE). IEEE 30. Barro S, Presedo J, Castro D, Fernández Delgado M, Fraga S, Lama M, Vila J (1999) Intelligent tele monitoring of critical-care patients. IEEE Eng Med Biol Mag 18(4):80–88 31. Jennings NR (2000) On agent-based software engineering. Artif Intell 117:277–296 32. Li Y, Lawley MA, Siscovick DS, Zhang D, Pagán JA (2016) Agent-based modeling of chronic diseases: a narrative review and future research directions. Prev Chronic Dis 13:150561. https:// doi.org/10.5888/pcd13.150561 33. Moreno A (2002) Medical applications of multi-agent systems [online]. Available: http://cyber. felk.cvut.cz/EUNITE03-BIO/pdf/Moreno.pdf 34. Miller K, Suresh S Applications of policy based agents in wireless body sensor mesh networks for patient health monitoring. Int J e Health Med Commun 2(2):24–45 35. Bergenti F, Poggi A, Tomaiuolo M (2016) Multi-agent systems for e-health and telemedicine. In: Encyclopedia of e-health and telemedicine. IGI Global, pp 688–699 36. De Mola F, Cabri G, Muratori N, Quitadamo R, Zambonelli F (2006) The Ubi medic framework to support medical emergencies by ubiquitous computing. Int Trans Syst Sci Appl 1(1):15–26 37. Bricon-Souf N, Newman C (2007) Context awareness in healthcare: a review. Int J Med Inf 76(1):2–12 38. Sinsky C, Colligan L, Li L, Prgomet M, Reynolds S, Goeders L, Westbrook J, Tutty M, Blike G (2016) Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med 165(11):753–760 39. Poslad S, Charlton P (2001) Standardizing agent interoperability: the FIPA approach. Multiagent systems and applications. Springer, Berlin, Germany, pp 98–117 40. Bellifemine F, Caire G, Poggi A, Rimassa G (2008) JADE: a software framework for developing multi-agent applications. lessons learned. Inf Softw Technol J 50:10–21 41. Mynatt ED, Essa I, Rogers W (2000) Increasing the opportunities for aging in place. In: Proceedings of the 2000 conference on universal usability, Arlington, VA, USA, pp 65–71 42. Liffick BW (2003) Assistive technology in computer science. In: Proceedings of the first international symposium on information and communication technologies, Dublin, Ireland, pp 46–51 43. Google Play Store 44. Fuhrer JP, Guinard D (2006) Building a smart hospital using RFID technologies. In: Proceedings of the first European conference on ehealth (ECEH06), Fribourg, Switzerland, pp 131–142 45. Mabry SL, Hug CR, Round RC (2004) Clinical decision support with IM-agents and ERMA multi-agents. In: Proceedings of the 17th IEEE symposium on computer-based medical systems (CBMS2004), Bethesda, MD, USA, pp 242–247 46. Racz P, Burgos JE, Inacio N, Morariu C, Olmedo V, Villagra V, Rui L, Aguiar VR, Stiller B (2007) Mobility and QoS support for a commercial mobile grid in Akogrimo. In: Mobile and wireless communications summit. Budapest, Hungary, pp 1–5 47. Schumacher M, Helin H, Schuldt H (2008) CASCOM: intelligent service coordination in the semantic web. Springer, Berlin, Germany 48. Bicer V, Kilic O, Dogac A, Laleci GB (2005) Archetype-based semantic interoperability of web service messages in the health care domain. Int J Semant Web Inf Syst 1(4):1–23

Chapter 9

Multi-agent Reinforcement Learning for Stock Market Strategy Analysis Akash Ranjan, Asim Kumar Mahadani, and Tarik A. Rashid

Abstract Stock market has always been uncertain in terms of prediction and liquidation, which is a major problem in stock market for any investor where a huge number of shares of a particular stock have been sold within a time period. Experts faced main issue in optimizing the liquidation which leads to obtain a proper modeling system than can handle the varieties of challenges of the stock market and provide viable strategic solution in the trading. Generally, the return on the invested amount is depend on the purchase price, and there are two key factors responsible for making money on stocks, first buy a stock on right time and second sell it on right time. So, to book a profit, both the decisions should be correct. Mostly, many investors have faced problem in selling a stock, where the common problem lies in the human nature of running toward making more profit. The most important thing in stock market is to make correct strategies and keep out all the human emotions in taking any decisions in terms of buy and sell and make very strategies call of buying a stock or in terms of sell a stock. All the strategies should be based on the market conditions and stock news, not on human emotions and greediness. There are mostly so many factors responsible for taking a call to sell a stock but among them there are three good reasons for taking a sell call is: first stock has been purchased by mistake, second there is a sudden good hike in the stock price, and third stock price reached to a reasonable price and meet the expectation. Along with these, there are also other factors involved in taking a sell call for a stock like influenced by any false news or any verbal spoof, these are not at all consider for making any decisions for any kind of investments. In this chapter, we use the multi-agent model by using deep reinforcement learning that allows us to capture high level of complexities comparing to other machine learning models and strategies, so that agents can train in better way for taking the decisions to sell a stock at right time.

A. Ranjan · A. K. Mahadani (B) Department of Computer Science and Engineering, Bankura Unnayani Institute of Engineering, Bankura, West Bengal, India T. A. Rashid University of Kurdistan Hewler, Kurdistan, Iraq e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 S. Gupta et al. (eds.), Multi Agent Systems, Springer Tracts in Human-Centered Computing, https://doi.org/10.1007/978-981-19-0493-6_9

197

198

A. Ranjan et al.

Keywords Multi-agent · Reinforcement learning · Stock price prediction · Deep learning

1 Introduction Stock analysis is always been a challenging task for market experts. This analysis strategy mainly comes to avoid the liquidation problem. When a huge number of shares for a stock have been sold for a certain time, frame is a challenging task to predict the appropriate time to come out from a stock. Mainly, machine learning models have been tried to analysis the strategy for liquidation, but a single model is not good enough to cover all the complex scenarios and to give a bestselling decision. Many of the analysis concentrates mainly depend upon the market, rate of growth, expense, etc. The company-based information dependent on their policies. Analysis mainly based on technical methods considers the previous charts and patterns to predict the fluctuation of stocks. Moreover, multiple agent deep learning can be termed as multi-agent system which comes with an upgrade in predicting stocks which decreases the uncertainty like right time to buy and sell the stocks. Prediction in stock market can be a hard nut to crack but with the support of multi-agent model where multiple agents analyze the different scenarios for a particular problem and train themselves to take correct decision at right time make our job easy and simple. It can be said, with the help of multi-agent system stock market, prediction will just become a calculative move for the financial experts. Bao et al. (2019) have studied multi-agent model by deep reinforcement learning where high level of difficulties have been taken into consideration in training the multi-agent model that will help the agents to take better selling decisions. In this paper, Almgren and Chriss model has been theoretically analyzed, and its mechanism has been used in training the multi-agents. Here, cooperative and competitive actions of agents have been analyzed to overcome the drawback of single-agent-based reinforcement learning models and develop an optimal trading strategy in solving the liquidation problems [1]. Hendricks et al. (2014) have studied reinforcement learning to enhance existing analytical solution for the optimal liquidation volume trajectory. In this paper, standard Almgren-Chriss model has been considered as a base model. Due to exponential increase in the spread and volume dynamics of market, a static Almgren-Chriss volume trajectory has been described for training a learning agent using Q-learning technique. Here, shortfall after post-trade implementation has been improved by up to 10.3% on the average as compared to the base model [2].

9 Multi-agent Reinforcement Learning …

199

Agogino et al. (2012) have studied intelligent air traffic flow management system. Here, a multi-agent algorithm has been explored where agents use reinforcement learning to reduce the congestion through local actions. Studies have been done on artificial and real historical data that results in receiving personalized rewards with reduce in congestion by up to 80% over the global reward and by up to 90% over the current industry approach [3]. Bloembergen et al. (2015) have been studied dynamical models that derived from various multi-agent reinforcement learning algorithms. A new learning algorithm has been introduced using evolutionary game theoretic tools. The evolutionary models help to study complex strategic interactions. In this paper, a proper roadmap has been given for analyzing the evolutionary dynamics of multi-agent learning [4]. Mihaylov et al. (2009) have studied a wireless sensor network (WSN) system where a reinforcement learning algorithm has been used to increase the autonomous lifetime of a wireless sensor network (WSN) and decrease its latency in a decentralized manner. WSNs consists of group of sensor nodes that collect the environmental data, in which the major challenges are limited power supply of the nodes and need for decentralized control. To tackle this problem, a reinforcement learning algorithm has been used to in each sensor node so that it can optimize the efficiency of a small group of surrounding nodes that at end helps to improve the performance of the whole system [5]. Pipattanasomporn et al. (2009) have studied about a multi-agent system and its design and implementation of it for providing intelligence to a distributed smart grid which are located at distributed level. In this paper, a detailed discussion has been mentioned for a multi-agent application development that consists of agent specification, application analysis, application design, and application realization. Here, a multi-agent system has been used to control a distributed smart grid in a simulated environment with a seamless transition from grid [6].

2 Purpose of Our Study Building the base for future multi-agent trading system is our primary vision. This is done by further exploration of Almgren and Chriss model [2] and working on its basic mechanism which can be helpful in having multi-agent trading system. Analyzing the competitive and cooperative behavior between the agents by adjusting the reward functions of each agent, which overcomes the limitation of single-agent-based reinforcement learning algorithms. Stimulating the trade and developing optimal trading policy with the viable limitations by applying reinforcement learning models, which shows the power of reinforcement learning methods in achieving solutions for practical liquidation issues.

200

A. Ranjan et al.

3 Multi-agent Reinforcement Learning The study of many artificial intelligence agents which cohabitate in an environment, associating toward an end goal. While association, it works mainly as social architecture in the animal kingdom. It also shows scenarios parallel to game theory. Multi-agent systems are not only a research method; it can be applied to solve many of the complex problems like human society mainly as urban and air traffic control (Agogino and Tumer 2012) [3], coordinating multiple robot (Ahmadi and Stone 2006; Claes et al. 2012) [4], sensing distribution (Mihaylov et al. 2014), [5] and energy distribution (Pipattanasomporn et al. 2009) [6]. To control an agent in an optimal way in an environment reinforcement system uses machine learning. The entire process consists of several stages beginning from a function or policy which agent learn from its observation and the response of environment which confirm whether is decision taken by agent is correct or not by the rewards. A function or policy is generally a network of neurals which having deep relationship among them that is why it is termed as “deep reinforcement learning.” The primary end point of any technique of deep reinforcement learning is to understand an appropriate function or policy which can receive the high percentage of rewards from environment for particular action. A reward can be termed as a dimensionless value that has been achieved after the immediate action by the environment. The entire procedure is presented in Fig. 1.

Fig. 1 Single-agent reinforcement learning model

9 Multi-agent Reinforcement Learning …

201

The typical example of reinforcement learning consists of wide diversity of different scenarios, mainly a role of the video game, i.e., Atari, in which every time a reward has been given to the character for successful attempt results into change in the score. Like that, a robot which is delivering items over the city where an agent has been rewarded the points for successfully completion of run and punishment has been given for taking a long time in deliver or the trading bot which trade stock and get reward for monetary gain.

3.1 Multi-agent Reinforcement Learning Applications Ability to learn how to play a multiplayer game is the most intense success in the field of AI which we used in the day to day life style. There are so many games that have used artificial intelligence in learning the games to play that include DOTA 2, StarCraft 2 and GO. Controlling the multiple agents for accomplishing an activity using reinforcement learning is termed as multi-agent reinforcement learning. Generally, it is almost similar in terms of learning a policy by the agent to get the reward as single-agent reinforcement learning. It is possible to use a policy that is centrally implemented for all agents which will communicate with the centralized server to take respective action (which is difficult to implement in the real life situations), so practically multi-agent reinforcement learning has been used which is decentralized. The entire process is presented in Fig. 2.

Fig. 2 Multi-agent reinforcement learning model

202

A. Ranjan et al.

Multi-agent deep reinforcement learning is the way to train model by using the deep neural networks that helps to learn policies in the multi-agent reinforcement systems.

3.2 Simulation Analysis of Optimal Execution To make an optimal execution strategy, two models mainly have been used in order to analyze the stock market strategies. Almgren-Chriss (AC) strategy: This strategy is better for analyzing the downward price trend. 1.

The time weighted average price (TWAP) strategy: This strategy has better performance as compared with Almgren-Chriss in upward price trend. In terms of analyzing the stock problem to minimize the cost of liquidation or maximize the benefit, a good execution strategy is to balance transaction cost and time risk where Almgren-Chriss strategy is mainly used which follows the arithmetic Brownian motion with linear impact function. Almgren-Chriss model assumes that execution strategy is time discrete and security price follows random walk. The optimal approach based on Almgren-Chriss model is to make the balance between minimizing price impact and minimizing time risks. It mainly consists of: 1.

2.

Risk minimization: It uses the variance, the quadratic variation or the VaR [7, 8] as risk measures and also maximizes its expected revenue of trading at the same time. Utility function approach: It uses the power function or exponential function [9] as a utility function and maximizes it.

Minimizing expected execution cost approach: Such as minimizing the expected execution costs by modeling the dynamic character of the distribution of bids and ask in the limit order book [10].

4 Models and Methods Considering a problem of liquidation where the large number of shares have been sold from a stock before time T to maximize the expected terminal wealth. Assumption can be taken that the dynamics of the stock is characterized by dS(t) = σ dB(t) − γ v(t)dt,

(1)

9 Multi-agent Reinforcement Learning …

203

where S(t) represents the price of one share of the stock at time t, and dB(t) represents the increment of Brownian motion, σ denotes the volatility of the stock and γ denotes the permanent impact. In Eq. 1, (t) is the trading rate that is characterized by v(t) = −

dx(t) , dt

(2)

where x(t) represents the number of holdings in the stock and x(t) satisfies x(0) = X, x(T ) = 0. According to the implement shortfall [11], our execution cost is expressed by  C = X S(0) −

T

(S(t) − ηv(t))dx(t),

(3)

0

where the η represents the temporary impact. According to the Almgren-Chriss mean–variance discretization model [12], we obtain the value function under the given period V (x(t)) = min{E[C] + λVar[C]} v∈A  T   γ v(u)x(u) + ηv 2 (u) + λσ 2 x 2 (u) du, = min v∈A

(4)

t

where v represents permissible control rate and A represents permissible control set, then we get HJB equation   ∂V ∂V 2 = min γ v(t)x(t) + ηv (t) − v(t) − v∈A ∂t ∂x + λσ 2 x 2 (t)

(5)

and the optimal trading trajectory has been given by  x(t) = X

eκ(T −t) − eκ(t−T ) eκ T − e−κ T

 =

sinh(κ(T − t)) X, sinh κ T

(6)

√ where κ = λσ 2 /η. That denotes when the trading time interval is very small, the influence of permanent impact on the strategy can be ignored. Then the trading rate can be represented by vκ = −

cosh(κ(T − t)) X sinh κ T

(7)

204

A. Ranjan et al.

As long as X > 0, the number of transactions n at each time t must be positive. That means, for the trading of large positions, the solution we will achieve must be a monotonically decreasing function, the rate of decline determined by the parameter κ.

4.1 Impact on Optimal Liquidation by Parameter κ Let us understand the clear picture in the change in trajectory with the change in the value of κ, let the time and number of liquidation stocks change with the range of [0,1]. We rescale the original condition, let Xˆ = 1, Tˆ = 1 and assume that transaction times is N = 240 during Tˆ and t = 1/N. Then the above parameters can be used to represent the trading trajectory

x(t) ˆ =



sinh κ( ˆ Tˆ − t) sinh κ Tˆ 

,

(8)

where 2/t 2 (cosh(κt)−1) ˆ = λσ 2 (η/Tˆ −γ /2N 2 ). The parameter κ of the optimal execution strategy just not only reflects the liquidation’s influence on the price process but also determines the shift rate of the optimal trading trajectory. So that, it can be represented as suppose κ change from small to large by simulation, and get a set of optimal trading trajectory curves. In Fig. 3 as κ continues to increase, the steeper the trading trajectory curves. The steep execution strategy curve here represents that at the beginning of the trading, Fig. 3 Impact on optimal trajectory by parameter κ

9 Multi-agent Reinforcement Learning …

205

the investors prefer to avoid risks and choose to execute a large number of positions to avoid the impact of subsequent price fluctuations. There are extreme cases where all positions are executed at first time, however, that is known that the market cannot absorb such a huge order resulting in large stock impact.

4.2 Different Deep Reinforcement Learning Approaches In real-world scenarios, most of the financial market applications use deep reinforcement learning that involves continuous or discrete state and action and use one of the below mentioned methods for learning: 1. 2. 3. 1.

Critic Only Method Actor Only Method Actor Critic Method Critic Only Method: The most common objective of critic only method is to solve a discrete action space problem by using Q-learning, deep Q-learning, and its improvements that will train an agent on a single asset or stock. In this method, the agent is using Q-value method to obtain best action and policy selection that will maximize the chances of desired future rewards as compare to the current state. Here rather than using a state-action value table for calculating, deep Q-learning reduces mean squared error (MSE) between target Q-values and helped neural networks to get better result. The main drawback of this method is it mainly works with finite and discrete state and action spaces that are practically not possible to deal with large portfolio of stocks because prices are always continuous in nature. • Q-learning: Q-learning is utilized to determine best appropriate action and policy selection by using the Q-function. It is used as per value-based reinforcement learning method. • DQN: Deep Q-learning (DQN) is used to optimize a neural-network to an estimated Q-value function. Here, a state is feed as an input and Q-value of permitted action is getting as an anticipated output.

2.

Actor Only Method: The main advantage of actor only method is that here, the optimal policy is learned by the agent directly by itself that means it does not require a neural network to learn Q-value, it directly learns the policy. Policy can be termed as a probability distribution of strategy for a particular state that helps in taking the approximate action. This method has been used to deal with continuous action only. • Policy Gradient: It is used to increase the rewards, i.e., total of the expected rewards by learning the optimal policy directly.

3.

Actor Critic Method: This method is widely used in finance where both actor and critic methods approaches are combined together to getting better result and

206

A. Ranjan et al.

overcome the individual drawbacks. In this method, an actor network has been represented as a policy, and critic network has been represented as a value function. Here, actor will update the policy probability distribution which is guided by critic which will estimate the value function. By continuous learning/training, actor will learn to take better actions and critic will able to evaluate those actions efficiently. Hence, this method has proved that it has the ability to learn and adapt as per the large and complicated environment. It is widely been used in video games environment. It finds that this method is a good fit to large stock trading portfolio [13]. • A2C: It is a classic type of method of actor critic Algo in which copies of same agent have been used to work in parallel and that helps in updating the gradients with different types of data samples. There is no interaction among the agents that are working in parallel, every agent interacting separately to the environment. • PPO: This method is used to check the policy gradient during updation of any policy gradient and ensures that there is not so much difference in the new policy as compared with previous policy. • DDPG: This method uses neural networks as a function approximator that combines both the frameworks of policy gradient and Q-learning.

4.3 Deep Reinforcement Learning Used for Trading Agent Actor Critic Algorithm: A2C is a classic method of actor critic approach which is used in the ensemble method as a component. It is used to increase the efficiency of updation of policy gradient. It aims to reduce the policy gradient’s variance by leveraging advantage function. Here, focus is not only on value function but also on critic network that calculate the value of advantage function. For evaluating any action, it has been taken into consideration the future perspective also where it is calculated how it can be making the result better instead of only depending on the present result of any action. So, it makes A2C model more robust by reducing the high variance of policy networks [14]. This algorithm uses the feature of an agent by copying it while working in parallel that helps in updating the gradients with different types of data samples. There is no interaction among the agents that are working in parallel, every agent interacting separately to the environment. After completion of the tasks of estimating the gradients for all the parallel agents, A2C acts a coordinator that will pass the average of the gradients over all the agents of global network. Then global network will upgrade both critic and actor network, and its presence improves the diversity in the training data set. Now, this can prove that A2C is a good model for trading stocks due to its solidity and it synchronized the gradient update faster, cost-effectively, and work better with larger batch sizes [15] (Fig. 4).

9 Multi-agent Reinforcement Learning …

207

Fig. 4 A2C model

DDPG: DDPG is a method of actor critic approach which is used in the ensemble method as a component that will maximize the return. This method uses neural networks as a function approximator that combines both the frameworks of policy gradient and Q-learning. DDPG overcome the challenges of dimensionality problem that DQN is suffered from. DQN learns indirectly from Q-values tables but DDPG learns directly from observations through policy gradients. It works better in continuous action space environment by mapping states with actions [16] (Fig. 5).

Fig. 5 DDPG model

208

A. Ranjan et al.

PPO: PPO is a method of actor critic approach which is used in the ensemble method as a component and this method is used to check the policy gradient during updation of any policy gradient and ensures that there is not so much difference in the new policy as compared with older policy. To simplify the objective function of Trust Region Policy Optimization (TRPO), it introduces a clipping technique to objective function. This method does not allow the large change in the policy outside of the clipped interval. It targets to minimize the normal and clipped objective. It is selected as a good model for trading stocks due to its fastness, simplicity, and stability to implement and tune. It enhances the strength of training policies of networks by limiting its policy upgrade at each level [17].

4.4 Multi-agent Approach for Stock Market Strategy Analysis Multi-agent deep reinforcement learning can be broadly used in stock market analysis and making strategies for future prediction for any stock. Huge buy and huge sell for any stock in particular time period is a major challenge to predict and finding right time to exit from a particular stock (Fig. 6). These challenges cannot be handled by using a training model which have only single agent.

Fig. 6 Multi-agent training model

9 Multi-agent Reinforcement Learning …

209

So, we have to apply multi-agent model in reinforcement model which can train as per the market condition and analyze effectively with better precision.

5 Implementation of Multi-agent Systems for Stock Market By applying the Almgren-Chriss model, the solution for an optimal liquidation strategy has been investigated [2] in a given time frame assets have been liquidated by the agents behind the hood. The stock market is impacted mainly by following three components: (a) temporary impact, (b) unaffected price process, and (c) permanent impact. Mean–variance eliminates all the stochastic problems of the price process that exists. This price process allows linear function of temporary and permanent prices. So, the model can serve the features of trading environment where agents can take the decisions on selling and the environment can return information on the price accordingly. So, for Optimal Execution of Portfolio Transactions, deep reinforcement learning was been used. To train the model, we have to perform following steps: A.

Get the default financial and Almgren and Chriss (AC) model parameters, which is in our case

210

A. Ranjan et al.

Financial Parameters Annual Volatility:

12 %

Bid-Ask Spread:

0.125

Daily Volatility:

0. 8%

Daily Trading Volume:

5,000,0 00

Almgren and Chriss Model Parameters Total Number of Shares for Agent1 to Sell:

500,000

Fixed Cost of Selling per Share:

$0.062

Total Number of Shares for Agent2 to Sell:

500,000

Trader's Risk Aversion for Agent 1:

1e-06

Starting Price per Share:

$50.00

Trader's Risk Aversion for Agent 2:

0.0001

Price Impact for Each

$2.5e-06

Permanent Impact Constant:

2.5e-07

Number of Days to Sell All the Shares:

60

Single Step Variance:

0.144

Number of Trades:

60

Time Interval between trades:

1.0

1% of Daily Volume Traded:

B.

Create simulation environment and initialize feed-forward DNNs for actor and critic models

9 Multi-agent Reinforcement Learning …

agent1 = Agent(state_size=environment.observation_space_dimension (), action_size=environment.action_space_dimension(),random_se ed = 1225) agent2 = Agent(state_size=environment.observation_space_dimension (), action_size=environment.action_space_dimension(),random_se ed = 108) C.

Train the model

# Create simulation environment env = syntheticChrissAlmgren.MarketEnvironment() # Initialize Feed-forward DNNs for Actor and Critic models agent1 =

211

212

A. Ranjan et al.

Agent(state_size=environment.observation_space_dimension (), action_size=environment.action_space_dimension(),random_se ed = 1225) agent2 = Agent(state_size=environment.observation_space_dimension (), action_size=environment.action_space_dimension(),random_se ed = 108) # Set the liquidation time lqt = 60 # Set the number of trades n_trades = 60 # Set trader's risk aversion tr1 = 1e-6 tr2 = 1e-6 # Set the number of episodes to run the simulation episodes = 1300 shortfall_list = [] shortfall_hist1 = np.array([]) shortfall_hist2 = np.array([]) shortfall_deque1 = deque(maxlen=100) shortfall_deque2 = deque(maxlen=100) for episode in range(episodes): # Reset the enviroment cur_state = environment.reset(seed = episode, liquid_time = lqt, num_trades = n_trades, lamb1 = tr1,lamb2 = tr2) # set the environment to make transactions environment.start_transactions() for i in range(n_trades + 1):

9 Multi-agent Reinforcement Learning …

# Predict the best action for the current state. cur_state1 = np.delete(cur_state,8) cur_state2 = np.delete(cur_state,7) #print(cur_state[5:]) action1 = agent1.act(cur_state1, add_noise = True) action2 = agent2.act(cur_state2, add_noise = True) #print(action1,action2) # Action is performed and new state, reward, info are received. new_state, reward1, reward2, done1, done2, info = environment.step(action1,action2) # current state, action, reward, new state are stored in the experience replay new_state1 = np.delete(new_state,8) new_state2 = np.delete(new_state,7) agent1.step(cur_state1, action1, reward1, new_state1, done1) agent2.step(cur_state2, action2, reward2, new_state2, done2) # roll over new state cur_state = new_state if info.done1 and info.done2: shortfall_hist1 = np.append(shortfall_hist1, info.implementation_shortfall1) shortfall_deque1.append(info.implementation_shortfall1) shortfall_hist2 = np.append(shortfall_hist2, info.implementation_shortfall2) shortfall_deque2.append(info.implementation_shortfall2) break if (episode + 1) % 100 == 0: # print average shortfall over last 100 episodes print('\rEpisode [{}/{}]\tAverage Shortfall for Agent1: ${:,.2f}'.format(episode + 1, episodes, np.mean(shortfall_deque1))) print('\rEpisode [{}/{}]\tAverage Shortfall for Agent2:

213

214

A. Ranjan et al.

${:,.2f}'.format(episode + 1, episodes, np.mean(shortfall_deque2))) shortfall_list.append([np.mean(shortfall_deque1),np.mean(shortfa ll_deque2)]) print('\nAverage Implementation Shortfall for Agent1: ${:,.2f} \n'.format(np.mean(shortfall_hist1))) print('\nAverage Implementation Shortfall for Agent2: ${:,.2f} \n'.format(np.mean(shortfall_hist2))) D.

Training model result

Episode [100/1300] $1,168,737.14 Episode [100/1300] $1,182,497.07 Episode [200/1300] $1,281,250.00 Episode [200/1300] $1,281,250.00 Episode [300/1300] $1,274,753.90 Episode [300/1300] $1,278,818.44 Episode [400/1300] $958,446.84 Episode [400/1300] $996,403.34 Episode [500/1300] $321,537.19 Episode [500/1300]

Average Shortfall for Agent1: Average Shortfall for Agent2: Average Shortfall for Agent1: Average Shortfall for Agent2: Average Shortfall for Agent1: Average Shortfall for Agent2: Average Shortfall for Agent1: Average Shortfall for Agent2: Average Shortfall for Agent1: Average Shortfall for Agent2:

9 Multi-agent Reinforcement Learning …

$321,944.71 Episode [600/1300] $331,625.64 Episode [600/1300] $328,738.83 Episode [700/1300] $302,789.39 Episode [700/1300] $296,596.55 Episode [800/1300] $305,151.05 Episode [800/1300] $301,542.19 Episode [900/1300] $343,508.22 Episode [900/1300] $342,052.92 Episode [1000/1300] $318,731.56 Episode [1000/1300] $317,495.71 Episode [1100/1300] $329,135.85 Episode [1100/1300] $333,255.71 Episode [1200/1300] $300,993.44 Episode [1200/1300] $301,320.57 Episode [1300/1300] $294,413.69 Episode [1300/1300] $292,937.04

215

Average Shortfall for Agent1: Average Shortfall for Agent2: Average Shortfall for Agent1: Average Shortfall for Agent2: Average Shortfall for Agent1: Average Shortfall for Agent2: Average Shortfall for Agent1: Average Shortfall for Agent2: Average Shortfall for Agent1: Average Shortfall for Agent2: Average Shortfall for Agent1: Average Shortfall for Agent2: Average Shortfall for Agent1: Average Shortfall for Agent2: Average Shortfall for Agent1: Average Shortfall for Agent2:

Average Implementation Shortfall for Agent1: $579,313.38 Average Implementation Shortfall for Agent2: $582,681.01

216

A. Ranjan et al.

6 Performance Analysis of Multi-agent Systems Here, implementation of expected shortfalls has been compared and is shown in Fig. 4; three agents are used for comparison namely $A, B1$ and $B2$. The result showed that agent A has higher expected shortfall than the result after addition of two expected shortfalls, i.e., $B_1$ & $B_2$ (Fig. 7). Trading trajectory: It showed the comparison of the original trading trajectories and the current trading trajectories when they have been trained in a multi-agent system which looks closer to each other (Fig. 8). Cooperative and Competitive Relationships: This shows the cooperative relationship result of two agents which has not better total expected shortfall as compared to the independent training of reward functions. If it is compared the same in competitive relationship, then the first thing is to learn how to minimize expected shortfall, and it leads to good competition which lead to significant shortfall incremental implementation (Fig. 9). Trading trajectory: It showed independent and competitor trainings where a competitor has been introduced in the independent training that improves the learning of host-agent to accommodate in new environs that take the sell decisions where all the shares of a particular stock has been sold in first two days (Fig. 10).

Fig. 7 Comparison of expected implementation shortfalls

9 Multi-agent Reinforcement Learning …

217

Fig. 8 Trading trajectory

Fig. 9 Cooperative and competitive relationships

7 Conclusion and Future Work We have proposed a multi-agent reinforcement learning approach to solve the problem of liquidation of stock. We conclude that two agents-based model is the better model for stock market analysis and helps to improve both the performance

218

A. Ranjan et al.

Fig. 10 Trading trajectory

and efficiency in the stock prediction. We proposed a multi-agent deep learning-based approach for predicting the stock prices. There is a huge scope in multi-agent training area where researchers can increase the agents to find the better result and can apply different types of algorithms and training models in multi-agent systems.

References 1. Bao W (2019) Multi-agent deep reinforcement learning for liquidation strategy analysis. arXiv: 1906.11046v1, pp 1–9 2. Hendricks D (2014) A reinforcement learning extension to the Almgren-Chriss framework for optimal trade execution. arXiv:1403.2229, pp 1–7 3. Agogino AK, Tumer K (2012) A multiagent approach to managing air traffic flow. Auton Agent Multi-Agent Syst 24:1–25 4. Bloembergen D (2015) Evolutionary dynamics of multi-agent learning: a survey. J Artif Intell Res 53:659–697 5. Mihaylov M (2009) Decentralized learning in wireless sensor networks. In: Adaptive and learning agents workshop, pp 2–4 6. Pipattanasomporn M (2009) Multi-agent systems in a distributed smart grid: design and implementation. In: Power systems conference and exposition, pp 1–6 7. Engle RF, Ferstenberg R (2007) Execution risk. J Portf Manag 33:34–44 8. Gatheral J, Schied A (2011) Optimal trade execution under geometric Brownian motion in the Almgren and Chriss framework. Int J Theor Appl Financ 14:353–368 9. Schied A, Schöneborn T (2009) Risk aversion and the dynamics of optimal liquidation strategies in illiquid markets. Financ Stochast 13:181–204 10. Predoiu S, Shaikhet G, Shreve SE (2011) Optimal execution in a general one-sided limit-order book. Siam J Financ Math 2:183–212

9 Multi-agent Reinforcement Learning …

219

11. Perold AF (1988) The implementation shortfall: paper versus reality. J Portf Manag 14:4–9 12. Almgren R, Chriss N (2001) Optimal execution of portfolio transactions. J Risk 3:5–40 13. Grondman I, Busoniu L, Lopes GAD, Babuska R (2012) A survey of actor-critic reinforcement learning: standard and natural policy gradients. IEEE Trans Syst Man Cybernetics Part C 1–15 14. Fayad A, Ibrahim M (2021) Behavior-guided actor-critic: improving exploration via learning policy behavior representation for deep reinforcement learning. arXiv:2104.04424, pp 1–9 15. Mnih V, Badia AP, Mirza M, Graves A, Lillicrap TP, Harley T, Silver D, Kavukcuoglu K (2016) Asynchronous methods for deep reinforcement learning. arXiv:1602.01783, pp 1–19 16. Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, Wierstra D (2019) Continuous control with deep reinforcement learning. arXiv:1509.02971, pp 1–14 17. Schulman J, Wolski F, Dhariwal P, Radford A, Klimov O (2017) Proximal policy optimization algorithms. arXiv:1707.06347, pp 1–12

Chapter 10

Multi-agent Systems: Future Initiatives Siddhartha Bhattacharyya and Indradip Banerjee

Abstract Multi-agent systems (MAS), as the name suggests, refer to the collaborative and collective engagement of two or more agents to make decisions out of mutual or shared interests. These agents interact with each other to target a problem and give it to fruition. Typical manifestations of MAS are exhibited in the form of self-sustaining driving, multi-robotic manipulation, gaming platforms, automatic tutoring to name a few. With the rapid growth and evolution of computational intelligence algorithms, the future of MAS seems to be ripe with a host of promising avenues. This chapter highlights the possible incarnations of MAS in the near future. Keywords Multi-agent systems · Human–computer interaction · Agents · Human-centered computing

1 Introduction Multi-agent systems (MAS) [1–3], as the name suggests, refer to the collaborative and collective engagement of two or more agents to make decisions out of mutual or shared interests. These agents interact with each other to target a problem and give it to fruition. Typical manifestations of MAS are exhibited in the form of selfsustaining driving, multi-robotic manipulation, gaming platforms, automatic tutoring to name a few. Computational intelligence has stemmed from the efforts invested to infuse human understanding in machines with a special focus on enabling machines to behave like human beings. The computationally intelligent paradigm has ushered in a new era of computing like humans making it more akin to human perception and human participation. Computational intelligence has made strongholds in almost all spheres of technological advancements ranging from scientific applications, engineering designs, and medical engineering practices. With the rapid growth S. Bhattacharyya (B) Rajnagar Mahavidyalaya, Rajnagar, Birbhum 731130, India I. Banerjee Department of Computer Science, University Institute of Technology, The University of Burdwan, Burdwan, India © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 S. Gupta et al. (eds.), Multi Agent Systems, Springer Tracts in Human-Centered Computing, https://doi.org/10.1007/978-981-19-0493-6_10

221

222

S. Bhattacharyya and I. Banerjee

and evolution of computational intelligence algorithms, the future of MAS seems to be ripe with a host of promising avenues.

2 Intelligent MAS Computational intelligence now appears to have infused far-flung possibilities in all domains of engineering, science, and medical fields, thanks to the constant evolution of intelligent algorithms and modalities. The evolved intelligent algorithms are capable of mimicking human activities and human understanding and can inculcate the human factor in existing solutions. One of the obvious fallouts of computational intelligence is the advent of humancentered computing (HCI) [4, 5], which refers to the study and development of human–computer systems based on mixed initiatives. To be precise, HCI can be thought of as a three-dimensional space comprising the human, computer, and environment as the key tenets. This upcoming computing paradigm has been made possible due to the convergence of multiple disciplines centered on both the concepts of understanding human perception and the design of computational artifacts. As such, the field of human-centered computing is multidisciplinary in nature, which encompasses a plethora of disciplines in the form of computing, human perception, psychology, cognitive intelligence, anthropology, communication, interactive graphics, and industrial design. Human-centered computing is primarily targeted to the design of system management centered on human perception and interconnections arising out of human activities. The greater stress is, however, on the human-centered perception rather than on the computing perspectives. Thus, this shift from the computing paradigm has been manifested in several other forms of computing methodologies like social computing, accessibility, and organizational computing. The resultant computing paradigm is thus dependent on the wide range of human-related aims and practices. This aspect of human-centered computing has necessitated a uniform design paradigm for multi-agent teams in a MAS, enabling the collaborating agents to operate and evolve in dynamic environments vis-à-vis maintaining cooperation and interaction with human beings in solving their associated problems. While the current MAS delves with supplementary interdependent multi-agent teams, infusion of computational intelligence can help in translating these teams into intelligent agents which can usher in an efficient interactive and collaborative environment while maintaining human intervention in the process. The possible manifestations may be in the form of optimizing the human–computer or human–agent interactions, optimizing the dynamic interactive environment, construing intelligent decision making on the part of the agents, and better learning of the interactive strategy [6].

10 Multi-agent Systems: Future Initiatives

223

Foresighted applications include distributed environments entailing intelligent Internet information systems, intelligent grids and sensor-based information networks, intelligent wearable devices, multimedia, and multi-modal human– computer interfaces, intelligent interfaces for information representation and visualization, intelligent computer-mediated human–human interaction, defining relevant semantic structures for multimedia information representation, communityspecific social HCI solutions, social orthotics, affective computing, knowledgedriven human–computer interaction, human-centered management science, and participatory action research.

References 1. Alqahtani H, Liu C, Kavakli-Thorne M, Kang Y (2019) An agent-based intelligent hcl information system in mixed reality. In: 28th international conference on information systems development, Toulon, France 2. Albrecht SV, Stone P (2018) Autonomous agents modelling other agents: a comprehensive survey and open problems. Artif Intell 258:66–95. https://doi.org/10.1016/j.artint.2018.01.002 3. Billard AG, Calinon S, Dillmann R (2016) Learning from humans. In: Siciliano B, Khatib O (eds) Springer handbook of robotics, Springer handbooks. Springer, Cham, Switzerland, pp 1995–2014. https://doi.org/10.1007/978-3-319-32552-1_74 4. Ramachandra N, Ahuja M, Rao RM, Dubash N (2021) Human centered computing in digital persona generation. In: Fu W, Xu Y, Wang SH, Zhang Y (eds) Multimedia technology and enhanced learning. ICMTEL 2021. Lecture notes of the institute for computer sciences, social informatics and telecommunications engineering, vol 388. Springer, Cham. https://doi.org/10. 1007/978-3-030-82565-2_32 5. Friedman B, Kahn Jr. PH, Borning A, Kahn PH (2006) Value sensitive design and information systems. Human–computer interaction and management information systems: foundations. ME Sharpe, New York, pp 348–372 6. Brown CM (1998) Human–computer interface design guidelines. Intellect Books, pp 2–3

Index

A Actuator, 39, 88 Agent, 88 Agent Communication Language (ACL), 119–121 Agent specification, 40 Agile agents, 101 Air traffic system, 11 Ambient intelligence, 35 A new scheme for medical diagnosis using MAS, 162 Applications layer, 94 Artificial intelligence, 89

B Background, 140 Behavior based layer, 118 Benchmark production system, 36 Bidirectional friendship link, 44 Body area network, 111 Building automation, 35

C Case based reasoning, 135 Cloud, 89 Cockpit, 11 Collaborative agents, 116, 117 Communication layer, 92 Computational Theory of Mind, 26 Computation tree logic, 34 Conceptual planning model, 37 Conclusion, 167 Confirm-message, 41 Connectivity agent, 106

Conveyor1-left, 39 Covid-19, 44 Crisp output parameter, 50 Cyber agent, 88

D Data gathering, 90 Decline message, 41 Defuzzification unit, 46 Distributed task allocation, 36 Domotics, 35

E Expert system design, 115, 122

F Federated Learning (FL), 22 Foundation for Intelligent Physical Agent (FIPA), 119, 121 Fuzzy vaccination multi-agent system, 36, 45

G Game Theory, 135 Generic-behavior, 38

H Hardware agents, 100 Hardware layer, 90 Heterogeneity/diversity, 90 Heterogeneous agents, 119

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 S. Gupta et al. (eds.), Multi Agent Systems, Springer Tracts in Human-Centered Computing, https://doi.org/10.1007/978-981-19-0493-6

225

226 Human agent, 101 Hybrid agents, 116, 118, 119

I IAPhome, 12, 13 IIoT, 108 Initiator agent, 40 Intelligent behavior, 88 Intelligent systems, 88 Interactive agents, 101 Interconnectivity, 90 Interface agents, 116, 117 Internet agents, 117 Internet of Things (IoT), 88 Introduction, 140 IoT agents, 99 IoT architecture, 88 IoT gateways, 93 Its-ctl, 36

K KAMRO, 8 Knowledge based system, 135 Knowledge Query and Manipulation Language, 121

L Learning from Demonstration (LfD), 5, 21–23 Literature Survey, 141 Low Power Wide Area Network (LPWAN), 92

M Machine-to-machine, 89 Mamdani fuzzy inference system, 44 Markoff’s resolution procedure, 26 Multi-agent platform, 88 Multi agent system, 87 Multi agent systems used in healthcare, 160 Musical scale, 5, 23, 26–29

N Navigation procedure management, 11

O Office robotization, 8

Index P Pandemic, 44 Participant agent, 40 Passive agents, 101 Pilot, 11 Platform agent, 106 Platform layer, 93 Position-power regulator, 10 Processing agent, 105 Protected computing, 103 Pull model, 44 Push model, 44

Q Qualitative distance calculus, 43 Qualitative trajectory calculus, 43

R Reactive agents, 116, 118, 135, 136 Resource-announce-message, 40 Rotate1-left, 39 Rotate1-right, 40

S SARS-CoV-2, 34 Security layer, 94 Self governing versatile automated systems, 8 Semantic behaviors, 43 Semantic matchmaking, 44 Sensors, 88 Service Oriented Architecture (SOA), 97 Severity of symptoms, 44 Short-range connectivity, 92 Simulation, 102 SIR model, 34 Smart Air Traffic System (SATS), 11 Smart city, 104 Smart environments, 88 Smart grid, 111 Smart healthcare, 110 Smart home, 107 Smart home communication, 13 Smart parking, 104 Smart things, 88 Smart waste management, 106 Social networks complexity, 43 Socio-psychological procedures, 19 Software agents, 100 Stag-hunt, 41 Structure mapping engine, 43

Index Supervisory control layer, 131 Supply chain management system, 120 Suppositional behaviors, 43

T Task allotment, 40 Tele health, 20 Temporary working memory, 50 Things-related services, 90 Triangular membership function, 47

227 U Urgency of vaccination, 50

V Vulnerable health system, 53

W World health organization, 53