Next Generation Healthcare Informatics (Studies in Computational Intelligence, 1039) 9789811924156, 9789811924163, 9811924155

Table of contents :
Preface
Contents
Editors and Contributors
Methods for the Recognition of Multisource Data in Intelligent Medicine: A Review and Next-Generation Trends
1 Introduction
2 Data Processing Steps
3 Machine Learning Methods
3.1 ID3
3.2 Linear Regression
3.3 Naive Bayesian
3.4 Support Vector Machines
3.5 K-NN
3.6 Artificial Neural Network
3.7 Long Short-Term Memory
4 Model Evaluation
4.1 Creating Confusion Matrix
4.2 Establishing Accuracy Criteria for Two-Class Modeling
4.3 n-Fold Cross Validation
4.4 Receiver Operating Characteristic
4.5 Kappa Coefficient
5 Multimodal Data Fusion
5.1 Early Fusion
5.2 Late Fusion
5.3 Intermediate Fusion
6 Singular Modeling Experiments
7 Multimodal Experiments
7.1 In Medical Data sets: Literature Studies Using Early Fusion Technique in Multimodal Decision-Based Data Fusion Technique
7.2 In Medical Data sets: Literature Studies Using Late Fusion Technique in Multimodal Decision-Based Data Fusion Technique
7.3 In Medical Data Sets: Literature Studies Using Intermediate Fusion Technique in Multimodal Decision-Based Data Fusion Technique
7.4 Contribution to Multimodal Models
8 Discussion and Conclusion
References
Deep Learning in Healthcare: Applications, Challenges, and Opportunities
1 Introduction
2 Deep Learning Applications in Healthcare
2.1 Medical Imaging and Diagnosis
2.2 Simplification of Clinical Trials
2.3 Drug Discovery
2.4 Enhanced Patient Monitoring and Health Records
2.5 Personalized Treatment
3 Deep Learning Frameworks in Healthcare
3.1 Convolutional Neural Networks (CNN)
3.2 Recurrent Neural Network (RNN)
3.3 Autoencoder (AE)
3.4 Deep Synthetic Minority Oversampling Technique (SMOTE)
3.5 Elastic Net
3.6 Generative Adversarial Network (GAN)
4 Data Types Used in the Automated Healthcare
4.1 Electronic Health Record (EHR)
4.2 Clinical Imaging
4.3 Genomics
5 Challenges of Using Deep Learning in Healthcare
5.1 The Volume of Healthcare Data
5.2 Quality of Healthcare Data
5.3 Handling Healthcare Data Stream
5.4 Temporality
5.5 Complexity in Domain
5.6 Interpretability
6 Opportunities of Using Deep Learning in Healthcare
6.1 Feature Enhancement
6.2 Federated Extrapolation
6.3 Privacy in Modeling
6.4 Integrating Expert Knowledge
6.5 Temporal Modeling
6.6 Explainable Modeling
6.7 Computation Complexity
6.8 Multitasking Using Deep Learning
6.9 Semi-Supervised Learning for Healthcare Data
7 Conclusions and Future Scopes
References
Examination of Health Data Depending on Creative Use of Optimization Methods and Machine Learning Algorithms
1 Introduction
2 Background
3 Literature Review
4 Experimentation
5 Results and Recommendations
6 Future Research Directions
7 Conclusion
References
Effect of Computation and Cognitive Bias in Healthcare Intelligence and Pharmacogenomics
1 Introduction
2 Related Research
3 The Evolution of Clinical Prediction Models
4 Bias and Its Effect in ML-Based Healthcare Predictions
4.1 Computational Bias
4.2 Effectiveness of Bias in Machine Learning Models
5 The Influence of Bias in Healthcare Predictions
5.1 Real-World Examples
5.2 Pharmacogenomics and Bias
6 Conclusion and Future Scope
References
Application of Genetic Algorithms in Healthcare: A Review
1 Introduction
2 Preliminary Concept
2.1 Problems in Healthcare
2.2 Genetic Algorithm
3 Variants of Genetic Algorithm in Healthcare
3.1 Binary GA in Healthcare
3.2 Chaotic GA in Healthcare
3.3 Parallel GA in Healthcare
3.4 Multiobjective GA in Healthcare
4 Applications of GA in Healthcare
4.1 Oncology
4.2 Radiology
4.3 Cardiology
4.4 Surgery
4.5 Obstetrics and Gynaecology
4.6 Radiotherapy
5 Conclusion and Future Scope
References
Decision-Making in Healthcare Nanoinformatics
1 Introduction
2 Data Integration and Knowledge Discovery
2.1 Development of Research Centers for Nanotechnology in Medicine
2.2 Information Technology for Nanotechnology
2.3 Development and Recognition of Standards
2.4 Translational Nanoinformatics
2.5 E-Health Records with Nano-Based Information
3 Decision-Making in Healthcare and Informatics
3.1 Assessment of Healthcare Decision-Making Capacity
4 Decision Analysis in Nanoinformatics
4.1 Multicriteria Decision Analysis
4.2 Value of Information
4.3 Weight of Information
4.4 Portfolio Decision Analysis
5 Nanoinformatics and Biomolecular Computing
5.1 DNA and RNA-Based Computing
5.2 RNA-Based Computers
5.3 Future of DNA and RNA Computing
6 Nanotechnology in Biomedical Applications
6.1 Nanodevices and Nanomedicines
6.2 Nanoinformatics for Precision Medicine
7 Nanotechnology to Handle COVID
8 Issues and Challenges
8.1 Obstacles for Implementation of Nanotechnology in Health Care
9 Future Scope
10 Conclusions
References
A Succinct Analytical Study of the Usability of Encryption Methods in Healthcare Data Security
1 Introduction
2 Encryption and Its Use in Healthcare Data
2.1 Public Keys
2.2 Private Keys
3 The Need of Encryption
3.1 Data Interception in Cyberspace
3.2 Lost and Stolen Unencrypted Devices
3.3 The Ease and Importance of Encryption
4 Different Types of Encryptions
4.1 Symmetric Cryptography
4.2 Asymmetric Cryptography
5 Existing Works in Healthcare Data Security Using Encryption
6 Example of a Health Data Networking Model
7 Problems with Cryptography
8 Encryption Likelihood of Health-Related Data
9 Conclusions
10 Future Scope
References
IoMT in Healthcare Industry—Concepts and Applications
1 Introduction
2 Literature Survey
3 Architecture of Healthcare IoT (HIoT)
4 HIoT Technologies
4.1 Identification Technology
4.2 Communication Technology
4.3 Location Technology
5 Technologies Enduing IoMT Implementation
5.1 Local Systems and Control Layer
5.2 Device Connectivity and Data Layer
5.3 Analytic Solutions Layer
6 Advantages and Disadvantages of IoMT
7 Open Issues in the Implementation of IoMT
8 Services of IoT in Health care
8.1 Ambient Assisted Living. Ambient Assisted Living
8.2 Mobile IoT
8.3 Wearable Devices
8.4 Community-Based Healthcare Services
8.5 Cognitive Computing
8.6 Adverse Drug Reaction
8.7 Blockchain
8.8 Child Health Information
9 Healthcare Applications of IoT
9.1 ECG Monitoring
9.2 Glucose Level Monitoring
9.3 Temperature Monitoring
9.4 Blood Pressure Monitoring
9.5 Oxygen Saturation Monitoring
9.6 Asthma Monitoring
9.7 Mood Monitoring
9.8 Medication Management
9.9 Wheelchair Management
9.10 Rehabilitation System
9.11 Other Notable Applications
10 Case Study—Internet of Medical Things (IoMT) for Orthopaedic in COVID-19 Pandemic
10.1 Working Process of IoMT for Orthopaedic During COVID-19
10.2 Digital Connectivity of Hospital During COVID-19 Pandemic Using IoMT
10.3 Key-Roles of IoMT in Orthopaedic Field During COVID-19 Pandemic
11 Conclusion and Future Scope
References
The Effect of Heuristic Methods Toward Performance of Health Data Analysis
1 Introduction
2 Heuristic Methods
2.1 Biology-based Algorithms
2.2 Swarm Intelligence Algorithms
3 Heuristic Methods for the Health Data Analysis
3.1 Heuristic Methods for the Problems of Missing Value and Unbalanced Dataset in Health Data
3.2 Heuristic Methods for Feature Selection in Health Data
3.3 Heuristic Methods for Detection and Prediction of Diseases in Health Data
4 Results
5 Conclusion
References
AI for Stress Diagnosis at Home Environment
1 Introduction
1.1 Challenges
1.2 Major Goals and Contributions
1.3 Overview of the Proposed Approach
1.4 Area of Application
1.5 Structure of the Paper
2 Related Works and Analysis
2.1 Keystroke Dynamics-Based Stress Detections
3 Proposed Method
3.1 Event Monitoring
3.2 Pre-processing
3.3 Feature Selection
3.4 Bootstrapping and Building Model
3.5 Classification and Decision
4 Dataset and Implementation
4.1 Dataset
4.2 Feature Extraction and Selection
4.3 Windowing and Sampling
4.4 Outlier Detection and Removal
4.5 Statistical Features Extraction
4.6 Normalization
4.7 Classification and Evaluation
5 Experimental Results
5.1 Performance of the Proposed Model
6 Discussion
7 Performance Comparison
7.1 Model Complexity
7.2 Challenges, Advances, and Opportunities
8 Conclusions
References
Contemporary Technologies to Combat Pandemics and Epidemics
1 Introduction
2 Background
3 Main Focus of the Chapter
4 Issues, Controversies, Problems
4.1 Hindrance During the Spanish Flu
4.2 Hindrance During the SARS (2003)
4.3 Hindrance During the Zika Virus
4.4 Hindrance During the COVID-19
4.5 Comparing the Statistics
5 Solutions and Recommendations
5.1 Role of Technology
5.2 Information Gathering and Privacy Protection
5.3 Current Status
6 Future Scope
7 Conclusions
References
Deep Learning for Diabetic Retinopathy Detection: Challenges and Opportunities
1 Introduction
2 The Fundus Image
3 Diabetic Retinopathy
4 Deep Learning
4.1 Artificial Neural Network
4.2 Convolutional Neural Networks
4.3 Generative Adversarial Network
4.4 U-Net
4.5 Transfer Learning
4.6 Ensemble Learning
5 Preprocessing
5.1 Fundus Image Denoising
5.2 Fundus Image Normalization
5.3 Fundus Image Color Channel Extraction
5.4 Fundus Image Contrast Enhancement
5.5 Fundus Image Cropping and Resizing
5.6 Data Augmentation
6 A Brief Comparison of the Techniques Proposed for DR Detection and Segmentation
7 DL Challenges in DR Classification/Segmentation
7.1 Data-Related Challenges
7.2 Challenges in Terms of Availability of Ophthalmologists
7.3 Production Issues
7.4 Black Box Problem
7.5 Class Imbalance Problem
7.6 Data Privacy and Legal Issues
8 DL Opportunities
9 Conclusion
10 Future Scope
References
Deep Reinforcement-Based Conversational AI Agent in Healthcare System
1 Introduction
2 Literature Review
3 Methodology
3.1 Natural Language Understanding
3.2 Dialog Management
3.3 Natural Language Generation
3.4 Textual Question Answering
3.5 Text Summarization
3.6 Visual Question Answering
3.7 Anomaly Detection
4 Results
5 Conclusion
References
Deep Learning Empowered Fight Against COVID-19: A Survey
1 Introduction
2 Methodology Used in This Survey
2.1 Sources
2.2 Keywords
2.3 Inclusion and Exclusion Criteria
3 Deep Learning-Based Systems Used for Diagnosis and Prevention of COVID-19
3.1 Collecting Samples
3.2 Radiology
3.3 Surgery
3.4 Medicine and Antiviral Therapeutics
3.5 Convalescent Plasma Therapy is a New Medication Approach
3.6 Prediction
3.7 Detection
3.8 Response and Recovery
3.9 Hospital
3.10 Drone Technology and Delivery of Goods
3.11 Surveillance
3.12 Robotics Technologies
4 Discussion
5 Summary
References
Application of GAN in Guided Imagery Therapy
1 Guided Imagery (GI) Therapy
2 Applications of Guided Image (GI) Therapy
3 Introduction to Generative Adversarial Networks (GAN)
4 Different Types of GAN
5 Application of DCGAN in Guided Imagery Therapy
6 The Proposed Methodology
7 Conclusion and Future Work
References
Digital Transformation in Healthcare Industry: A Survey
1 Introduction
1.1 Healthcare with AI and ML
2 Healthcare Industry Application Programming Interface
2.1 Importance of APIs in the Development of Human Health
3 Challenges in Digital Transformation of Healthcare
4 Future of Healthcare Technology
4.1 Technology Trends in Healthcare
5 Contribution of Digital Transformation
6 Relevant Case Studies
6.1 Case Study of Africa
6.2 Case Study of Poland
7 Conclusions and Scope for Future Studies
References
Application of Deep Learning in Mental Disorder: Challenges and Opportunities
1 Introduction
2 Mental Disorder
2.1 Fundamental of Mental Disorder
2.2 Taxonomy of Mental Disorder
3 Deep Learning Techniques and Architecture
3.1 Fundamental of Deep Learning in Healthcare
3.2 Architecture of Deep Learning
3.3 Deep Learning Architecture for Mental Disorder
4 Challenges and Opportunities
4.1 Challenges in Mental Disorder
4.2 Challenges and Limitations When Applying Deep Learning to Mental Disorder
4.3 Data Creation and Annotation
4.4 How Accurately and Precisely Measure Mental Disorder Similarity
4.5 Quality Global Healthcare
5 Deep Learning Implementation
5.1 Mental Disorder Dataset
5.2 Experiments and Performance Measures Used
5.3 Experimental Results and Discussion
6 Conclusions and Future Scope of Research
References

Citation preview

Studies in Computational Intelligence 1039

B. K. Tripathy Pawan Lingras Arpan Kumar Kar Chiranji Lal Chowdhary   Editors

Next Generation Healthcare Informatics

Studies in Computational Intelligence Volume 1039

Series Editor Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Poland

The series “Studies in Computational Intelligence” (SCI) publishes new developments and advances in the various areas of computational intelligence—quickly and with a high quality. The intent is to cover the theory, applications, and design methods of computational intelligence, as embedded in the fields of engineering, computer science, physics and life sciences, as well as the methodologies behind them. The series contains monographs, lecture notes and edited volumes in computational intelligence spanning the areas of neural networks, connectionist systems, genetic algorithms, evolutionary computation, artificial intelligence, cellular automata, selforganizing systems, soft computing, fuzzy systems, and hybrid intelligent systems. Of particular value to both the contributors and the readership are the short publication timeframe and the world-wide distribution, which enable both wide and rapid dissemination of research output. Indexed by SCOPUS, DBLP, WTI Frankfurt eG, zbMATH, SCImago. All books published in the series are submitted for consideration in Web of Science.

More information about this series at https://link.springer.com/bookseries/7092

B. K. Tripathy · Pawan Lingras · Arpan Kumar Kar · Chiranji Lal Chowdhary Editors

Next Generation Healthcare Informatics

Editors B. K. Tripathy School of Information Technology and Engineering Vellore Institute of Technology Vellore, India

Pawan Lingras Department of Mathematics and Computer Science St. Mary’s University Halifax, NS, Canada

Arpan Kumar Kar Bharti School of Telecommunications Technology and Management Indian Institute of Technology New Delhi, India

Chiranji Lal Chowdhary School of Information Technology and Engineering Vellore Institute of Technology Vellore, India

ISSN 1860-949X ISSN 1860-9503 (electronic) Studies in Computational Intelligence ISBN 978-981-19-2415-6 ISBN 978-981-19-2416-3 (eBook) https://doi.org/10.1007/978-981-19-2416-3 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

B. K. Tripathy would like to dedicate this book to his beloved teacher Prof. (Dr.) Narahari Parhi (on his 80th birth year) for his suggestions at a critical juncture, which changed his focus in career and was instrumental in putting him in the present position. Chiranji Lal Chowdhary would like to dedicate this book to his beloved wife for all her support in life, without which he would not have reached the present position.

Preface

Healthcare informatics is one among the primary focus of researchers and industry experts due to its significant effects on society and has emerged as a growing area of interest among researchers worldwide. In transforming health care, obtaining awareness and actionable perspectives from diverse, high-dimensional and heterogeneous biomedical data have been a constant challenge. In modern biomedical research, data generated are in various forms, including electronic health records, imaging, omics, sensor data and text, which are nuanced, heterogeneous, poorly annotated and typically unstructured. The last decade has witnessed previously unforeseen advances in artificial intelligence techniques in numerous fields. Machine learning applications in health care range from disease detection to personalized services at the patient level. In this volume, we are going to present several topics under the current status of research in healthcare informatics as stated below. There are few comprehensive studies in which data obtained from different sources are used together to define diseases. Since each disease is disease-specific to the individual, it is necessary to establish systems that assist the physician by taking steps to reflect on preventive treatment for the early diagnosis of the disease by evaluating the data of each individual individually and extracting the meaningful features related to the disease. The multimodal decision-based data fusion technique is promising for effectively classifying individual data in disease detection. In Chapter “Methods for the Recognition of Multisource Data in Intelligent Medicine: A Review and Next-Generation Trends”, the study is about the systematic literature review that uses new generation techniques for a personal, reliable and sensitive healthcare system that can determine the course of the disease by processing data obtained from different digital sources by the artificial intelligence methods. Deep learning is helping medical researchers and professionals to uncover hidden data patterns and to better support the healthcare sector. It uses large collection of data such as patient information, health reports, insurance records and produces best results. The next chapter, “Deep Learning in Healthcare: Applications, Challenges, and Opportunities” acts as a foundation for illuminating these issues and highlights opportunities for methods of the deep learning group to contribute to health care.

vii

viii

Preface

Fast decision-making-based medical systems have a huge ability to reduce treatment costs, refine the standard patient healthcare and reduce duplication and failures. But, perhaps some of the most important challenges is rebooting the healthcare system is the problem of dealing with big data of storage medium, necessary information retrieval within a short period of time, cost-effective treatment options and many more. Chapter “Examination of Health Data Depending on Creative Use of Optimization Methods and Machine Learning Algorithms” discusses on the techniques developed in this direction and proposes a comparative analysis of them. Pharmacogenomics (personalized future medicine and lifesaving therapy) is the study of how genes affect a person’s response to drugs. This new field is emerging with collaborative approach by pharmacology (the science of drugs), genomics (the study of genes and their functions) and machine intelligence (AI technologies). The machine learning techniques, which are relevant to pharmacogenomics, are highlighted in Chapter “Effect of Computation and Cognitive Bias in Healthcare Intelligence and Pharmacogenomics”. Cancer is a wide category of diseases that is caused by the abnormal, uncontrollable growth of cells, and it is the second leading cause of death globally. Screening, early diagnosis and prediction of recurrence gives patients the best possible chance for successful treatment. Genetic algorithms (GAs) are meta-heuristics that belong to the class for evolutionary algorithms. GAs can find the optimal or near-optimal solutions in huge, difficult search spaces and are widely used for search and optimization. Chapter “Application of Genetic Algorithms in Healthcare: A Review” discusses the use of GAs to analyse, review, address and compare the developments in cancer detection, so that the best techniques can be utilized by medical professionals. Nanotechnology has become one of the most sought after areas of research in recent days. Due to increase in data due to extensive research in this area and to take benefit from that huge amount of data, the necessity of informatics is of utmost importance. The process of building nano-informatics repositories, decisionmaking processes get pivotal positions. Among several associated interdisciplinary field of applications, healthcare sector is of utmost importance. The applications of computing in nano-medicine are challenging and mostly unexplored. Nanomedicines are more reliable and biocompatible. It provides better precision and accuracy in targeting the place to deliver the drug more safely. Rapid invention of new nano-medicines with better therapeutic range and advancement of better in-vivo pharmacokinetic properties show that the research is in the right direction. Chapter “Decision-Making in Healthcare Nanoinformatics” provides many insights related to nanotechnology in health care, nano-informatics and decision-making methodologies involved in it. Security of healthcare data is a basic requirement. This can be achieved through encryption. In Chapter “A Succinct Analytical Study of the Usability of Encryption Methods in Healthcare Data Security”, various aspects related to application of encryption in this clinical sector are presented. Importance of encryption in healthcare domain along with its associated challenges is highlighted through the three techniques of public key encryption (PKE), symmetric key encryption (SKE) and attribute-based encryption (ABE).

Preface

ix

The desires of people for a modern lifestyle and easy access to resources have brought the concept of smart cities. Internet of Things (IoT) is an important aid to achieve this. Internet of Medical things (IoMT) is the version of IoT employed in the field of medical sciences. Use of neural networks (NN) and deep learning accelerators (DLAs) have brought improvements in the field of IoMT to a great extent. Chapter “IoMT in Healthcare Industry—Concepts and Applications” presents these concepts with satisfaction of quality of services following low energy consumption using machine learning (ML) algorithms and different mapping approaches. Heuristic methods are useful in many optimization problems, and health data analysis is no exception. Using heuristic methods in combination of other methods increases the performance of the algorithms. In Chapter “The Effect of Heuristic Methods Toward Performance of Health Data Analysis”, the combinations of heuristic algorithms in conjunction with other machine learning algorithms on healthcare data are analysed. Neurocognitive disorders like those developing from emotional stress are needed to be detected earlier in order to prevent fatal consequences. Detecting this disease in a laboratory environment is challenging to diagnose and monitor properly for a longer period. But artificial intelligence-based technologies provide a scope for thinking to measure stress at home by using the way the user interacts with a phone that can detect stress continuously for a longer period. In Chapter “AI for Stress Diagnosis at Home Environment”, an ensemble model of classification is proposed which depends upon precision of the sensors attached to the smartphones. Repeated evaluations of the proposed approach achieved high accuracies. This ensemble approach is fast, accurate, reliable, capable of running long and compatible with any low-configured and space–memory constraints devices like smartphones. Also, comparisons are made with existing approaches. The online mode of the proposed approach can be used in distance-based diagnosis, better treatment, therapy management and future reference. Directions for possible extension of the proposed approach have been outlined which are likely to be more effective. Pandemics and epidemics cause terrors in the human society due to the loss of human lives they cause due to their terrible effects. However, while epidemic is mostly confined to specific regions pandemic is worldwide in coverage. Other points of contention to distinguish these two are the spread of infectious, contagious or viral illnesses. More importantly, their progress is sudden and there is little preparedness. In Chapter “Contemporary Technologies to Combat Pandemics and Epidemics”, the authors have aimed to analyse various pandemics and epidemics till date and finding out how technology has helped to mitigate them. To aid this, they have focused on studying the role played by technology in maintaining the economy during past pandemics and predicting how any future pandemic can be handled by eliciting previous mistakes and by improving current technology aided by artificial intelligence. Several tech-driven outlooks for predicting and dealing with outbreaks via smart, robust and cost-effective solutions have been proposed in the literature. All these are presented in this chapter which can help in developing effective strategies and action plans to combat future pandemics and epidemics.

x

Preface

Chapter “Methods for the Recognition of Multisource Data in Intelligent Medicine: A Review and Next-Generation Trends” deals with diabetic retinopathy, which is one of the retinal disorders, caused due to high sugar levels in the blood. For the automatic detection of diabetic retinopathy, several techniques have been proposed. Deep learning provides one such technique. In Chapter “Deep Learning for Diabetic Retinopathy Detection: Challenges and Opportunities”, different approaches in this direction are compared and their efficiencies are analysed. Conversational AI is a sub-domain of artificial intelligence that deals with speechbased or text-based AI agents that have the capability to simulate and automate conversations and verbal interactions. Reinforcement learning, which has striking similarities with conversational AI, has boosted research on this topic. It has been established in many cases that hybrid approaches are more efficient than their components if combined suitably. In Chapter “Deep Reinforcement-Based Conversational AI Agent in Healthcare System”, an hybrid architecture using deep reinforcement learning is discussed which has shown improved results on the tasks of intent classification, entity recognition, dialogue management, state tracking, information retrieval and natural language response generation. At present, we all are feeling and living under the shadow of COVID-19. This virus impacts a person’s respiratory system and creates patchy white shadows in the lungs, which leads to pneumonia and goes further to death. Detection of the disease is very important to go for treatments which possibly save the patient. Its outbreak has necessitated and motivated several scholars in the field of image processing to come up with the state-of-the-art deep learning research in the field of pandemic. Comparative analyses of these methods are presented in Chapter “Deep Learning Empowered Fight Against COVID-19: A Survey” of this volume. It explores various safe, reliable and effective deep learning tools that can be used in COVID-19 applications. The mostly used imagines are X-ray, CT scan and MRI in COVID-19 with deep learning methods. Therapy that uses “Guided Images” along with visualization to restore or increase the positivity in behaviours and thoughts with stress reduction is referred to as guided imagery therapy. Generative adversarial networks (GANs) can be defined as the process of generating the output by learning the pattern and the regularities of the input data in such a manner that it is not possible to identify the origin of the output. Chapter “Application of GAN in Guided Imagery Therapy” presents the techniques to utilize GAN to generate images that can be used by the therapist for “Guided Imagery Therapy”.

Preface

xi

All the players in the vast and complex healthcare ecosystem are stepping up their digitization and digital transformation efforts. Chapter “Digital Transformation in Healthcare Industry: A Survey” sheds light on opportunities and challenges of healthcare digitization before elaborating on the root causes of harming the digitalization. Mental illness is a type of health condition that changes a person’s mind, emotions or behaviour (or all three) and has been shown to impact an individual’s physical health. In Chapter “Application of Deep Learning in Mental Disorder: Challenges and Opportunities”, the state-of-the-art machine learning approaches are presented by considering the trends, gaps and challenges in mental health questions and then proposed several deep learning methods. In the next chapter, the suitability of deep learning applications to address the mental health issues in terms of developing plans for effective mental health treatment, identifying behavioural biomarkers and predicting crises with utmost caution are discussed. This edited volume is targeted for undergraduate and graduate students, practitioners, researchers, clinicians and data scientists who are interested in getting information on latest developments in the field of health care. This will bring the researchers to the frontiers of the literature on healthcare informatics, so that they can build up their problems for further research. Vellore, India Halifax, USA New Delhi, India Vellore, India

B. K. Tripathy Pawan Lingras Arpan Kumar Kar Chiranji Lal Chowdhary

Contents

Methods for the Recognition of Multisource Data in Intelligent Medicine: A Review and Next-Generation Trends . . . . . . . . . . . . . . . . . . . . Sengul Bayrak and Eylem Yucel

1

Deep Learning in Healthcare: Applications, Challenges, and Opportunities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jyotismita Chaki

27

Examination of Health Data Depending on Creative Use of Optimization Methods and Machine Learning Algorithms . . . . . . . . . . Ça˘gda¸s Özer and Zeynep Orman

45

Effect of Computation and Cognitive Bias in Healthcare Intelligence and Pharmacogenomics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . G. K. Panda, I. K. Sahu, and D. Sahu

57

Application of Genetic Algorithms in Healthcare: A Review . . . . . . . . . . . Sahil Sharma and Vijay Kumar

75

Decision-Making in Healthcare Nanoinformatics . . . . . . . . . . . . . . . . . . . . . R. K. Mohanty and B. K. Tripathy

87

A Succinct Analytical Study of the Usability of Encryption Methods in Healthcare Data Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Abhishek, Hrudaya Kumar Tripathy, and Sushruta Mishra IoMT in Healthcare Industry—Concepts and Applications . . . . . . . . . . . . 121 Anirban Mitra, Utpal Roy, and B. K. Tripathy The Effect of Heuristic Methods Toward Performance of Health Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Hatice Nizam Ozogur and Zeynep Orman AI for Stress Diagnosis at Home Environment . . . . . . . . . . . . . . . . . . . . . . . . 173 Soumen Roy, Utpal Roy, Devadatta Sinha, and Rajat Kumar Pal

xiii

xiv

Contents

Contemporary Technologies to Combat Pandemics and Epidemics . . . . . 197 Aviral Jain, Ipsita Goel, Sahaj Maheshwari, and B. K. Tripathy Deep Learning for Diabetic Retinopathy Detection: Challenges and Opportunities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 N. Jagan Mohan, R. Murugan, and Tripti Goel Deep Reinforcement-Based Conversational AI Agent in Healthcare System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 Pradnya S. Kulkarni, Andrew Stranieri, Ameya Mahableshwarkar, and Mrunalini Kulkarni Deep Learning Empowered Fight Against COVID-19: A Survey . . . . . . . 251 Chiranji Lal Chowdhary and Harpreet Kaur Channi Application of GAN in Guided Imagery Therapy . . . . . . . . . . . . . . . . . . . . . 265 Biswa Ranjan Samal and Mrutyunjaya Panda Digital Transformation in Healthcare Industry: A Survey . . . . . . . . . . . . . 279 Harpreet Kaur Channi, Prateek Shrivastava, and Chiranji Lal Chowdhary Application of Deep Learning in Mental Disorder: Challenges and Opportunities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295 Sumitra Mallick and Mrutyunjaya Panda

Editors and Contributors

About the Editors Dr. B. K. Tripathy received his Ph.D. degree in 1983. During his student career, he received 3 gold medals for standing first at graduation level, standing first at postgraduate level, and being adjudged as the best postgraduate of the year from Berhampur University, Odisha. He has the distinction of receiving the national scholarship at PG level, UGC (Government of India) fellowship for pursuing his research, DST (Government of India) fellowship for pursuing M.Tech. (Computer Science) in Pune University, and the SERC fellowship (DOE, Government of India) for joining IIT Kharagpur as a visiting fellow. As recognition of his distinguished research in the field of Mathematics, he was included by American Mathematical Society as a reviewer for Mathematical Reviews and later honored with an honorary membership of American Mathematical Society in 1992. A year later, he was selected as a reviewer for Zentralblutt fur Mathematik (Germany). At present, Dr. Tripathy is a reviewer for over 100 international journals from all top publishing houses world over. He has published over 600 research articles in several international journals, proceedings of international conferences, edited research volume chapters, and national conference proceedings. Some of his publications are in journals like Information Sciences, Applied Soft Computing, IEEE Access, Journal of Mathematical Analysis and Applications, Bulletin of the Malaysian Mathematical Society, Indian Journal of Pure and Applied Mathematics, Journal of Educational Technology and Society, International Journal of Earth Sciences and Engineering, Kybernetes, World Applied Sciences Journal, International Journal of Communication Systems, Journal of Web Engineering and Evolutionary Intelligence, only to name a few. As an evidence of his research supervising ability, Prof. Tripathy has supervised over 52 candidates for Ph.D. degrees, M.Phil. degrees, and M.Tech./M.S. by research degrees. Dr. Tripathy is a member of more than 20 professional bodies like a senior member of IEEE, a senior member of ACM, a senior member of IRSS, a senior life member of CSI, Indian Academy of Mathematics, IEEE Communication Society, Indian Mathematical Society, ACM Compute News group, Indian Science Congress Association,

xv

xvi

Editors and Contributors

International Science and Technology group, and IEEE fuzzy logic society. He has evaluated the Ph.D. thesis of several universities from India. He has been sanctioned with three research projects from central agencies of India. Pawan Lingras is a graduate of IIT Bombay with graduate studies from University of Regina. He is currently a professor and a director of Computing and Data Analytics at Saint Marys University, Halifax. He is also internationally active having served as a visiting professor at Munich University of Applied Sciences, IIT Gandhinagar, as a research supervisor at Institut Superieur de Gestion de Tunis, as a scholar-inresidence, and as a Shastri Indo-Canadian scholar. He has delivered more than 60 invited talks at various institutions around the world. He has authored more than 210 research papers in various international journals and conferences. He has also co-authored three textbooks and co-edited two books and eight volumes of research papers. His academic collaborations/co-authors include academics from Canada, Chile, China, Germany, India, Poland, Tunisia, UK, and USA. His areas of interests include artificial intelligence, information retrieval, data mining, web intelligence, and intelligent transportation systems. He has served as the general co-chair, the program co-chair, the review committee chair, the program committee member, and a reviewer for various international conferences on artificial intelligence and data mining. He is also on editorial boards of a number of international journals. His research has been supported by Natural Science and Engineering Research Council (NSERC) of Canada for twenty-five years, as well as other funding agencies including NRC-IRAP and MITACS. His total research and development grants as of May 2020 exceed $2.5 million with $2 million coming in the last five years. He has collaborated with more than 25 companies on projects ranging from feasibility studies to product development. He has supervised 135 research students (1 PDF, 9 Ph.D., 77 Master’s, 48 UG). He has coached highly successful academic teams for computer programming as well as software development competitions. He has also served on the NSERC’s Computer Science peer-review committee. He has been awarded an Alumni Association Excellence in Teaching Award, Student Union’s Faculty of Science Teaching Award, and President’s Award for Excellence in Research at Saint Mary’s University. Prof. Arpan Kumar Kar is Amar S Gupta Chair Professor in Indian Institute of Technology Delhi, India. Within IIT Delhi, he shares a joint appointment in the Department of Management Studies and School of Artificial Intelligence. Administratively he is the Chair of Corporate Relations (DMS) and is a member of Board of Academic Programme and Institute of Eminence Committee, among others. His research interests are in the domain of data science, digital transformation, internet ecosystems, social media and ICT-based public policy. He has authored over a 170 peer reviewed articles and edited 8 research monographs, all of which are highly cited. He is the recipient of Research Excellence Award by Clarivate Analytics (Web of Science) for highest citations from 2015–2020. He is the recipient of Basant Kumar Birla Distinguished Researcher Award for the highest count of ABDC A*/ABS 4/FT50 level publications in India between the period

Editors and Contributors

xvii

2014–2019. In terms of teaching cases, he is the recipient of the Best Seller Award from Ivey Cases/Harvard Business Publishing in 2020. He has received numerous other awards as well from reputed organizations like 3 International Federation of Information Processing Best Research Papers, Association of Computing Machinery ICEGOV Best Research Paper, Tata Consultancy Services Best Research Project, Project Management Institute Research Scholar, Association of Indian Management Schools Faculty Research, IIT Delhi Teaching Excellence, and many more best paper awards. Prior to joining IIT Delhi, he has worked in IIM Rohtak, Cognizant Business Consulting and IBM India Research Laboratory. He is the Editor in Chief of International Journal of Information Management Data Insights, published by Elsevier. He is also Associate/Coordinating Editor in International Journal of Electronic Government Research, Journal of Public Affairs, Information Systems Frontiers and Global Journal of Flexible Systems Management. He has undertaken over 40 research, consultancy and training projects from organizations like BASF, PWC, Fidelity, EY, Facebook, CIPPEC, BitGrit, Government of India (DST, MOTA, MOT, MEITY, MHRD, etc). Chiranji Lal Chowdhary is an associate professor in the School of Information Technology and Engineering at VIT University, where he has been since 2010. He received a B.E. (CSE) from MBM Engineering College at Jodhpur in 2001 and M.Tech. (CSE) from the M. S. Ramaiah Institute of Technology at Bangalore in 2008. He received his Ph.D. in Information Technology and Engineering from the VIT University, Vellore, in 2017. From 2006 to 2010, he worked at M. S. Ramaiah Institute of Technology in Bangalore, eventually as a lecturer. His research interests span both computer vision and image processing. Much of his work has been on images, mainly through the application of image processing, computer vision, pattern recognition, machine learning, biometric systems, deep learning, soft computing, and computational intelligence. He has given few invited talks on medical image processing. Professor Chowdhary is the editor/co-editor of 8 books and is the author of over fifty articles on computer science. He filed two patents deriving from his research.

Contributors Abhishek School of Computer Engineering, Kalinga Institute of Industrial Technologies, Bhubaneswar, Odisha, India Sengul Bayrak Istanbul Sabahattin Zaim University, Istanbul, Turkey Jyotismita Chaki School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, India Harpreet Kaur Channi Department of Electrical Engineering, Chandigarh University, Mohali, Punjab, India

xviii

Editors and Contributors

Chiranji Lal Chowdhary School of Information Technology and Engineering, Vellore Institute of Technology, Vellore, India Ipsita Goel VIT, Vellore, Tamil Nadu, India Tripti Goel Bio-Medical Imaging Laboratory, Department of Electronics and Communication Engineering, National Institute of Technology Silchar, Silchar, Assam, India N. Jagan Mohan Bio-Medical Imaging Laboratory, Department of Electronics and Communication Engineering, National Institute of Technology Silchar, Silchar, Assam, India Aviral Jain VIT, Vellore, Tamil Nadu, India Mrunalini Kulkarni MIT World Peace University, Pune, India Pradnya S. Kulkarni MIT World Peace University, Pune, India Vijay Kumar Computer Science and Engineering, National Institute of Technology, Hamirpur, Himachal Pradesh, India Ameya Mahableshwarkar MIT World Peace University, Pune, India Sahaj Maheshwari Wipro, Bengaluru, India Sumitra Mallick Department of Computer Science and Applications, Utkal University, Bhubaneswar, India Sushruta Mishra School of Computer Engineering, Kalinga Institute of Industrial Technologies, Bhubaneswar, Odisha, India Anirban Mitra Department of CSE, ASETK, Amity University Kolkata, Newtown, West Bengal, India R. K. Mohanty School of Computer Science and Engineering, VIT, Vellore, Tamil Nadu, India R. Murugan Bio-Medical Imaging Laboratory, Department of Electronics and Communication Engineering, National Institute of Technology Silchar, Silchar, Assam, India Hatice Nizam Ozogur Department of Computer Engineering, Istanbul Kultur University, Istanbul, Turkey Zeynep Orman Department of Computer Engineering, Istanbul UniversityCerrahpasa, Istanbul, Turkey Ça˘gda¸s Özer Istanbul Bilgi University, Istanbul, Turkey Rajat Kumar Pal Department of Computer Science and Engineering, University of Calcutta, Kolkata, India

Editors and Contributors

xix

G. K. Panda School of Biotechnology, MSB, Utkal University, Bhubaneswar, Odisha, India Mrutyunjaya Panda Department of Computer Science and Applications, Utkal University, Bhubaneswar, India Biswa Ranjan Samal Department of Computer Science and Application, Utkal University, Bhubaneswar, India Soumen Roy Department of Computer Science and Engineering, University of Calcutta, Kolkata, India Utpal Roy Department of Computer and System Sciences, Siksha-Bhavana, VisvaBharati, Santiniketan, West Bengal, India D. Sahu Bank of America, Addison, TX, USA I. K. Sahu Faculty of MCA, KA College, Ganjam, Odisha, India Sahil Sharma Computer Science and Engineering, Thapar Institute of Engineering and Technology, Patiala, India Prateek Shrivastava Electrical Engineering Department, Chandigarh University, Mohali, Punjab, India Devadatta Sinha Department of Computer Science and Engineering, University of Calcutta, Kolkata, India Andrew Stranieri Federation University, Ballarat, Australia B. K. Tripathy VIT, Vellore, Tamil Nadu, India; School of Information Technology and Engineering, VIT, Vellore, Tamil Nadu, India Hrudaya Kumar Tripathy School of Computer Engineering, Kalinga Institute of Industrial Technologies, Bhubaneswar, Odisha, India Eylem Yucel Istanbul University—Cerrahpasa, Istanbul, Turkey

Methods for the Recognition of Multisource Data in Intelligent Medicine: A Review and Next-Generation Trends Sengul Bayrak

and Eylem Yucel

Abstract The use of technological innovations in medicine has led to an increase in efficiency in disease recognition and to guide human life by facilitating. Intelligent medical systems can easily diagnose the disease predictions that real physicians may overlook by establishing a connection between the disease and the symptoms. These developments have created the need for new hardware and software technologies to process of big data in medical science. Today, technological support is necessary, especially for patients with neurological disorders such as epilepsy and Alzheimer’s, requiring constant attention, care, and treatment. New technological innovations are needed to develop new artificial intelligence modules to process and define the big data from different data sources in digital medical applications. In the applications, innovative technological developments are needed by using artificial intelligence methods to process and define big data from different data sources. This study is about the systematic literature review that uses new generation techniques for a personal, reliable, and sensitive healthcare system that can determine the course of the disease by processing data obtained from different digital sources by the artificial intelligence methods. Keywords Intelligent medicine · Multisource data · Neurological disorders · Artificial intelligence methods

1 Introduction In recent years, interactive studies between artificial intelligence and medical sciences have found application. The increase in studies developed with machine learning methods to assist the doctor in diagnosing diseases has led to the interest in studies S. Bayrak (B) Istanbul Sabahattin Zaim University, Istanbul 34303, Turkey e-mail: [email protected] E. Yucel Istanbul University—Cerrahpasa, Istanbul 34320, Turkey e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 B. K. Tripathy et al. (eds.), Next Generation Healthcare Informatics, Studies in Computational Intelligence 1039, https://doi.org/10.1007/978-981-19-2416-3_1

1

2

S. Bayrak and E. Yucel

between the two sciences. Classification of diseases with the mathematical methods approaches developed using feature engineering methods has created a solid alternative to clinical methods, and new studies with better results are needed in the literature. In current studies, it has been possible to detect diseases by processing data obtained from various data sources for many diseases. However, in today’s world, data from different data sources is now called big data. Extracting meaningful disease-related features from big data is a challenge. The 5 V, volume, variety, velocity, veracity, and value, are the essential characteristics for the processing of data [1]. Day by day, scientists are challenged by the problem of extracting meaningful data from big data from different sources. In recent years, studies have generally been used to classify diseases. It has been determined that the mathematical methods developed for the data in the available databases related to the diseases generally deal with a single model. Data from different sources has different characteristics by nature. It is not practical to classify the entire data set with a single classifier by combining all the features. Instead, categorizing the features into different groups and combining the classifiers’ outputs produces more successful classification results [2]. Data obtained from different sources can include numerical and categorical data and signal, image, video, and text data. In this study, studies in the literature that can accept multiple data inputs such as numeric, categorical, signal, video, image, and text data can model mixed data inputs end-to-end and evaluate the model’s performance using multiple data inputs have been searched. We reviewed the literature based on medical data sets, primarily neurological disorders studies, under two criteria: 1—applied singular modeling to data from multisource information, 2—applied multimodal decision-based modeling to data from multisource information. This study is organized as follows: data processing steps and application of feature engineering steps are in Sect. 2, machine learning methods are described in Sect. 3, model evaluation methods are in Sect. 4, and multimodal decision-based data fusion in Sect. 5 and literature is given singular model and multimodal in Sect. 6 and Sect. 7. Lastly, Sect. 8 is about discussion and conclusion.

2 Data Processing Steps The first step in data processing is data collection. It is the systematic gathering quality data to solve a problem in a particular field. Thus, it provides accurate and reliable results to the relevant field questions by making statistical analyses of the collected quality data [3]. In the data collected by the data collection process, if the data does not have the desired characteristics, the data must be cleaned. There are various techniques for data cleaning. Records with missing values are removed from the data set, or a generally constant value is used instead. This value is assigned for the missing data by taking the numeric values in the column containing the relevant data variable. Instead of all the numerical values of the relevant data variable, the average value of

Methods for the Recognition of Multisource Data in Intelligent …

3

Fig. 1 Processing steps of data

the sample data belonging to a class can be calculated and assigned instead of the missing value. Regression estimation can be made for missing data [4]. Transformation (normalization) reduces the data selected from the database to ranges such as 0–1. There are min–max normalization, zero-mean normalization, and decimal normalization methods for normalization operations. Feature engineering is the creation of new feature sets from raw data to increase the predictive power of the learning method. The selection of significant features in the data obtained from different sources affects the classification result [5]. Data preparation is a laborious process. According to a survey conducted in Forbes on this subject, data preparation takes 79% of data scientists’ time [6]. The processing steps can be seen in Fig. 1. Feature engineering steps are required to prepare the appropriate input data set compatible with machine learning method requirements and improve the performance of machine learning models. Imputation, use of outliers, division, log transform, one-hot encoding, grouping operations, feature splitting, and scaling are techniques used in feature engineering. Implementation steps for feature engineering are shown in Fig. 2. According to Fig. 2, the raw data obtained from different sources needs to be cleaned and converted to be suitable for modeling. After these processes, the features used in the modeling are selected, and the classification process is performed [7].

3 Machine Learning Methods Supervised learning is created with training and test data sets. Each record in the training data set has properties and class information. In supervised learning, a model is created with information about which class the data belongs to. The data that is not included in the training data set is modeled as a test data set, and the success of

4

S. Bayrak and E. Yucel

Fig. 2 Feature engineering application steps

the model is evaluated according to the test data set. The mathematical equation of the instructional learning process is as in (1). D = {x1 , x2 , . . . , xn }; Y = {y1 , y2 , . . . , ym }

(1)

According to (1), D represents a database and xi record data in the database, Y represents m class set. Thus, f : D → Y and each xi must belong to a class. According to (2), each Y j is a separate class and each class contains its data records [8, 9].   Y j = xi | f (xi ) = Y j , 1 ≤ i ≤ n, 1 ≤ j ≤ m ve xi ∈ D

(2)

3.1 ID3 The ID3 method uses the concept of entropy in (3) to find the most distinctive feature in the data set. Entropy is the maximum value of anticipation and takes values [10] between [0–1]. H ( p1 , p2 , . . . , pn ) =



( pi log(1/ pi ))

(3)

According to (3), while < p1 , p2 , . . . , pn > expresses probabilities, the sum of probabilities is 1. The ID3 method uses the difference value of the data set before and after the split to make the correct classification. Priority node and branching are determined according to the calculated large value. This difference is the gain (S) and is calculated according to (4). Gain(D; S) = H (D) −

n  i=1

P(Di )H (Di )

(4)

Methods for the Recognition of Multisource Data in Intelligent …

5

3.2 Linear Regression It is a statistical method that estimates the functional information y of the linear relationship between two dependent x variables [11–13]. With this method, when the value of even one of the x variables is known, the estimation can be made for other x variables. The curve is fitted to calculate the trend of the relationship between the variables. The least-squares method is used for the curve to fit the trend among the data. The least-squares method is expressed by the line with the slightest error for each data point. The sum of the squared differences between the output and target values is calculated by (5). According to (5), error (cost) values with E, output values with yi , and weight values with 0 , 1 are calculated. E(0 , 1 ) =

N 1  (h i (x) − yi )2 2N i=1

(5)

The error value is expected to be minimum for each problem modeled by the linear regression method. It iteratively converges toward the local minimum of the function with the gradient reduction method. In (6), the first-order derivative is calculated, and the gradient reduction method is used to calculate the local minimum of the function. y = x 1 w1 + x 2 w2 + . . . + x N w N + b

(6)

3.3 Naive Bayesian It is a statistical method that calculates the probability of new data with an unknown class entering any of the existing classes. In a two-class classification problem, Bayes’ theorem is calculated as in (7) [14, 15]. P(C1 |xi ) =

P(xi |C1 )P(C1 ) P(xi |C1 )P(C1 ) + P(xi |C2 )P(C2 )

(7)

According to (7), C1 and C2 represent two separate classes. P(C1 |xi ) calculates the probability that the value xi belongs to class C1 . P(xi ) is the number of times that xi is found in the database. The values of P(C1 ) and P(C2 ) are the number of times the classes C1 and C2 in the database. For a problem with m classes, Bayes’ theorem is as in (8). m      P(xi ) = P xi |C j P C j j=1

(8)

6

S. Bayrak and E. Yucel

3.4 Support Vector Machines It is the classification of data based on estimating the most appropriate function with the help of a linear or nonlinear function. For nonlinear problems, support vector machines (SVM) calculations are made with kernel functions. Since our study is a two-class classification problem, mathematical operations are given according to 2-class classification. D data set with n elements is (x1 , y1 ), (x2 , y2 ), . . . , (xn , yn ) and y ∈ {+1, −1}. According to Fig. 3, it is necessary to choose the one with the largest space between hyperplanes for the classification of data [16, 17]. According to Fig. 4, the median value of the H1 and H2 hyperplanes, H0 is the linear hyperplane separating the two classes as the optimal separation hyperplane, and it can be expressed as in (9). H0 =

n  i=1

Fig. 3 Linearly separable data set in two-dimensional space

Fig. 4 Largest possible gap between linearly separable classes

wi xi + b = 0

(9)

Methods for the Recognition of Multisource Data in Intelligent …

7

3.5 K-NN According to the distance calculation, it is a classifier that finds which class a new data belongs to by using a database with certain classes. Selection can be made for k data with the smallest Euclidean distance. Euclidean distance equation is expressed according to (10) [18].   n   2 xik − x jk d(i, j) =

(10)

k=1

In the method, k nearest neighborhood parameter is determined beforehand. The distance of the data whose class is desired to all the data is calculated and sorted. The smallest k pieces of data are selected. The most frequently repeated category value is calculated as the class of new data [19, 20].

3.6 Artificial Neural Network Information and signals are transmitted by axon connections between neurons [21– 24]. The artificial neural network (ANN) method was developed by being inspired by the neurological structure of the human brain. Neurons of the human brain can send signals through the connections between neurons. ANN creates its own rules by learning from a given an example. Learning is realized with a learning rule that changes or adjusts the connection weights of the network depending on the input samples or the outputs of these inputs. The network is trained according to the samples by changing the weight component values and producing the appropriate response when compared with any sample again. It is essential to use a good model in learning and change the weights according to this model. Changing these weights is repeated until a certain error rate. Finally, the weight values are recorded for use in testing. The multilayer perceptron is a forward propagation ANN system in which more than one layer is used between the input and output layers. In these intermediate layers, called the hidden layer, there are processing elements whose nodes are not directly connected to the input and output layers. When n-dimensional input samples are entered here; if xi = [x1 , x2 , . . . , xn ]T similarly desired m–dimensional output samples specify as dk = [d1 , d2 , . . . , dm ]T . If the xi values are the output values of the neurons in the i layer, the total input value to a neuron in the j layer is calculated as in (11). net j =

n  i=1

w ji xi

(11)

8

S. Bayrak and E. Yucel

Transfer function output value of neuron j in the hidden layer is calculated as in (12).   y j = f j net j

(12)

The total input to the k neuron in the output layer is calculated as in (13). netk =

J 

wk j y j

(13)

j=1

The nonlinear output of a k neuron in the output layer is calculated as in (14). ok = f k (netk ); k = 1, 2, . . . , m

(14)

The output value obtained from the network is compared with the actual output value, and the ek error is calculated as in (15). ek = (dk − ok )

(15)

dk and ok are the target value of any k neuron in the output layer and the actual output values obtained from the network, respectively. For each sample data, the total squared error is calculated as in (16). E=

1 (dk − ok )2 2 k

(16)

3.7 Long Short-Term Memory The ANN method calculates the gradients of the error value calculated from the network and then updates the weights. In a backpropagation ANN network, from the output layer to the input layer, there may be no problem updating the weights in a simple neural network, but problems called gradient loss or gradient burst may be encountered [25–28]. Back to the gradient values, the values may get exponentially smaller, causing the gradient loss problem or exponentially more prominent, causing the exploding gradient problem. One solution is to apply the smoothed linear unit as an activation function to avoid this problem. There are time steps in the long shortterm memory method (LSTM) network, but for each step, there is extra information called “memory” in the LSTM network [28]. Therefore, in a neural network with LSTM cell sigmoid function, “forget gate (f ),” “input gate (I),” and “output gate (O),” in a neural network with tanh function, “candidate layer (C),” a vector “hidden

Methods for the Recognition of Multisource Data in Intelligent …

9

Fig. 5 General structure of the LSTM network in time step t

state (H)” consists of “memory state (C)” components, which is a vector. Figure 5 shows the LSTM network diagram at time step t.

4 Model Evaluation It compares of classification accuracies between models by developing different models to determine the most suitable classification model for the data set.

4.1 Creating Confusion Matrix This section compares of the class of the actual data with the predicted class values in classification operations. In the confusion matrix of the two-class data set, the rows in Table 1 represent the class labels for the actual values, and the columns represent the predicted class labels [29]. According to Table 1, in a two-class model as patient and healthy; Table 1 Confusion matrix

Predicted value: patient

Predicted value: healthy

Real class label: patient

TP

FP

Real class label: healthy

FN

TN

10

S. Bayrak and E. Yucel

TP prediction value indicates the number of successful predictions for the patient class (true positive). FP prediction value indicates the number of failed predictions for the patient class (false positive). FN predictive value represents the number of unsuccessful predictions for the healthy class (false negative). TN predictive value represents the number of successful predictions for the healthy class (true negative).

4.2 Establishing Accuracy Criteria for Two-Class Modeling In two-class classification models, accuracy, sensitivity, specificity, and precision, F1 measure criteria are used for modeling performance [30]. These criteria are calculated according to (17–21). TP + TN TP + TN + FP + FN

(17)

Sensitivity =

TP TP + FN

(18)

Specificity =

TN TN + FP

(19)

Precision =

TP TP + FP

(20)

2TP 2TP + FP + FN

(21)

Accuracy =

F1 =

4.3 n-Fold Cross Validation The data set is divided into n equal subsets and n − 1 sets are reserved for training and 1 for testing. As shown in Table 2, this process is repeated n times, and the average accuracy value obtained each time is calculated as the accuracy value of the relevant classification model [31].

Methods for the Recognition of Multisource Data in Intelligent …

11

Table 2 n—validation by fold cross method (n = 10) 1. step

1. step

2. step

3. step

4. step

5. step

6. step

7. step

8. step

9. step

10. step

Test

Train

Train

Train

Train

Train

Train

Train

Train

Train

2. step

Train

Test

Train

Train

Train

Train

Train

Train

Train

Train

3. step

Train

Train

Test

Train

Train

Train

Train

Train

Train

Train Train

4. step

Train

Train

Train

Test

Train

Train

Train

Train

Train

5. step

Train

Train

Train

Train

Test

Train

Train

Train

Train

Train

6. step

Train

Train

Train

Train

Train

Test

Train

Train

Train

Train Train

7. step

Train

Train

Train

Train

Train

Train

Test

Train

Train

8. step

Train

Train

Train

Train

Train

Train

Train

Test

Train

Train

9. step

Train

Train

Train

Train

Train

Train

Train

Train

Test

Train

10. step

Train

Train

Train

Train

Train

Train

Train

Train

Train

Test

4.4 Receiver Operating Characteristic In the sensitivity of a classification model, the false-negative value must be reduced to zero. The receiver operating characteristic (ROC) analysis is the overall success curve of the classification model that can be obtained with different sensitivity— false positive ratio value pairs. The area under the curve shows the accuracy of the classification model. The area under the curve of AUC, TP, and FP is calculated according to (22) [32].



 TP TN d 1− AUC = ∫ TN + FP 0 TP + FN 1

(22)

4.5 Kappa Coefficient Pr(a) and Pr(e) is a statistical method that calculates the agreement between two different observations [33]. Pr(a) is the ratio of the observed fits to the total value for the two observation values. Pr(e) is the probability that the expected value will occur by chance. Kappa value is calculated according to (23). Kappa =

Pr(a) − Pr(e) 1 − Pr(e)

(23)

The Kappa value can take a value between -1 and 1. The agreement between the two observations is evaluated as follows [34, 35]. If the Kappa value is