Video Data Analytics for Smart City Applications: Methods and Trends 9815123718, 9789815123715

Table of contents :
Cover
Title
Copyright
End User License Agreement
Contents
Foreword
Preface
List of Contributors
Comprehensive Analysis of Video Surveillance System and Applications
Nand Kishore Sharma1,*, Surendra Rahamatkar1 and Abhishek Singh Rathore2
INTRODUCTION
LITERATURE REVIEW
OPEN CHALLENGES & ISSUES
DISCUSSION, TRENDS, AND FUTURE WORK
CONCLUSION
CONSENT FOR PUBLICATION
CONFLICT OF INTEREST
ACKNOWLEDGEMENT
REFERENCES
Compressed Video-Based Classification for Efficient Video Analytics
Sangeeta1,*, Preeti Gulia1 and Nasib Singh Gill1
INTRODUCTION
RELATED WORK
PROPOSED MODEL
COMPRESSION NETWORK
CLASSIFICATION NETWORK
EXPERIMENTS
Experimental Setup
Dataset
Implementation Details
Loss Function
Evaluation
EXPERIMENTAL RESULTS AND DISCUSSION
Experimental Results and Analysis of Compression Network
Experimental Results and Analysis of Classification Network
CONCLUSION
CONSENT FOR PUBLICATION
CONFLICT OF INTEREST
ACKNOWLEDGEMENT
REFERENCES
Object Detection and Tracking: Exploring a Deep Learning Approach and Other Techniques
Samuel Oluyemi Owoeye1,*, Folasade Durodola1 and Jethro Odeyemi1
INTRODUCTION
OBJECT DETECTION AND TRACKING WITH CONVOLUTIONAL NEURAL NETWORK (CNN) AND IMAGE PROCESSING
EXAMPLE OF CNN MODEL ARCHITECTURES FOR OBJECT DETECTION
R-CNN
Limitations of R-CNN
Fast R-CNN
Advantages of Fast R-CNN
Faster R-CNN
Mask R-CNN
R-FCN
Single Shot Detector (SSD)
Advantages of SSD
You Only Look Once (YOLO)
DEVELOPMENT OF THE NEURAL NETWORK ARCHITECTURE
Hyperparameter Values For Each Model Layer
First Convolutional Layer
Second, Third, Fourth, and Fifth Convolutional Layers
OBJECT DETECTION WITH HAAR CASCADE CLASSIFIER
Haar Features Calculation
Integral Images
Adaboost Training
Haar Cascade Classifier Experiments for Face and Eyes Detection
CANNY EDGE DETECTION
Noise Reduction
Gradient Calculation
Non-Maximum Suppression
Double Threshold
Edge Tracking by Hysteresis
BACKGROUND REMOVAL
Background Generation
Background Modeling
Background Model Update
Foreground Detection
MOTION DETECTION USING KNN BACKGROUND SUBTRACTION
CONCLUSION
CONSENT FOR PUBLICATION
CONFLICT OF INTEREST
ACKNOWLEDGEMENT
REFERENCES
Introduction and Overview of Key Enabling Technologies for Smart Cities and Homes
Karthika Dhanasekar1,* and Deepika Muni Krishna Reddy2
INTRODUCTION
TRENDS IN SMART CITIES AND HOMES
Smart cities
Smart Mobility and Smart Traffic Management
Smart Environment
Smart Living
Smart Economy
Smart Governance
Smart People
Smart Homes
Programmable and Zone-based Smart Thermostat
Wireless Power
Automatic Door Locks
Advanced Security System
CHALLENGES IN SMART CITIES AND HOMES
Security
IoT Challenges
Fragmentation of Standards
Processing big data
Scalability
SURVEY OF MAJOR KEY ENABLING TECHNOLOGIES FOR SMART CITIES AND HOMES
Internet of Things
Smart Dust
Smartphones
Cloud Computing
Smart Grid
SMART CITY DATA PLANE CHALLENGES
Compatibility Between Smart City Devices
Simplicity
Mobility and Geographic Control
SOFTWARE-DEFINED NETWORK-BASED SMART CITY NETWORK MANAGEMENT
Communications in Smart Grids
Purpose of the Smart Grid
THE DIFFERENT SEGMENTS OF THE SMART GRID
CONCLUSION
CONSENT FOR PUBLICATION
CONFLICT OF INTEREST
ACKNOWLEDGEMENT
REFERENCES
Intelligent Processing: Scope and Application of Smart Agriculture in Smart City
Geetanjli Khambra1,2,*, Sini Shibu1, Archana Naik1 and Dileep Singh2
INTRODUCTION
INTERNET OF THINGS (IOT)
INTELLIGENT PROCESSING
Intelligent Processing in Agriculture
INTELLIGENT PROCESSING MODEL
App Development
App Modules
Integration with sensors
Soil Moisture Sensor
Humidity and Temperature Sensor
Wind velocity Sensor
ADVANTAGES OF THE MODEL
CONCLUSION
CONSENT OF PUBLICATION
CONFLICT OF INTEREST
ACKNOWLEDGEMENTS
REFERENCES
Challenges and Security in Terahertz Band for Wireless Communication
Kannadhasan Suriyan1,* and Nagarajan Ramalingam2
INTRODUCTION
CHALLENGES IN TERAHERTZ BAND
TERAHERTZ RADIATION SOURCES
TERAHERTZ COMMUNICATION SYSTEMS
CONCLUSION
CONSENT FOR PUBLICATION
CONFLICT OF INTEREST
ACKNOWLEDGEMENT
REFERENCES
Empirical Impact of AI and IoT on Performance Management: A Review
Shahnawaz Ahmad1,*, Shabana Mehfuz2 and Javed Beg2
INTRODUCTION
Contribution of this Study
LITERATURE REVIEW
Industry 4.0 Revolution Through AI and IoT
AI Intersects with IoT
Future Potential Growth of Performance Management Through AI and IoT
Theoretical Underpinning
RESEARCH METHODS
RESULT AND DISCUSSION
Systematic Review
Thematic Review
DISCUSSION
CONCLUSION AND FUTURE SCOPE
CONSENT FOR PUBLICATION
CONFLICT OF INTEREST
ACKNOWLEDGEMENT
REFERENCES
A Review of Progress Status in Achieving the Jal Jeevan Mission Goals in the State of Chhattisgarh
Surendra Rahamatkar1,*, Sweta Patnaik2 and Satyendra Patnaik3
INTRODUCTION
BACKGROUND OF DRINKING WATER POLICIES THROUGHOUT HISTORY
JAL JEEVAN MISSION
POLICY GUIDELINES AT CENTRE
DECENTRALISED MANAGEMENT OF WATER IN CHHATTISGARH
OBJECTIVES OF THE STUDY
RESEARCH METHODOLOGY
RESULTS AND DISCUSSION
Capacity Monitoring of the Project
Functional Household Tap Connections (FHTC)
Surveillance and Monitoring of Water Quality
Institutional Mechanism
NATIONAL LEVEL - NATIONAL JAL JEEVAN MISSION (NJJM)
State Level - State Water and Sanitation Mission (SWSM)
District Level - District Water and Sanitation Mission (DWSM)
Village Level - Gram Panchayat/Village Water & Sanitation Committee/ Pani Samiti
CONCLUSION
AUTHOR'S CONTRIBUTION
FUNDING
INSTITUTIONAL REVIEW BOARD STATEMENT
DATA AVAILABILITY STATEMENT
CONSENT FOR PUBLICATION
CONFLICT OF INTEREST
ACKNOWLEDGEMENT
REFERENCES
Subject Index
Back Cover

Citation preview

IoT and Big Data Analytics (Volume 1) Video Data Analytics for Smart City Applications: Methods and Trends Edited by Abhishek Singh Rathore

Department of Computer Science & Engineering, Faculty of Science, Shri Vaishnav Vidyapeeth Vishwavidyalaya, Indore, India

Surendra Rahamatkar

Faculty of Engineering & Technology, Faculty of Information Technology, Amity School of Engineering & Technology, IQAC Amity University- Chhattisgarh, Raipur, India

Syed Imran Ali

Engineering Department, Computer Engineering, University of Technology and Applied Sciences, Al Musannah, Sultanate of Oman

Ramgopal Kashyap

Department of Computer Science & Engineering, Amity University Chhattisgarh, Raipur, India

& Nand Kishore Sharma

Department of Computer Science & Engineering, Pranveer Singh Institute of Technology, Kanpur, (U.P.), India

IoT and Big Data Analytics (Volume 1) Video Data Analytics for Smart City Applications: Methods and Trends Editors: Abhishek Singh Rathore, Surendra Rahamatkar, Syed Imran Ali, Ramgopal Kashyap and Nand Kishore Sharma ISBN (Online): 978-981-5123-70-8 ISBN (Print): 978-981-5123-71-5 ISBN (Paperback): 978-981-5123-72-2 ©2023, Bentham Books imprint. Published by Bentham Science Publishers Pte. Ltd. Singapore. All Rights Reserved. First published in 2023.

BSP-EB-PRO-9789815123708-TP-134-TC-08-PD-20230420

BENTHAM SCIENCE PUBLISHERS LTD.

End User License Agreement (for non-institutional, personal use) This is an agreement between you and Bentham Science Publishers Ltd. Please read this License Agreement carefully before using the book/echapter/ejournal (“Work”). Your use of the Work constitutes your agreement to the terms and conditions set forth in this License Agreement. If you do not agree to these terms and conditions then you should not use the Work. Bentham Science Publishers agrees to grant you a non-exclusive, non-transferable limited license to use the Work subject to and in accordance with the following terms and conditions. This License Agreement is for non-library, personal use only. For a library / institutional / multi user license in respect of the Work, please contact: [email protected].

Usage Rules: 1. All rights reserved: The Work is the subject of copyright and Bentham Science Publishers either owns the Work (and the copyright in it) or is licensed to distribute the Work. You shall not copy, reproduce, modify, remove, delete, augment, add to, publish, transmit, sell, resell, create derivative works from, or in any way exploit the Work or make the Work available for others to do any of the same, in any form or by any means, in whole or in part, in each case without the prior written permission of Bentham Science Publishers, unless stated otherwise in this License Agreement. 2. You may download a copy of the Work on one occasion to one personal computer (including tablet, laptop, desktop, or other such devices). You may make one back-up copy of the Work to avoid losing it. 3. The unauthorised use or distribution of copyrighted or other proprietary content is illegal and could subject you to liability for substantial money damages. You will be liable for any damage resulting from your misuse of the Work or any violation of this License Agreement, including any infringement by you of copyrights or proprietary rights.

Disclaimer: Bentham Science Publishers does not guarantee that the information in the Work is error-free, or warrant that it will meet your requirements or that access to the Work will be uninterrupted or error-free. The Work is provided "as is" without warranty of any kind, either express or implied or statutory, including, without limitation, implied warranties of merchantability and fitness for a particular purpose. The entire risk as to the results and performance of the Work is assumed by you. No responsibility is assumed by Bentham Science Publishers, its staff, editors and/or authors for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products instruction, advertisements or ideas contained in the Work.

Limitation of Liability: In no event will Bentham Science Publishers, its staff, editors and/or authors, be liable for any damages, including, without limitation, special, incidental and/or consequential damages and/or damages for lost data and/or profits arising out of (whether directly or indirectly) the use or inability to use the Work. The entire liability of Bentham Science Publishers shall be limited to the amount actually paid by you for the Work.

General: 1. Any dispute or claim arising out of or in connection with this License Agreement or the Work (including non-contractual disputes or claims) will be governed by and construed in accordance with the laws of Singapore. Each party agrees that the courts of the state of Singapore shall have exclusive jurisdiction to settle any dispute or claim arising out of or in connection with this License Agreement or the Work (including non-contractual disputes or claims). 2. Your rights under this License Agreement will automatically terminate without notice and without the

need for a court order if at any point you breach any terms of this License Agreement. In no event will any delay or failure by Bentham Science Publishers in enforcing your compliance with this License Agreement constitute a waiver of any of its rights. 3. You acknowledge that you have read this License Agreement, and agree to be bound by its terms and conditions. To the extent that any other terms and conditions presented on any website of Bentham Science Publishers conflict with, or are inconsistent with, the terms and conditions set out in this License Agreement, you acknowledge that the terms and conditions set out in this License Agreement shall prevail. Bentham Science Publishers Pte. Ltd. 80 Robinson Road #02-00 Singapore 068898 Singapore Email: [email protected]

BSP-EB-PRO-9789815123708-TP-134-TC-08-PD-20230420

CONTENTS FOREWORD ........................................................................................................................................... i PREFACE ................................................................................................................................................ ii LIST OF CONTRIBUTORS .................................................................................................................. iv CHAPTER 1 COMPREHENSIVE ANALYSIS OF VIDEO SURVEILLANCE SYSTEM AND APPLICATIONS ..................................................................................................................................... Nand Kishore Sharma, Surendra Rahamatkar and Abhishek Singh Rathore INTRODUCTION .......................................................................................................................... LITERATURE REVIEW .............................................................................................................. OPEN CHALLENGES & ISSUES ............................................................................................... DISCUSSION, TRENDS, AND FUTURE WORK ..................................................................... CONCLUSION ............................................................................................................................... CONSENT FOR PUBLICATION ................................................................................................ CONFLICT OF INTEREST ......................................................................................................... ACKNOWLEDGEMENT ............................................................................................................. REFERENCES ............................................................................................................................... CHAPTER 2 COMPRESSED VIDEO-BASED CLASSIFICATION FOR EFFICIENT VIDEO ANALYTICS ............................................................................................................................................ Sangeeta, Preeti Gulia and Nasib Singh Gill INTRODUCTION .......................................................................................................................... RELATED WORK ......................................................................................................................... PROPOSED MODEL .................................................................................................................... COMPRESSION NETWORK ...................................................................................................... CLASSIFICATION NETWORK .................................................................................................. EXPERIMENTS ............................................................................................................................. Experimental Setup ................................................................................................................. Dataset .......................................................................................................................... Implementation Details ................................................................................................. Loss Function ................................................................................................................ Evaluation ..................................................................................................................... EXPERIMENTAL RESULTS AND DISCUSSION ................................................................... Experimental Results and Analysis of Compression Network ............................................... Experimental Results and Analysis of Classification Network .............................................. CONCLUSION ............................................................................................................................... CONSENT FOR PUBLICATION ................................................................................................ CONFLICT OF INTEREST ......................................................................................................... ACKNOWLEDGEMENT ............................................................................................................. REFERENCES ............................................................................................................................... CHAPTER 3 OBJECT DETECTION AND TRACKING: EXPLORING A DEEP LEARNING APPROACH AND OTHER TECHNIQUES ........................................................................................ Samuel Oluyemi Owoeye, Folasade Durodola and Jethro Odeyemi INTRODUCTION .......................................................................................................................... OBJECT DETECTION AND TRACKING WITH CONVOLUTIONAL NEURAL NETWORK (CNN) AND IMAGE PROCESSING ..................................................................... EXAMPLE OF CNN MODEL ARCHITECTURES FOR OBJECT DETECTION ............... R-CNN .................................................................................................................................... Limitations of R-CNN .................................................................................................... Fast R-CNN ............................................................................................................................

1 2 5 12 12 13 14 14 14 14 18 19 21 22 23 27 29 29 29 29 29 30 30 30 31 34 34 34 34 34 37 37 39 40 40 40 40

Advantages of Fast R-CNN ........................................................................................... Faster R-CNN ......................................................................................................................... Mask R-CNN .......................................................................................................................... R-FCN .............................................................................................................................................. Single Shot Detector (SSD) .................................................................................................... Advantages of SSD ........................................................................................................ You Only Look Once (YOLO) ............................................................................................... DEVELOPMENT OF THE NEURAL NETWORK ARCHITECTURE ................................. Hyperparameter Values For Each Model Layer ..................................................................... First Convolutional Layer ............................................................................................. Second, Third, Fourth, and Fifth Convolutional Layers ............................................... OBJECT DETECTION WITH HAAR CASCADE CLASSIFIER ........................................... Haar Features Calculation ....................................................................................................... Integral Images ........................................................................................................................ Adaboost Training .................................................................................................................. Haar Cascade Classifier Experiments for Face and Eyes Detection ....................................... CANNY EDGE DETECTION ....................................................................................................... Noise Reduction ...................................................................................................................... Gradient Calculation ............................................................................................................... Non-Maximum Suppression ................................................................................................... Double Threshold .................................................................................................................... Edge Tracking by Hysteresis .................................................................................................. BACKGROUND REMOVAL ....................................................................................................... Background Generation .......................................................................................................... Background Modeling ............................................................................................................ Background Model Update ..................................................................................................... Foreground Detection ............................................................................................................. MOTION DETECTION USING KNN BACKGROUND SUBTRACTION ............................ CONCLUSION ............................................................................................................................... CONSENT FOR PUBLICATION ................................................................................................ CONFLICT OF INTEREST ......................................................................................................... ACKNOWLEDGEMENT ............................................................................................................. REFERENCES ............................................................................................................................... CHAPTER 4 INTRODUCTION AND OVERVIEW OF KEY ENABLING TECHNOLOGIES FOR SMART CITIES AND HOMES ................................................................................................... Karthika Dhanasekar and Deepika Muni Krishna Reddy INTRODUCTION .......................................................................................................................... TRENDS IN SMART CITIES AND HOMES ............................................................................. Smart cities .............................................................................................................................. Smart Mobility and Smart Traffic Management ........................................................... Smart Environment ........................................................................................................ Smart Living .................................................................................................................. Smart Economy ............................................................................................................. Smart Governance ......................................................................................................... Smart People ................................................................................................................. Smart Homes ........................................................................................................................... Programmable and Zone-based Smart Thermostat ...................................................... Wireless Power .............................................................................................................. Automatic Door Locks ................................................................................................... Advanced Security System .............................................................................................

41 41 41 42 42 42 43 43 44 44 44 46 47 47 47 48 48 48 49 49 49 49 50 51 51 51 51 51 51 52 52 52 52 54 54 56 56 57 57 58 58 58 59 59 60 60 61 61

CHALLENGES IN SMART CITIES AND HOMES .................................................................. Security ................................................................................................................................... IoT Challenges ........................................................................................................................ Fragmentation of Standards .................................................................................................... Processing big data ................................................................................................................. Scalability ............................................................................................................................... SURVEY OF MAJOR KEY ENABLING TECHNOLOGIES FOR SMART CITIES AND HOMES ........................................................................................................................................... Internet of Things .................................................................................................................... Smart Dust .............................................................................................................................. Smartphones ............................................................................................................................ Cloud Computing .................................................................................................................... Smart Grid ............................................................................................................................... SMART CITY DATA PLANE CHALLENGES ......................................................................... Compatibility Between Smart City Devices ........................................................................... Simplicity ................................................................................................................................ Mobility and Geographic Control ........................................................................................... SOFTWARE-DEFINED NETWORK-BASED SMART CITY NETWORK MANAGEMENT ............................................................................................................................ Communications in Smart Grids ............................................................................................. Purpose of the Smart Grid ....................................................................................................... THE DIFFERENT SEGMENTS OF THE SMART GRID ........................................................ CONCLUSION ............................................................................................................................... CONSENT FOR PUBLICATION ................................................................................................ CONFLICT OF INTEREST ......................................................................................................... ACKNOWLEDGEMENT ............................................................................................................. REFERENCES ............................................................................................................................... CHAPTER 5 INTELLIGENT PROCESSING: SCOPE AND APPLICATION OF SMART AGRICULTURE IN SMART CITY ..................................................................................................... Geetanjli Khambra, Sini Shibu, Archana Naik and Dileep Singh INTRODUCTION .......................................................................................................................... INTERNET OF THINGS (IOT) ................................................................................................... INTELLIGENT PROCESSING ................................................................................................... Intelligent Processing in Agriculture ...................................................................................... INTELLIGENT PROCESSING MODEL ................................................................................... App Development ................................................................................................................... App Modules ........................................................................................................................... Integration with sensors .......................................................................................................... Soil Moisture Sensor ............................................................................................................... Humidity and Temperature Sensor ......................................................................................... Wind velocity Sensor .............................................................................................................. ADVANTAGES OF THE MODEL .............................................................................................. CONCLUSION ............................................................................................................................... CONSENT OF PUBLICATION ................................................................................................... CONFLICT OF INTEREST ......................................................................................................... ACKNOWLEDGEMENTS ........................................................................................................... REFERENCES ...............................................................................................................................

61 61 61 62 62 62 63 63 63 63 64 65 65 66 66 66 67 67 68 68 69 69 69 69 69 72 72 73 74 75 77 78 80 80 80 81 81 81 82 82 82 82 82

CHAPTER 6 CHALLENGES AND SECURITY IN TERAHERTZ BAND FOR WIRELESS COMMUNICATION .............................................................................................................................. 85 Kannadhasan Suriyan and Nagarajan Ramalingam

INTRODUCTION .......................................................................................................................... CHALLENGES IN TERAHERTZ BAND ................................................................................... TERAHERTZ RADIATION SOURCES ..................................................................................... TERAHERTZ COMMUNICATION SYSTEMS ........................................................................ CONCLUSION ............................................................................................................................... CONSENT FOR PUBLICATION ................................................................................................ CONFLICT OF INTEREST ......................................................................................................... ACKNOWLEDGEMENT ............................................................................................................. REFERENCES ............................................................................................................................... CHAPTER 7 EMPIRICAL IMPACT OF AI AND IOT ON PERFORMANCE MANAGEMENT: A REVIEW .............................................................................................................. Shahnawaz Ahmad, Shabana Mehfuz and Javed Beg INTRODUCTION ......................................................................................................................... Contribution of this Study ....................................................................................................... LITERATURE REVIEW .............................................................................................................. Industry 4.0 Revolution Through AI and IoT ......................................................................... AI Intersects with IoT ............................................................................................................. Future Potential Growth of Performance Management Through AI and IoT ........................ Theoretical Underpinning ....................................................................................................... RESEARCH METHODS ............................................................................................................... RESULT AND DISCUSSION ....................................................................................................... Systematic Review .................................................................................................................. Thematic Review .................................................................................................................... DISCUSSION .................................................................................................................................. CONCLUSION AND FUTURE SCOPE ...................................................................................... CONSENT FOR PUBLICATION ................................................................................................ CONFLICT OF INTEREST ......................................................................................................... ACKNOWLEDGEMENT ............................................................................................................. REFERENCES ............................................................................................................................... CHAPTER 8 A REVIEW OF PROGRESS STATUS IN ACHIEVING THE JAL JEEVAN MISSION GOALS IN THE STATE OF CHHATTISGARH ............................................................. Surendra Rahamatkar, Sweta Patnaik and Satyendra Patnaik INTRODUCTION .......................................................................................................................... BACKGROUND OF DRINKING WATER POLICIES THROUGHOUT HISTORY .......... JAL JEEVAN MISSION ............................................................................................................... POLICY GUIDELINES AT CENTRE ........................................................................................ DECENTRALISED MANAGEMENT OF WATER IN CHHATTISGARH .......................... OBJECTIVES OF THE STUDY .................................................................................................. RESEARCH METHODOLOGY .................................................................................................. RESULTS AND DISCUSSION ..................................................................................................... Capacity Monitoring of the Project ......................................................................................... Functional Household Tap Connections (FHTC) ................................................................... Surveillance and Monitoring of Water Quality ...................................................................... Institutional Mechanism .......................................................................................................... NATIONAL LEVEL - NATIONAL JAL JEEVAN MISSION (NJJM) ................................... State Level - State Water and Sanitation Mission (SWSM) ................................................... District Level - District Water and Sanitation Mission (DWSM) .......................................... Village Level - Gram Panchayat/Village Water & Sanitation Committee/ Pani Samiti ........ CONCLUSION ............................................................................................................................... AUTHOR'S CONTRIBUTION .....................................................................................................

85 87 90 94 96 96 96 96 96 99 100 102 103 103 104 105 105 106 107 107 107 108 108 108 108 109 109 112 113 115 116 117 118 119 120 121 121 121 124 125 125 126 126 126 127 128

FUNDING ........................................................................................................................................ INSTITUTIONAL REVIEW BOARD STATEMENT ............................................................... DATA AVAILABILITY STATEMENT ...................................................................................... CONSENT FOR PUBLICATION ................................................................................................ CONFLICT OF INTEREST ......................................................................................................... ACKNOWLEDGEMENT ............................................................................................................. REFERENCES ...............................................................................................................................

128 128 128 129 129 129 129

SUBJECT INDEX .................................................................................................................................... 130

i

FOREWORD Data is everything and nowadays, Big Data and Big Data Analytics are given more attention in the field of research. The smart city paradigm completely relies on data and smart frameworks. This framework comprehends the physical infrastructure, networking system, centralized computing center, data storage, and higher-level domain use-cases. The physical infrastructure requires sensors, devices, and cameras to capture and generate data. In the smart city, also known as an innovative city, the devices and sensors involved in smart city applications generate a massive amount of multi-media data especially Video Data. The involvement of multimedia sensors enables to obtain precise and concrete information. This book titled “IoT and Big Data Analytics” Vol. 1 (Video Data Analytics for Smart City Applications: Methods and Trends) aims to explore the various applications and use cases of Big Data & Analytics and processing for smart city applications. The technologies like Machine Learning, Deep Learning, IoT, WSN, IoT, and AI have been contributing to the application and efficient processing of data. Object detection and tracking, intelligent processing, video compression, the performance of IoT & AI in business growth, key enabling technologies and video analytics need to be explored more. The information processing is both challenging and interesting. The researchers are working with passion, and dedication to develop new methods and algorithms for information analysis and to provide a solution to remain in the new direction. The book will provide a valuable window and wideranging exposure to concerned technologies and their architectural frameworks for smart city applications. How these technologies are serving themselves to fulfilling the needs of a smart city is also discussed, for the research and industry personnel. This book is a good step in that direction and will surely help researchers to find relevant information in one place.

Vishal Nagar Department of Computer Science & Engineering, Pranveer Singh Institute of Technology, Kanpur, India

ii

PREFACE This book aims to explore the various applications and use cases of Video Data Analytics and processing for Smart City Applications. Everyday things are embedded together with the help of software, and sensors through the internet to collect and exchange data. Hence these devices generate a massive amount of data. The smart city paradigm entirely depends on data. In the context of video data, cameras capture huge data, and later these video recordings are effectively used in various smart city applications such as surveillance systems to counter potential threats. The generated data is heterogeneous and sparse in most cases. The processing of this Big Data in real-time is a matter of concern. With the participation of multimedia sensors, it is possible to obtain more precise and concrete information. Enormous challenges come in the data collection, analysis, and distribution.Various trending technologies like Machine Learning and Deep Learning are contributing to real-time data processing. Hence, these technologies should also be explored for efficient processing. The video analytics field is continuously in evolution, because of speedy hardware and software progress that is making the technology more reachable and valued. To transform a large amount of video into actionable intelligence as complementary mechanisms, an effective algorithm and model are required. The book will provide comprehensive coverage of the latest and trending technologies like machine learning, deep learning, blockchain, AI, etc. The book will highlight the advances and trends in various technologies by targeting the research and industry that are involved in video data analytics via addressing extensive themes. This book will surely demonstrate how technologies are serving their use-cases in the smart city. Interested researchers and academicians will get to know about the latest happenings and research, and more research possibilities can be explored. The industry professional may also benefit from this book by connecting the research with industry needs. This book will be proven as a useful resource for the target audience by outlining the promising future research and providing references to newcomers in the field of video data analytics.

Abhishek Singh Rathore Department of Computer Science & Engineering, Faculty of Science, Shri Vaishnav Vidyapeeth Vishwavidyalaya, Indore, India Surendra Rahamatkar Faculty of Engineering & Technology, Faculty of Information Technology, Amity School of Engineering & Technology, IQAC Amity University- Chhattisgarh, Raipur, India Syed Imran Ali Engineering Department, Computer Engineering, University of Technology and Applied Sciences, Al Musannah, Sultanate of Oman

iii

Ramgopal Kashyap Department of Computer Science & Engineering, Amity University Chhattisgarh, Raipur, India & Nand Kishore Sharma Department of Computer Science & Engineering, Pranveer Singh Institute of Technology, Kanpur, (U.P.), India

iv

List of Contributors Abhishek Singh Rathore

Shri Vaishnav Vidyapeeth Vishwavidyalaya, , Indore, India

Archana Naik

Department of Computer Applications, The Bhopal School of Social Sciences, Bhopal, M.P, India

Karthika Dhanasekar

Department of Computer Science, SDNB Vaishnav College for Women, Chennai,

Dileep Singh

School of Engineering and Technology, Jagran Lake City University, Bhopal, M.P, India

Folasade Durodola

Department of Mechatronics Engineering, Federal University of Agriculture, Abeokuta, Nigeria

Geetanjli Khambra

Department of Computer Applications, The Bhopal School of Social Sciences, Bhopal, M.P, India

Javed Beg

Oracle, Noida, India

Deepika Muni Krishna Reddy

Department of Computer Science, Crescent University, Chennai, India

Nand Kishore Sharma

Amity University Chhattisgarh, Raipur, India

Nasib Singh Gill

Department of Computer Science & Applications, Maharshi Dayanand University, Rohtak, India

Jethro Odeyemi

Department of Mechatronics Engineering, Federal University of Agriculture, Abeokuta, Nigeria

Preeti Gulia

Department of Computer Science & Applications, Maharshi Dayanand University, Rohtak, India

Nagarajan Ramalingam

Department of Electrical and Electronics Engineering, Gnanamani College of Technology, Tamilnadu, India

Kannadhasan Suriyan

Department of Electronics and Communication Engineering, Cheran College of Engineering, Tamilnadu, India

Samuel Oluyemi Owoeye

Department of Mechatronics Engineering, Federal University of Agriculture, Abeokuta, Nigeria

Sangeeta

Department of Computer Science & Applications, Maharshi Dayanand University, Rohtak, India

Satyendra Patnaik

Amity University Chhattisgarh, India

Shahnawaz Ahmad

Department of Electrical Engineering, Jamia Millia Islamia, New Delhi, India

Shabana Mehfuz

Oracle, Noida, India

Sini Shibu

Department of Computer Applications, The Bhopal School of Social Sciences, Bhopal, M.P, India

Surendra Rahamatkar

Amity School Engineering & Technology, Amity University Chhattisgarh, India

Sweta Patnaik

WASH Specialist, UNICEF, Chhattisgarh, India

IoT and Big Data Analytics, 2023, Vol. 1, 1-17

1

CHAPTER 1

Comprehensive Analysis of Video Surveillance System and Applications Nand Kishore Sharma1,*, Surendra Rahamatkar1 and Abhishek Singh Rathore2 1 2

Amity School of Engineering & Technology, Amity University Chhattisgarh, Raipur, India Shri Vaishnav Vidyapeeth Vishwavidyalaya, Indore, India Abstract: In this growing age of technology, various sensors are used to capture data from their nearby environments. The captured data is multimedia in nature. For example, CCTV cameras are used in those places where security matters or where continuous monitoring is required. Hence object detection, object recognition, and face recognition became key elements of city surveillance applications. Manual surveillance seems time-consuming and requires huge space to store the data; hence video surveillance has a significant contribution to unstructured big data. All surveillance techniques and approaches are based on Object Tracking, Target Tracking, Object Recognition, and Object Mobile Tracking Systems (OMTS). The main difficulty, however, lies in effectively processing them in real time. Therefore, finding a solution still needs careful consideration. This paper mainly targeting to the smart city surveillance system and inspects all existing surveillance systems based on various tremendous technologies like a wireless sensor network, machine learning, and Deep Learning. The author discovered the problems in the existing methods and summarized them in the paper. The motive is to point out the various challenges and offer new research prospects for the multimedia-oriented surveillance system over the traditional surveillance system for the smart city network architecture. The thorough survey in this paper starts with object recognition and goes toward action recognition, image annotation, and scene understanding. This comprehensive survey summarizes the comparative analysis of algorithms, models, and datasets in addition to targeting the methodologies.

Keywords: Deep learning, Face-recognition , Image annotation, Multimedia, Machine learning, OMTS, Object detection, Smart city, Video surveillance. Corresponding author Nand Kishore Sharma: Amity School of Engineering & Technology, Amity University Chhattisgarh, Raipur, India; E-mail: [email protected] *

Abhishek Singh Rathore, Surendra Rahamatkar, Syed Imran Ali, Ramgopal Kashyap & Nand Kishore Sharma (Eds.) All rights reserved-© 2023 Bentham Science Publishers

2 IoT and Big Data Analytics, Vol. 1

Sharma et al.

INTRODUCTION Surveillance may be outlined in a variety of ways, including vehicle monitoring at roadside traffic areas as an intelligent traffic management system, theft monitoring & identification, capturing abnormal happenings, monitoring of widely open critical areas, and crowd analysis. Apart from that, it is also important in the smart healthcare system to perform hospital surveillance, secure hospital facilities, detect patient emotion and sentiment, detect patient fraud, and analyze hospital traffic patterns. The video surveillance system has progressed. Now it is not only aiming to capture and show the video but has upgraded towards an autonomous and intelligent system. Only cutting-edge algorithms have made this possible. Because the purpose of all these algorithms was not only to classify images or videos but also to enhance them. Thus, a modern surveillance system workflow is mentioned in Fig. (1). As a result, video surveillance makes a significant contribution to unstructured big data.

Fig. (1). Surveillance System workflow.

Intelligent surveillance is the main application of a Smart city, and the objective of the Smart city is to improve the quality of human lives. Smart environments contain sensors & devices that are network-connected and work together to perform operations. Though the last decade, the Internet of Things (IoT) with Machine Learning & Deep Learning has received so much attention. The cause for the accomplishment of this much attention is the services and capabilities offered by it. The IoT is an interconnection between everyday objects and computing machines. It enables them to communicate in many smart city applications, where smart surveillance is one of them. Intelligent Transport Management Systems, Intelligent Traffic Management systems, Vehicular Ad-Hoc Networks, and Carto-Car Communication are a few examples of IoT. Here, Intelligence indicates the best utilization of data. This data is generated by aggregating the knowledge and then converted into information through modeling. After that, this information is used for further processing.

Comprehensive Analysis

IoT and Big Data Analytics, Vol. 1 3

In reality, all of this generated data is also known as multimedia data, such as audio and video. Their combination also makes the computation more energy efficient. Wireless Multimedia Sensors are used to collect multimedia data. However, Wireless Multimedia Sensors have exceptional issues such as high bandwidth and energy consumption. Other issues observed include quality of service (QoS), data processing, and compression at the node level. Object detection and recognition schemes have emerged in recent years as a solution for reducing the size of multimedia data at the node level. The strategy used in it is based on motion detection. The camera only starts recording when it detects motion; otherwise, it does not record. As a result, unnecessary recording and storage are avoided. As a result, no overhead exists. However, it is inefficient in some ways because it requires the user’s involvement to process forwarded data, so that alert decisions can be made. A simple object classification with few details can work. It is based on a genetic algorithm-based classifier. The classifier used only two features of the objects: 1. Video frames: specifically, the shape of the minimum bounding box, also known as video annotation of the object. 2. The frame rate of the observed region. This method was tested in a simulated environment on three types of objects: humans, animals, and vehicles. The observation was that as the audio was added, the noise count increased [1]. The evolution of IoT presents enormous challenges in data collection, data analysis, and distribution in the direction of more productive use of information [2]. According to the survey, video surveillance systems have advanced technically over three generations. The survey said that the video surveillance systems have technically progressed as: ●

●

The first generation was introduced in the 1960s with Analogue Close Circuit TV (CCTV). That was primarily for indoor surveillance applications. But the limitations were the recording and distribution process. Digital imaging began to expand surveillance systems in the 1980s. Advances in this area include compression and distribution as well as surveillance algorithms. The system included object tracking as well as an event alert system.

4 IoT and Big Data Analytics, Vol. 1 ●

Sharma et al.

The third-generation surveillance systems with fully automated and wide-area surveillance were investigated in the 2000s. The goal was to provide inference frameworks and behavioral analysis.

Systems are usually classified into three generations

There are several categorizations of video surveillance that can be drawn to fulfill the requirement. Systems are usually classified into three generations as shown in Fig. (2). And the below Table 1 is representing the same categorization.

The First CCTV based camera that only recorded video on tape Digital cameras with low-level image processing (like perimeter intrusion detection, abandoned object detection, etc.) with IP-based networks Multi-view intelligent surveillance systems with semantic information extraction

Fig. (2). Surveillance system generations. Table 1. The categorization of the Video Surveillance Systems. Criteria

Categories of System

Image gain obtained

• Number of cameras like one or more. • Cameras are fixed or movable.

Applications

• To track and recognize the objects. • Re-identification, and behavioral analysis.

Architecture

• Unaided systems, • Cloud-based and distributed system.

The systems try to emulate the method by which individuals recognize activities and classify them. For instance, backdrop or foreground classification is a typical event detection pre-processing technique. The technique tries to differentiate between static and dynamic foreground scenes. The development of a surveillance system is heavily influenced by the caliber of the acquired input. Some of the video sensor's key characteristics include resolution, frame per second (FPS), and contrast.

Comprehensive Analysis

IoT and Big Data Analytics, Vol. 1 5

LITERATURE REVIEW Low-cost, compact cameras and microphones are now being introduced by recent technological advancements. And research started to get more precise real-world information. The outcome was distributed Wireless Multimedia Sensor Network. Multimedia sensors make it simpler to gather more accurate and thorough data for a variety of smart city applications. A multilayer automatic surveillance system architecture is suggested in a study [3] for outdoor applications. The system has two layers at the node level. Scalar sensors having the ability to sense motion, vibration, and sound are present in the first layer. This layer activates the second layer to record audio and video. Scalar and multimedia sensors gather data, which is then analyzed and combined in three separate layers. In computer vision and robotics, automatic human action recognition in any video stream has been a highly addressed challenge. Majorly works were focused on classifying the segmented clips, joint detection, and action recognition. But none of them were dealing with wireless camera networks. To deal with this issue [4], an efficient system is presented by using a wireless smart camera network. The approach was based on Deformable Part Models (DPMs) for object detection in the images. Later, the framework was extended with tight integration inside a centralized video analysis system named Deformable Keyframe Model (DKM) from single-view and multi-view video settings to joint detection and action recognition. The DKM was validated on two different datasets- the publicly available dataset, and Bosch Multiview Complex Action (BMCA) dataset. In a study [5], a deep learning technique was introduced to predict the type of action in smart cities. Four different types of audio datasets were used: crowded city audio, home appliance sound, household item sound, and human action sound. The analysis of video surveillance was covered in a study [6]. The study classifies it into two categories, abnormal and normal, as well as object and action recognition. Deep learning architectures is the main topic of this survey. CNN, auto-encoders, and their combinations are the most often used deep learning models for surveillance analysis. The video data is increasing because of the many networked cameras located in public places around the world for security [7]. During surveillance, these public surveillance cameras generated large amounts of data in order to capture human behaviors. As a result, generating a large amount of data necessitated a large space or data warehouse, but storing this large amount of surveillance data for an extended period of time is difficult. It is preferable to have the results of the anal-

6 IoT and Big Data Analytics, Vol. 1

Sharma et al.

ysis rather than large amounts of video data. Thus, storage space could be reduced. Apart from deep learning, methods have also been identified. Methods are divided into two subgroups: deep learning-based and non-deep learning-based. The Surveillance Video Analysis System (SVAS) [8] is concerned with the automatic action recognition and deduction of its complexity. The event detection procedure is divided into low and high levels. The low-level analysis detects both people and objects, and the results are used in higher-level analysis to detect the event. The architecture proposed in a study [9] includes Event and Action model learning, Action detection, Complex event model learning, and Complex event detection. A hybrid event model is the Interval-Based Spatio-Temporal Model (IBSTM). In addition to these techniques, Markov logic networks, Bayesian networks, and threshold models are also used. To handle the dataset, which is generated from moving cameras and multi-camera, the SVAS approach can be modified. In particular, in areas like calibration and noise reduction, additional enrichments are preferred when handling complicated events. A rule-based system detects numerous abnormal activities in videos. Motion patterns are used to identify the features [10]. It has been stated that abnormal events are detected by the system’s training and following the dominant set property. External rules are required to recognize the object. In video surveillance, abnormal trajectory, and event detection work together. Existing methods are either trajectory-based or pixel-based. However, a proposal [11] encompasses both. The proposal includes object and group tracking, trajectory filtering, gridbased analysis, and abnormal behavior detection. A study [12] proposed AMDN for self-learning feature representations. Using stacked denoising autoencoders, the model learns appearance features and motion features. Throughout learning, several one-class SVMs are trained. The anomaly score of each input is predicted by this SVM. These scores are then added together to detect the abnormal event. A structure based on autoencoders and CNN was proposed [13]. An autoencoder is made up of an encoder and a decoder. The architecture enables the use of low-level frames with high-level appearance and extracts the features from them. Reconstruction errors are used to represent anomaly scores. A study [14] examined fundamental deep learning techniques such as Convolutional Neural Network, Autoencoder, and Sparse Coding. In a study [15], 100 different approaches to deep learning were surveyed. The goal was to identify individuals using various biometric modalities. And the majority of the research was focused on biometrics. Five physiological biometrics (face,

Comprehensive Analysis

IoT and Big Data Analytics, Vol. 1 7

fingerprint, palm print, and iris) and four behavioral biometrics (voice, signature, gait, and keystroke) are included. Commonly used Datasets in various biometric modalities are: The Labeled Faces in the Wild (LFW) dataset is extensively used for face recognition. It consists of 13,323 images of 5,749 celebrities. All are taken in an unrestricted setting. The YouTube Faces (YTF) dataset, which contains 3,425 YouTube videos of 1,595 celebrities, is extensively utilized for face recognition. The videos are organized into 5,000 video pairs divided into ten splits. AR Face is a popular dataset for face recognition and facial attribute recognition. It includes 4,000 frontal face images of 126 people with various lighting, occlusions, and facial expressions. MORPH is primarily used to estimate face attributes. It includes two face albums. Each face has characteristics such as age, gender, and ethnicity. Album 1 contains 1,724 photographs of 515 people. Album 2 contains 55,134 photographs of 13,000 people. IJB-A contains 5,396 images and 20,412 video frames from 500 different people. Face recognition is the primary application for this dataset. The Adience dataset is used to estimate age and gender. It includes 26,580 unrestricted face images of 2,284 people. To calculate noticeable age, the ChaLearn 2015 dataset is used. There are 2,476 training images and 1,136 validation images in total. Fig. (3) represents the statistical summary of commonly used datasets in various biometric modalities. A study [16] proposed a deep bi-directional Long Short-Term Memory (BiLSTM) model. The model's goal was to provide an explanatory description for the image. The model was evaluated using the Flickr8K, Flickr30K, MSCOCO, and Pascal1K datasets. Another study [17] proposed an innovative Deep Multi-View Feature Learning method (DMVFL). The Cross-view Quadratic Discriminant Analysis (XQDA) metric was used to create it. A CNN-based method for classifying traffic conditions was developed [18]. To classify levels of traffic congestion in ITS, an optimal set of filters and a CNN configuration are proposed. Where the ITS is responsible for traffic management. It has been validated using traffic video data. The process is done with different camera angles and illumination.

8 IoT and Big Data Analytics, Vol. 1

Sharma et al.

Commonly used Datasets in various biometric modalities DATA VOLUME

60,000 50,000 40,000 30,000 20,000 10,000 0

Labeled Faces in the Wild (LFW) No of Data 13,323

YouTube Faces (YTF)

AR Face database

MORPH

IJB-A

Adience

ChaLearn 2015

3,425

4,000

56,858

25,808

26,580

3612

Fig. (3). Commonly used datasets in various biometric modalities.

For community safety, the surveillance necessitated the re-identification of people (re-id). Researchers are interested in the process of automatically identifying the individual concerned in the video. A study [19] used three public person re-id data sets - PRID-2011, iLIDS-VID, and MARS - to address the image to video person re-id. The probe is an image in this case, and the gallery is made up of videos captured by nonoverlapping cameras. A study [20] proposes an efficient re-id approach based on a highly discriminative hybrid person representation. The framework is tested using publicly available small-scale and large-scale person re-id datasets. The small scale dataset includes- datasets- VIPeR, PRID450s, and GRID. While, the large-scale person re-id datasets include- CUHK01, Market1501, and DukeMTMC-ReID. A study [21] proposes a new face verification algorithm that begins by selecting rich frames from a video by using discrete wavelet transform and entropy computation. The entire procedure is broken down into frame selection, feature extraction, and face verification. The procedure was tested on two well-known video benchmark datasets: YouTube Faces and the Point and Shoot Challenge. A study [22] addressed a situation, which was defined as a sequence of event grounds that can be enhanced with low-level object-based features and directional grounds. A research work [23] proposed an automatic and comprehensive system for integrated video object tracking for trajectory-based event identification. It

Comprehensive Analysis

IoT and Big Data Analytics, Vol. 1 9

classifies the trajectories of the tracked objects to provide event detection while reliably tracking numerous targets. Tracking applications include both human surveillance and traffic monitoring. Finding the video of the person who has been searched for is the goal of face video retrieval. But the issue is the intra-class variances in facial features. a revolutionary deep convolutional neural network has been introduced to learn discriminative binary hash videos for the same [24]. In addition to person reidentification and event identification, action recognition is adopting by many researchers over the past decades. The experiment carried out in a study [25] for action recognition showed that deep activation has obtained cutting-edge performance as feature vectors and attribute learners. The method was developed using multiple-stream deep neural networks. In addition, a joint semantic preserving action attribute learning framework has been implemented. The result was the generation of generic features to identify the actions from depth videos. The large-scale TRECVID MED 2014 video dataset has been used [26] to detect high-level events. A study [27] analyzed the scene using an unsupervised approach to find the anomaly in the video data. That footage of the traffic area was captured by static surveillance. One deep convolutional architecture was presented [28] in order to detect anomalous behavior of the people. The unified framework was used as the framework. The framework's goal was to increase detection speed without sacrificing recognition accuracy. In addition to activity recognition, a fascinating and popular area of research is the localization and classification of images. Fig. (4) represents the performance of various deep learning models for activity recognition. A comprehensive analysis [29] highlighted how deep learning was used in transportation applications to identify traffic signs. Furthermore, it predicted the traffic flow and speed, which helped to estimate the travel time. Additionally [30], the use of benchmarked datasets in deep learning for object detection has been discussed. The object detection datasets are summarized as follows: ●

●

●

Microsoft COCO (Microsoft Common Objects in Context) dataset contains 3,28,000 images, of which 2.5 million have a label. There are 91 different types of objects. ImageNet dataset contains 14,197,122 images in 480 X 410 resolution. The dataset is based on the WordNet hierarchy. Synset is another name for WordNet; each synset is defined by 1,000 images. The CIFAR-10 dataset includes 6,000 total images of 10 different types of

10 IoT and Big Data Analytics, Vol. 1

●

●

●

●

Sharma et al.

objects at a resolution of 32 x 32. Out of these 60,000 images, 50,000 are used for training and the remaining 10,000 are used for testing. The CIFAR-100 dataset contains 600 images in each of its 100 categories of objects. Out of the 600 images in each category, 500 are used for training and 100 are used for testing. CUB-200-2011 (Caltech-UCSD Birds-200-2011) has 200 categories of bird species. There are 11,788 images total, each with a single bounding box, 15-part locations, and 312 binary characteristics. Caltech-256 dataset consists of 256 categories of the object. There are a total of 30,607 images, with 80 images for each object type. Nevertheless, it is not advised for object localization. ILSVRC (ImageNet Large Scale Visual Recognition Challenge) has 200 object categories, which is ten times as many as in PASCAL VOC.

Performance of Deep Learning Models in Activity Recognition Hierarchical Dirichlet Process (HDP) Deformable Keyframe Model (DKM) Surveillance Video Analysis System Joint Semantic Preserving Action Attribute... Trajectory-based Methods Appearance and Motion DeepNet (AMDN) Gaussian Mixture Model (GMM) 76% 78% 80% 82% 84% 86% 88% 90% 92% 94% 96%

Accuracy

Gaussian Mixture Model (GMM)

Appearance and Motion DeepNet (AMDN)

Trajectorybased Methods

90%

92.42%

88.70%

Joint Semantic Preserving Action Attribute Learning Framework 87.88%

Surveillance Deformable Hierarchical Keyframe Dirichlet Video Analysis Model Process System (DKM) (HDP) 93.33%

82.00%

86.57%

Fig. (4). Performance Accuracy of Different Methods for Scene Understanding & Activity Recognition.

The Static Experienced Based Adaptive One-Shot Network (SENet) was put up as a solution to the issue of generating a single sample and costly dataset for object detection [31]. Deep neural networks and cluster analysis were the foundations of the network. The human mobility prediction model is an important component in the development of a smart city [32]. Human mobility can be routine and irregular. Routine human mobility consumes a sizable portion. Sometimes it becomes a challenge to manage them effectively. Therefore, to make an accurate prediction,

Comprehensive Analysis

IoT and Big Data Analytics, Vol. 1 11

the system will work the majority of the time. Annotations then come into play for these kinds of problems. Image annotation is labeling an image with keywords that describe its contents. This approach aids in the intelligent retrieval of pertinent images using a straightforward query representation. Annotating a moving object in a video is typically an extremely time-consuming and tedious task. A study [33] defines an effective modern object annotation tool for annotating different types of video footage. The use of automatic image annotation (AIA) becomes essential for managing the continuously increasing number of digital images. The most popular approaches, including performance evaluation metrics and annotation datasets, are reviewed [34]. Additionally, there was a discussion on ways to get around those restrictions. Machine Learning Assisted Image Annotation (MAIA) can keep an eye on the environment in order to get around the problem of insufficient training data [35]. By combining autoencoder networks and Mask R-CNN, the technique makes it possible for human observers to annotate enormous image sets considerably more quickly than in the past. An intelligent system anticipates that the object should define itself, unlike a conventional system. But to make a system so intelligent that the object it is targeting and surveillance, describes itself, becomes a matter of advanced research. Again, the convolutional neural network-based model is proposed [36] for both encoder ad decoder. This model worked with image captioning. The entire research was conducted on the Microsoft Common Objects in Context (MSCOCO) dataset. But there are many objects in any video, and it would not be very useful to describe them all at once. Therefore, it would be more correct to first decide which object to target and select it. Real-time object detection will play a very important role in this whole process. Object Detection Using Depth Sensors (ODDS) is a technique for real-time object detection [37]. It utilized raw depth data and was embedded on the GPU (NVIDIA Jetson TX1). This approach can be seen as going above and beyond the Deep Learning approach. Vehicle detection and counting have emerged as important areas of research and challenges in intelligent highway management, owing to the wide range of vehicle sizes. This challenge has a direct impact on the accuracy of vehicle counting. An effective vehicle detection framework based on convolutional neural networks was developed [38] to identify and categorize vehicles. The latest dataset of highway vehicles with 11,129 images and 57,290 annotated instances was published to address the problem of multi-object tracking and counting [39]. But in addition to effective algorithms and good quality datasets, power management is also essential to overall performance. In this context, an energy-efficient Object Tracking Sensor Network based on Wireless Sensor Network is represented by

12 IoT and Big Data Analytics, Vol. 1

Sharma et al.

[40]. The cognitive computing-based CIoT-based smart city network (CIoT-Net) [41] is also able to process a large amount of smart city data effectively. OPEN CHALLENGES & ISSUES IoT devices generate massive amounts of data. Data is real-time, varies in structure, and may have uncertain attribution. Handling such data is difficult because the overall performance is proportional to the features of the data management service. The importance of visual data for object extraction cannot be overstated. Surveillance systems must deal with a variety of challenges, including algorithmic and infrastructure challenges. To offer more robust and trustworthy services, systems must adapt to evolving technologies like cloud computing systems. The trend will also necessitate the incorporation of various surveillance systems to extract concrete information from the captured scenes. This will necessitate the development of new communication protocols, adapted databases, query languages, and data formats among surveillance agents. Finally, more precise algorithms will be needed, specifically from the perspective of behavioral analysis and the detection of abnormal activities. Model selection and weighting are critical considerations. Otherwise, a poor choice can produce worse. DISCUSSION, TRENDS, AND FUTURE WORK Several studies have been conducted in recent years. A significant portion of existing work provides solutions that are tailored to the context. It is necessary to identify all types of actions, behavior, and movement. Extensive research has been carried out over the last several decades to address the issue of intelligent (or semi-intelligent) surveillance. Object tracking, object recognition, object reidentification, and image enhancement were the main subtasks investigated. Many excellent studies have been proposed within this framework. So far, much work remains to be done. Fig. (5) shows the identified contexts as application areas. Researchers in video analytics can upgrade the surveillance by focusing on the topics depicted in Fig. (6). The main objectives that have been identified to demonstrate the topic's relevance are listed below. Existing methods dealt with each problem separately. However, none of them addressed all of the objectives as a feature. To effectively analyze the video in real-time, the method should be able to solve all the above-listed problems. This would result in the real-time detection of objects. The high computational resources are available for fast computation, and by utilizing them, deep learningbased solutions can be implemented.

Comprehensive Analysis

IoT and Big Data Analytics, Vol. 1 13

Fig. (5). Application Areas identified for Video Surveillance.

Fig. (6). Main objective for video analysis and relevance to the surveillance.

CONCLUSION The author studied various types of surveillance and the techniques involved in surveillance in this chapter. All video surveillance techniques are examined to determine their status. The author's goal is to identify datasets and algorithms.

14 IoT and Big Data Analytics, Vol. 1

Sharma et al.

Their analysis will undoubtedly highlight future research challenges as well as opportunities for researchers. Aside from that, the literature review covers a wide range of applications. Tables have been created to list the techniques, tools, and datasets identified through the survey. The survey starts with traditional surveillance and progresses to advanced video surveillance. The comprehensive analysis began with a broad overview to demonstrate the various demanding purposes. Automatically monitoring large critical open areas is a challenge that must be overcome. This paper covers all of the fundamentals as well as advanced surveillance techniques. The fundamental components and requirements for video surveillance are addressed to provide a systematic research approach and to demonstrate how researchers can begin their research in this field. This survey emphasizes object recognition and action detection as key surveillance terms. The summarization includes methods, algorithms, approaches for video surveillance other than deep learning with challenges, and datasets. CONSENT FOR PUBLICATION Not applicable. CONFLICT OF INTEREST The authors declare no conflict of interest, financial or otherwise. ACKNOWLEDGEMENT Declared none. REFERENCES [1]

M. Koyuncu, A. Yazici, M. Civelek, A. Cosar, and M. Sert, "Visual and auditory data fusion for energy-efficient and improved object recognition in wireless multimedia sensor networks", IEEE Sens. J., vol. 19, no. 5, pp. 1839-1849, 2019. [http://dx.doi.org/10.1109/JSEN.2018.2885281]

[2]

B. Javed, M. Iqbal, and H. Abbas, "Internet of things (IoT) design considerations for developers and manufacturers", 2017 IEEE International Conference on Communications Workshops (ICC Workshops), 2017 Paris, France [http://dx.doi.org/10.1109/ICCW.2017.7962762]

[3]

A. Yazici, M. Koyuncu, S.A. Sert, and T. Yilmaz, "A fusion-based framework for wireless multimedia sensor networks in surveillance applications", IEEE Access, vol. 7, pp. 88418-88434, 2019. [http://dx.doi.org/10.1109/ACCESS.2019.2926206]

[4]

N. Naikal, P. Lajevardi, and S. Sastry, "Joint detection and recognition of human actions in wireless surveillance camera networks", 2014 IEEE International Conference on Robotics and Automation (ICRA) Vol.6, 4747-4754, 2014 [http://dx.doi.org/10.1109/ICRA.2014.6907554]

[5]

M.G.H. AL Zamil, S. Samarah, M. Rawashdeh, A. Karime, and M.S. Hossain, "Multimedia-oriented action recognition in Smart City-based IoT using multilayer perceptron", Multimedia Tools Appl., vol. 78, no. 21, pp. 30315-30329, 2019.

Comprehensive Analysis

IoT and Big Data Analytics, Vol. 1 15

[http://dx.doi.org/10.1007/s11042-018-6919-z] [6]

V. Tsakanikas, and T. Dagiuklas, "Video surveillance systems-current status and future trends", Comput. Electr. Eng., vol. 70, pp. 736-753, 2018. [http://dx.doi.org/10.1016/j.compeleceng.2017.11.011]

[7]

L. Tay, A.T. Jebb, and S.E. Woo, "Video capture of human behaviors: Toward a Big Data approach", Curr. Opin. Behav. Sci., vol. 18, pp. 17-22, 2017. [http://dx.doi.org/10.1016/j.cobeha.2017.05.026]

[8]

K. Kardas, and N.K. Cicekli, "SVAS: Surveillance video analysis system", Expert Syst. Appl., vol. 89, pp. 343-361, 2017. [http://dx.doi.org/10.1016/j.eswa.2017.07.051]

[9]

G. Sreenu, and M.A. Saleem Durai, "Intelligent video surveillance: A review through deep learning techniques for crowd analysis", J. Big Data, vol. 6, no. 1, p. 48, 2019. [http://dx.doi.org/10.1186/s40537-019-0212-5]

[10]

S. Chaudhary, M.A. Khan, and C. Bhatnagar, "Multiple anomalous activity detection in videos", Procedia Comput. Sci., vol. 125, pp. 336-345, 2018. [http://dx.doi.org/10.1016/j.procs.2017.12.045]

[11]

S. Cosar, G. Donatiello, V. Bogorny, C. Garate, L.O. Alvares, and F. Bremond, "Toward abnormal trajectory and event detection in video surveillance", IEEE Trans. Circ. Syst. Video Tech., vol. 27, no. 3, pp. 683-695, 2017. [http://dx.doi.org/10.1109/TCSVT.2016.2589859]

[12]

D. Xu, Y. Yan, E. Ricci, and N. Sebe, "Detecting anomalous events in videos by learning deep representations of appearance and motion", Comput. Vis. Image Underst., vol. 156, pp. 117-127, 2017. [http://dx.doi.org/10.1016/j.cviu.2016.10.010]

[13]

M. Ribeiro, A.E. Lazzaretti, and H.S. Lopes, "A study of deep convolutional auto-encoders for anomaly detection in videos", Pattern Recognit. Lett., vol. 105, pp. 13-22, 2018. [http://dx.doi.org/10.1016/j.patrec.2017.07.016]

[14]

Y. Guo, Y. Liu, A. Oerlemans, S. Lao, S. Wu, and M.S. Lew, "Deep learning for visual understanding: A review", Neurocomputing, vol. 187, pp. 27-48, 2016. [http://dx.doi.org/10.1016/j.neucom.2015.09.116]

[15]

K. Sundararajan, and D.L. Woodard, "Deep learning for biometrics", ACM Comput. Surv., vol. 51, no. 3, pp. 1-34, 2019. [http://dx.doi.org/10.1145/3190618]

[16]

C. Wang, H. Yang, and C. Meinel, "Image captioning with deep bidirectional LSTMs and multi-task learning", ACM Trans. Multimed. Comput. Commun. Appl., vol. 14, no. 2s, pp. 1-20, 2018. [http://dx.doi.org/10.1145/3115432]

[17]

D. Tao, Y. Guo, B. Yu, J. Pang, and Z. Yu, "Deep multi-view feature learning for person reidentification", IEEE Trans. Circ. Syst. Video Tech., vol. 28, no. 10, pp. 2657-2666, 2018. [http://dx.doi.org/10.1109/TCSVT.2017.2726580]

[18]

T. Pamula, "Road traffic conditions classification based on multilevel filtering of image content using convolutional neural networks", IEEE Intell. Transp. Syst. Mag., vol. 10, no. 3, pp. 11-21, 2018. [http://dx.doi.org/10.1109/MITS.2018.2842040]

[19]

D. Zhang, W. Wu, H. Cheng, R. Zhang, Z. Dong, and Z. Cai, "Image-to-video person re-identification with temporally memorized similarity learning", IEEE Trans. Circ. Syst. Video Tech., vol. 28, no. 10, pp. 2622-2632, 2018. [http://dx.doi.org/10.1109/TCSVT.2017.2723429]

[20]

N. Perwaiz, M. Moazam Fraz, and M. Shahzad, "Person re-identification using hybrid representation reinforced by metric learning", IEEE Access, vol. 6, pp. 77334-77349, 2018. [http://dx.doi.org/10.1109/ACCESS.2018.2882254]

16 IoT and Big Data Analytics, Vol. 1

Sharma et al.

[21]

G. Goswami, M. Vatsa, and R. Singh, Face verification via learned representation on feature-rich video frames., IEEE Trans. Inf. Forensics Security, vol. 12, no. 7, pp. 1686-1698, 2017. [http://dx.doi.org/10.1109/TIFS.2017.2668221]

[22]

E. Şaykol, U. Güdükbay, and Ö. Ulusoy, "Scenario-based query processing for video-surveillance archives", Eng. Appl. Artif. Intell., vol. 23, no. 3, pp. 331-345, 2010. [http://dx.doi.org/10.1016/j.engappai.2009.08.002]

[23]

H.Y. Cheng, and J.N. Hwang, "Integrated video object tracking with applications in trajectory-based event detection", J. Vis. Commun. Image Represent., vol. 22, no. 7, pp. 673-685, 2011. [http://dx.doi.org/10.1016/j.jvcir.2011.07.001]

[24]

Z. Dong, C. Jing, M. Pei, and Y. Jia, "Deep CNN based binary hash video representations for face retrieval", Pattern Recognit., vol. 81, pp. 357-369, 2018. [http://dx.doi.org/10.1016/j.patcog.2018.04.014]

[25]

C. Zhang, Y. Tian, X. Guo, and J. Liu, "DAAL: Deep activation-based attribute learning for action recognition in depth videos", Comput. Vis. Image Underst., vol. 167, pp. 37-49, 2018. [http://dx.doi.org/10.1016/j.cviu.2017.11.008]

[26]

C. Tzelepis, D. Galanopoulos, V. Mezaris, and I. Patras, "Learning to detect video events from zero or very few video examples", Image Vis. Comput., vol. 53, pp. 35-44, 2016. [http://dx.doi.org/10.1016/j.imavis.2015.09.005]

[27]

V. Kaltsa, A. Briassouli, I. Kompatsiaris, and M.G. Strintzis, "Multiple hierarchical dirichlet processes for anomaly detection in traffic", Comput. Vis. Image Underst., vol. 169, pp. 28-39, 2018. [http://dx.doi.org/10.1016/j.cviu.2018.01.011]

[28]

K.E. Ko, and K.B. Sim, "Deep convolutional framework for abnormal behavior detection in a smart surveillance system", Eng. Appl. Artif. Intell., vol. 67, pp. 226-234, 2018. [http://dx.doi.org/10.1016/j.engappai.2017.10.001]

[29]

Y. Wang, D. Zhang, Y. Liu, B. Dai, and L.H. Lee, Enhancing transportation systems via deep learning: A survey., Transp. Res., Part C Emerg. Technol., vol. 99, pp. 144-163, 2019. [http://dx.doi.org/10.1016/j.trc.2018.12.004]

[30]

A.R. Pathak, M. Pandey, and S. Rautaray, "Application of deep learning for object detection", Procedia Comput. Sci., vol. 132, pp. 1706-1717, 2018. [http://dx.doi.org/10.1016/j.procs.2018.05.144]

[31]

Z. Wei, and F. Wang, "Adaptive cascade single-shot detector on wireless sensor networks", EURASIP J. Wirel. Commun. Netw., vol. 2019, no. 1, p. 150, 2019. [http://dx.doi.org/10.1186/s13638-019-1440-2]

[32]

Z. Fan, X. Song, T. Xia, R. Jiang, R. Shibasaki, and R. Sakuramachi, "Online deep ensemble learning for predicting citywide human mobility", Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 2, no. 3, pp. 1-21, 2018. [http://dx.doi.org/10.1145/3264915]

[33]

S. Kletz, A. Leibetseder, and K. Schoeffmann, "A comparative study of video annotation tools for scene understanding", In: Proceedings of the 10th ACM Multimedia Systems Conference, 2019, pp. 133-144. [http://dx.doi.org/10.1145/3304109.3306223]

[34]

P.K. Bhagat, and P. Choudhary, "Image annotation: Then and now", Image Vis. Comput., vol. 80, pp. 1-23, 2018. [http://dx.doi.org/10.1016/j.imavis.2018.09.017]

[35]

M. Zurowietz, D. Langenkämper, B. Hosking, H.A. Ruhl, and T.W. Nattkemper, "MAIA—A machine learning assisted image annotation method for environmental monitoring and exploration", PLoS One, vol. 13, no. 11, p. e0207498, 2018. [http://dx.doi.org/10.1371/journal.pone.0207498] [PMID: 30444917]

Comprehensive Analysis

IoT and Big Data Analytics, Vol. 1 17

[36]

C. Poleak, and J. Kwon, "Parallel image captioning using 2D masked convolution", Appl. Sci. (Basel), vol. 9, no. 9, p. 1871, 2019. [http://dx.doi.org/10.3390/app9091871]

[37]

N. Mithun, S. Munir, K. Guo, and C. Shelton, "ODDS: Real-time object detection using depth sensors on embedded GPUs", 2018 17th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) p. 230–241, 2018 Porto. [http://dx.doi.org/10.1109/IPSN.2018.00051]

[38]

L. Chen, F. Ye, Y. Ruan, H. Fan, and Q. Chen, "An algorithm for highway vehicle detection based on convolutional neural network", EURASIP J. Image Video Process., vol. 2018, no. 1, p. 109, 2018. [http://dx.doi.org/10.1186/s13640-018-0350-2]

[39]

H. Song, H. Liang, H. Li, Z. Dai, and X. Yun, "Vision-based vehicle detection and counting system using deep learning in highway scenes", Eur. Trans. Res. Rev., vol. 11, no. 1, p. 51, 2019. [http://dx.doi.org/10.1186/s12544-019-0390-4]

[40]

L. Paris, and M. Anisi, "An energy-efficient predictive model for object tracking sensor networks", 2019 IEEE 5th World Forum on Internet of Things (WF-IoT), 2019 Limerick, Ireland [http://dx.doi.org/10.1109/WF-IoT.2019.8767195]

[41]

J. Park, M.M. Salim, J.H. Jo, J.C.S. Sicato, S. Rathore, and J.H. Park, "CIoT-Net: A scalable cognitive IoT based smart city network architecture", Human-centric Computing and Information Sciences, vol. 9, no. 1, p. 29, 2019. [http://dx.doi.org/10.1186/s13673-019-0190-9]

18

IoT and Big Data Analytics, 2023, Vol. 1, 18-36

CHAPTER 2

Compressed Video-Based Efficient Video Analytics

Classification

for

Sangeeta1,*, Preeti Gulia1 and Nasib Singh Gill1 1

Department of Computer Science & Applications, Maharshi Dayanand University, Rohtak, India Abstract: Videos have become a crucial part of human life nowadays and share a large proportion of internet traffic. Various video-based platforms govern the mass consumption of videos through analytics-based filtering and recommendations. Various video-based platforms govern their mass consumption by analytics-based filtering and recommendations. Video analytics is used to provide the most relevant responses to our searches, block inappropriate content, and disseminate videos to the relevant community. Traditionally, for video content-based analytics, a video is first decoded to a large raw format on the server and then fed to an analytics engine for metadata generation. These metadata are then stored and used for analytic purposes. This requires the analytics server to perform both decoding and analytics computation. Hence, analytics will be fast and efficient, if performed over the compressed format of the videos as it reduces the decoding stress over the analytics server. This field of object and action detection from compressed formats is still emerging and needs further exploration for its applications in various practical domains. Deep learning has already emerged as a de facto for compression, classification, detection, and analytics. The proposed model comprises a lightweight deep learning-based video compression-cumclassification architecture, which classifies the objects from the compressed videos into 39 classes with an average accuracy of 0.67. The compression architecture comprises three sub-networks i.e. frame and flows autoencoders with motion extension network to reproduce the compressed frames. These compressed frames are then fed to the classification network. As the whole network is designed incrementally, the separate results of the compression network are also presented to illustrate the visual performance of the network as the classification results are directly dependent on the quality of frames reconstructed by the compression network. This model presents a potential network and results can be improved by the addition of various optimization networks.

Keywords: Compression, CNN, ConvGRU, Deep learning, PSNR, SSIM, Video. * Corresponding author Sangeeta: Department of Computer Science & Applications, Maharshi Dayanand University, Rohtak, India; E-mail: [email protected]

Abhishek Singh Rathore, Surendra Rahamatkar, Syed Imran Ali, Ramgopal Kashyap & Nand Kishore Sharma (Eds.) All rights reserved-© 2023 Bentham Science Publishers

Compressed Video-Based

IoT and Big Data Analytics, Vol. 1 19

INTRODUCTION Videos take the maximum share of the internet content and it is growing exponentially day by day [1]. A lot of modern high-resolution formats of images and videos have been developed and used by cell phones to capture and store them. Various recent applications like autonomous drones, self-driving cars, and warehouse inventory systems are largely driven by video content. Videos store and convey a lot of information rather than still images convey. Emotions [2], predicting some future course of action [3], spatial awareness, and temporal context be better represented and conveyed by the videos [4]. These domains are less exploited and still emerging. Several deep learning-based techniques for video analysis have been developed but they are quite simpler and primarily comprise CNNs. The techniques designed for image analysis are further extended for decomposing videos into consecutive still frames. But these techniques are not able to outperform the traditional handdesigned schemes [5, 6]. Deep Learning based object detection in images results in promising outputs but its transformative implication on video chores like action recognition is yet to be explored. Two arguments may support this concern. Firstly, the information density in the videos is low. A compression rate ranging from 1 GB to 222 GB can be achieved for a 720p video of one hour. Videos contain a lot of redundant information and this redundancy put some additional challenges before CNNs in extracting significant information from the videos. Moreover, it also results in slow training. Secondly, temporal feature learning is hard with RGB images only. A number of explorations have been made employing RNNs and 3D and 2D CNNs for video processing as RGB stills, but result in only marginal improvements [5, 7]. Further, the usage of pre-computed optical flow has enhanced the performance [8]. Action recognition plays an important role in video analytics. Here, the type of action is predicted based on the movements in the given video. As video accumulates rich information than still images, understanding or action recognition from videos stimulates new explorations in the field of vision and deep learning. Traditionally, for video content-based analytics, a video is first decoded to a large raw format on a server and then fed to an analytics engine for metadata generation. These metadata are then stored and used for analytics purposes. This requires the analytics server to perform both decoding and analytics computation. Fig. (1) illustrates the whole process. The decoding of compressed videos for object/action detection increases the computation task of the analytics server. The efficient detection, directly from the compressed videos will surely enhance the whole process. This same concern has

20 IoT and Big Data Analytics, Vol. 1

Sangeeta et al.

motivated us to propose this model. This model comprises an integrated network of compression and classification sub-networks. It exploits the compressed formats for classification tasks instead of decoding into raw format. The proposed scheme is presented in below Fig. (2).

Fig. (1). Traditional scheme of object/action detection from videos.

Fig. (2). Proposed scheme of object/action detection from videos.

The traditional compression codecs take into account the similarity between successive frames. They keep some of the important frames only and reproduce the remaining frames using residual error and motion vectors of the preserved frames. Our model comprises a flow autoencoder for efficient motion vector compression and processing. The frame autoencoder eradicates the redundancy and insignificant data and makes important signals prominent. The motion vectors provide better motion information than simple RGB frames. In addition, motion signals did not take into account the spatial differences, for example, if two persons are doing the same thing but in varied lighting and attires, they will generate the same motion signals. Hence, the generalization will get better and improves training efficiency due to lesser variation. The proposed model exploits some time-sensitive variations rather than i.i.d. frames in addition to the spatial features. This way of constraining information helps to address the dimensionality overhead. Avoiding continual processing of near duplicates and the use of only true signals also enhances the efficiency of the model. Lastly, as the classification operates directly on compressed formats of the videos, this saving of decompression overload also adds to the efficiency of the network. YouTube UGC data set is used to train the network. 600+ UGC content labels are used for studying the relationship between UGC content and perceptual quality. The model efficiently classifies the content into 39 labels with an accuracy of 0.67. The

Compressed Video-Based

IoT and Big Data Analytics, Vol. 1 21

proposed approach represents an efficient, fast, and simpler compression-cumclassification network without making use of 3D convolutions, Recurrent Neural Networks, or any complex fusion. RELATED WORK Deep Learning-based action recognition from videos is a recent field of advancement and explorations. In the traditional detection techniques, manually designed schemes like Histogram of Optical Flow or Histogram of Oriented Gradients are used [9, 10]. These techniques take into account the important features among frames and their intelligent fusion gleaned from intense trajectories [6, 11, 12]. Among all the hand-crafted techniques, some are performing well and are widely used. These days, iDT is also used for motion corrections in cameras. Deep learning has contributed notably in the domain of action recognition from videos during the last few years. These advancements evolved from the enhanced performance of deep learning-based image representations. The temporal features can be modeled relatively easily. Some of the prominent techniques make predictions based on sub-sampling a few frames followed by average pooling [13, 14]. In addition to the temporal CNNs [15] and RNNs [16, 17], some new feature extraction schemes have also been proposed [18]. Such techniques cannot able to perform better than existing traditional schemes like average pooling due to the addition of extra computations. For the efficient modeling of the temporal features, some new 3D CNN-based networks were also proposed [19]. Though, these networks result in some improved performance with an increased number of parameters and enhanced computation time. It has been observed that these schemes are not adequate to extract all the temporal features. The performance of the networks improved with the usage of precomputed optical flow [20]. True representation of input contributes significantly to the performance over simple RGB frames. But all these techniques apply to the raw frame-by-frame format of the videos and cannot be applied to the compressed format. The techniques used for the compression of the videos will directly affect the action recognition or classification from their compressed formats. The better the quality of re-constructed frames, the better will be the detection results. Efficient compression algorithms are required for better detection or classification tasks. The exponentially growing video content over the internet urges the requirement of more competent compression techniques. H.264, HEVC, and MPEG-4 are the most widely used video codecs these days. Their main working idea lies in the similarity between the consecutive frames of the videos. They retain only some of the important frames and store the difference between the consecutive frames

22 IoT and Big Data Analytics, Vol. 1

Sangeeta et al.

instead of storing them all. Generally, the frames of a video are categorized into Intracoded or I-frames, Predictive or -frames, and Bi-directional or B-frames. Iframes are fully retained and compressed normally. For the P-frames, only the ‘change’ is stored about the last frame. This ‘change’ is called a motion vector and it represents the change in the position of the block of pixels among two consecutive frames at a particular instant. In the B-frame, there are bi-directional computations of the motion vectors, hence also called special P-frames. They may refer to the next future frame. Both P-frames and B-frames can be compressed easily as they comprise only changes and a lesser dynamic range [21]. Though the traditional video codecs are performing quite well and are widely used, some improvements have also been achieved through deep learning-based enhancements to them. As they are block designed, they cannot be the end–to –end optimized but the overall performance improvement can only be achieved through individual module optimizations. The next-generation video codecs will comprise pure end-to-end trainable and optimizable deep learning-based approaches. In this work, we have used a deep learning-based end-to-end network to compress the video frames using frame and flow autoencoders with a motion extension network. The results of this compression network are efficient and competitive with the enhanced visual quality of the reconstructed frames. The field of deep learning-based detection or recognition from videos is very less explored and is still evolving. Only limited works are available in this field. Though, some of the discussed works present improved detection but utilize hand-crafted modules, not deep learning [22 - 25]. MVCNN, a deep learningbased network was proposed that relies on the transmission of motion information from the flow network to the motion vector network [25]. But this network still requires the decompressed format of the video frames in addition to the flow vectors. So, this network also does not take into account the compressed representation. But in our proposed model, the classification network will directly fetch the compressed representation from the compression network and classify accordingly with a finer accuracy rate. PROPOSED MODEL The proposed model is designed to make the classification directly from the compressed format of the videos. Deep learning is emerging as a potential tool for the next generation of end-to-end trainable and optimizable video codecs. Deep Learning already presents a breakthrough in the analytics domain. The proposed model comprises an end-to-end trained pure video compression-cu-classification network. The entire network is an integration of two sub-networks i.e. compression network and the classification network. The compression network efficiently compresses the video frames and then feeds this compressed

Compressed Video-Based

IoT and Big Data Analytics, Vol. 1 23

representation to the classification network to classify the objects detected into various categories. The overview of the proposed model is given in Fig. (3). The detailed architecture and internal configurations of the compression and classification network have been given in the below sections, respectively. Compression Network

Classification Network

Fig. (3). Overview of Proposed Model.

COMPRESSION NETWORK The video compression network of the model is designed to compress the video frames in such a way that classification can be made efficiently from the compressed format. It primarily comprises three sub-networks namely: 1. Frame Autoencoder 2. Flow Autoencoder 3. Motion Extension Network The Frame Autoencoder is employed to compress the video frames. The encoder of this sub-network takes the video frames as input and encodes them into the binary format. The encoder is five layers, comprising four 2D-CNN layers and one ConvGRU layer. ConvGRU layer is incorporated to enhance the compression quality as it encompasses the properties of both CNN and RNN. The binary format is quantized and then fed into the decoder for reconstruction. Like the encoder, the decoder also comprises four 2D-CNN layers along with one ConvGRU layer. The residual image is calculated by the difference between the original image and the recreated image. Residual image is again passed through the autoencoder network to encode into compressed binary format. This process is repeated based on the level of required compression quality. The network exploits multiple emissions to produce frames of varying compression quality. The same has been depicted in Fig. (4) and the detailed configuration of Convolutional and deconvolutional layers has been given in Table 1.

24 IoT and Big Data Analytics, Vol. 1

Sangeeta et al.

Fig. (4). The model of the Compression Network.

The Flow Autoencoder is used for efficient motion compensation. Optical flow is a widely used and potent tool for effective temporal representation. Among various learning-based optical flow methods, we have used Farneback flow estimation method in our model as it is well-tested and has lower training requirements. Firstly, the motion among the consecutive frames is computed and represented in flow vectors. The flow vectors are then fed into the flow autoencoder for efficient motion information compression. They are compressed using a standard CNN-based encoder network with Generalized Divisive Normalization layers as the nonlinearity. A CNN-based decoder network with Inverse GDN as the nonlinearity is used to decompress the flow vectors. The decompressed flow vector is then finally fed to the motion extension network for frame reconstruction. A detailed internal description has been given in Table 2.

Compressed Video-Based Table 1. Network parameters of Frame Autoencoder.

IoT and Big Data Analytics, Vol. 1 25

26 IoT and Big Data Analytics, Vol. 1

Sangeeta et al.

Table 2. Network Parameters of Flow Autoencoder.

The motion extension network has been used for the final reconstruction of the frames. The next frame is reconstructed using three inputs namely - the flow vector, the previous frame, and the current frame. This network employs Convolutional layers only. Below Fig. (5) depicts the internal configuration of this network. The compressed representation generated by the compression network is fed to the classification network for further processing. The quality of reconstructed frames directly affects the classification task. If the compression rate is increased beyond a limit, subsequently it will make the detection task more complex and hence accuracy of the detection network will get reduced.

Compressed Video-Based

IoT and Big Data Analytics, Vol. 1 27

Fig. (5). Motion Extension Network.

CLASSIFICATION NETWORK The proposed classification network utilizes the compressed representation of the videos for classification. This is a seven-layered network employing MobileNet Convolutional layers. The overview of the network is given in Fig. (6). It has been clearly shown that the MobileNet-based classification network is classifying the

28 IoT and Big Data Analytics, Vol. 1

Sangeeta et al.

videos based on N class attributes using binarized coding. The detailed internal configuration of the network has been given in Table 3. IP(n) IP(n-1)

Frame AutoEncoder Optical Flow Estimator

Binarized Encodings Flow AutoEncoder Video Encoder Network

MobileNetv2 Layers

N Video class attributes

Analytics/Classification Network

Video Decoder Network

OP(n) OP(n-1)

Fig. (6). The model of Compression-cum-classification Network. Table 3. Network parameters of the classification network.

Compressed Video-Based

IoT and Big Data Analytics, Vol. 1 29

The whole network is end-to-end trained and designed using a deep learning approach only. It is trained with the UGC dataset having 600+ content labels but we have chosen only 39 labels based on the frequent occurrence of the labels in multiple videos. When trained with the given video dataset, the network successfully classified the videos into chosen 39 classes with an average accuracy of 0.67. The proposed method is also more efficient, fast, and simpler as it avoids the decoding overhead over the analytics network. EXPERIMENTS Experimental Setup Dataset A dataset comprising 20s long 571 small videos out of a total of 826 videos from YouTube UGC has been used to train the network. The remaining video clips have been used for testing and validation. Videos of varying quality have been chosen i.e. 480p, 360p, and 720p. The frame size has been chosen as 64x64. Hence video clips of all quality are firstly rescaled to the chosen format, and then training is performed. Video frames are taken randomly during training but while testing the clips are chosen from the start. The model has been trained with a randomized emission step training strategy with emission steps varying from 1 to 10. The addition of each emission step improves the output but affects the compression efficiency. Implementation Details For the implementation purpose, a single T4, K80, or P100 GPU has been used to train the network on the Google Collaboratory platform. The frames have been kept to the size of 64 x 64. Adam Optimizer is used to train the neural network with 10e-4 being the learning rate. λ1 is taken as one and λ2 be 10. The first frame encoder is trained for 100 epochs. Then the complete network is trained end-t-end for 70 epochs. During the training of the frame encoder with 100 epochs; the learning rate has been divided by ten at the 50th, 70th and 90th epoch. For the whole model training, only 70 epochs have been used after stacking the pre-trained framer encoder first and the learning rate has been altered at the 35th and 55th epoch by dividing with ten. Loss Function The goal of the network is to reduce the structural distortion between the input video and the output. We have also used the mean squared error (MSE) loss function to reduce the color distortion in decompressed images.

30 IoT and Big Data Analytics, Vol. 1

Sangeeta et al.

‫ ܮ‬ൌ ‫ܮ‬௦௦௜௠ ൅ ߙ‫ܮ‬௠௦௘ where MSE error is evaluated as: ௡

‫ܮ‬௠௦௘ ሺ‫ݕ‬ǡ ‫ݕ‬

ᇱሻ

ͳ ൌ ෍ሺ‫ ݕ‬െ ‫ݕ‬௜ᇱ ሻଶ ܰ ଴

and SSIM error is evaluated based upon three comparison measurements, luminance (l), contrast (c), and structure (s): ‫ܮ‬௦௦௜௠ ሺ‫ݕ‬ǡ ‫ ݕ‬ᇱ ሻ ൌ ሾ݈ሺ‫ݕ‬ǡ ‫ ݕ‬ᇱ ሻǤ ܿሺ‫ݕ‬ǡ ‫ ݕ‬ᇱ ሻǤ ‫ݏ‬ሺ‫ݕ‬ǡ ‫ ݕ‬ᇱ ሻሿ Evaluation The performance of the compression network is measured using four parameters. SSIM i.e. Structural Similarity Index Measure and PSNR i.e. Peak Signal to Noise Ratio has been used to measure the visual quality of the reconstructed frames. The temporal distortion among the frames has been evaluated by Flow EPE i.e. End Point Error. Moreover, the reconstruction time of individual frames has been measured by the TPF i.e. Time Per Frame parameter. The accuracy of the classification determines the efficiency of the classification network. EXPERIMENTAL RESULTS AND DISCUSSION Experimental Results and Analysis of Compression Network The proposed model has been designed and implemented in incremental order. Firstly, the compression network is designed, and its performance is evaluated separately. The qualitative performance i.e. visual quality of the reconstructed images is measured using SSIM and PSNR. The quantitative performance is measured by Flow End Point Error and TPF. The network is trained with a random emission step strategy with the number of emission steps ranging from 1 to 10. The values of all four parameters have been measured for each emission step and have been presented below in Tables 4 to 7. It can be observed that SSIM and PSNR have the least values at first emission and their values are increasing with each additional emission step. Both SSIM and PSNR have a maximum value at the 10th emission step. It infers that the visual quality of the frame is improved with the addition of multiple emissions. Flow End Point Error measures the error in motion information among subsequent frames. Table 6 represents the flow error at each emission step. The obtained values show that the error is also

Compressed Video-Based

IoT and Big Data Analytics, Vol. 1 31

gradually decreasing with each new emission step. But we can also see some irregular patterns that the error value is minimum at the 9th emission step i.e. 0.148 instead at the 10th emission. The time elapsed in reconstruction has been given in Table 7. Though above-obtained values infer that the inclusion of multiple emission steps is improving the visual quality of the images and reducing the flow error but with some nominal increment in frame reconstruction time. The value of TPF is minimum at 1st emission and with gradual increment, reaches maximum i.e. 0.029 at the last emission. The average performance of the network over ten emission steps has been given in Table 8. These values depict the improved and efficient performance of the network both qualitatively and quantitatively. Table 4. SSIM values per emission. SSIM

1

Compression Network

2

3

4

5

6

7

8

9

10

0.709 0.819 0.874 0.91 0.932 0.948 0.957 0.961 0.963 0.963

Table 5. PSNR values per emission. PSNR

1

2

3

4

5

6

7

8

9

10

Compression Network

20

22.5

24.1

25.5

26.7

27.8

28.4

28.9

29.1

29.2

4

5

6

7

Table 6. Flow EPE values per emission. Flow EPE

1

Compression Network

2

3

8

9

10

0.822 0.577 0.368 0.276 0.226 0.253 0.189 0.17 0.148 0.17

Table 7. TPF values per emission. Time Per Frame Compression Network

1

2

3

4

5

6

7

8

9

10

0.0243 0.0248 0.0254 0.0258 0.0263 0.0268 0.0274 0.0279 0.0285 0.029

Table 8. Average Performance Values. -

Avg. SSIM

Avg. PSNR

Avg. EPE

Avg. TPF

Compression Network

0.9036

26.22

0.3199

0.02662

Experimental Results and Analysis of Classification Network The performance of the classification network has been evaluated with the accuracy of the predicted labels of the videos. Firstly, the network is trained with the UGC dataset. This dataset has 600+ labels for studying the relationship between UGC content and perceptual quality. Each video was assigned 12

32 IoT and Big Data Analytics, Vol. 1

Sangeeta et al.

candidate YT8M labels and then refined through a crowd-sourcing subjective test. Every label on each video was voted on by more than 10 subjects.

Fig. (7). Votes v/s Labels.

To reduce the labels with fewer examples, we have selected labels that have at least 25 videos and 50+ cumulative votes. Based on these criteria, we got 39 filtered good labels. All the selected labels are mentioned below. The same has been given in Fig. (7). Total Labels:39 ['Vehicle', 'Game', 'Photography', 'Video game', 'Animation', 'Musician', 'Naruto', 'Car', 'Food', 'Guitar', 'Animal', 'Sports', 'Drawing', 'Vlog', 'Association football', 'Gaming', 'Musical ensemble', 'Music video', 'NewsClip', 'Trailer (promotion)', 'String instrument', 'Minecraft', 'Orchestra', 'Piano', 'Concert', 'HowTo', 'Lecture', 'Art', 'Motorsport', 'CoverSong', 'Strategy video game', 'Call of Duty, 'Christmas', 'MusicVideo', 'VerticalVideo', 'Dance', 'Cartoon', 'LiveMusic', 'Prayer'] Video Count: [ 155, 209, 132, 144, 117, 137, 79, 66, 47, 103, 53, 106, 78, 85, 59, 95, 89, 60, 60, 44, 67, 42, 43, 28, 58, 60, 63, 55, 35, 59, 49, 44, 33, 52, 54, 32, 36, 55, 25] The performance of the trained network is measured with the accuracy achieved in classification. Fig. (8) represents the accuracy of the classified labels. The model successfully classified the video set into given labeled sets with an average accuracy of 0.66. The individual precision description of all selected 39 labels has been given in Table 9.

Compressed Video-Based

IoT and Big Data Analytics, Vol. 1 33 .

Accuracy: 0.664887382972897 Fig. (8). Accuracy of the classification network. Table 9. Individual precision description of all selected 39 labels. precision . Vehicle Game . Photography . Video game . Animation . . Musician . Naruto . Car . Food Guitar . . Animal Sports . Drawing . . Vlog Association football . . Gaming . Musical ensemble Music Video . NewsClip . Trailer (promotion) . String instrument . Minecraft . . Orchestra Piano . . Concert HowTo . Lecture . Art . . Motorsport Cover Song . Strategy Video game . Call of Duty . Christmas . MusicVideo . . VerticalVideo . Dance . Cartoon . LiveMusic . Prayer

recall

fl-score

support

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

0.38 0.56 0.28 0.84 0.54 0.80 0.72 0.60 0.92 0.71 0.66 0.42 0.70 0.50 0.51 0.85 0.66 0.65 0.80 0.59 0.66 0.84 0.54 0.55 0.73 0.59 0.55 0.71 0.91 0.55 0.67 0.79 0.64 0.64 0.76 0.75 0.78 0.70 0.80

0.75 0.83 0.71 0.71 0.72 0.90 0.67 0.73 0.70 0.86 0.74 0.65 0.77 0.64 0.66 0.76 0.85 0.82 0.67 0.61 0.91 0.74 0.77 0.64 0.91 0.65 0.67 0.75 0.83 0.81 0.71 0.86 0.55 0.69 0.63 0.66 0.69 0.82 0.64

0.50 0.67 0.40 0.77 0.62 0.85 0.69 0.66 0.80 0.78 0.70 0.51 0.73 0.56 0.58 0.80 0.75 0.73 0.73 0.60 0.76 0.78 0.63 0.59 0.81 0.62 0.60 0.73 0.87 0.65 0.69 0.83 0.59 0.67 0.69 0.70 0.74 0.76 0.71

155 209 132 144 117 137 79 66 47 103 53 106 78 85 59 95 89 60 60 44 67 42 43 28 58 60 63 55 35 59 49 44 33 52 54 32 36 55 25

micro avg macro avg weighted avg

0.59 0.66 0.64

0.75 0.74 0.75

0.66 0.69 0.68

2808 2808 2808

34 IoT and Big Data Analytics, Vol. 1

Sangeeta et al.

The proposed video compression-cum-classification model is quite simpler, efficient, and fast. The model is trained, tested, and evaluated on a particular dataset. In the future, its performance can also be measured over varying datasets. The videos are of small size, the proposed model can be further enhanced or scaled for larger or long videos. The performance can also be improved using various optimization techniques also. More work can be done on precision improvement. CONCLUSION The proposed model represents a pure deep learning based video compressioncum-classification network that directly utilizes the compressed representation of the videos. It makes the detection or recognition task faster by avoiding the decompressing overhead. The compression removes the redundant and insignificant information among the subsequent video frames. It makes the data more robust with lessened dimensionality and enhanced relevance, eventually leading to effective computation and improved detection performance. The results demonstrate that the model exhibits improved accuracy with compressed data. In short, this proposed method is quite simpler, fast, and accurate and put forth new directions for further explorations in this avenue. CONSENT FOR PUBLICATION Not applicable. CONFLICT OF INTEREST The authors declare no conflict of interest, financial or otherwise. ACKNOWLEDGEMENT The first author acknowledges UGC (University Grants Commission), India for providing JRF (Junior Research Fellowship) to carry out this work. REFERENCES [1]

C. V. networking Index. Forecast and methodology. 2016-2021, white paper. San Jose, CA, USA, 2016.

[2]

G.A. Sigurdsson, G. Varol, X. Wang, A. Farhadi, I. Laptev, and A. Gupta, "Hollywood in homes: Crowdsourcing data collection for activity understanding", European Conference on Computer Vision. p. 510–526, 2016. [http://dx.doi.org/10.1007/978-3-319-46448-0_31]

[3]

M. Mathieu, C. Couprie, and Y. LeCun, "Deep multi-scale video prediction beyond mean square error", International Conference on Learning Representations, 2016.

[4]

M. Pollefeys, D. Nistér, J.M. Frahm, A. Akbarzadeh, P. Mordohai, B. Clipp, C. Engels, D. Gallup, S.J. Kim, P. Merrell, C. Salmi, S. Sinha, B. Talton, L. Wang, Q. Yang, H. Stewénius, R. Yang, G. Welch,

Compressed Video-Based

IoT and Big Data Analytics, Vol. 1 35

and H. Towles, "Detailed real-time urban 3d reconstruction from video", Int. J. Comput. Vis., vol. 78, no. 2-3, pp. 143-167, 2008. [http://dx.doi.org/10.1007/s11263-007-0086-4] [5]

A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei, "Large-scale video classification with convolutional neural networks", Conference on Computer Vision and Pattern Recognition, 2014. [http://dx.doi.org/10.1109/CVPR.2014.223]

[6]

H. Wang, and C. Schmid, "Action recognition with improved trajectories", International Conference on Computer Vision, 2013.

[7]

D. Tran, J. Ray, Z. Shou, S-F. Chang, and M. Paluri, "ConvNet architecture search for spatiotemporal feature learning", arXiv, p. 1708.05038, 2017. [http://dx.doi.org/10.48550/arXiv.1708.05038]

[8]

J. Carreira, and A. Zisserman, "Quo vadis, “Action recognition? A new model and the kinetics dataset", Conference on Computer Vision and Pattern Recognition, 2017. [http://dx.doi.org/10.1109/CVPR.2017.502]

[9]

N. Dalal, and B. Triggs, "Histograms of oriented gradients for human detection", Conference on Computer Vision and Pattern Recognition, 2005. [http://dx.doi.org/10.1109/CVPR.2005.177]

[10]

I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld, "Learning realistic human actions from movies", Conference on Computer Vision and Pattern Recognition, 2008.

[11]

X. Peng, C. Zou, Y. Qiao, and Q. Peng, "Action recognition with stacked fisher vectors", European Conference on Computer Vision. pp 581–595, 2014.

[12]

H. Wang, A. Kläser, C. Schmid, and C.L. Liu, "Dense trajectories and motion boundary descriptors for action recognition", Int. J. Comput. Vis., vol. 103, no. 1, pp. 60-79, 2013. [http://dx.doi.org/10.1007/s11263-012-0594-8]

[13]

K. Simonyan, and A. Zisserman, "Two-stream convolutional networks for action recognition in videos", Conference on Neural Information Processing Systems, 2014.

[14]

L. Wang, Y. Xiong, Z. Wang, Y. Qiao, D. Lin, X. Tang, and L. Van Gool, "Temporal segment networks: Towards good practices for deep action recognition", European Conference in Computer Vision, 2016. [http://dx.doi.org/10.1007/978-3-319-46484-8_2]

[15]

C-Y. Ma, M-H. Chen, Z. Kira, and G. AlRegib, "TS-LSTM and temporal-inception: Exploiting spatiotemporal dynamics for activity recognition", arXiv, p. 1703.10667, 2017. [http://dx.doi.org/10.48550/arXiv.1703.10667]

[16]

J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell, "Long-term recurrent convolutional networks for visual recognition and description", Conference on Computer Vision and Pattern Recognition, 2015. [http://dx.doi.org/10.1109/CVPR.2015.7298878]

[17]

J. Yue-Hei Ng, M. Hausknecht, S. Vijayanarasimhan, O. Vinyals, R. Monga, and G. Toderici, "Beyond short snippets: Deep networks for video classification", Conference on Computer Vision and Pattern Recognition. pp. 4694-4702, 2015. [http://dx.doi.org/10.1109/CVPR.2015.7299101]

[18]

R. Girdhar, D. Ramanan, A. Gupta, J. Sivic, and B. Russell, "ActionVLAD: Learning spatio-temporal aggregation for action classification", Conference on Computer Vision and Pattern Recognition. p. 3165-3174, 2017. [http://dx.doi.org/10.1109/CVPR.2017.337]

[19]

D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, "Learning spatiotemporal features with 3d convolutional networks", International Conference on Computer Vision. Vol. 1, P. 4489-4497,

36 IoT and Big Data Analytics, Vol. 1

Sangeeta et al.

2015. [http://dx.doi.org/10.1109/ICCV.2015.510] [20]

C. Feichtenhofer, A. Pinz, and A. Zisserman, "Convolutional two-stream network fusion for video action recognition", Conference on Computer Vision and Pattern Recognition. p. 1933-1941, 2016. [http://dx.doi.org/10.1109/CVPR.2016.213]

[21]

I.E. Richardson, Video codec design: Developing image and video compression systems. John Wiley & Sons, 2002. [http://dx.doi.org/10.1002/0470847832]

[22]

V. Kantorov, and I. Laptev, "Efficient feature extraction, encoding and classification for action recognition", Conference on Computer Vision and Pattern Recognition, 2014. [http://dx.doi.org/10.1109/CVPR.2014.332]

[23]

O. Sukmarg, and K.R. Rao, "Fast object detection and segmentation in MPEG compressed domain", TENCON Proceedings, Intelligent Systems and Technologies for the New Millennium, 2000. [http://dx.doi.org/10.1109/TENCON.2000.892290]

[24]

B.U. Töreyin, A.E. Çetin, A. Aksay, and M.B. Akhan, "Moving object detection in wavelet compressed video", Signal Process. Image Commun., vol. 20, no. 3, pp. 255-264, 2005. [http://dx.doi.org/10.1016/j.image.2004.12.002]

[25]

Y. Boon-Lock, and L. Bede, "Rapid scene analysis on compressed video", IEEE Trans. Circ. Syst. Video Tech., vol. 5, no. 6, pp. 533-544, 1995. [http://dx.doi.org/10.1109/76.475896]

IoT and Big Data Analytics, 2023, Vol. 1, 37-53

37

CHAPTER 3

Object Detection and Tracking: Exploring a Deep Learning Approach and Other Techniques Samuel Oluyemi Owoeye1,*, Folasade Durodola1 and Jethro Odeyemi1 1

Department of Mechatronics Engineering, Federal University of Agriculture, Abeokuta, Nigeria Abstract: Object detection and tracking have a wide range of uses, for example, in security and surveillance systems to deter and investigate crimes, for traffic monitoring, and for communication through video sharing. In a smart city, data is continuously being obtained through various means. In terms of video data, the data collected through cameras and other digital devices need to be analyzed to derive useful information from it. Hence, the concept of object detection and tracking comes into play. This chapter looks into developing various frameworks for object detection and tracking in the context of video data. We will be working with a database of 1,939 pencil images. These images will be used to train a neural network that performs image classification tasks. Various object detection methods are implemented such as Convolution Neural Network (CNN) image classification, Canny edge detection, object detection using the Haar Cascade classifier, and background subtraction. The experiments are carried out with Python programming language, TensorFlow, and OpenCV library.

Keywords: CNN , Canny edge , Haar cascade , Image processing , Neural network , Open CV , Object detection , Python . INTRODUCTION Video analytics is of high importance in today’s world. By analyzing visual data, we attempt to detect, and track objects of particular interest, and interpret the action of the object. For instance, in the area of surveillance and security, video analytics can be used to monitor each person entering a store or designated area which can help in the prevention of crimes or criminal investigation. Visual analytics can also be applied in traffic management, automated parking systems [1], pedestrian detection, optical character recognition [2], vehicle counting [3], and several other areas. Video analytics brings about new visual data collection, Corresponding author Samuel Oluyemi Owoeye: Department of Mechatronics Engineering, Federal University of Agriculture, Abeokuta, Nigeria; E-mail: [email protected]

*

Abhishek Singh Rathore, Surendra Rahamatkar, Syed Imran Ali, Ramgopal Kashyap & Nand Kishore Sharma (Eds.) All rights reserved-© 2023 Bentham Science Publishers

38 IoT and Big Data Analytics, Vol. 1

Owoeye et al.

storage, and analysis methods. In large environments, capturing data and subsequently analyzing it can be a challenge. For instance, a camera can capture images of cars every second at a traffic light. Now imagine a human physically monitoring this data every second around the clock. This is not feasible, hence the need for modern techniques of video analysis. Object detection and tracking is a field of computer vision that aims at detecting occurrences of a semantic object with a given label in digital videos or images, for example, detection of human’s facial features such as eyes, nose, and mouth, detection of houses, cups, bridges, or road lanes. This field is ever-growing, and more technologies are being developed to help improve the accuracy by which objects are detected. Object detection for visual data essentially works by initializing a tracking process which is then continually applied to each video frame. A common approach to object detection is using a deep-learning neural network. This concept will be explained in-depth further in the chapter, but basically, deep learning mimics the natural way by which humans assimilate and gain knowledge. The most common neural network used when dealing with visual data is the convolution neural network (CNN). This chapter discusses fundamental aspects of object detection using the convolutional neural network and other image processing techniques such as the Haar Cascade classifier, Canny edge detection as well as the background subtraction method of image tracking. The objective of this chapter is to develop a framework for object detection and tracking using deep learning and various image-processing techniques. The specific aims are; a. Developing a convolutional neural network to perform image classification tasks in a continuous stream of data. b. Using Haar Cascade classifier to track various human facial features. c. Using Canny edge detection algorithm for object recognition. d. Image tracking using background subtraction techniques. Porikli & Yilmaz [4], in 2012, provided research that discussed major developments in the detention and tracking of objects. In their research, several methods of object detection and tracking were discussed such as matrix decomposition where images were re-vectorized and used in background modeling. Karmakar et al. [5] developed a tracking system for dynamic objects that are based on vision guidance, using the guidance laws approach of the

Object Detection and Tracking

IoT and Big Data Analytics, Vol. 1 39

rendezvous cone. The system made use of a point estimator for localizing targets. The concept of background subtraction was explored by Wren et al. [6] who proposed to model each pixel with a single Gaussian distribution. However, using just one pixel was not effective for dynamic screens because multiple points may be seen as a pixel due to repeated object motions. In a study [7], a system was described by which a single human can be detected within a crowded environment. The method made use of local appearance structures with their geometrical relations using the top-down segmentation approach that is determined by the probability of each pixel occurring. OBJECT DETECTION AND TRACKING WITH CONVOLUTIONAL NEURAL NETWORK (CNN) AND IMAGE PROCESSING Building a deep learning model capable of analyzing picture input and classifying it into relevant classes can be used to discover objects. CNN is the most common neural network type used for image classification problems in deep learning. Convolutional neural networks (Fig. 1) are examples of artificial neural networks that can take in an image as input and assign learnable weights and biases to distinct aspects/objects in the picture to distinguish one from the other.

Fig. (1). CNN algorithm to classify handwritten digits [8].

CNN is highly efficient in dealing with image data than other neural network models. For example, trying to feed an image to an Artificial Neural Network can be very tasking for several reasons. ANN requires a large number of parameters (over 10,000 for very small images), you’ll also end up losing all 2D information when you flatten out the image, therefore the model will be far less accurate than if done with a CNN. In other models to solve these issues with ANN, CNN makes

40 IoT and Big Data Analytics, Vol. 1

Owoeye et al.

use of a convolution layer created when multiple image filters are used on the source image. This convolution layer aids the model in automatically figuring out the best filter values. Unlike ANN, where all neurons are fully connected, CNN helps to reduce parameters by focusing on local connections only. In addition to the convolutional layers, CNN employs an extra layer known as a pooling layer. This layer is important because when dealing with colored images and lots of filters, the number of parameters could be too much for the system to process. The pooling layer works by accepting an input convolution layer and then subsampling by picking maximum values with each stride. The pooling layer helps the algorithm reduce the number of parameters by dropping lots of information. Another layer with works with CNN is the Dropout layer. During training, units are randomly dropped to prevent the algorithm from gaining too much information, hence avoiding overfitting. EXAMPLE OF CNN MODEL ARCHITECTURES FOR OBJECT DETECTION R-CNN In the paper [9], an algorithm that combines regions proposals with convolutional neural networks, named Regions with CNN (R-CNN) feature is described. This object identification system combines two approaches: bottom-up region proposal using convolutional neural networks with a high capacity to locate and segment objects, and also supervising the pre-training process for an auxiliary task when labeled data is limited. The method works by taking an input image and extracting roughly 2000 bottomup recommendations. It computes CNN features for each proposal, then classifies each region using class-defined linear support vector machines. With PASCAL VOC 2010, the method gets a mean average accuracy of 53.7 percent [9]. Limitations of R-CNN a. Object detection is slow because, for each object proposal, a ConvNet forward pass is performed. b. Because of the usage of VGG16, which runs quite deep, training is costly in terms of both space and time. Fast R-CNN As an update to the preceding R-CNN, Ross Girshick created a Fast R-CNN for

Object Detection and Tracking

IoT and Big Data Analytics, Vol. 1 41

detecting objects [10]. Fast R-CNN aims to increase detection accuracy while improving training and testing speed. An input picture with several regions of interest (RoI) is fed into a fully connected convolutional network, to create a convolutional feature map, each RoI is treated using maximum pooling layers. The softmax activation function is then used to extract the RoI feature vector for each feature map [10]. Advantages of Fast R-CNN a. It has a higher mean average precision than R-CNN. b. Less disk storage is required. c. The training is done in a single stage with a multi-task loss. Faster R-CNN Regional Proposal Network (RPN) was added to the design, in which the full image of the convolutional features is shared with the network for detection and gives a costless proposal region [11]. The Faster R-CNN is made up of two segments: a deep convolutional network that proposes areas and a rapid R-CNN detector that utilizes those regions. The regional proposal network (RPN) accepts images as inputs and generates rectangular item proposals as output. Mask R-CNN This architecture as presented in this paper [12], is an extension of the earlier described Faster R-CNN. class box

RolAlign conv

Fig. (2). Mask R-CNN architecture [12].

conv

42 IoT and Big Data Analytics, Vol. 1

Owoeye et al.

This model, as illustrated in Fig. (2), adds a segmentation mask prediction to each region of interest, resulting in two outputs: a predicted class label and a bounding box. R-FCN Like the R-CNN and the Fast R-CNN, a Region-based Fully Convolutional Network (R-FCN) [13] is a region-based detection, as shown in Fig. (3). But unlike the previous models, which use an expensive per-region subnetwork, this one is entirely convolutional, with almost all computation shared across the whole picture. It produces a better outcome than the Fast R-CNN.

Fig. (3). R-FCN architecture [13].

Single Shot Detector (SSD) It predicts items in photos using a single deep neural network, as the name suggests [14]. Advantages of SSD 1. Easy to train. 2. SSD architecture eliminates the need for proposal generation and connects all computations to a single network. 3. It is much faster because it makes use of a combined network for both training and inference.

Object Detection and Tracking

IoT and Big Data Analytics, Vol. 1 43

You Only Look Once (YOLO) This architecture [15] describes a novel method for detecting objects. Rather than using classifiers, the YOLO architecture (Fig. 4) treats object recognition as a regression issue with spatially separated bounding boxes and associated class probabilities. The base model processes pictures at 45 frames per second, whereas the network's latest version, Fast YOLO, processes images at 155 frames per second.

Fig. (4). YOLO architecture [15].

DEVELOPMENT OF THE NEURAL NETWORK ARCHITECTURE To build our custom image classifier using a convolutional network, we collected data that was used to form a database consisting of 1,939 images of a pencil. The neural network aimed to be able to detect whether or not a pencil is present in a stream of video data. The architecture which consists of five convolutional layers is summarized by the model shown in Table 1. Table 1. Model Summary. Layers

Features

First convolution layer

Size of filter: (3,3), Filters: 32

Second convolution layer

Size of filter: (3, 3), Filters: 64.

Third convolution layer

Size of filter: (3, 3), Filters: 64.

Pooling layer

Size of pool: (2,2)

Fourth convolution layer

Size of filter: (3, 3), Filters: 64.

Fifth convolution layer

Size of filter: (3, 3), Filters: 64.

Pooling layer

Size of pool: (2,2)

44 IoT and Big Data Analytics, Vol. 1

Owoeye et al.

(Table ) cont.....

Layers

Features

Flatten

2D to 1D array

Three dense layers

128,64,1

Dropout layers

0.2

Hyperparameter Values For Each Model Layer First Convolutional Layer A 2D convolutional layer reads input images in shapes 200 by 200 pixels. Hyperparameters for this layer were set as follows: ● ●

● ●

32 filters – representing the dimensionality of the output space. (3,3) filter size – representing the height and width of the 2D convolutional window. 200 by 200 input shape. Rectified Linear Unit (ReLU) activation function is described by the equation below [16]. (1)

Second, Third, Fourth, and Fifth Convolutional Layers These have the same hyperparameters as the first convolutional layer but with 64 filters. i. Three pooling layers: The layers are set with a 2 by 2 pool size after each convolutional layer. This layer helps prevent overfitting by providing an abstracted form of representation. It calculates the maximum values in each patch of each feature map. ii. Flatten layer: Flattens the input by converting the data to a 1-D array. iii. Three dense layers: These are fully connected layers. The first is set with 128 nodes which connect the output to a hidden layer. The second has 64 nodes and the third has just one since it is a binary classification task. iv. Dropout layer: Helps to avoid overfitting the model during training by turning off 0.2 (20%) of the neutrons.

Object Detection and Tracking

IoT and Big Data Analytics, Vol. 1 45

The prediction results of this mode (Fig. 5) showed an initial accuracy of 78%. After hyperparameters tuning and data augmentation, we had an accuracy of 85% which was rather impressive.

Fig. (5). Training and Validation Loss.

The custom classifier built was loaded through the OpenCV library for real-time analytics. Figs. (6 and 7) show the original and the output at a specific frame of a video stream respectively.

Fig. (6). Original frame.

46 IoT and Big Data Analytics, Vol. 1

Owoeye et al.

Fig. (7). Output frame.

OBJECT DETECTION WITH HAAR CASCADE CLASSIFIER The Haar cascade classifiers are an effective way of object detection and tracking. A Haar cascade classifier is simply a machine-learning program that identifies objects in images or videos. Paper [16] describes it as a machine learning approach for visual detection with the capability to process images or videos extremely rapidly with high accuracy rates. The flow chart for the Haar cascade classifier is shown in Fig. (8). Cascade Classifier Stage one

trainCascadeObjectDetector function

fx

Stage two Stage three

vision.CascadeObjectDetector System Object

stored as an XML file

Fig. (8). Cascade classifiers flow chart [17].

Each stage of the classifier is a collection of weak learners. These weak learners are trained using boasted training which allows for highly accurate classifiers and provides either positive (whether the object is found) or negative (whether the object is not found) output. This classifier works in three steps:

Object Detection and Tracking

IoT and Big Data Analytics, Vol. 1 47

Haar Features Calculation Haar cascade classifiers make use of simple features rather than actual pixels to classify images. This is because features (see Fig. (9)) help encode ad-Hoc domain knowledge which is difficult to learn using limited training data. Feature calculations are done by adding the pixel densities in each region and calculating the differences between each sum.

Fig. (9). Examples of Haar features [16].

Integral Images Creating integral images allows for fast feature evaluation. It does so by creating array references for small rectangles rather than computing at every pixel. Adaboost Training There are lots of features in every image. In order to decide the best features to evaluate, a training method called Adaboost is applied. The working principle of Adaboost training is shown in Fig. (10). To construct strong classifiers, it uses a mixture of weak classifiers. A window is moved over the input picture to calculate the Haar features for each subsection of weak learners.

48 IoT and Big Data Analytics, Vol. 1

Owoeye et al.

Fig. (10). How Adaboost training works [18].

Haar Cascade Classifier Experiments for Face and Eyes Detection For this experiment, the author made use of pre-trained faces and eyes. Haar Cascade classifiers are saved as an extensible markup language (XML) file. These pre-trained machine learning models were loaded through the OpenCV library and fed to an image frame. The output was seen in the form of green squares showing the bounding boxes of the eyes region while the blue square represents the face bounding box. CANNY EDGE DETECTION Another method of object detection is what is known as Canny edge detection. A multi-stage method is used by the canny edge detector to identify ranges of edges in an image [19]. The algorithm works in five steps: Noise Reduction Canny edge detection results are highly susceptible to noise. By applying a Gaussian blur to the image, we can smoothen it out. This is represented by the equation below. (2)

Object Detection and Tracking

IoT and Big Data Analytics, Vol. 1 49

Gradient Calculation Edge corresponds to sharp changes in pixel values. Gradient calculation helps detect these changes by calculating the gradient of the image at different frames. Non-Maximum Suppression This is performed to thin out the edges. The algorithm loops through all the points on the gradient intensity matrices and finds the pixels with the maximum values. Double Threshold This step is performed to identify three types of pixels: strong, weak, and nonrelevant. Strong pixels have the highest intensities, weak pixels have lower intensities and the other pixels are the non-relevant pixels. Edge Tracking by Hysteresis Finally, weak pixels are transformed into strong ones, if and only if strong pixels are located around the weak pixels. Figs. (11 and 12), respectively show the original frame and resulting image using the Canny edge detection algorithm.

Fig. (11). Original frame.

50 IoT and Big Data Analytics, Vol. 1

Owoeye et al.

Fig. (12). Coins detected after applying the Canny edge detection algorithm.

BACKGROUND REMOVAL Background removal or subtraction (Fig. 13) is a technique mostly used in video analytics whereby the background of the video is modeled and that model is used to detect moving objects.

Fig. (13). Background subtraction process [20].

Object Detection and Tracking

IoT and Big Data Analytics, Vol. 1 51

Background Generation This processes the input frames and provides the background image to be modeled. Background Modeling This defines the model for background representation [21]. Background Model Update This handles the changes which occur over time. Foreground Detection This divides each pixel into sets of background or foreground. This process outputs a binary mask. MOTION DETECTION USING KNN BACKGROUND SUBTRACTION The KNN background subtraction method was implemented using the inbuilt OpenCV library [22]. KNN is a non-parametric classification machine learning technique that works by finding the distances between a specific query and the test data. It then selects the number of examples (K) closest to the query. KNN background subtraction method functions in two steps: ● ●

It initializes the background. It uses the KNN algorithm to track changes in each frame by capturing the most recent information in a given frame and continuously updating the background.

CONCLUSION Video analytics has a lot of applications in a smart city. In this chapter, the author has discussed several methods of object detection and tracking from video data. The author built a custom classifier using a convolutional neural network for object classification tasks which yielded a very high accuracy. Experiments were also performed using Haar cascade classifiers and other image processing techniques such as background separation and canny edge detection algorithm which gave excellent results. This project's future goals might include enhanced object identification and tracking algorithms, both in terms of accuracy and speed of execution.

52 IoT and Big Data Analytics, Vol. 1

Owoeye et al.

CONSENT FOR PUBLICATION Not applicable. CONFLICT OF INTEREST The authors declare no conflict of interest, financial or otherwise. ACKNOWLEDGEMENT Declared none. REFERENCES [1]

H.G. Jung, D.S. Kim, P.J. Yoon, and J. Kim, Structure Analysis Based Parking Slot Marking Recognition for Semi-automatic Parking System: Structural, Syntactic, and Statistical Pattern Recognition. Springer, 2006. [http://dx.doi.org/10.1007/11815921_42]

[2]

A. Permaloff, and C. Grafton, "Optical Character Recognition", PS Polit. Sci. Polit., vol. 25, no. 3, pp. 523-531, 1992. [http://dx.doi.org/10.1017/S1049096500036040]

[3]

A. Alsanabani, M. Ahmed, and A. AL-Smadi, "Vehicle counting using detecting-tracking combinations: A comparative analysis", In: 4th International Conference on Video and Image Processing, 2020, pp. 48-54.

[4]

F. Porikli, and A. Yilmaz, Object detection and tracking in Video Analytics for Business Intelligence. vol. 409. Springer: Berlin, Heidelberg, 2012, pp. 3-41. [http://dx.doi.org/10.1007/978-3-642-28598-1_1]

[5]

P. Karmakar, K. Dhal, W. J. Beksi, and A. Chakravarthy, "Vision-Based Guidance for Tracking Dynamic Objects", 2021 International Conference on Unmanned Aircraft Systems (ICUAS), 2021.Athens, Greece [http://dx.doi.org/10.1109/ICUAS51884.2021.9476712]

[6]

C. Wren, A. Azarbayejani, T. Darell, and A. Pentland, "Pfinder: Real-time tracking of the human body", In: IEEE Transactions on Pattern Analysis and Machine Intelligence. vol. 19 p. 39-40, 1997.

[7]

B. Leibe, E. Seemann, and B. Schiele, "Pedestrian detection in crowded scenes", IEEE Conference on Computer Vision and Pattern Recognition, 2005.

[8]

S. Saha, "A Comprehensive Guide to Convolutional Neural Networks – the ELI5 way", Available from: https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-theeli5-way-3bd2b1164a53.

[9]

R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation", 2014 IEEE Conference on Computer Vision and Pattern Recognition Columbus, OH, USA p. 580-587, 2014. [http://dx.doi.org/10.1109/CVPR.2014.81]

[10]

R. Girshick, "Fast R-CNN", 2015 IEEE International Conference on Computer Vision (ICCV) Santiago, Chile vol. 1, P. 1440-1448, 2015. [http://dx.doi.org/10.1109/ICCV.2015.169]

[11]

S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks", IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137-1149, 2017. [http://dx.doi.org/10.1109/TPAMI.2016.2577031] [PMID: 27295650]

Object Detection and Tracking

IoT and Big Data Analytics, Vol. 1 53

[12]

H. Kaiming, G. Gkioxari, P. Dolla, and R. Girshick, "Mask R-CNN", 2017 IEEE International Conference on Computer Vision (ICCV). 2018 Venice, Italy pp. 2961-2969, 2017.

[13]

J. Dai, Y. Li, H. Kaiming, and J. Sun, R-FCN: Object detection via region-based fully convolutional networks., arXiv, 2016.

[14]

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, "SSD: Single Shot MultiBox Detector", In: Computer Vision – ECCV 2016. ECCV 2016. Lecture Notes in Computer Science Springer: Cham. pp 21–37, 2016. [http://dx.doi.org/10.1007/978-3-319-46448-0_2]

[15]

J. Redmon, S. Divvala, and R. Girshick, "You only look once: unified, real-time object detection", 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas, NV, USA vol 1, P. 779-788, 2016. [http://dx.doi.org/10.1109/CVPR.2016.91]

[16]

V. Nair, and G.E. Hinton, "Rectified Linear Units Improve Restricted Boltzmann Machines", International Conference on Machine Learning, pp. 807-814, 2010.

[17]

P. Viola, and M. Jones, "Rapid Object Detection using a Boosted Cascade of Simple Features", Conference on Computer Vision and Pattern Recognition. vol. 2, P. 511, 2001. [http://dx.doi.org/10.1109/CVPR.2001.990517]

[18]

A. Mittal, "Haar cascades explained", Available from: https://medium.com/analytics-vidhya/haa-cascades-explained-38210e57970d

[19]

Packt, "Extending machine learning algorithms- AdaBoost classifier", Available from: https://m.youtube.com/watch?v=BoGNyWW9-mE

[20]

Wikipedia, "Canny edge detector", Available from: https://en.m.wikipedia.org/wiki/Canny_edge_ detector

[21]

A. Murzova, "Background subtraction with OpenCV and BGS libraries", LearnOpenCV, 2021. Available from: https://learnopencv.com/background-subtraction-with-opencv-and-bgs-libraries/

[22]

C.V. Open, "How to use background subtraction methods", Available from: https://docs.opencv. org/4.x/d1/dc5/tutorial_background_subtraction.html

54

IoT and Big Data Analytics, 2023, Vol. 1, 54-71

CHAPTER 4

Introduction and Overview of Key Enabling Technologies for Smart Cities and Homes Karthika Dhanasekar1,* and Deepika Muni Krishna Reddy2 1 2

Department of Computer Science, SDNB Vaishnav College for Women, Chennai, India Department of Computer Science, Crescent University, Chennai, India Abstract: The Internet has come a long way in the previous three decades. It began as a content-sharing platform for early websites, portals, and search engines. With the globalization of Internet users, the importance and diversity of Internet material have increased; the Internet is increasingly connecting people rather than just connecting information. Other services, such as voice over Internet Protocol (VoIP) and video chat, as well as social networks, have enabled the social web. The end-user has changed as a result of the shift from information consumers to contributors and producers. This trend has accelerated with the introduction of smartphones and Internet of Things (IoT) devices. Smartphones have ushered in a new era of communication, in addition to data. End-users may access information from wherever they need to connect with their environment for basic everyday activities thanks to a new type of communication. Cabs may be summoned in large cities by pressing a button on a mobile device and using an app like Uber. The Internet connects things, dwellings, cities, and everything else that can be linked, with early information associated with individuals and later users engaging with other users via the social Web. As a result, a smart platform with a wide range of smart services emerges. This chapter examines the growing trend of connecting things and the issues they provide in the context of smart homes and cities, as well as a summary of key technological capabilities.

Keywords: Internet of things (IoT), ICT, Information and communications technology (ICT), SCDs (smart city departments), Smart homes, Smart cities, Smart governance, Smart economy, Smart obility, Smart grid, Security, Wireless power transfer (WPT). INTRODUCTION According to recent estimates, half of the world's population now lives in or near cities. These cities account for three-quarters of global energy consumption and greenhouse gas emissions. By 2050, the world's population will have increased by Corresponding author Karthika Dhanasekar: Department of Computer Science, SDNB Vaishnav College for Women, Chennai, India; E-mail: [email protected]

*

Abhishek Singh Rathore, Surendra Rahamatkar, Syed Imran Ali, Ramgopal Kashyap & Nand Kishore Sharma (Eds.) All rights reserved-© 2023 Bentham Science Publishers

Introduction and Overview

IoT and Big Data Analytics, Vol. 1 55

30%, with cities accounting for 70% of the total population. Town planners and urban architects have sought to construct a vision for the future city based on the concept of a smart city [1]. Acceptance of these new concepts will be determined by the timely and successful delivery of a wide range of services. The way forward is to take advantage of the current surge in networked IT. Smart cities are aided in their expansion by technology breakthroughs such as the Internet of Things (IoT), public Wi-Fi access, all-around mobile coverage, 4G and 5G networks, and smartphones. Individuals and smart cities are bridging the digital divide using IoT and smartphones. These two ubiquitous and democratized technologies will make services available to everyone. Over the next ten years, smart cities will make significant investments. The development of technologies to assist smart cities is expected to cost roughly $100 billion. By 2020, most of these technologies will be commonplace. Aside from smart cities, the notion of smart housing is gaining traction. ICT is used in smart houses to take home and living to a new level. Intelligent dwellings make use of ICT as well. Clever gadgets link houses to the internet and digital services, which range from basic utility information like weather predictions to intelligent algorithms that manage and optimize residential energy usage [2]. There are no two intelligent grids that are the same. The first is that a significant effort must be made to build a smart grid environment in which utilities may participate without conflict; the second is that common features must be found for this sharing of common practices and standards to be practicable. Although it is still in its infancy, the intelligent grid concept is evolving [1 - 4]. This immaturity is caused in part by a lack of “already there” strong corporate references, and in part by the fact that utility conditions change so quickly that, while consumption scenarios may appear to be the same, an implementation may differ substantially. A reference to the modernization of the structure, as well as significant advances in electrical system sensors and controllers, may be identified as common components of all smart grid concepts. These components, which are constantly present in the grid, were built during network installation, monitoring, and local operation, as well as from remote, centralized control centers. Following that, these components were incorporated into automated systems capable of making difficult judgments based on large amounts of data. Even though smart grids require new infrastructure such as grids and algorithms (intelligence), ICTs [5 - 10] and this new infrastructure, telecommunications serve as the glue that holds it all together.

56 IoT and Big Data Analytics, Vol. 1

Dhanasekar and Krishna Reddy

Telecommunication for utilities, on the other hand, is not a novel notion. Many telecommunication enterprises (traffic rights, pipelines, poles, fiber optics, telecommunications services, and so on) are now built on the essential infrastructure provided by these companies, and a few are even utility spin-offs. Telecommunication networks were built due to a lack of public transportation networks, or specially those that met critical commercial criteria. Even though today's telecommunications networks are a valuable tool for utilities dealing with operations in hundreds of thousands of power distribution stations, earlier communication standards are unlikely to be cost-effective. The challenge becomes even more apparent when telecommunication coverage is not provided consistently and affordably by a single technology, but rather by a situational combination of commercial and governmental telecommunication solutions. In both cases, the challenge will be to seamlessly integrate a wide range of technologies over several important networks. Global life expectancy is increasing at an alarming rate, especially in developed and newly industrialized countries, with most projections suggesting that this increase will approach 10 million people in the next ten years or so. Seniors are typically cared for in the hospitals or at home, where they are closely monitored. To satisfy the needs of this population, intelligent homes and cities might be employed efficiently and cost-effectively. ●

●

●

Due to the following causes, there has been a boom in interest in intelligent homes and cities in recent years: increased global use of ICT devices and technology by consumers and businesses; green economy; growing interest in environmental conservation and cutting CO2 emissions; Economic growth has been phenomenal in populous countries like China, India, and Brazil (China and India account for 40% of the world's population). The number of people over 65 who want intelligent houses and cities to live peacefully, healthily, and affordably is rapidly increasing in many countries, particularly in Japan, Europe, and China.

TRENDS IN SMART CITIES AND HOMES Smart cities Some characteristics distinguish a city as “intelligent.” Summary reports are provided [3]: Immersive city services that leverage real-time data sensing and knowledge technologies to collect and analyze all data.

Introduction and Overview

IoT and Big Data Analytics, Vol. 1 57

Get quick and simple access to data from a variety of interconnected domains. It has not only gotten greener as people have moved further into cities, but it is also crucial for defining urban growth in the following decade. The intelligent city's supports and trend themes are depicted in Fig. (1).

Fig. (1). Smart City Trends.

Smart Mobility and Smart Traffic Management Intelligent mobility is an important part of every smart city initiative. It will be easier for city residents and visitors to move around. In addition, getting out of town is a breeze. Any transit into or out of the city is meticulously planned to ensure the general public's comfort [6]. Because of developments in real-time computer processing, analysis, and decision-making, intelligent traffic management is now possible. The data may be accessed by intelligent GPS-enabled traffic management systems, CCTV image data for important roads and intersections, GIS, weather and road conditions, historical data on road traffic, and other systems. Smart Environment The intelligent environment has self-sufficiency, environmental adaptation, and a simple human connection. A smart environment is not possible without the rapid advancement of computer technology. Devices, networks, middleware, and applications are the four pillars of the latter. Gadgets include smartphones, cell phones, boxes, and other electronic devices. They collect and transport data to cloud-based systems, which process it and provide intelligent insights to endusers. The networking kernel, also known as penetrating middleware, is used to connect to various cloud services [8]. The user may simply access the

58 IoT and Big Data Analytics, Vol. 1

Dhanasekar and Krishna Reddy

heterogeneous infrastructure thanks to the middleware. Apps created on top of this architecture give genuine value-added services to end-users via additional IoT devices at home or via smartphones. Smart Living The goal is to create an attractive place to live, work, and visit in an intelligent city. The smart city's profitability necessitates a good quality of life. Housing, culture, security, and education are all important aspects of living in a responsible city. Without these crucial components, it is impossible to implement an intelligent city project using ICT alone. One of the most pressing issues is how to grow the city while meeting people's needs for more time. Smart Economy Smart economics is a new field that studies how cities attract and compete in areas like innovation, art, culture, productivity, and, most significantly, global appeal. Smart Governance ICT plays a critical role in the development of social value in the management of intelligent cities. International organizations began to see governance as a form of the political system by the end of the 1990s. When information systems begin to play a larger role in our daily lives and municipal infrastructure, however, this traditional worldview is called into question, and in some circumstances, totally vanishes. The five pillars outlined below [9] were suggested for evaluating governance practices: 1. Openness 2. Participation 3. Accountability 4. Effectiveness 5. Coherence To properly apply these five principles, contemporary Information Systems must be used to connect with people. Fig. (2) depicts the importance of ICT in intelligent urban administration.

Introduction and Overview

IoT and Big Data Analytics, Vol. 1 59

Fig. (2). Role of ICT in Smart City Governance: Smart City House.

Fig. (2) depicts an ICT-based social value generation system. The “smart town's residence” is what it's called. Furthermore, data and networking constitute the cornerstone of the smart city project, which is comprised of three primary pillars that support effective administration, modify social organizational structures, and inform or guide people in their daily decisions. As a result, smart and efficient services such as renewable energy, fair employment, and enhanced overall quality of life are possible. Furthermore, real-time data and changes in priority management, urban planning, emergency management, budgeting, and forecasting are required for intelligent government. To ease the load, it also relies on strategic planning and enhanced health care. Finally, it ensures that data on energy generation and consumption is collected and monitored so that improved management plans may be developed. Smart People People in the city should have additional technical skills to participate in, benefit from, and develop their smart city in this context [10]. Smart Homes Households with many brains are a novel phenomenon. An intelligent house, once futuristic, will be able to accommodate a variety of services. Fig. (3) illustrates several important smart home trends.

60 IoT and Big Data Analytics, Vol. 1

Dhanasekar and Krishna Reddy

Fig. (3). Smart Home Trends.

Programmable and Zone-based Smart Thermostat Google programmable thermostats were recently bought by many start-ups, including Nest. Nest offers a fully automated thermostat that uses advanced machine learning algorithms to help you save money on energy [11]. ●

It is managed by a smartphone or web-based app for better reporting if no identified occupiers are present.

Wireless Power The concept of a Smart Home is based on a variety of gadgets and goods (IoT). This makes it extremely difficult to power them all with electrical connectors or lithium batteries. WPT (wireless power transmission) is a technology that uses magnetic fields to deliver energy across short distances. There are two forms of WPT: non-radiative and radiative. The near-field nonradiative variant is now used in consumer items such as electric toothbrushes, smart cards, radio-frequency identification (RFID), and modernizers. Because it targets small devices in tight locations, this version is more suitable for smart homes. Long-distance peer-topeer (P2P) power transfer, on the other hand, uses the radiative approach. Laser beams or microwave technology are used to support it. These, on the other hand, are not appropriate for use at home.

Introduction and Overview

IoT and Big Data Analytics, Vol. 1 61

Automatic Door Locks Without automated door locks, smart homes would be incomplete. Sensor technology may be used to identify and authenticate inhabitants, allowing doors to be locked and unlocked without the use of keys. This technology is already used to open doors in today's automobiles. Advanced Security System The necessity of security in today's modern home cannot be overstated. All of these scenarios are simple to implement with advanced security systems that rely on new technologies like image and video processing and face detection, by remotely monitoring home activities with a smartphone, receiving alerts that the home has been accessed when it should not have been accessed, and sending a live feed to local law enforcement authorities when violations are detected and confirmed. CHALLENGES IN SMART CITIES AND HOMES Security The attack surface is enormous, especially considering that the bulk of these IoT devices in their present state require security patching [12 - 15]. For such a largescale repair operation, the typical technique of dealing with configuration errors applies. There should be some severe security problems if these devices are linked. IoT Challenges Every smart home and smart city idea rely on IoT devices. Although the Internet of Things offers several benefits, it also has several drawbacks: ●

●

In terms of data, there are still various challenging hurdles to overcome, such as determining who owns the data. Who can benefit from the information and how? Which standards/formats must be adhered to? The distribution of responsibility for data among all organizations participating in the supplied services remains unknown due to the lack of entirely agreed-upon protocols and standards. It's critical to understand the client connection limits if a service is provided by several companies. In a smart home, who would the customer consider to be the primary relationship holder if we look at a thermostat? Is it better to use the thermostat provided by the firm or the one provided by the utility provider?

62 IoT and Big Data Analytics, Vol. 1

Dhanasekar and Krishna Reddy

There will be no meaningful response unless and until service providers establish specified service level agreements (SLAs). Fragmentation of Standards To advance toward a more intelligent city, local governments must develop better governance concepts and methods to address the following questions: ● ● ●

●

How will governments set smart city targets and measure their achievement? How will the data gathered from infrastructure services be shared? What are the risks of implementing smart city services? What are our options for getting rid of them? How can all parties agree on the definition of intelligence and the way it should be delivered?

Responding to these concerns necessitates multi-level participation. Data consumption, standard conceptual modeling, and interoperability are all addressed by these strategies. There haven't been a lot of innovative city criteria. The following are the key points: ●

●

PAS 180 aims to increase communication and understanding of visions, compatibility, and common organizational difficulties across a variety of stakeholders; BS ISO 27000 also contains information security best practices. This permits huge volumes of data to be protected across a variety of network boundaries. ISO/IEC 29100 addresses the problem of privacy risk management.

To provide a complete and consistent level of service, policymakers increasingly need to agree on standards. Processing big data IoT devices, sensors, RFID chips, and intelligent energy meters, among other things, will be used to provide value-added services to residents and homeowners in smart homes and communities. On the other hand, these devices generate vast amounts of data and data sets that are too big for traditional data processing methods to handle. Scalability As the number of intelligent devices in smart homes and cities grows, large-scale installations may face interoperability and scalability issues due to their numerous interconnections. This is one of the key reasons for the convergence of the norms

Introduction and Overview

IoT and Big Data Analytics, Vol. 1 63

that govern these sentient beings. One of the scaling issues is the paucity of large scale testbeds that allow for the validation of diverse idea proofs in real-world contexts. SURVEY OF MAJOR KEY ENABLING TECHNOLOGIES FOR SMART CITIES AND HOMES Internet of Things The Internet of Things (IoT) is a critical technology that will shape the digital world's future, including smart cities and intelligent worlds. It's a mesh network that sends and receives data from peers or sends and receives data from a service provider [15]. Many connected goods, such as microwaves, webcams, refrigerators, and other household appliances that employ RFID technology and cutting-edge software and sensors, are currently available. IoT devices may be identified and operated via local area network (LAN) and wide area network (WAN) networks. This allows for the creation of a wider range of items as well as more efficient connections between physical infrastructure and digital systems. Many researchers predict that by 2020, the number of IoT devices will have surpassed 10 billion. With so many IoT devices on the market, detailed IoT and digital business policies are essential. By offering cloud-based IoT services and devices, Microsoft, IBM, Cisco, Siemens, and Google have already stepped in to aid in the formulation and implementation of such rules. Smart Dust Micro-electric dust particles are minuscule particles of micro-electric dust (MEMS). These are tiny robots with the ability to detect everything from microvibrations to chemical composition. Internet of Things (IoT) objects, such as intelligent dust, are a sort of IoT item [16]. They operate on a wireless network and may be deployed in a variety of locations without generating any problems. Their sizes allow them to converse in centimetres. As a result, many intelligent dust things are required throughout wide areas. For communication and relay semi-conductivity, each comprises a laser diode and a MEMS beam reflector. Because these small particles are so light, air currents can easily transport them from one location to another. It's exceedingly difficult to detect after it's been installed, but it's much more difficult to get rid of once it's been installed. Smartphones People's lives are being more side-tracked by smart mobile gadgets. Today's smartphones include GPS, gyroscope, microphone, camera, and accelerometer processors, which together supply a wealth of raw data. Smartphones are built on

64 IoT and Big Data Analytics, Vol. 1

Dhanasekar and Krishna Reddy

the foundations of intelligent home and municipal ICT infrastructures. They are used to send and receive data, as well as to access and use digital services. Smartphones play an important part in smart cities and houses, as shown in [16 - 18]: ●

●

●

They use the sensitivity of recorded data to bridge the gap between physical and virtual reality. Your cell phones come with cloud services like Google (Android) and iCloud (Apple). Off-load servers increase the phone's storage capacity. Advances in mobile technology, such as LTE (4G) networks, may provide highspeed access virtually anywhere. It would be easier to transmit and link data between fields if this were the case.

Different programs, such as TV sets, lights, window shopping, parking doors, and security cameras, are now controlled by smartphones in an intelligent dwelling. Mobile phones may also be used to communicate with their cities, obtain realtime information, and connect to municipal and public transportation systems. Cloud Computing Cloud computing is quickly establishing itself as the de-facto platform for delivering information to customers. Today's comprehensive computing is mostly supported by smartphones and IoT devices to convert raw data into valuable information. Traditional computer concepts are no longer suitable in this situation. There are three types of cloud computing [19]: ●

●

●

SaaS: Most people utilize software as a service. It sells software to individuals and businesses on a subscription basis. Software as a service (SaaS) reduces the time it takes to deploy software by eliminating the requirement for downloads, installation, and maintenance. SaaS can be accessed via the internet, smartphone apps, or high-tech vehicles such as Tesla or BMW. PaaS: To create and run time, the environmental software platform is extensively utilized as a service. The SaaS software development, integration, and usage environment are where SaaS software is created, integrated, tested, and utilized. PaaS relieves novices and entrepreneurs from the burden of managing application servers, middleware, and other low-tech components, letting them concentrate on generating their goods and services. Cloud computing as a service (IaaS) is a critical component of the infrastructure. This sort of company provides shared or specialized hardware to many customers. Customers may request capacity-on-demand using a simple administrative interface or existing triggers (network traffic, CPU load, etc.).

Introduction and Overview

IoT and Big Data Analytics, Vol. 1 65

This is perfect for businesses looking to cut their IT costs. Storage, communication, networking, service monitoring, analysis, data, and other services are provided by the XaaS cloud services combination (Everything as a service). Smart Grid An intelligent grid is a contemporary system of ICT and digital networks that collect grid data such as generated and utilized power, as well as construct live consumer models for real-time grid optimization. The following is an overview of the intelligent grid's primary features [20]: ●

●

●

●

Load balancing in smart grid aims to improve the detection and self-remediation of electricity grid problems, reducing the intervention of service providers; Smart grid reliability aims to improve problem detection and self-treatment of electricity networks, thereby reducing intervention by maintenance employees [21, 22]; The smart grid allows the use and reuse of renewables (e.g., a smart grid may alert the smart customer that power consumption is reduced during peak periods, allowing the conserved capacity to be used to serve other top customers); The smart grid is a topic that is receiving a lot of attention and research right now. The Intelligent Grid, for example, is a crucial endeavour that incorporates measurement and dissemination tools, methodology, and standards. Many organizations throughout the world, such as the Modern Grid Initiative, Grid Wise, Grid Works, Solar Cities, and others, do additional research.

SMART CITY DATA PLANE CHALLENGES The Smart City Network boosts capacity and output while lowering latency, battery consumption, equipment prices, and dependability. For the administration of numerous devices, network density and autonomous network management frameworks are being investigated [23, 24]. The concept of connected SCDs has numerous components (Smart City Departments). Potential dangers and breakdowns would be avoided if control and preventative measures were implemented. Customer satisfaction would grow, except for the intermediate periods. Understanding how different types of equipment are connected provides for a more precise assessment of physical processes. The relationships between various forms of data may be better understood, allowing for the implementation of more effective models. The SCD network, on the other hand, has several drawbacks.

66 IoT and Big Data Analytics, Vol. 1

Dhanasekar and Krishna Reddy

Compatibility Between Smart City Devices The controller's NE (Network Element) connection would be the same as the previous communication frame. Communication between the NE and the SCD, as well as the results, are the main concerns. In the future, smart cities will be made up of an enormous number of networks of intelligent devices that generate massive amounts of raw data. Because the majority of SCDs are applicationbased, they come in a variety of shapes and sizes, as well as a focus on certain goals. These CDs require a specified layer structure at the communication level, which necessitates a specific electrical capacity [25]. The huge network load needs the deployment of additional NEs as the number of SCDs grows and interacts more often, emphasizing the relevance of the heterogeneity issue. Controllers in a typical communication architecture must identify many switches. Aside from the wide range of gadgets, the normal differences across countries and even areas are inconvenient. Simplicity The difficulty is exacerbated by the variety and dynamic nature of SCDs. Each SCD has its unique set of applications and processes because many SCDs are application and specialty devices. Because the newest switches have a difficult-t-relate control plane, these criteria are challenging to meet (their OS). Applications cannot be applied to the switches because of the hard connection. This issue emphasizes the need for management simplicity. Software-based application control is necessary to tackle the management problem. Engineers should be able to build features into NEs that allow them to administer this application-based SCD network. Mobility and Geographic Control SCDs will be used in the future to conduct a variety of tasks in intelligent cities, including perception, monitoring, and performance [26]. As a result, smart devices are dispersed throughout a large geographic area. Regularly, dispersed devices generate a large volume of data, which should be delivered simultaneously throughout these units. Due to the large number of packets, a high degree of communication is required. When many switches and connections must be shifted over a large area, these devices are employed. In a typical network, each switch decides based on local data. The switch, on the other hand, tries to send the packet through a different path, notwithstanding the possibility of a connection failure. For example, a commercial SCD network application like fleet management may require real-time location and fuel usage of the company's truck fleet. As it drives through multiple switches on its business path, the truck's smart

Introduction and Overview

IoT and Big Data Analytics, Vol. 1 67

devices must communicate continually. To accomplish such great mobility, a centralized control mechanism and a thorough image of global topological networks are required. SOFTWARE-DEFINED NETWORK-BASED SMART CITY NETWORK MANAGEMENT Future SCD network applications in intelligent cities face the following main barriers [27]: ● ● ● ●

A lack of centralized control with complete topology information; Simplicity of integration and application; Large system needs quantities; and Compatibility between SCD and NE.

Software-centered control methods must be developed to solve these challenges. The SDN is a framework for this development process that has been presented. The control and data planes are separated in SDN's logically centralized design. The Open Flow protocol is used to connect the two separate planes. Communications in Smart Grids In an electricity system densely packed with sensors and controllers, the former will offer us a large quantity of data, allowing us to respond. Consider a group of isolated islands that are unable to communicate and must make decisions only based on limited knowledge of the local situation. Let us look at the consequences of uncoordinated activity [28]. Telecommunications must be used to bring intellectual and global viewpoints to the forefront. Furthermore, remote control or automated controls are possible in situations where one technique or the other is currently more feasible owing to pragmatism, advancement, or even trust in telecommunications' capabilities. For smart grid-related telecommunication services, the following attributes will be developed: ●

●

●

Depending on the distance involved, services can be offered locally, regionally, nationally, or worldwide. Depending on the bandwidth used, services might be narrowband or broadband. The number of bits per second available on the channel will influence this factor. The service's maximum affordable latency should be established by the maximum delay that communication allows.

68 IoT and Big Data Analytics, Vol. 1 ●

Dhanasekar and Krishna Reddy

The service will be characterized in terms of latency, with the change in latency between each communication attempt being monitored.

Depending on the nature of the media or telecommunications technology employed, the service will use dedicated or shared capacity. Dedicated capacity is always used with the same (up to a predetermined limit) characteristics regardless of how many users are on the system. With technology-shared capacity, this is not the case. Depending on the type of telecommunication infrastructure supporting the service, it may also be delivered over wired or wireless (radio frequency) networks. Purpose of the Smart Grid Smart grids have a synchronized and pragmatic goal of reducing consumption in customer programs (demand-side management is an indirect source of generation) and improving network operation and maintenance efficiency (both to reduce costs and increase grid available parameters) by allowing smart devices to be integrated into the grid [29]. The ICT difficulty is to make the devices actual (electronics), access them from remote locations (contact), interact with, and share data to automate decisionmaking and action (applications). THE DIFFERENT SEGMENTS OF THE SMART GRID An electrical system is a fundamental concept. Power plants (the primary source of energy), transmission lines (for power transmission), and substations (for managing electricity voltage levels to accomplish performance — transportation losses reduction) make up the transmission system. Finally, there should be a specific reference to the distribution system's endpoint, the points of supply where energy is provided, for the reasons stated previously in this chapter [30]. All of the components have the notion of a substation. A substation is a part of any power transmission or distribution network that uses transformers to convert high voltage to medium voltage and subsequently to low voltage (or vice versa). This is a basic premise definition with a few additional features (switching transmission or distribution circuits into and out of the grid system, measuring electric power qualities, protecting from lightning and other electrical surges, etc.). The substations are divided into four categories based on their functions:

Introduction and Overview ●

●

●

IoT and Big Data Analytics, Vol. 1 69

Their job is to use proper transformers to carry electricity from power plants to the grid. These substations serve as transmission line linking nodes for a varying number of transmission lines. Their primary purpose is to reduce transmission voltage to the level required for local distribution.

CONCLUSION Smart cities and smart homes have been hot topics for more than a decade. Smart cities and smart households are enabled by cloud computing and the Internet of Things, two important building block technologies. A new strategy based on profiting from ICT breakthroughs is required to meet the growing demand for useful and trustworthy services. The terms “smart city” and “smart house” are not interchangeable. To offer digital services, the former allows devices to communicate with one another and share data. The first generation of smart home devices has arrived, addressing issues such as energy efficiency, home security, and home automation. Smart grids and smart transportation are gaining popularity in the smart city community. Obstacles must be overcome, however, by implementing standardized and compatible systems. CONSENT FOR PUBLICATION Not applicable. CONFLICT OF INTEREST The authors declare no conflict of interest, financial or otherwise. ACKNOWLEDGEMENT Declared none. REFERENCES [1]

M.S. Obaidat, "Key enabling ICT systems for smart homes and cities: the opportunities and challenges", Proceedings of the 2014 IEEE international conference on network infrastructure and digital content (IC-NIDC 2015), 2015 Beijing, China

[2]

"Siemens. Wired for an urban world", Available from: http://www.siemens.com/innovation/ en/home/pictures-of-the-future/infrastructure-and-finance/smart-cities-trends.html

[3]

"Siemens. Why cities are getting smarter", Available from: http://www.siemens.com/innovation/ en/home/pictures-of-the-future/infrastructure-and-finance/smart-cities-facts-and-forecasts.html

[4]

S. Singh, "Smart cities – a $1.5 trillion market opportunity", Available from: http://www.forbes. com/sites/sarwantsingh/2014/06/19/smart-cities-a-1-5-trillion-market-opportunity/

[5]

M. Weiser, "The computer for the 21st century", Sci. Am., vol. 265, no. 3, pp. 94-104, 1991. [http://dx.doi.org/10.1038/scientificamerican0991-94] [PMID: 1675486]

70 IoT and Big Data Analytics, Vol. 1

Dhanasekar and Krishna Reddy

[6]

"Smart environment", Wikipedia, The Free Encyclopedia. Wikimedia Foundation, Inc, 2015. Available from: https://en.wikipedia.org/wiki/Smart_environment

[7]

"Technishe Universität Wein. http://www.smart-cities.eu/model_5.html

[8]

M.C. Poloncarz, Initiatives for a smart economy, 2013. Available from: http://www2.erie.gov/environment/sites/www2.erie.gov.environment/files/uploads/pdfs/SmartEconom y%20for%20Web3.pdf

[9]

E. Ferro, B. Caroleo, M. Leo, M. Osella, and E. Pautasso, The role of ICT in smart cities governance. Conference for e-democracy and open government, 2013, p. 133.

[10]

D. Kaufmann, A. Kraay, and P. Zoido-Lobatón, "Governance matters", Finance Dev., vol. 37, no. 2, pp. 10-13, 2000.

[11]

K.A. Armstrong, "Rediscovering civil society: the European Union and the white paper on governance", Eur. Law J., vol. 8, no. 1, pp. 102-132, 2002. [http://dx.doi.org/10.1111/1468-0386.00144]

[12]

A. Caragliu, C. Del Bo, and P. Nijkamp, "Smart cities in Europe", J. Urban Technol., vol. 18, no. 2, pp. 65-82, 2011. [http://dx.doi.org/10.1080/10630732.2011.601117]

[13]

TUWEIN Team, Technishe Universität Wein, European smart cities, 2003. Available from: http://www.smart-cities.eu/model_4.html

[14]

R. Tseng, B. von Novak, S. Shevde, and K.A. Grajski, "Introduction to the alliance for wireless power loosely-coupled wireless power transfer system specification version 1.0", Proceedings of the 2013 IEEE wireless power transfer conference, WPT’13, 2013 p. 79 Perugia, Italy [http://dx.doi.org/10.1109/WPT.2013.6556887]

[15]

S. Alfino, The role of standards in smart cities, 2013. Available from: http://www.bsigroup.com/LocalFiles/en-GB/smart-cities/resources/BSI-smart-cities-report-The-Role-o f-Standards-in-Smart-Cities-UK-EN.pdf

[16]

L. Atzori, A. Iera, and G. Morabito, "The Internet of things: a survey", Comput. Netw., vol. 54, no. 15, pp. 2787-2805, 2010. [http://dx.doi.org/10.1016/j.comnet.2010.05.010]

[17]

Xiaohang Wang, ChungYau Chin, S. Hettiarachchi, Daqing Zhang, and D. Zhang, "Semantic space: an infrastructure for smart spaces", IEEE Pervasive Comput., vol. 3, no. 3, pp. 32-39, 2004. [http://dx.doi.org/10.1109/MPRV.2004.1321026]

[18]

M. Hilbert, and P. López, "The world’s technological capacity to store, communicate, and compute information", Science, vol. 332, no. 6025, pp. 60-65, 2011. [http://dx.doi.org/10.1126/science.1200970] [PMID: 21310967]

[19]

D. Soldo, A. Quarto, and V. Di Lecce, "M-DUST: an innovative low-cost smart PM sensor", Proceedings of the 2012 IEEE international, instrumentation and measurement technology conference, I2MTC’12 pp. 1823-1828, 2012.

[20]

C. Balakrishna, "Enabling technologies for smart city services and applications", Proceedings of the 6th IEEE international conference on next-generation mobile applications, services, and technologies, NGMAST’12, 2012 pp. 223-7 . [http://dx.doi.org/10.1109/NGMAST.2012.51]

[21]

K. Al-Begin, IMS: A development and deployment perspective. John Wiley & Sons Ltd: New Jersey, 2009. [http://dx.doi.org/10.1002/9780470750001]

[22]

T. Choudhury, G. Borriello, S. Consolvo, D. Haehnel, B. Harrison, B. Hemingway, J. Hightower, P.P. Klasnja, K. Koscher, A. LaMarca, J.A. Landay, L. LeGrand, J. Lester, A. Rahimi, A. Rea, and D.

European

smart

cities",

Available

from:

Introduction and Overview

IoT and Big Data Analytics, Vol. 1 71

Wyatt, "The mobile sensing platform: an embedded system for activity recognition", IEEE Pervasive Comput., vol. 7, no. 2, pp. 32-41, 2008. [http://dx.doi.org/10.1109/MPRV.2008.39] [23]

C-W. Tsai, A. Pelov, M-C. Chiang, C-S. Yang, and T-P. Hong, "A brief introduction to classification for smart grid", Proceedings of the 2013 IEEE international conference on systems, man, and cybernetics, SMC’13, p. 2905 [http://dx.doi.org/10.1109/SMC.2013.495]

[24]

Wikipedia, The Free Encyclopedia., 2015. Available from: https://en.wikipedia.org/wiki/Internet_ of_Things

[25]

"RITA Office of Research, Development, and Technology, U.S. Department of Transportation. 2012 urban mobility report released with new congestion measures", Available from: https://www.rita.dot.gov/utc/utc/sites/rita.dot.gov.utc/files/utc_spotlights/pdf/spotlight_0313.pdf

[26]

S. Moores, "Keynote speech: The challenges of making cities ‘smart’ in advanced democracies", In: IFSEC International vol. 2015. London, UK, 2015.

[27]

"Amsmarterdamcity", Available from: http://amsterdamsmartcity.com

[28]

"BCN smart city", Available from: http://smartcity.bcn.cat/en

[29]

"Birmingham smart city", Available from: http://www.birmingham.gov.uk/smartcity

[30]

"The city of Vienna, smart city Wien", Available from: https://smartcity.wien.gv.at/site/en

72

IoT and Big Data Analytics, 2023, Vol. 1, 72-84

CHAPTER 5

Intelligent Processing: Scope and Application of Smart Agriculture in Smart City Geetanjli Khambra1,2,*, Sini Shibu1, Archana Naik1 and Dileep Singh2 Department of Computer Applications, The Bhopal School of Social Sciences, Bhopal, M.P. India 2 School of Engineering and Technology, Jagran Lake City University, Bhopal, M.P. India 1

Abstract: By far, the Internet has been a major revolution in the field of communication technology as it allowed more and more devices and objects to get connected. The Internet of Things (IoT) has evolved rapidly as it can support intelligent data processing and can help in automation. IoT is gradually being used in various automation processes and also to make intelligent decisions. This paper analyses the various ways in which IoT can be used in agriculture. Intelligent system implementation in the field of agriculture can lead to a lot of advantages such as minimization of human intervention and resource wastage, maximization of profits, and efficient utilization of resources. This paper focuses on the model implementation and analysis of the implementation of intelligent systems in agriculture.

Keywords: App development, Artificial intelligence (AI), Communication, Dashboard, Intelligent processing, Internet of things (IoT), Networking, Robotics, Sensor, Smart farming, User interface. INTRODUCTION The Internet was a major revolution in the field of communication technology as it allowed people to get connected and share data easily. With the development of technology, more and more devices and objects are being connected to the Internet very rapidly. Nowadays, the Internet of Things (IoT) is gaining a lot of prominence in the technological world because it can support intelligent data processing with minimum human intervention. IoT is regarded as the next technological revolution in the information industry after the launch of the Internet. IoT is gradually being used to automate various processes, to analyze crucial parameters, and to take intelligent decisions. The volume of data generated * Corresponding author Geetanjli Khambra: Department of Computer Applications, The Bhopal School of Social Sciences, Bhopal, M.P. India; E-mail: [email protected]

Abhishek Singh Rathore, Surendra Rahamatkar, Syed Imran Ali, Ramgopal Kashyap & Nand Kishore Sharma (Eds.) All rights reserved-© 2023 Bentham Science Publishers

Intelligent Processing

IoT and Big Data Analytics, Vol. 1 73

by the things connected through the Internet is increasing exponentially with every passing day. This has resulted in an increase in intelligent processing machines that can make real-time decisions based on sensor data. The major concern for the implementation of IoT is the storage, retrieval, and analysis of huge amounts of data that are being generated through these connected devices. Another major concern is regarding the security of the data. IoT has the caliber to completely change the present Internet scenario and hence it is important to understand the underlying technologies that support IoT. INTERNET OF THINGS (IOT) The IoT is a group of intelligent machines which connect objects or things to the Internet for swapping information and providing communication via sensing tools according to a set of protocols. It achieves the goal of intelligently identifying, tracking, locating, managing, and monitoring things. As depicted in Fig. (1), the IoT integrated technologies include Software Applications, Sensing Technology, Network systems, Data Analytics, Hardware and Software Systems, and Global Positioning Systems. The Internet of Things (IoT) is a network of virtual and physical things that may be utilized in several ways to innovate and facilitate a range of useful tasks. The Internet of Things is a calculative and processing concept that draws an idea of everyday observable things being associated with the Internet and being able to communicate with other devices. The term is closely identified with RFID as the process of communication, although it may also consider other wireless and sensor technologies or QR codes. The ability to provide real-time communication between objects or things makes IoT a very desirable technology. The scope of implementing IoT is also very vast as it can be integrated into a host of objects for communication, control, and automation. The IoT is an important technological advancement because an object that can constitute itself digitally becomes more useful and important than the objects by itself. No longer does the object just relate to its user, but it gets associated with the surrounding objects, machines, and database. When so many objects or things behave in unison, they possess “ambient intelligence”. In an IoT-based environment, smart devices are used to enhance the existing processes and introduce new approaches to gather data and using it efficiently. It provides better decision-making and improved performance in the system. But, there is a major concern about the security and privacy of sensitive personal data. IoT provides a boundless imagination which put it on an edge of reshaping the current forms of the internet into an altered and integrated version. The types of communication we see nowadays are either human-human or human-device, but the IoT assures an extent future by enabling the internet technologies and the sensor networks where

74 IoT and Big Data Analytics, Vol. 1

Khambra et al.

the type of communication is machine-machine (M2M). The usage of IoT in various applications is expected to increase rapidly in the coming era.

Fig. (1). IoT Integrated Technologies.

INTELLIGENT PROCESSING The IoT has initiated to unite intelligence and autonomous control into its environment. It could build into a non-detected and open network of autoorganized structures. IoT is utilized for smart applications and innovative technologies by using intelligent processes. The data generated by sensors are being analyzed and intelligent decisions are being taken in real-time. With the use of intelligence processing, IoT bridges the space between the materialistic and digital world by providing connection and communication between things. Intelligent processing is gaining a lot of popularity as it facilitates automation and requires less human intervention. With a huge quantity of data being produced every second, demand for intelligent processing machines is increasing at a fast speed. Intelligent Process Automation (IPA) is the application of new technologies related to Artificial Intelligence including Cognitive automation, Machine Learning, and Computer Vision to Robotic Process Automation. This union of

Intelligent Processing

IoT and Big Data Analytics, Vol. 1 75

technologies results in automation capabilities that significantly increase business value and provide competitive advantages for the end-users. The major aim of IPA is to enable organizations and businesses to automate processes that involve unstructured content including text, audio, and images. This is done without a rule-based decision-making process or large training data sets that are difficult to obtain. Natural Language Processing (NLP) is an Intelligent Automation Solution. NLP is the ability of software to understand human language in its written and spoken form which is known as natural language. It is a part of Artificial Intelligence (AI). NLP has existed for more than half a century and has originated from the field of linguistics. NLP helps systems communicate with humans in their own language and handles other language-related tasks. For example, NLP makes it possible for systems to hear speech or read text interpret it, deduce sentiments, and determine which parts are significant. The application areas of intelligent processing are very vast and they can be integrated for a lot of automation work and decision-making. Intelligent Processing in Agriculture In the paper titled “Internet-of-Things (IoT)-Based Smart Agriculture: Toward Making the Fields Talk” [1] the authors have highlighted the effectiveness of wireless sensors and IoT in the domain of agriculture. They have also listed the challenges that may arise when combining IoT technology with traditional farming techniques. Various IoT devices with wireless sensors and communication techniques used in agricultural applications are studied and analyzed elaborately by the authors. The sensors available for particular agricultural applications like preparation of soil, status monitoring of crops, irrigation, pest, and insect detection, and control are also enumerated. They've shown that this technology can help farmers at every stage of the farming process, from sowing through harvesting, packing, and shipping. In addition, this study article considers the practice of unmanned floating motor vehicles for agricultural observation and other important applications such as crop yield optimization. The use of state-of-the-art architectural platforms based on IoT used in the fields is also elaborated. Finally, based on a thorough review of technologies, the authors have identified current and future trends in the application of IoT in agriculture. The authors of study [2] proposed an integrated system platform design that incorporates the Internet of Things, data mining, cloud computing, and other related technologies. They've proposed a new application for it in smart agriculture. The experimental framework and simulation design show that the IoT for agriculture monitoring system's core features may be successfully used.

76 IoT and Big Data Analytics, Vol. 1

Khambra et al.

Furthermore, the innovation that results from combining these diverse technologies contributes significantly to lower the cost of system development and assuring its dependability through improved security. The authors have built a decision support system to anticipate agricultural production by means of IoT in a paper titled “Agricultural Production System Based on IoT” [2]. Their system is an integrated system that aids the processes of agriculture from the sowing of seeds to the selling of agricultural products. The authors created an automated irrigation system to maximize the use of water for farm crops in their research study [3]. A dispersed wireless network of soilmoisture and temperature sensors installed in the root zone of the plants makes up the integrated system. A gateway unit, in addition to these, processes sensor data, initiates actuators and sends data to a specialized web application. The authors created an algorithm that controls the amount of water used in irrigation by programming temperature and soil moisture threshold values into a microcontroller-based gateway. The system is powered by photovoltaic panels and includes a bidirectional communication link based on a cellular-Internet interface that permits data analysis and irrigation scheduling via a web page. When compared to traditional agricultural irrigation practices, the automated technology saved up to 90% of water in a sage crop field after 136 days of testing. The authors concluded that the system can be useful in water-scarce geographical areas due to its energy efficiency and low cost. The authors focused their study [4] on effective water management, which is a key problem in many agricultural systems in arid locations with little rainfall. Distributed in-field sensor-based irrigation systems might be a viable option for farm-specific irrigation control, allowing farmers to increase output while conserving water. In their article, they describe the design and installation of a variable rate irrigation system, as well as a wireless sensor network and software enabling real-time in-field sensing and control of a site-specific precision irrigation system. The authors offer Wireless Sensor Network (WSN) as the best approach to handle problems connected to land monitoring, resource optimization in farming and decision support in their study work [5]. Their method offers farmers real-time agricultural information to assist them in making informed decisions. Precision agricultural systems based on IoT technology are explored utilizing WSN and Internet technologies, with an emphasis on the precision irrigation system's network architecture, hardware architecture, and software process control. In a feedback loop, the software monitors data from the sensors and triggers the control devices depending on pre-defined threshold levels. WSN use in precision

Intelligent Processing

IoT and Big Data Analytics, Vol. 1 77

agriculture will improve resource usage while also increasing agricultural production. In the research paper [5] the authors briefly present the prominent IoT platforms used for precision agriculture, focusing on their advantages and disadvantages. The work presented in the paper can be used as a basic method for identifying and choosing an IoT platform solution for smart telemonitoring systems in agriculture. The authors of the study “IoT Based Smart Agriculture System” [6] designed a system that practices sensors and Arduino boards to efficiently observe humidity, temperature, wetness, and animal activity in agricultural fields that might harm crops. In the event of a discrepancy, the system sends an SMS as well as a notification to the farmer's smartphone through Wi-Fi/3G/4G via the application designed for the purpose. The suggested system includes a duplex communication link based on a cellular Internet interface that allows for data examination and irrigation scheduling through an Android app [7]. INTELLIGENT PROCESSING MODEL Intelligent processing can be applied to nearly all activities related to agriculture. The life cycle of agriculture is shown below in Fig. (2).

Fig. (2). Life-cycle of Agriculture.

78 IoT and Big Data Analytics, Vol. 1

Khambra et al.

The industry heavily relies on Artificial Intelligence technologies to monitor soil and growing conditions, help yield healthier crops, control pests, organize data for better decision-making by farmers, ease the workload of farming, and upgrade an extensive variety of other agriculture-connected tasks supporting the complete food supply chain [8].

Fig. (3). Use of Technologies in Agriculture.

Fig. (3) depicts the use of technologies in agriculture. A basic intelligent processing model for agriculture would require the following: App Development In the farm setting, various sensors are implanted to monitor crucial parameters which will interact with the app as shown in Fig. (4). A smartphone based app is developed with connectivity to IoT equipment on the farm which will transmit data over the Internet to the farmer regarding various parameters such as soil moisture content, the temperature of the farm, humidity of air, weather conditions, and forecast. The app design is shown in Fig. (5). depending on the soil moisture substance alert, the farmer could operate the irrigation supply system with the click of an app button [9]. Additionally, the framework will also allow customized settings for each crop and give alerts for fertilizers, pesticides, etc. The dashboard of the framework will also have features to connect the farmer to agricultural experts who would be able to see the crops over a video calling facility and give expert advice to the farmer. A utility program will also be designed to provide the farmer with the highest prices of the crop in the local markets [10]. The intelligent model for agriculture has been shown in Fig. (5).

Intelligent Processing

Fig. (4). Intelligent model for agriculture.

Fig. (5). App Design.

IoT and Big Data Analytics, Vol. 1 79

80 IoT and Big Data Analytics, Vol. 1

Khambra et al.

App Modules ●

●

●

●

●

●

●

●

●

Farm Control: The various parameter values such as soil moisture, temperature, humidity, wind velocity, etc. will be quantitatively available to the farmer and based on this data, he will be able to take intelligent decisions for the crops. Additionally, this data will be used to control IoT applications such as automated irrigation control systems, temperature control mechanisms in greenhouses, etc. The data will also be stored on the cloud for further analysis and decision control [ 11]. Crop Progression: This module will let the farmer keep a check on the crop progression. He can keep a track of which crop was sowed when? and which crop will be ready for harvest and when? Weather Forecasting: Real-time weather forecasting will be available and alerts will be given to the farmer. The weather forecast may be short-term (daily basis) and long-term (seasonal changes). This information will be useful for the farmer to plan the sowing and harvesting of crops at appropriate times [12]. Chat Community: The farmer will be able to connect and chat with other farmers and experts through this module. Calendar: The farmer can set up his calendar to monitor the crops. It will also give alerts for the spraying of pesticides and insecticides at an appropriate time. Statistical Analysis: This module will give the farmer all statistical views of previous crop yields, expenses, income, profits, resource utilization, etc. Market Connectivity: This module will be designed to provide the farmer with the highest prices of the crop in the local markets. He can be connected to multiple markets and can choose to sell his crops at a market that offers the best price for his produce [13]. Browser: The browser will enable the farmer to have internet connectivity so that he can use it for online purchases of equipment/machinery, seeds, fertilizers, pesticides, etc. after proper market analysis. Settings: A general settings module would be available on the dashboard which can be customized as per the user's requirement.

Integration with sensors The app is integrated with the following sensors for effective parameter controlling: Soil Moisture Sensor Soil moisture sensors are used to calculate the volumetric water ratio of the soil. Soil moisture sensors indirectly measure the volumetric water content by using other properties of the soil as a proxy for the moisture content, such as dielectric

Intelligent Processing

IoT and Big Data Analytics, Vol. 1 81

constant, electrical resistance, or neutron interaction, because direct gravimetric measurement of free soil moisture necessitates the removal, drying, and weighing of a sample [14]. Humidity and Temperature Sensor A humidity sensor, commonly known as a hygrometer detects, measures, and reports moisture as well as air temperature. Relative humidity is defined as the percentage of moisture in the air to the greatest quantity of moisture at a given air temperature. It can be tracked using the farmer's smartphone. Wind velocity Sensor This sensor is used on the farm to monitor the wind velocity. If it crosses a set threshold, then it sounds like an alert on the smartphone of the farmer [15]. ADVANTAGES OF THE MODEL ●

●

●

Ease of use of technology for the farmers as the framework is app-based and the GUI is user-friendly. The better market price for the crops produced as the farmer has connectivity to multiple markets. Efficient resource utilization as electricity and water can be used only when needed.

The above model can be implemented as an intelligent processing model in agriculture using IoT. It aims to facilitate notified and exact decision-making by the farmers. With the advancement in Communication Technology, low-cost smartphones are affordable even to farmers in remote areas. The operative ease of these smartphones with multi-lingual apps further makes communication with the farmers easier than before [16]. This chapter discusses the use of IoT and intelligent processing in smart farming, as well as the opportunities that come with it, such as the semantic integration of data from various sources such as sensors, social media, connected farms, governmental alerts, and regulations, to ensure increased production and productivity while maximizing the use of resources such as water, electricity, and fertilizers. Interoperability between sensors, processes, data streams, and web-based services will be provided by the application. The overall impact of the program will be visible through the profitability and economic upliftment of the farmers as the program will provide knowledge-based assisted farming platforms to them. They will be able to take more informed decisions based on the data available on their smartphones. Thus, the above method will result in generating a very useful framework to assist farmers in agriculture. IoT is still an evolving technology in India and the

82 IoT and Big Data Analytics, Vol. 1

Khambra et al.

challenges about the security and interoperability remain the major concerns. More and more research in this field must be promoted to achieve greater goals [17]. In the realm of agriculture and allied industries, artificial intelligence (AI) is unquestionably a developing technology. The current agricultural system has been elevated to a new level thanks to AI-based equipment and machinery. This technology has significantly enhanced real-time monitoring, harvesting, processing, and marketing, as well as agricultural yield. The use of intelligent processing in agriculture will have a lot of benefits, especially in an agricultureoriented country like India. The industry must work together – open source efforts and partnerships and through standards groups, to address these challenges, and drive innovations for using IoT in agriculture. This will surely have a tremendous impact on the productivity and well-being of our farmers [18]. CONCLUSION As elaborated in the chapter, Intelligent processing can be easily implemented for increasing efficiency and production in the field of agriculture. A lot of manual work can be automated and remote monitoring facilities could be given to the farmers. It is important to bridge the digital divide that exists among farmers so that they can benefit from the latest technology. The system must be robust and cost-effective to meet the challenges faced by the farmers. CONSENT OF PUBLICATION Not applicable. CONFLICT OF INTEREST The authors declare no conflict of interest, financial or otherwise. ACKNOWLEDGEMENTS The authors are grateful to Dr. Fr. John P J, Principal, The Bhopal School of Social Sciences, Bhopal for providing R&D support, and necessary institutional facilities and to the chancellor, vice-chancellor, director, and head of the department of Jagran Lake City University, Bhopal, for their continuous guidance, and encouragement for the above research work. The authors are also thankful to all the authors whose literature has been considered in this research paper. REFERENCES [1]

M. Ayaz, M. Ammad-Uddin, Z. Sharif, A. Mansour, and E.H.M. Aggoune, "Internet-of-Things (IoT)based smart agriculture: Toward making the fields talk", IEEE Access, vol. 7, pp. 129551-129583,

Intelligent Processing

IoT and Big Data Analytics, Vol. 1 83

2019. [http://dx.doi.org/10.1109/ACCESS.2019.2932609] [2]

M. Lee, J. Hwang, and H. Yoe, "Agricultural production system based on IoT", 2013 IEEE 16Th international conference on computational science and engineering, pp. 833-837, 2013.

[3]

J. Gutiérrez, J.F. Villa-Medina, A. Nieto-Garibay, and M.A. Porta-Gándara, "Automated irrigation system using a wireless sensor network and GPRS module", IEEE Trans. Instrum. Meas., vol. 63, no. 1, pp. 166-176, 2014. [http://dx.doi.org/10.1109/TIM.2013.2276487]

[4]

Y. Kim, R.G. Evans, and W.M. Iversen, "Remote sensing and control of an irrigation system using a distributed wireless sensor network", IEEE Trans. Instrum. Meas., vol. 57, no. 7, pp. 1379-1387, 2008. [http://dx.doi.org/10.1109/TIM.2008.917198]

[5]

I. Marcu, C. Voicu, A.M.C. Drăgulinescu, O. Fratu, G. Suciu, C. Balaceanu, and M.M. Andronache, "Overview of IoT basic platforms for precision agriculture", International Conference on Future Access Enablers of Ubiquitous and Intelligent Infrastructures, pp. 124-137, 2019. [http://dx.doi.org/10.1007/978-3-030-23976-3_13]

[6]

G. Sushanth, and S. Sujatha, "IoT based smart agriculture system", 2018 International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET), pp. 1-4, 2018.

[7]

S. Jaiganesh, K. Gunaseelan, and V. Ellappan, "IoT agriculture to improve food and farming technology", 2017 Conference on Emerging Devices and Smart Systems (ICEDSS), pp. 260-266, 2017. [http://dx.doi.org/10.1109/ICEDSS.2017.8073690]

[8]

S. Raj, S. Sehrawet, N. Patwari, and K.C. Sathiya, "IoT based model of automated agricultural system in India", 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI).Tirunelveli, India pp. 88-93, 2019. [http://dx.doi.org/10.1109/ICOEI.2019.8862749]

[9]

K. Patil, and N. Kale, "A model for smart agriculture using IoT", 2016 International Conference on Global Trends in Signal Processing, Information Computing and Communication (ICGTSPICC), pp. 543-545, 2016. [http://dx.doi.org/10.1109/ICGTSPICC.2016.7955360]

[10]

A. Czyzewski, A. Kaczmarek, and B. Kostek, "Intelligent processing of stuttered speech", J. Intell. Inf. Syst., vol. 21, no. 2, pp. 143-171, 2003. [http://dx.doi.org/10.1023/A:1024710532716]

[11]

V. Barannik, A. Krasnorutskiy, Y.N. Ryabukha, and D. Okladnoy, "Model intelligent processing of aerial photographs with a dedicated key features interpretation", 2016 13th International Conference on Modern Problems of Radio Engineering, Telecommunications and Computer Science (TCSET).Lviv, Ukraine pp. 736-738, 2016. [http://dx.doi.org/10.1109/TCSET.2016.7452167]

[12]

K. Lakhwani, H. Gianey, N. Agarwal, and S. Gupta, "Development of IoT for smart agriculture a review", In: Emerging trends in expert applications and security Springer, 2019, pp. 425-432. [http://dx.doi.org/10.1007/978-981-13-2285-3_50]

[13]

M. Pathan, N. Patel, H. Yagnik, and M. Shah, "Artificial cognition for applications in smart agriculture: A comprehensive review", Artificial Intelligence in Agriculture, vol. 4, pp. 81-95, 2020. [http://dx.doi.org/10.1016/j.aiia.2020.06.001]

[14]

K. Sekaran, M.N. Meqdad, P. Kumar, S. Rajan, and S. Kadry, "Smart agriculture management system using internet of things", TELKOMNIKA (Telecommunication Computing Electronics and Control), vol. 18, no. 3, pp. 1275-1284, 2020. [http://dx.doi.org/10.12928/telkomnika.v18i3.14029]

[15]

J. Lin, Z. Shen, A. Zhang, and Y. Chai, "Blockchain and IoT based food traceability for smart agriculture", Proceedings of the 3rd International Conference on Crowd Science and Engineering, pp.

84 IoT and Big Data Analytics, Vol. 1

Khambra et al.

1-6, 2018. [http://dx.doi.org/10.1145/3265689.3265692] [16]

F. Bu, and X. Wang, "A smart agriculture IoT system based on deep reinforcement learning", Future Gener. Comput. Syst., vol. 99, pp. 500-507, 2019. [http://dx.doi.org/10.1016/j.future.2019.04.041]

[17]

Q. Wu, Y. Liang, Y. Li, and Y. Liang, "Research on intelligent acquisition of smart agricultural big data", 2017 25th International Conference on Geoinformatics.Buffalo, NY, USA pp. 1-7, 2017. [http://dx.doi.org/10.1109/GEOINFORMATICS.2017.8090913]

[18]

H. Bhardwaj, P. Tomar, A. Sakalle, and U. Sharma, Artificial Intelligence and Its Applications in Agriculture With the Future of Smart Agriculture Techniques.Artificial Intelligence and IoT-Based Technologies for Sustainable Farming and Smart Agriculture., I.G.I. Global, Ed., , 2021, pp. 25-39. [http://dx.doi.org/10.4018/978-1-7998-1722-2.ch002]

IoT and Big Data Analytics, 2023, Vol. 1, 85-98

85

CHAPTER 6

Challenges and Security in Terahertz Band for Wireless Communication Kannadhasan Suriyan1,* and Nagarajan Ramalingam2 Department of Electronics and Communication Engineering, Cheran College of Engineering, Tamilnadu, India 2 Department of Electrical and Electronics Engineering, Gnanamani College of Technology, Tamilnadu, India 1

Abstract: As society generates, transmits, and consumes information, wireless data traffic has grown dramatically in recent years. As a result of this shift, there has been a surge in demand for even faster wireless networking that can be used anywhere and at any time. Over the last three decades, the amount of data sent wirelessly has tripled every eighteen months, reaching the capacity of wired networks. Wireless Terabit-persecond (Tbps) connections will be a reality in the next five to ten years, if current trends continue. Support for these extraordinarily high data speeds will need advanced physical layer technologies, notably present spectral bands. Terahertz Band networking is envisioned as a vital wireless technology to meet this need, alleviating bandwidth depletion and power limits in present wireless networks, and allowing a deluge of longawaited applications across a variety of industries. THz is a spectral band with frequencies ranging from 0.1 to 10 GHz. Despite the fact that the frequency ranges immediately below and above this band (microwaves and far infrared, respectively) have attracted a lot of attention, this is still one of the least explored contact bands.

Keywords: Bandwidth, Cellular networks and 5G communication, Spectroscopy, Terahertz. INTRODUCTION The THz Band's very wide bandwidth allows for the development of a wide range of modern technologies in both traditional networking and new nanoscale connection paradigms. Any of these implementations may be predicted in advance, while others will undoubtedly arise as technology advances. THz Band networking is used by next-generation small cells, such as hierarchical and heterogeneous cellular networks. Small cells would be able to communicate at Corresponding author Kannadhasan Suriyan: Department of Electronics and Communication Engineering, Cheran College of Engineering, Tamilnadu, India; E-mail: [email protected]

*

Abhishek Singh Rathore, Surendra Rahamatkar, Syed Imran Ali, Ramgopal Kashyap & Nand Kishore Sharma (Eds.) All rights reserved-© 2023 Bentham Science Publishers

86 IoT and Big Data Analytics, Vol. 1

Suriyan and Ramalingam

ultra-high speeds with coverage ranges of up to 10 meters using the THz Band. In both indoor and outdoor contexts, the operating structure of these tiny cells incorporates static and smartphone involvement. THz Band networking allows ultra-high-speed wired networks, such as fibre optic cables, to communicate with personal wireless devices like laptops and tablet-like devices (no speed difference between wireless and wired links). This will make bandwidth-intensive apps more accessible to static and mobile users, particularly in indoor situations. Synchronization is a crucial activity in cooperative communication, both at the receiver and across several transmitters, and it becomes more difficult in ultrabroadband networks for the following reasons. To begin with, sampling at the Nyquist rate and performing complex signal processing operations at Tbps data rates are both very difficult. Second, various users' local oscillators create carrier frequencies in different ways, resulting in large frequency offsets across systems [1 - 5]. Short-range communication employing pulse-based modulation methods may be accomplished using low-complexity noncoherent analogue detectors like the energy detector and auto-correlation receiver. A more sophisticated non-coherent receiver is being developed using a Continuous-time Moving Average symbol identification algorithm in this approach. In terms of symbol error rate, this symbol detection approach outperforms previous detection systems for pulsebased modulations, lowering the synchronization need. For ultra-broadband communication in the THz Band, longer communication distances need robust and trustworthy synchronization mechanisms. In the near future, it is expected that the Terahertz Band (0.1–10 THz) will meet the need for Tbps wireless communication. THz Band connectivity can overcome the current spectrum shortages and power limitations in wireless systems, enabling a variety of applications such as ultra-fast massive data transfers between neighboring devices or high-definition videoconferencing across mobile personal devices in small cells [6 - 10]. In addition, the THz Band might allow new nanoscale networking paradigms like Wireless Nano Sensor Networks and the Internet of Nano-Things. From a computer standpoint, this research examines the state of the art in THz Band technology, covering various system transceiver topologies as well as new ultra-broadband and exceptionally wide antenna array designs. In terms of channel modelling and at various levels of the protocol stack, from the physical to the transport layers, we've also uncovered connection difficulties and proposed viable solutions. Finally, we examined the current status of experimental and simulation platforms and highlighted the most major impediments to their implementation. Due to a paucity of good sources and detectors in this THz gap, the THz (1012 Hz) section of the electromagnetic spectrum from around 100 GHz to 10 THz was essentially inaccessible until recently [11 - 15]. Since the 1960s,

Challenges and Security

IoT and Big Data Analytics, Vol. 1 87

astrophysics has been a driving force behind the development of detectors in this frequency range, since the rotational spectra of several gases of astrophysical and environmental relevance lie within this band. In millimeter and submillimeter wave astronomy, observational astronomers used cryogenic detectors to capture spectra (named after the wavelengths). The development of photoconductive and electro-optic methods for creating and monitoring radiation in the THz frequency range was sparked by a pioneering work on picosecond photoconductivity in silicon. As a result, there has been a boom in interest in this frequency range, and a lot of work has been done for developing efficient sources, sensitive detectors, and appropriate modulators in this range [16 - 18]. CHALLENGES IN TERAHERTZ BAND Lower-frequency sources depend on electric generation regulated by classical electron transport, such as RFs for AM and FM radio and microwaves. At these wavelengths, most dielectric materials are transparent, enabling radio reception and wireless communication within. For example, radar imaging has a resolution in the order of the wavelength and is often restricted to a few centimeters. The optical domain, which encompasses IR emission, visible light, and UV, is indicated by higher frequencies in the spectrum. Light is created through quantum transitions, which may provide exceptionally high intensities when used by lasers. The rules of this regime govern how electromagnetic radiation propagates in free space. The phrase “THz communication” may refer to either successful data rates exceeding 1 terabit per second (usually on an optical carrier) or communication using a THz carrier wave, which is the subject of this article. THz frequencies are appealing for a number of reasons, including frequency band availability and communications bandwidth. Although larger bandwidths can be obtained at optical wavelengths through point-to-point optical communications, THz frequencies are appealing for a number of reasons, including frequency band availability and communications bandwidth. According to the United States Frequency Allocation table, the Federal Communications Commission has yet to award frequencies above 300 GHz. THz networking is still in its infancy since the first data transmission in this frequency range was just recently validated. THz imaging is an interesting application that has spurred a great deal of THz research. The ability to image from dielectrics with a resolution of a few hundred of micrometers has several uses. Many systems, like the one at APL, use a photoconductive emitter to provide THz imaging with time-domain experimental setups by scanning an area via the focus of a single pixel. In the whole picture, each pixel possesses both a time domain and a spectrum waveform (with phase). The average time it takes to capture a photograph is in the tens of minutes. A full-field time-domain picture

88 IoT and Big Data Analytics, Vol. 1

Suriyan and Ramalingam

may be acquired using a large-aperture electro-optic crystal with a CCD sensor, as shown recently. THz imaging was used by researchers at APL and RPI to identify antipersonnel mines buried to a depth of a few centimeters in fairly damp, granular soil. Theoretical models created for materials science may be used for the problem of THz scattering and propagation in granular media since TDS provides critical amplitude and phase information. In foams and ceramics, as well as sand and silt, phase knowledge extraction will illustrate the transition between Rayleigh scattering and multiple scattering. While the significant absorption of THz imaging in living tissue (attenuation coefficient = 220 cm–1) limits its medical use, studies have shown that it can distinguish skin cancer (basal cell carcinoma) from healthy skin. Flaws in the computer chips and the sprayed-on foam insulation of the Space Shuttle's external fuel tank have also been revealed using 30 THz photography. In the electromagnetic spectrum, the terahertz (THz) frequency range is defined as the frequency range between 1011 and 1013 Hz: 1 THz = 1012 Hz. Such invisible radiation is safe for the human body, bimolecular, food, and pharmaceutics due to the low photon energy. Comparable approaches are being developed and applied for a variety of purposes, including civil protection, consistency, and process management. Following the development of femtosecond lasers in the 1980s, THz research grew increasingly vigorous due to the creation of a modern emission and detection technology, the photoconductive switch. The photoconductive switch used as a THz antenna works on the following principle: Two symmetric strip lines with a 5 m separation inserted in the centre of the symmetrically positioned dipole on a semiconducting substrate. The emitter antenna receives a DC bias voltage source (Ub), resulting in an electric bias field between them. The substrate becomes briefly conductive when the antenna is blasted with a femtosecond (fs) laser pulse based on the dipole difference. The bias voltage will accelerate the produced short-lifetime carriers, resulting in a sub-picosecond current transient flowing through the antenna. The form and length of this transient are determined by the laser pulse and physical parameters of the semiconductor, notably the lifespan of the produced free charge carriers. The new transient creates a pulsed electromagnetic wave of the same duration to be transmitted into free space, according to Maxwell's principle. The THz pulse is employed for both spectroscopic studies in this case. The antenna is turned off until the next laser pulse comes after a normal 10 ns delay, at which point it is turned on again. In the other direction, a non-biased antenna may be used as a detector to monitor and quantify freely travelling THz waves. Instead of a DC voltage, the antenna is linked to the input of a nanoampere meter in this scenario.

Challenges and Security

IoT and Big Data Analytics, Vol. 1 89

Although succinonitrile's static and low frequency (1 GHz) permittivity has been studied (both in its pure form and with various ionic dopants), there have been no studies of succinonitrile in the far-IR (THz) region, especially in its lowtemperature rigid crystal structure. Furthermore, the solid crystal structure of succinonitrile is unknown, with contradicting findings from several X-ray powder diffraction experiments. Terahertz spectroscopy is a useful instrument for investigating low-frequency lattice vibrational modes in molecular crystalline materials including explosives, pharmaceuticals, and therapies. The observed THz absorption peaks may be attributed to distinct lattice vibrational modes in the crystal by comparing the measured THz absorption peaks to resonances predicted by solid-state density functional theory (DFT). The assignment of modes aids in the semi-empirical London dispersion power adjustments, which are essential for accurate structural DFT calculations. THz time-domain spectroscopy is used to explain the lattice phonon modes of the stiff crystal phase succinonitrile (THz- TDS). Using semiempirical adjustments for London dispersion powers, the calculated modes are compared to solid-state DFT simulations. THz radiation is being used to improve materials for modern solar cells, and it might eventually be a crucial component of airport security scanners. Keeping track of all the innovations was easier while the field was still expanding; but, now that the field has become too large, keeping up with the vast array of new discoveries and applications that are arising is becoming more challenging. As THz science and technology matures into a more mature and diverse subject, it is critical to provide a road map to better define the field's scope and potential courses. We took help of an international panel of experts to develop 18 parts that cover the majority of essential topics of THz research and technology. THz frequency QCLs remain the only source capable of considerable performance beyond 1 THz, despite the numerous obstacles ahead. Many new uses and functionalities of these instruments would be discovered as a result of the possible advancements described above, ranging from basic science to advanced study, including their use as new sources for nonlinear optics, photography, spectroscopy, and trace gas analysis, among other things. Some features of how these devices work, such as their high-temperature efficiency, pulse creation dynamics, and ability to construct extremely large frequency combs, are unclear. In addition, QCLs will be integrated with current breakthroughs in detector technology, such as coherent detection and nanodetection technologies, to create efficient, inexpensive, and light THz systems. In the last ten years, laser-based THz outputs have progressed significantly. THz pump experiments are currently being used to research chemicals that were previously difficult to investigate. In the context of extremely powerful THz bursts, when the instantaneous electromagnetic fields exceed the DC breakdown

90 IoT and Big Data Analytics, Vol. 1

Suriyan and Ramalingam

values by a substantial amount, scaling up current single sources might improve our understanding of material behavior. By explicitly addressing low energy mutual phenomena in the 5–15 THz band, developing superior pump technologies might offer up new routes of control in complicated material systems. THz science has a promising future. The search for light-weight, dependable, low-cost, medium-to-high-power THz vacuum electron devices (VEDs) is a fascinating journey through a multidisciplinary area on the leading edge. New designs, components, and manufacturing procedures are emerging as a result of a growing number of research organizations throughout the world striving to overcome the daunting problem of high-power-density VEDs with submillimeter measurements. The future availability of low-energy, high-power-density THz VED sources would enable revolutionary advancements in the field of THz applications, where THz radiation's exceptional and unique capacity is currently limited by the power, scale, weight, availability, and/or cost of current product options. TERAHERTZ RADIATION SOURCES Accelerator-based THz radiation sources have advanced toward consumer facilities in recent years, with either unprecedented large average energies, up to the tens of watts and even kilowatts, or, more recently, high peak fields and/or peak fields at high repeat frequencies, up to 10 MV cm and beyond. These sources are designed to work with laser-based and other table-top THz sources, which have lower average pressures, peak fields, and repetition frequencies. From early brilliance-limited THz spectroscopy through flux-limited research and experiments into nonlinear dynamics caused by intense transient THz fields, the applications of such sources have grown. The related passive components implemented with either guided-wave structures or free-space quasi-optics will eventually limit the overall efficiency of front-end THz subsystems as the performance of active devices and circuits that make up sources, amplifiers, active modulators, and detectors will be improved. Existing and innovative passive device technologies such as impedance/amplitude/phase matching networks, power couplers, filters, antennas, polarizers, and even switches must all find a place in the ever-expanding cost-performance device spectrum. To stay up with advances in active technology, custom passive solutions, preferably without cryogenic cooling, would be necessary. Despite the fact that THz time-domain spectroscopy has progressed to the point that it is now used in hundreds of testing laboratories across the world, THz technologies still have a long way to go. The answer might be improved photoconductive semiconductor quantum structures and mode-locked edgeemitting semiconductor lasers. Industrial processing as well as cutting-edge manufacturing processes such as 3D printing might have an effect. Static and

Challenges and Security

IoT and Big Data Analytics, Vol. 1 91

time-resolved terahertz spectroscopy will continue to be prominent methods for researching semiconductors and semiconductor nanostructures. It may be used to examine ever-smaller regions in the future, as well as to disrupt network equilibrium. When combined with metamaterials, photoexcited semiconductors may be used as transitory optical components such as tunable filters. In recent years, THz microscopy has evolved from a proof-of-concept technique to a wellestablished diagnostic tool with distinctive properties. Only a few of the topics covered include local material features of inhomogeneous media and nanoparticles, as well as evanescent fields limited to the surfaces of subwavelength objects. Near-field signal-to-noise ratios may increase as THzproducing technologies and near-field tip design improve. As a result, the capacity to detect minor changes in the local dielectric function would improve. Meanwhile, cleaner sample conditions may help increase THz microscopy spatial resolution. The ultraclean cryogenic activity at ultrahigh vacuum might improve the THz-STM. It'll usher in a brand-new era of ultrafast, ultrasmall exploration. Following the introduction of THz time-domain spectroscopy, there was a rush to make unanticipated assumptions about THz biological capabilities, with the unfortunate effect that some wildly inaccurate findings were published. The biomedical community has expressed concern as a result of these concerning findings. This skepticism is exacerbated by a natural reluctance to new technology. As a result, engagement and cooperation between optical engineers and physicists, as well as biologists and medical doctors, have been restricted. Despite this, partnerships have resulted in more THz publications being published in biomedical journals. To take the field to the next level, global cooperation of biological professionals will be required to identify precise research goals. Using terahertz technology in medical imaging has potential benefits. Because of the low photon intensity, the emission is non-ionizing; tissue scattering is minimal; high sensitivity to water content allows for disease differentiation; time-domain structures may provide quasi-3D information; and the wide frequency spectrum allows for the investigation of a variety of diagnostic parameters. While there are a variety of other well-established clinical imaging technologies, as well as those that are making their way from the lab to the clinic, there are still a number of fascinating clinical challenges where terahertz might be used to aid clinical decision-making. Correct detection of tumor margins, for example, must be improved in cancer surgical excision, particularly in senior patients. Despite the fact that no substantial commercial advancements in terahertz medical applications have been made yet, a variety of hurdles remain, ranging from recognizing contrast to creating suitable systems. Terahertz technology is still in its infancy, therefore, specialised applications are likely to arise. Over the last decade, the area of terahertz NDT and molecular spectroscopy applications has increased dramatically, demonstrating that it is becoming a widely accepted

92 IoT and Big Data Analytics, Vol. 1

Suriyan and Ramalingam

measurement tool. Novel terahertz radiation implementations are being created in this manner, and they are already being employed in real-world applications. Due to scientific applications like as astronomy and Earth monitoring, THz technologies have a long history in space. Increased commercial use of space, such as for weather forecasting and future ultra-high frequency telecommunications, offers a lot of promise. Next-generation instrumentation, on the other hand, would be compatible with satellite payload systems with minimal mass, small volume, and low power. For better sensitivity and imaging capabilities, as well as improved optical signal processing, higher-frequency sensors would be created. These advancements provide significant hurdles, including detector component upgrades, circuit downsizing, skilled machining, lightweight composites, and enhanced cooling technologies, to name a few. Solving these issues will allow for more widespread use of the THz domain from space, and there is a lot of work being done in this area all around the world. THz research in space will grow in the next decades, delivering both technological and economic benefits. The concept of standoff imaging systems was used to show the improvements and potential hazards in THz protection systems. Though solidstate transistors will most certainly replace Schottky diodes as sources and receivers in active systems, the enormous number of receivers (thousands) necessary for reasonable contrast in passive systems above 150 GHz implies that bolometer arrays will remain crucial until MMICs can compete on packing. We expect integrated circuit and packaging technologies to continue to be the top research challenges for both active and passive devices in order to attain lower costs and more generally accessible capabilities, with cautious optimism. Regardless of the matter of the route THz technology, THz systems will provide sensory modalities that are just not conceivable in any other electromagnetic spectrum as shown in Fig. (1). The need for a bandwidth to facilitate highcapacity wireless data transmission is a major driver for THz communication system development. Short-range and indoor wireless networks will benefit from a plethora of communication windows in the spectrum below 400 GHz.

Fig. (1). Terahertz Band.

Challenges and Security

IoT and Big Data Analytics, Vol. 1 93

The development of lightweight and effective THz generators with continuous wave output power levels up to 100 mW, as well as tiny electronically steerable antenna arrays to minimize wireless connection degradation, are the two most important obstacles in commercialization of this technology. Terahertz (THz) waves, often referred to as undeveloped electromagnetic waves, have recently achieved significant scientific breakthroughs. Apart from laser-based THz manufacturing, progress is being made in THz-producing methods employing electronic devices and accelerators, and a variety of THz optical components are now being produced. THz is being employed in an increasing number of applications. The terahertz (THz) region of the electromagnetic spectrum has been the focus of study over the last 30 years. This spectral area is often defined by the lower limit of 100 GHz, which is about the frequency at which vector network analyzers become troublesome and expensive, and the top limit of 10 THz, which is roughly the lowest frequency accessible from a conventional lead salt laser diode. Despite the fact that the 1980s had long been regarded to be a fruitful period for both fundamental and applied physics, the discipline remained constrained. Many significant and remarkable achievements were made throughout this time period, but only a few may be included here. Despite this, many technical problems associated with both the creation and detection of radiation in this range hampered international research efforts, leading to the coining of the term “terahertz distance.” This term was used to characterize the sensitivity of common detectors10 and the power of common sources. It was meant to convey the general idea that, although technical advancement had evolved at lower (microwave) and higher (infrared and optical) frequencies, the THz spectrum had not. New research organizations faced a considerable barrier to entry as a consequence. The late 1980s saw the development of a new spectroscopic approach, which resulted in a large increase in jumpin operations. However, it took many years for the first commercially viable portable device to be launched. While scholars work to increase the available power of spectrum below 10 GHz, we may be approaching physical capacity limits as well as the economic viability of expanding capacity in the low and mid-band spectrum below 10 GHz. The signal-to-interference-noise ratio is increasing because of advances in modulation and coding technologies, as well as small cells, antenna beam shaping, virtualization, and other approaches (SINR). The only way to expand capacity if we approach physical limits in terms of SINR reduction is to increase available bandwidth. Despite the importance of such initiatives, they face a variety of economic and political problems since bandwidth below 10GHz has already been allotted, assigned, and is often utilized by many users in space and time. Furthermore, far greater channel bandwidths are necessary to allow B5G efficiency gains (in terms of power, data throughput, and latency) and more scalable wireless network architectures. Historically, radio, mobile broadband,

94 IoT and Big Data Analytics, Vol. 1

Suriyan and Ramalingam

public safety, positioning, radar, and military activities have all been assigned to bands below 10GHz in a dynamic network of paired and unpaired frequency bands of variable widths (measured in MHz). As a consequence, obtaining huge continuous networks, which are needed for higher data rate services, becomes more expensive, if not impossible. The electronics industry has progressed to the next stage of development. As a consequence of this process, photonic circuits, high-precision infrared sensors, quantum computing, and a slew of other technologies will emerge that would have been unthinkable just a decade ago. Issues that humanity has yet to resolve are at the centre of our endeavor. A Terahertz diode, for example, is a diode that can operate in the Terahertz range. As the next big thing, this strategy has a lot of potential. This article looks at the most recent advancements in the area and gives a quick rundown of the fundamentals. Terahertz diodes are diodes with a Terahertz frequency of operation. The response time of Schottky diodes and other pn-junction diodes is slow. They are unable to adapt to such high frequencies, necessitating the development of a new kind of diode designed specifically for switching at this frequency. The usage of THz diodes in Rectenna applications is another important reason for their current study. Rectennas are signal-rectifying antennas that are also known as rectifying antennas. Instead of utilizing photovoltaics, we'll capture light using rectifying antennas and then immediately correct it with these diodes to generate energy from sunlight. TERAHERTZ COMMUNICATION SYSTEMS There has been a huge shift in the economy and a transition to alternative energy choices such as solar in recent years, owing to climate change and our overdependence on the oil industry. Despite the backing of numerous governments, solar cells have yet to become a mass-market item for the reasons stated. In this case, rectennas come in handy. Rectennas are not subject to the SQ limit and do not need the use of expensive Silicon components. As a consequence, they will be able to solve the problem and make solar cells available to the general public. However, as previously said, no one currently understands how to construct an efficient Rectenna. Make no mistake: there have been multiple efforts, as we shall detail later in this article, but until a strong and cost-effective THz diode is developed, this kind of solar cell will remain an untested game-changer. The Metal-Insulator-Metal (MIM) diode and the Geometric or Ballistic diode are the two main types of diodes that have gained favor in the scientific community. It's just an insulator placed between two metals with very distinct work functions. The thickness of the insulator is less than ten nanometers. The goal of 6G communications is to boost data rates and latency efficiency while enabling pervasive access. Furthermore, 6G communications would use cutting-edge

Challenges and Security

IoT and Big Data Analytics, Vol. 1 95

methodologies to provide cutting-edge networking experiences like virtual presence and universal presence, which would be available from everywhere. Notable innovations in 6G communications include holographic messaging, flying networks, and teleoperated vehicles. Furthermore, 6G is touted to be more efficient and secure than traditional wireless networks. THz and AI, on the other hand, are the most exciting of all 6G-related technical advancements. In the field of cellular networks, these improvements are deemed to be ground-breaking. In order for these new technologies to be implemented into future networks, business professionals would need to dramatically improve their architectural concepts. The wireless transition from “connected things” to “connected knowledge” is projected to undergo a change that would distinguish 6G broadband networks different from earlier generations. Furthermore, 6G communications may have capabilities beyond mobile Internet, allowing for pervasive AI connectivity from the central network, which includes data centers, to distribution backhauls, and ultimately to end users. In other words, the transition would not be restricted to a single domain, but would usher in a new age of cross-domain collaboration between computer technology and wireless communication. Meanwhile, in the design and optimization of 6G networks, topologies, protocols, and operations, artificial intelligence (AI) will be critical. The millimeter wave (mmWave) spectrum was proposed and utilized to overcome the spectrum bottleneck in 4G communications. Unfortunately, holographic images' bandwidth requirements are incompatible with the current spectrum frequency. This obviously raises concerns like spatial spectral efficiency and transmission frequency ranges. As a result, a wide bandwidth is needed, which may be met by THz bands, which are classified as a distance band between microwave and optical spectra and are the focus of this research. THz waves are extremely short-wavelength high-frequency waves. In comparison to mm waves, THz waves have a higher frequency. As a result, 6G may be thought of as an ultra-dense network with increased capacity that can combine numerous technologies to execute and fulfill a broad variety of service orchestration needs. THz band coordination may be used for a wide range of applications, both macro and micro scale. Video transmission necessitates a huge amount of bandwidth, which is one of the reasons why internet video delivery services for prior wireless generations were delayed. With the recent increase in mobile data use (i.e., up to 24 Gbps for an uncompressed ultra-high definition [UHD] video and up to 100 Gbps for uncompressed 3D UHD video), existing high-rate systems, such as IEEE 802.11ad mm Wave Wi-Fi at 60 GHz, can only offer an average of 6.8 Gbps. In other words, the current channels' bandwidth has been depleted, necessitating the hunt for a new spectrum. The THz bands are obviously the only viable alternative, since they can provide multigigahertz continuous bandwidths adequate to sustain uncompressed video data

96 IoT and Big Data Analytics, Vol. 1

Suriyan and Ramalingam

rates of multi-Gbps and Tbps. Because the temperature and ambient light have only a tiny influence on Radar at THz frequencies, it is preferable over light or infrared-based imaging, such as Light Detection and Ranging (LIDAR). LIDAR performs well in terms of picture resolution but fails in the presence of natural phenomena such as fogs, rain, and clouds. These natural occurrences, however, do not exclude the use of THz radar while driving or flying in bad weather. The visual quality of HD video resolution radar will be equivalent to that of television, with an operating frequency of several hundred gigahertz. It may be used in combination with lower-frequency radars (below 12.5 GHz), which give improved range identification but poor image quality. CONCLUSION As a consequence, THz waves may be able to convey data faster at the cost of a shorter signal transmission wavelength. THz waves, which are completely implemented and integrated into cellular networks, may clearly address the problem of poor data throughput or latency that 5G will not be able to manage. As a result, service providers may seek THz waves to overcome the coming bandwidth bottleneck in the lower electromagnetic spectrum. With the rise of a smart society, technology will extend to a vast digital gadgets’ age, culminating in a multitude of devices being connected, resulting in massive amounts of data. As robotic and autonomous drone systems evolve, 6G is projected to provide huge power and a secure wireless connection for human to machine (H2M) and machine to machine (M2M) communications. In the future, the current Internet of Things (IoT) paradigm will be superseded by the Internet of All (IoE) paradigm. CONSENT FOR PUBLICATION Not applicable. CONFLICT OF INTEREST The authors declare no conflict of interest, financial or otherwise. ACKNOWLEDGEMENT Declared none. REFERENCES [1]

Q.J. Gu, Z. Xu, H.Y. Jian, B. Pan, X. Xu, M.C.F. Chang, W. Liu, and H. Fetterman, "CMOS THz generator with frequency selective negative resistance tank", IEEE Trans. Terahertz Sci. Technol., vol. 2, no. 2, pp. 193-202, 2012. [http://dx.doi.org/10.1109/TTHZ.2011.2181922]

[2]

R. Al Hadi, H. Sherry, J. Grzyb, Y. Zhao, W. Forster, H.M. Keller, A. Cathelin, A. Kaiser, and U.R.

Challenges and Security

IoT and Big Data Analytics, Vol. 1 97

Pfeiffer, "A 1 k-pixel video camera for 0.7–1.1 terahertz imaging applications in 65-nm CMOS", IEEE J. Solid-State Circuits, vol. 47, no. 12, pp. 2999-3012, 2012. [http://dx.doi.org/10.1109/JSSC.2012.2217851] [3]

K. Shinohara, D.C. Regan, Y. Tang, A.L. Corrion, D.F. Brown, J.C. Wong, J.F. Robinson, H.H. Fung, A. Schmitz, T.C. Oh, S.J. Kim, P.S. Chen, R.G. Nagele, A.D. Margomenos, and M. Micovic, "Scaling of gan HEMTs and schottky diodes for submillimeter-wave mmic applications", IEEE Trans. Electron Dev., vol. 60, no. 10, pp. 2982-2996, 2013. [http://dx.doi.org/10.1109/TED.2013.2268160]

[4]

C. Campbell, M. Kao, and S. Nayak, "High efficiency Ka-band power amplifier MMICs fabricated with a 0.15 nm GaN on SiC HEMT process", In: IEEE MTT-S International Microwave Symposium, 2012.

[5]

M. Micovic, A. Kurdoghlian, A. Margomenos, D. Brown, K. Shinohara, S. Burnham, I. Milosavljevic, R. Bowen, A. Williams, and P. Hashimoto, "92–96 GHz GaN power amplifiers", In: IEEE MTT-S International Microwave Symposium Digest, 2012, pp. 1-3.

[6]

V. Radisic, K.M.K.H. Leong, X. Mei, S. Sarkozy, W. Yoshida, and W.R. Deal, "Power amplification at 0.65 THz using inp HEMTs", IEEE Trans. Microw. Theory Tech., vol. 60, no. 3, pp. 724-729, 2012. [http://dx.doi.org/10.1109/TMTT.2011.2176503]

[7]

I. Kallfass, J. Antes, D. Lopez-Diaz, S. Wagner, A. Tessmann, and A. Leuther, "Broadband active integrated circuits for terahertz communication", In: 18th European Wireless Conference European Wireless, 2012, pp. 1-5.

[8]

Y. Kawano, H. Matsumura, S. Shiba, and M. Sato, "Convergence of Terahertz Sciences in Biomedical Systems",

[9]

J.F. Federici, B. Schulkin, F. Huang, D. Gary, R. Barat, F. Oliveira, and D. Zimdars, "THz imaging and sensing for security applications—explosives, weapons and drugs", Semicond. Sci. Technol., vol. 20, no. 7, pp. S266-S280, 2005. [http://dx.doi.org/10.1088/0268-1242/20/7/018]

[10]

H.B. Liu, H. Zhong, N. Karpowicz, Y. Chen, and X.C. Zhang, "Terahertz spectroscopy and imaging for defense and security applications", Proc. IEEE, vol. 95, no. 8, pp. 1514-1527, 2007. [http://dx.doi.org/10.1109/JPROC.2007.898903]

[11]

F. Ospald, W. Zouaghi, R. Beigang, C. Matheis, J. Jonuscheit, B. Recur, J-P. Guillet, P. Mounaix, W. Vleugels, P.V. Bosom, L.V. González, I. López, R.M. Edo, Y. Sternberg, and M. Vandewal, "Aeronautics composite material inspection with a terahertz time-domain spectroscopy system", Opt. Eng., vol. 53, no. 3, p. 031208, 2013. [http://dx.doi.org/10.1117/1.OE.53.3.031208]

[12]

T. Yasui, T. Yasuda, K. Sawanaka, and T. Araki, "Terahertz paintmeter for noncontact monitoring of thickness and drying progress in paint film", Appl. Opt., vol. 44, no. 32, pp. 6849-6856, 2005. [http://dx.doi.org/10.1364/AO.44.006849] [PMID: 16294957]

[13]

D.H. Auston, K.P. Cheung, and P.R. Smith, "Study of high-power wideband terahertz-pulse generation using integrated high-speed photoconductive semiconductor switches", Plasma Science, IEEE Transactions, vol. 37, no. 1, pp. 219-228, 2009.

[14]

Y.S. Lee, Principles of Terahertz Science and Technology. Springer, 2009.

[15]

C. Lombardi, S. Manzini, A. Saporito, and M. Vanzi, "A physically based mobility model for numerical simulation of nonplanar devices", Computer-Aided Design of Integrated Circuits and Systems, IEEE Transactions, vol. 7, no. 11, pp. 1164-1171, 1988. [http://dx.doi.org/10.1109/43.9186]

[16]

M. Lundstrom, Fundamentals of carrier transport. Cambridge University Press, 2009.

[17]

J.D. Morse, R.P. Mariella, G.D. Anderson, and R.W. Dutton, "Picosecond optoelectronic gating of silicon bipolar transistors by locally integrated gaas photoconductive devices", vol. 12, no. 7, pp. 379-

98 IoT and Big Data Analytics, Vol. 1

Suriyan and Ramalingam

381, 1991. [18]

Z. Piao, M. Tani, and K. Sakai, "Carrier dynamics and terahertz radiation in photoconductive antennas", Japanese Journal of Applied Physics, vol. 39, no. 1, pp. 96-100, 2000. [http://dx.doi.org/10.1143/JJAP.39.96]

IoT and Big Data Analytics, 2023, Vol. 1, 99-111

99

CHAPTER 7

Empirical Impact of AI and IoT on Performance Management: A Review Shahnawaz Ahmad1,*, Shabana Mehfuz2 and Javed Beg2 1 2

Department of Electrical Engineering, Jamia Millia Islamia, New Delhi, India Oracle, Noida-India Abstract: Purpose In general, the role of AI and IoT in increasing the performance level of businesses has been discussed here to establish understandable data with several objectives regarding future potentials, business growth, and relevant challenges. Therefore, how different industrial businesses are using these two variables in their workplace cultures have been prioritized in the entire study. Research Method Secondary qualitative research method has been followed throughout the review work by aligning the systematic review and thematic analysis to get a clear overview of the topic. All the data have been collected between the published year- 2017- 2021 based on several inclusion criteria for numerous relevant measures. The researcher has counted on all essential methodological tools such as positivism philosophy, descriptive design, and deductive approach to make the study reliable. Findings The business sector units are focusing on their various departmental improvements such as Human Resources, supply chain, logistics, data transfer, etc. and others have been found to work efficiently with the help of AI and IoT automation sensors' productivity. Furthermore, it has been proved that the selection of topics has been beneficial to get expected findings by using various journals in similar regard. Corresponding author Shahnawaz Ahmad: Department of Electrical Engineering, Jamia Millia Islamia, New Delhi-110025, India; E-mail: [email protected]

*

Lamture Yeshwant Ramrao All rights reserved-© 2023 Bentham Science Publishers

100 IoT and Big Data Analytics, Vol. 1

Ahmad et al.

Conclusion Hence, the conclusion section has illustrated how efficiently the researcher achieved the knowledge of the topic by focusing on its various relatable factors to improve the performances in businesses. Researching such a topic based on the current global scenario is found advantageous to move with the interest for further studies.

Keywords: AI, IoT, Industrial growth , Performance management , Technologies. INTRODUCTION Corporate performance management is adopting changes with the new technologies, especially to advance data analytics through Artificial Intelligence (AI). In previous times, it was not possible to create a correlation between the information and achieve a deeper insight which created challenges so many times to take effective decisions with potential development. This Industry 4.0 era has been found to depend on networks for being connected to the world, which has been possible due to the Internet of Things (IoT). Different IoT drivers to influence performance management can be seen in Fig. (1).

Fig. (1). IoT drivers to influence the performance management [1].

Intelligent communication across global networks has become possible with the help of IoT and the performance management of Human Resources has become efficient as well. Process automation works with high efficiency as it can work repetitively with a similar level of consistency by maintaining a standard that is not possible for humans [2]. With the help of computer software, the implementation of robotic process automation has made performances faster and errorless such as secure bank transactions, data transfer from one to another cloud, and others that play crucial roles in business performance betterment. There is no clear definition for the word IoT because it is so broad and extensively used. Several organizations and researchers have defined the Internet of Things. The

Empirical Impact

IoT and Big Data Analytics, Vol. 1 101

International Telecommunication Union (ITU) described it in 2012 as “a global infrastructure for the information society, enabling advanced services by interconnecting (physical and virtual) things based on, existing and evolving, interoperable information and communication technologies” [3]. There's no denying that AI has become one of the most influential issues in a variety of fields. The Merriam-Webster dictionary defines artificial intelligence as “the capability of a machine to imitate intelligent human behavior” [4]. The main goal of AI is to develop new models and ways that can perform intelligent tasks. In most cases, AI is based on experiments in which the researcher uses the computer system as a lab to validate and verify their assumptions [5]. Integrating the IoT with AI has the potential to create a powerful solution that can address a variety of IoT difficulties caused by the massive amount of data created by billions of IoT devices. Traditional analytical approaches will not be able to analyze these massive amounts of data; instead, adopting various AI and machine learning methods will be able to analyze and extract useful information, allowing you to reap the full benefits of IoT data [6]. With AI's tremendous analytic capabilities, many governments and businesses have begun to use AI technologies to uncover the value of massive amounts of IoT data. Although the discovery of fog/edge computing reduced response time, allowing real-time-based IoT applications to grow dramatically, AI-based techniques will be able to give a greater performance in a much shorter time frame. Furthermore, combining AI and IoT can improve security by not just combating external attacks but also providing an effective means to predict them. The Internet of Things (IoT) is a cutting-edge technology that has the potential to transform our lives, businesses, and economies. The Internet of Things generates a plethora of digitized services and applications that outperform traditional solutions in various ways. The following are some of the traits that these applications and services have in common [7 - 9]. Features of IoT systems include sensing capabilities, connectivity, large scale networks [10], dynamic system, intelligence capabilities, Big data, unique identity, autonomous decisions, and heterogeneity. The centralized IoT architecture, on the other hand, poses several difficulties (i.e., Single Point of Failure, Security [11], Privacy [12], Inflexibility [13], Cost [14], Scalability [15], Access and Diversity [16]). It, for example, has scalability concerns since it can't keep up with the increasing influx of IoT devices. Furthermore, it raises a slew of security and privacy concerns [17]. Fig. (2) depicts the market revenue after adopting AI and IoT technologies. In comparison to other traditional technologies, AI is making machine learning procedures 20x faster. By promoting safer connectivity among devices through the internet and AI, pattern identification and fault detection during data collection are stimulated enough through advanced sensors. With the growing performance, the business market revenue is also increasing from $9.51 billion to $34.87 billion in 2021 and it is predicted that in 2025, it will cross $110 billion

102 IoT and Big Data Analytics, Vol. 1

Ahmad et al.

[18]. Hence, it can be said that AI and IoT have a great impact on the performance management of corporate firms.

Fig. (2). Market revenue after adopting AI and IoT [18].

Contribution of this Study The main aim of this study is to establish a clear understanding of the impacts of AI and IoT on business performance management that has been followed with contributions as follows; ● ●

● ● ●

● ●

To define the role of AI and IoT to incorporate business growth. To explore the uses of AI and IoT in the identification of faults and achieving advantage. To address the future potential of business growth through AI and IoT. Investigating the most up-to-date research and studies on IoT and AI. Discuss the necessity for IoT to be integrated with AI, as well as how AI helped alleviate problems with centralized IoT architecture. Investigating the effects of AI integration on IoT and performance management. Discussing future IoT and AI research directions with performance management.

The remainder of this paper is organized as follows: an overview of the literature review; research method; discussion of the results; future research directions of the IoT with AI and conclusion of the paper.

Empirical Impact

IoT and Big Data Analytics, Vol. 1 103

LITERATURE REVIEW The Internet of Things (IoT), often known as the Internet of Everything or the Industrial Internet, is a new technology paradigm that allows machines and devices to communicate with one another across a global network. IoT produces trends, gives a sound planning strategy, and is known as innovative technology, according to some studies [19, 20], and IoT is not the sole buzzword for the organization. IoT is considered one of the most essential areas for forthcoming technology due to its capabilities, and many businesses are paying attention. Five important IoT technologies are widely used in IoT-based services and products, according to [21]: WSNs, RFIDs, cloud computing, middleware, and IoT application software are all examples of wireless sensor networks (WSNs). IoT application software, on the other hand, enables device engagement, and humanto-device communication modalities are dependable and stable. According to some studies [22 - 23], the ultimate goal of IoT is to develop a global system architecture that facilitates the exchange of products, services, and information. Some businesses have used the Internet of Things (IoT) or the Industrial Internet of Things (IIoT) to gather real-time data, resulting in more efficient operations. IoT can improve the ability to integrate suppliers, customers, and intraorganizational logistical activities, according to the idea of organizational capacities [24 - 26]. Simply put, the Internet of Things (IoT) will link the physical and digital worlds by synchronizing data and physical flow. As a result, a company's supply chain integration may improve [27]. Industry 4.0 Revolution Through AI and IoT Complications and competition in global industries are rising so fast that almost every industry alongside the IT industry is adopting AI and IoT in their business practices. It has been noted that business performance and management styles are dependent on numerous variables that enforce those to use these automation sensors for different reasons by focusing on the business goals and market scenarios [28]. Therefore, the sector-wise division and performance management variables have been noted in the below-mentioned Table 1. Table 1. Industrial sector-wise use of AI and IoT for performance management [28]. Industries Healthcare Manufacturing • Retail • •

Automation Usage Regular tracking and monitoring of the essential equipment for ensuring better and quality performances to serve the population with the help of remote sensors.

104 IoT and Big Data Analytics, Vol. 1

Ahmad et al.

(Table ) cont.....

Industries • •

Transportation Logistics

Oil and gas Construction • Manufacturing • Cities • •

Telecoms Another machining environment • •

Automation Usage As per [29], robotics and drone services are helping in hindering the performance of supply chain management that plays a crucial role in such industries for serving populations with on-time delivery and a higher degree of safety. As the complications are huge in these sectorial works, automation helps in finding issues and repetitively taking initiative.

By using Augmented reality, it helps in providing remote support to specialists and technicians.

AI Intersects with IoT AIOT, which can be found at the edge or in the cloud, is the junction of AI and IoT. The goal of the technology, also known as Artificial Intelligence of Things, is to improve IoT operations by increasing human-machine interactions, data management, and analytics. Then, by using AI, we may use superior IoT components, which leads to strong decision-making. The combination of AI with IoT yields an extremely strong technology in the future, implying that everything will be based on the internet, making our lives easier than before. There will be no time delay in meetings because IoT devices will remind you, and you can imagine that when it is combined with AI, it will also result in decision-making based on your schedules. As far as the problem of time delay is resolved, it also solves the problem of traffic in the same way that the model predicts the traffic area and how to resolve it. Assume that traffic signals operate automatically and that no physical staff is required, as a result, while future uses may exist, we cannot predict how the world will alter. In today's world, the Internet of Things (IoT) is a relatively new technology. Everything in the world today is made up of IoT, so we can control our air conditioners, televisions, fans, switches, and other devices with our smartphones instead of using their separate remote controls. Business performance management must be entertained from two different aspects such as predictive analytics and perspective analytics where the management decides about future activities based on its consequences. In that case, AI helps in defining the modern and advanced technologies for adaptation b)' working as a sensor to predict the flaws and take initiatives towards effective actions [30]. IoT in this instinct is found internally connected with AI as IoT defines the way data is used and forwarded to Al through sensors for making decisions regarding efficient performances.

Empirical Impact

IoT and Big Data Analytics, Vol. 1 105

Future Potential Growth of Performance Management Through AI and IoT The below-mentioned diagram is giving a clear idea that the number of organizations is increasing with the passing of time for reducing insecurities and any risks in workplace environmental measures. By taking into account this measurement, it is predicted that by 2025, the scenario might involve around 19% of the global market with Al’s performance. Furthermore, the financial services and achieving the expected revenue margin would be easier in the next 5 to 10 years in every industry such as manufacturing, healthcare, retail, and others which would be recorded as a big shift of technologies in today's business platform [31]. Machine learning in that condition indicates an effective move towards high-end technological edge devices for serving people with Al algorithm computing data and relying on powerful servers and central data storage. Fig. (3) presents the market estimate and machine learning estimation through Al and IoT from 2017 to 2025. Al semiconductor total available market.' 5 billion Al Non-Al 362 65 240 17 223

256 32 224

Al semiconductor total available market, % 7 93

295

Estimated Al semiconductor total available market CAGR," 2017-25, % 18-19

11

19

88

81

5x

3-4

2017

2020E

2025E

2017

2020E

2025E

Non-Al

Al

Fig. (3). Market estimate and machine learning estimation through Al and IoT (2017-2025) [31].

Theoretical Underpinning For making effective decisions regarding business performance, it is important to calculate those in a mathematical and logical segment so that a secured and sustained strategy can be developed. The Game theory is the most favorable theory, chosen by economists for reducing technological complexities by making general differentiation [32]. It helps in understanding the situation of business performance and potential hazards that seek a winning game by following the rules adhesively. In that AI has developed an appropriate algorithm rule for protecting the game and making the player win that has performed with proper guidelines. The digital developments of game theory and AI have been found with several applications such as machine learning algorithms, multi-agent AI systems,

106 IoT and Big Data Analytics, Vol. 1

Ahmad et al.

manipulation-resistant systems, and reinforcement learning. In the meantime, with the help of Nash equilibrium, AI helps industries to address numerous dynamic problems and solve them with effective solutions by taking into account the rules of Game theory as shown in Fig. (4). RESEARCH METHODS A secondary qualitative research method has been followed to conduct this research work as the topic selection was done on a global basis in which secondary phenomena would be easier to collect and get vast knowledge. In between, positivism philosophy has been used to align the researcher's concept of the topic along with collected data sources with the help of a deductive approach for selecting the most appropriate journals for better deployment [33]. The systematic research review is a useful methodological tool that helps in selecting data resources based on several inclusion and exclusion criteria to collect the data with critical appraises [34]. In this research work, alongside the systemic review, a thematic analysis has been done based on which descriptive design has been aligned to interpret the information. Online resources that are published after 2017 in English on the same or nearly the same topics have been integrated with this paper to maintain reliability and validity with high visibility criteria.

Perfect information vs.Imperfect information

Symmetric vs. Asymmetric Cooperative vs. Noncooperative

Game Theory Zero-sum vs. Nonzero-sum

Simultaneous vs.Sequential

Fig. (4). The Game Theory with AI.

Empirical Impact

IoT and Big Data Analytics, Vol. 1 107

RESULT AND DISCUSSION Systematic Review The systematic review based on findings and significance can be seen in Table 2. Table 2. Systematic Review (Source: Learner). Resources

Findings

Significance

[35]

This article found the negative growth of social and environmental sustainability due to the high death rate and unemployment that has raised concerns about AI and IoT in a real-time dataset with a stable pandemic performance management in businesses.

The article consumed data based on the most recent worldwide activities while the working culture is changing and workers are seeking a higher degree of security.

[36]

A digitized workplace environment has been prioritized to showcase the performance of the Human Resource department by using AI technologies that transformed the traditional business SHRM into digital SHRM.

Big data analytics was found with a major concern in this high-end technological edge, this article found it important to represent the transformation with clarity.

[37]

Future potentials with the AI performance through 5G networks have been found beneficial to explore the opportunities with better solutions.

As this article focused on addressing the challenges as well as opportunities, an entire overview has been gained.

Thematic Review Theme 1: Industrial growth through AI in emerging Human Resource practices is found with effective strategies to enhance the business processes and performances are directly controlled by the HRM members with a larger operational activity that would be performed with Big Data analytics, IoT through AI sensors. Therefore, the research has revealed that business growth is continuously rising after the adaptation of IoT automation technologies in worldwide businesses from different sectors that the error statistic graph is lowering down and the performances are flourishing [38]. It has helped in decreasing the workload of employees which has resulted in high employee retention as they only need to know the operative system of IoT and thereafter, it is the duty of the software to address the errors and solve problems in repetition whenever needed. Theme 2: Security enhancement by IoT security technologies has become easier in the organization, especially during money transactions as cases of confidentiality and data hacking-related challenges were comparatively higher in the last decade. With the help of IoT technologies, business management team members have become able to use the software according to their needs and personalize it with required security measures. A study [39] stated in the research

108 IoT and Big Data Analytics, Vol. 1

Ahmad et al.

work that through data storage in Cloud Computing, data transfer and data maintenance have become efficient so that employees do not get blamed by the seniors for their shortcomings in addressing the safety flaws that can result in customer loss along with decreased revenue margins. DISCUSSION Both the systematic review and thematic analysis have been performed based on the secondary data resources that have been collected from authentic online resources based on research requirements. As the main purpose of this work is to prove the significance of AI and IoT for business performance management, chosen articles in the systematic review section found similar findings that are making the concept clear against the topic. Besides, thematic review has been performed based on predefined objectives to meet the goal of the research aim. Hence, it is clearly showing the impact of AI and IoT on enhancing potential business performances. CONCLUSION AND FUTURE SCOPE The study is making it clear that technological advancement is essential in this era to be compatible with the current needs of businesses and enhance their performance level to ensure positive growth shortly. From the analysis, it has been emphasized that global business analytics are concerned about performance management with AI and IoT that is not only reducing the time and but also helping in maintaining consistency in any field of work. Furthermore, it is easy to conclude the concept of AI and IoT in industrial growth and business performances has become crucial and necessary to be used with automation. As the technology is growing faster, researching such a topic that is related to Al and IoT on behalf of business performance growth would be beneficial. For future studies, it would help in increasing the interest level of future readers to get more ideas on the same concept. The clear concept of its power to address the flaws and resolve those in a proper manner would foster industrial growth through high security with a proper understanding of prioritizing relevant variables and adequate recommendations to meet predictive outcomes. CONSENT FOR PUBLICATION Not applicable. CONFLICT OF INTEREST The authors declare no conflict of interest, financial or otherwise.

Empirical Impact

IoT and Big Data Analytics, Vol. 1 109

ACKNOWLEDGEMENT Declared none. REFERENCES [1]

A. Mynzhasova, C. Radojicic, C. Heinz, J. Kelsch, C. Grimm, J. Rico, K. Dickerson, R. Garcia-Castro, and V. Oravec, "Drivers, standards, and platforms for the IoT: Towards a digital VICINITY", 2017 Intelligent Systems Conference (IntelliSys), IEEE., pp. 170-176, 2017. [http://dx.doi.org/10.1109/IntelliSys.2017.8324287]

[2]

D. Saju Mathew, Artificial Intelligence (AI): Bringing a New Revolution in Human Resource Management (HRM)., 2021.

[3]

ITU, Overview of the T/recommendations/rec

[4]

Artificial Intelligence Definition of Artificial Intelligence by Merriam-Webster, Available from: https: //www.merriam-webster.com/dictionary/artificialintelligence [accessed on 24 April 2020].

[5]

R. Reddy, "The challenge of artificial intelligence", Comput. Long. Beach. Calif, vol. 29, pp. 86-98, 1996.

[6]

A.G. Alzahrani, A. Alenezi, H. Atlam, and G.B. Wills, "A framework for data sharing between healthcare providers using blockchain", Proceedings of the 5th International Conference on Internet of Things, Big Data and Security (IoTBDS 2020), pp. 349-358, 2020.Prague, Czech Republic [http://dx.doi.org/10.5220/0009413403490358]

[7]

H.F. Atlam, R.J. Walters, and G.B. Wills, "Internet of Things: State-of-the-art, Challenges, Applications, and Open Issues", International Journal of Intelligent Computing Research, vol. 9, no. 3, pp. 928-938, 2018. [http://dx.doi.org/10.20533/ijicr.2042.4655.2018.0112]

[8]

H.F. Atlam, and G.B. Wills, "intersections between IoT and distributed ledger", In: In Advances in Organometallic Chemistry vol. 60. Elsevier BV: Amsterdam, The Netherlands, 2019, pp. 73-113.

[9]

H.F. Atlam, and G.B. Wills, Technical aspects of blockchain and IoT.Advances in Organometallic Chemistry. vol. Vol. 60. Elsevier BV: Amsterdam, The Netherlands, 2019, pp. 1-39.

[10]

"Statista. Internet of Things (IoT) Connected Devices Installed Base Worldwide from 2015 to 2025 (in Billions)", Available from: https://www.statista.com/statistics/471264/iot-number-of-connected devices-worldwide/ [accessed on 8 October 2020].

[11]

H.F. Atlam, and G.B. Wills, IoT Security, Privacy, Safety, and Ethics.Intelligent Sensing, Instrumentation and Measurements. Springer Science and Business Media LLC: Berlin, Germany, 2019, pp. 123-149.

[12]

M. Conoscenti, A. Vetro, and J.C. De Martin, "Peer to peer for privacy and decentralization in the internet of things", In Proceedings of the 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C), pp. 288-290, 2017.Buenos Aires, Argentina [http://dx.doi.org/10.1109/ICSE-C.2017.60]

[13]

H. Atlam, R. Walters, and G. Wills, "Fog computing and the internet of things: A review", Big Data and Cognitive Computing, vol. 2, no. 2, p. 10, 2018. [http://dx.doi.org/10.3390/bdcc2020010]

[14]

T.M. Fernandez-Carames, and P. Fraga-Lamas, A review on the use of blockchain for the internet of things.IEEE Access, vol. 6, pp. 32979-33001, 2018. [http://dx.doi.org/10.1109/ACCESS.2018.2842685]

[15]

H.F. Atlam, and G.B. Wills, "An efficient security risk estimation technique for Risk-based access control model for IoT", Internet of Things, vol. 6, p. 100052, 2019.

Internet

of

Things..

Available

from:

https://www.itu.int/ITU-

110 IoT and Big Data Analytics, Vol. 1

Ahmad et al.

[http://dx.doi.org/10.1016/j.iot.2019.100052] [16]

H.F. Atlam, A. Alenezi, R.J. Walters, G.B. Wills, and J. Daniel, "Developing an Adaptive Risk-Based Access Control Model for the Internet of Things", In Proceedings of the 2017 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), pp. 655-661, 2017.Exeter, UK [http://dx.doi.org/10.1109/iThings-GreenCom-CPSCom-SmartData.2017.103]

[17]

H.F. Atlam, R.J. Walters, and G.B. Wills, "Intelligence of Things: Opportunities & Challenges", Proceedings of the 2018 3rd Cloudification of the Internet of Things (CIoT), pp. 1-6, 2018.Paris, France

[18]

S.C. Mukhopadhyay, S.K.S. Tyagi, N.K. Suryadevara, V. Piuri, F. Scotti, and S. Zeadally, "Artificial Intelligence-Based Sensors for Next Generation IoT Applications: A Review", IEEE Sens. J., vol. 21, no. 22, pp. 24920-24932, 2021. [http://dx.doi.org/10.1109/JSEN.2021.3055618]

[19]

T.M. Ghazal, M.K. Hasan, M.T. Alshurideh, H.M. Alzoubi, M. Ahmad, S.S. Akbar, B. Al Kurdi, and I.A. Akour, "IoT for smart cities: Machine learning approaches in smart healthcare—A review", Future Internet, vol. 13, no. 8, p. 218, 2021. [http://dx.doi.org/10.3390/fi13080218]

[20]

C.P. Tang, T.C.K. Huang, and S.T. Wang, "The impact of Internet of things implementation on firm performance", Telemat. Inform., vol. 35, no. 7, pp. 2038-2053, 2018. [http://dx.doi.org/10.1016/j.tele.2018.07.007]

[21]

I. Lee, and K. Lee, "The Internet of Things (IoT): Applications, investments, and challenges for enterprises", Bus. Horiz., vol. 58, no. 4, pp. 431-440, 2015. [http://dx.doi.org/10.1016/j.bushor.2015.03.008]

[22]

H.M. Alzoubi, M. Vij, A. Vij, and J.R. Hanaysha, "What leads guests to satisfaction and loyalty in UAE five-star hotels? AHP analysis to service quality dimensions", ENLIGHTENING TOURISM. A PATHMAKING JOURNAL, vol. 11, no. 1, pp. 102-135, 2021. [http://dx.doi.org/10.33776/et.v11i1.5056]

[23]

A. Abubakar, "Mediating effect of supply chain integration strategy on the relationship between internet of things capability and financial performance of manufacturing companies", KASU Journal of Supply Chain Management, vol. 1, pp. 48-71, 2020.

[24]

B.A. Kurdi, M. Alshurideh, and T.A. afaishat, "Employee retention and organizational performance: Evidence from banking industry", Management Science Letters, vol. 10, no. 16, pp. 3981-3990, 2020. [http://dx.doi.org/10.5267/j.msl.2020.7.011]

[25]

M. Alshurideh, A. Bataineh, B. Alkurdi, and N. Alasmr, "Factors affect mobile phone brand choicesstudying the case of Jordan Universities Students", Int. Bus. Res., vol. 8, no. 3, 2015. [http://dx.doi.org/10.5539/ibr.v8n3p141]

[26]

Nicholson Alshurideh, "The effect of previous experience on mobile subscribers’ repeats purchases behavior", Eur. J. Soil Sci., vol. 30, no. 3, 2012.

[27]

T. De Vass, H. Shee, and S.J. Miah, "The effect of “Internet of Things” on supply chain integration and performance: An organisational capability perspective", AJIS Australas. J. Inf. Syst., vol. 22, pp. 1-29, 2018. [http://dx.doi.org/10.3127/ajis.v22i0.1734]

[28]

R.T. Yarlagadda, "Applications management using Al Automation", International Journal of Creative Rese1U" ch Thoughts (UCRT), ISSN, pp. 2320-2882, 2021.

[29]

N. Tsolakis, D. Zissis, S. Papaeflhimiou, and N. Korfiatis, "Towards AI-driven environmental sustainability: an application of automated logistics in container port terminals", Int. J. Prod. Res., pp. 1-21, 2021.

Empirical Impact

IoT and Big Data Analytics, Vol. 1 111

[30]

SK. Shanna, N. Gayathri, S.R. Kumar, and R.K. Modanval, "Techniques in robotics for automation using AI and IoT", Al and IoT-Based Intelligent Automation in Robotics, pp. 15-33, 2021.

[31]

J. Steinhoff, "Machine Learning and Fixed Capital: The Contemporary AI Industry", In: Automation and Autonomy Palgrave Macmillan: Champp. 133-170.

[32]

J. Newton, "Evolutionary game theory: A renaissance", Games (Basel), vol. 9, no. 2, p. 31, 2018. [http://dx.doi.org/10.3390/g9020031]

[33]

M. Marsonet, "Philosophy and logical positivism", Academics International Scientific Journal, vol. I0, no. 19, pp. 32-36, 2019. [http://dx.doi.org/10.7336/academicus.2019.19.02]

[34]

Z. Munn, M.D.J. Peters, C. Stern, C. Tufanaru, A. McArthur, and E. Aromataris, "Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach", BMC Med. Res. Methodol., vol. 18, no. 1, p. 143, 2018. [http://dx.doi.org/10.1186/s12874-018-0611-x] [PMID: 30453902]

[35]

F. Alam, A. Almaghthawi, I. Katib, I, A. Albeshr, and R. Mehmood, IResponse: An AI and IoTenabled framework for autonomous COVID- 19 pandemic management sustainability, vol. 13, no. 7, p. 3797, 2021.

[36]

C. Zehir, and T. Karaboga, "The transformation of human resource management and its impact on overall business performance: big data analytics and AI technologies in strategic HRM", In: Digital business strategies in blockchain ecosystems Springer: Cham, 2020, pp. 265-279.

[37]

J.S. Park, and J.H. Park, "Future Trends of IoT, 5G Mobile Networks, and AI: Challenges, Opportunities, and Solutions", Journal of Information Processing Systems, vol. 16, no. 4, pp. 743-749, 2020.

[38]

G. Rana, and R. Sharma, "Emerging human resource management practices in Industry 4.0", Strategic HR Rev., vol. 18, no. 4, pp. 176-181, 2019. [http://dx.doi.org/10.1108/SHR-01-2019-0003]

[39]

L. Xiao, X. Wan, X. Lu, Y. Zhang, and D. Wu, "IoT security techniques based on machine learning: How do IoT devices use AI to enhance security?", IEEE Signal Process. Mag., vol. 35, no. 5, pp. 4149, 2018. [http://dx.doi.org/10.1109/MSP.2018.2825478]

112

IoT and Big Data Analytics, 2023, Vol. 1, 112-129

CHAPTER 8

A Review of Progress Status in Achieving the Jal Jeevan Mission Goals in the State of Chhattisgarh Surendra Rahamatkar1,*, Sweta Patnaik2 and Satyendra Patnaik3 Amity School Engineering & Technology, Amity University Chhattisgarh, India WASH Specialist, UNICEF, Chhattisgarh, India 3 Amity Innovation & Incubator, Amity University Chhattisgarh, India 1 2

Abstract: The water governance process is seen as a mechanism to solve both the deficiencies of the broader governance process and the Millennium Development Goals (MDGs). Millions of people in India rely on unstable, low-quality water supplies that are expensive and far from their homes. Water scarcity has a wide range of socioeconomic consequences. Women and girls are typically entrusted with the responsibility of collecting water for domestic purposes. Providing rural communities with piped water for drinking and domestic needs is an important and difficult endeavor. In light of the above Amity University Chhattisgarh partnering with UNICEF Chhattisgarh is entrusted with the responsibility of providing technical & monitoring support to the Mission Directorate, Jal Jeevan Mission (JJM), and Public Health & Engineering Department (PHED), Government of Chhattisgarh along with Institutional Capacity Building, planning, monitoring and reporting for effective implementation of safe rural drinking water facilities across the Chhattisgarh State. Materials and Methods In this perspective, the prime objective of the project was to strengthen the institutional capacity building, and program monitoring in 28 districts of Chhattisgarh through the establishment of the State Program Management Unit (SPMU) at the Directorate of Jal Jeevan Mission (JJM) and Project Management Unit (DPMU) at 14 district levels with light support to 14 additional districts. The method used for data collection was primary and secondary. The findings of the study are presented in suitable tables and graphs. Results Results of the study show the progress in solving drinking water problems in rural areas of Chhattisgarh since the inception of the monitoring project i.e. 1st June 2021. Corresponding author Surendra Rahamatkar: Amity School Engineering & Technology, Amity University Chhattisgarh, India; E-mail: [email protected]

*

Abhishek Singh Rathore, Surendra Rahamatkar, Syed Imran Ali, Ramgopal Kashyap & Nand Kishore Sharma (Eds.) All rights reserved-© 2023 Bentham Science Publishers

A Review of Progress

IoT and Big Data Analytics, Vol. 1 113

Conclusion In this paper, an attempt has been made to report the progress of the implementation of the JJM scheme in partnership with UNICEF Chhattisgarh.

Keywords: District planning, Jal jeevan mission, Rural drinking water, Sustainability. INTRODUCTION Sustainable rural drinking water is a common goal in India and around the world, ranging from individual households to village, district, state, and national levels. The Government of India launched the Jal Jeevan Mission in December 2019 intending to provide functional household tap connections (FHTP) to everyone by 2024 as a highly ambitious goal. As on 12th February 2022, only 46.59% of the rural household throughout India had tap connections although the number is increasing rapidly. It varies widely amongst states, ranging from 17.71% to 100%. Chhattisgarh state with its 28 districts had only 17.71% of the rural household had tap connections as shown in Fig. (1). Although tap connections are significant, they are not the only indicator of safe drinking water availability. According to the Ministry of Housing and Urban Affairs, the benchmark for urban water supply is 135 liters per capita per day (lpcd). Under the Jal Jeevan Mission, the minimum service delivery of 55 lpcd has been set for rural regions, which states may increase to a higher level. Do rural households get the full 55 liters per capita per day (lpcd) that current national requirements require? Is this quantity received every year, or do they require tankers during dry months and drought years? Do different classes, castes, and tribal habitations receive the same level of service in villages? Do they collect enough fair water tariffs? Is it still necessary for women and girls to fetch water from faraway wells or ponds? Is the water safe to drink? 933 water samples were collected from National Hydrograph Stations across Chhattisgarh to test the chemical purity of groundwater. These samples were taken in May 2018, during the pre-monsoon season, when ion concentrations were at their highest. pH, EC, CO3, HCO3, CI, SO4, F, TH, Ca, Mg, Na, K, PO4, and Si were all determined in the water samples [1]. These are some of the most pressing issues facing India's rural drinking water sustainability today. It's worth noting that India is quickly urbanizing, and the lines between urban and rural are blurring. Rural areas are classified as those with fewer than 5000 persons, less than 75 percent of the male workforce engaged in nonagricultural occupations, and population densities of less than 400 people per square kilometer, according to the Indian Census. Rurban or peri-urban villages are becoming more common. Rural residential water service

114 IoT and Big Data Analytics, Vol. 1

Rahamatkar et al.

must meet a national level of 55 lpcd, whereas peri-urban regions must meet a standard of 70 lpcd, and urban areas must meet a standard of 135 lpcd.

Chhattisgarh (17.71%) 0%-10%

11%-25%

26%-50%

51%-75%

76%-