Deep Learning Algorithms 9781774691830

Table of contents :
Cover
Title Page
Copyright
DECLARATION
ABOUT THE EDITOR
TABLE OF CONTENTS
List of Contributors
List of Abbreviations
Preface
Section 1: Methods and Approaches for Deep Learning
Chapter 1 Advancements in Deep Learning Theory and Applications: Perspective in 2020 and Beyond
Abstract
Introduction
Deep Network Topologies
Application of Deep Learning
Modern Deep Learning Platforms
Training Algorithms
Routine Challenges of Deep Learning
Available Open-Source Datasets
References
Chapter 2 Deep Ensemble Reinforcement Learning With Multiple Deep Deterministic Policy Gradient Algorithm
Abstract
Introduction
Background
Methods
Results and Discussion
Conclusions
References
Chapter 3 Dynamic Decision-Making For Stabilized Deep Learning Software Platforms
Abstract
Introduction
Stabilized Control for Reliable Deep Learning Platforms
The Use of Lyapunov Optimization for Deep Learning Platforms
Emerging Applications
Conclusions
Acknowledgements
References
Chapter 4 Deep Learning For Hyperspectral Data Classification Through Exponential Momentum Deep Convolution Neural Networks
Abstract
Introduction
Feature Learning
Structure Design of Hyperspectral Data Classification Framework
Exponential Momentum Gradient Descent Algorithm
Experiment and Analysis
Conclusion
Acknowledgments
References
Chapter 5 Ensemble Network Architecture for Deep Reinforcement Learning
Abstract
Introduction
Related Work
Ensemble Methods for Deep Reinforcement Learning
Experiments
Conclusion
References
Section 2: Deep Learning Techniques Applied in Biology
Chapter 6 Fish Detection Using Deep Learning
Abstract
Introduction
Literature Review
Materials and Methods
Data Augmentation
Results and Discussion
Conclusion
Acknowledgments
References
Chapter 7 Can Deep Learning Identify Tomato Leaf Disease?
Abstract
Introduction
Related Work
Materials and Methods
Experiments and Results
Conclusion
Acknowledgments
References
Chapter 8 Deep Learning For Plant Identification In Natural Environment
Abstract
Introduction
Proposed Bjfu100 Dataset and Deep Learning Model
Experiments and Results
Resnet26 on Flavia Dataset
Conclusion
Acknowledgments
References
Chapter 9 Applying Deep Learning Models to Mouse Behavior Recognition
Abstract
Introduction
The Mouse Behavior Dataset
Experiments and Results
Conclusions
Acknowledgements
References
Section 3: Deep learning Applications in Medicine
Chapter 10 Application of Deep Learning in Neuroradiology: Brain Hemorrhage Classification Using Transfer Learning
Abstract
Introduction
Related Work
Convolutional Neural Network
Transfer Learning
Materials and Methods
Results and Discussion
Limitations
Conclusion
References
Chapter 11 A Review of the Application of Deep Learning in Brachytherapy
Abstract
Introduction
Organ Delineation and Segmentation
Segmentation and Reconstruction of the Applicator (Interstitial Needles)
Dose Calculation
Application of Treatment Planning System
Others
Conclusions
References
Chapter 12 Exploring Deep Learning and Transfer Learning for Colonic Polyp Classification
Abstract
Introduction
Materials and Methods
Results and Discussion
Conclusion
Acknowledgments
References
Chapter 13 Deep Learning Algorithm For Brain-Computer Interface
Abstract
Introduction
Critical Review of the Related Literature
Comparison of Classification Algorithms
Discussion
Methodology
Conclusion
References
Section 4: Deep Learning in Pattern Recognition Tasks
Chapter 14 The Application of Deep Learning In Airport Visibility Forecast
Abstract
Introduction
Deep Learning
The Establishment of Prediction Model
Predictive Effect Test
Conclusions
References
Chapter 15 Hierarchical Representations Feature Deep Learning For Face Recognition
Abstract
Introduction
Images Preprocessing
Feature Extraction
Designing the Classifiers of Supervised Learning
Designing the Classifier Combining Unsupervised and Supervised Learning
Experiments
Conclusion
Acknowledgements
References
Chapter 16 Review of Research on Text Sentiment Analysis Based on Deep Learning
Abstract
Introduction
Brief Review on the Research Progress of Text Sentiment Analysis
Introduction to Text Sentiment Analysis Based on Deep Learning
Summary and Prospect
References
Chapter 17 Classifying Hand Written Digits With Deep Learning
Abstract
Introduction
Digit Classification with Deep Networks
Experiment
Conclusions
References
Chapter 18 Bitcoin Price Prediction Based on Deep Learning Methods
Abstract
Introduction
Dataset Exploration
Pre-Processing
Models
Results
Conclusion and Discussion
References
Index
Back Cover

Citation preview

本书版权归Arcler所有

本书版权归Arcler所有

Deep Learning Algorithms

本书版权归Arcler所有

本书版权归Arcler所有

Deep Learning Algorithms

Edited by: Zoran Gacovski

ARCLER

P

r

e

s

s

www.arclerpress.com

Deep Learning Algorithms Zoran Gacovski

Arcler Press 224 Shoreacres Road Burlington, ON L7L 2H2 Canada www.arclerpress.com Email: [email protected]

HERRN(GLWLRQ2 ISBN: (HERRN)

This book contains information obtained from highly regarded resources. Reprinted material sources are indicated. Copyright for individual articles remains with the authors as indicated and published under Creative Commons License. A Wide variety of references are listed. Reasonable efforts have been made to publish reliable data and views articulated in the chapters are those of the individual contributors, and not necessarily those of the editors or publishers. Editors or publishers are not responsible for the accuracy of the information in the published chapters or consequences of their use. The publisher assumes no responsibility for any damage or grievance to the persons or property arising out of the use of any materials, instructions, methods or thoughts in the book. The editors and the publisher have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission has not been obtained. If any copyright holder has not been acknowledged, please write to us so we may rectify. Notice: Registered trademark of products or corporate names are used only for explana        © 2022 Arcler Press ISBN: 978-1-77469-183-0 (Hardcover) Arcler Press publishes wide variety of books and eBooks. For more information about Arcler Press and its products, visit our website at www.arclerpress.com

本书版权归Arcler所有

DECLARATION Some content or chapters in this book are open access copyright free published research work, which is published under Creative Commons License and are indicated with the citation. We are thankful to the publishers and authors of the content and chapters as without them this book wouldn’t have been possible.

本书版权归Arcler所有

本书版权归Arcler所有

ABOUT THE EDITOR

Dr. Zoran Gacovski has earned his PhD degree at Faculty of Electrical engineering, Skopje. His research interests include Intelligent systems and Software engineering, fuzzy systems, graphical models (Petri, Neural and Bayesian networks), and IT security. He has published over 50 journal and conference papers, and he has been reviewer of renowned Journals. Currently, he is a professor in Computer Engineering at European University, Skopje, Macedonia.

本书版权归Arcler所有

TABLE OF CONTENTS

List of Contributors .......................................................................................xv List of Abbreviations .................................................................................... xxi Preface.................................................................................................. ....xxiii Section 1: Methods and Approaches for Deep Learning Chapter 1

Advancements in Deep Learning Theory and Applications: Perspective in 2020 and Beyond ............................................................... 3 Abstract ..................................................................................................... 3 Introduction ............................................................................................... 4 Deep Network Topologies.......................................................................... 8 Application of Deep Learning .................................................................. 11 Modern Deep Learning Platforms ............................................................ 14 Training Algorithms.................................................................................. 17 Routine Challenges of Deep Learning ...................................................... 19 Available Open-Source Datasets .............................................................. 21 References ............................................................................................... 24

Chapter 2

Deep Ensemble Reinforcement Learning With Multiple Deep Deterministic Policy Gradient Algorithm....................................... 29 Abstract ................................................................................................... 29 Introduction ............................................................................................. 30 Background ............................................................................................. 32 Methods .................................................................................................. 34 Results and Discussion ............................................................................ 39 Conclusions ............................................................................................. 50 References ............................................................................................... 51

Chapter 3

Dynamic Decision-Making For Stabilized Deep Learning Software Platforms .................................................................................. 55 Abstract ................................................................................................... 55 Introduction ............................................................................................. 56 Stabilized Control for Reliable Deep Learning Platforms .......................... 57 The Use of Lyapunov Optimization for Deep Learning Platforms ............. 63 Emerging Applications ............................................................................. 68 Conclusions ............................................................................................. 69 Acknowledgements ................................................................................. 70 References ............................................................................................... 71

Chapter 4

Deep Learning For Hyperspectral Data Classification Through Exponential Momentum Deep Convolution Neural Networks ................ 73 Abstract ................................................................................................... 73 Introduction ............................................................................................. 74 Feature Learning ...................................................................................... 75 Structure Design of Hyperspectral Data Classification Framework ........... 76 Exponential Momentum Gradient Descent Algorithm .............................. 77 Experiment and Analysis .......................................................................... 80 Conclusion .............................................................................................. 86 Acknowledgments ................................................................................... 87 References ............................................................................................... 88

Chapter 5

Ensemble Network Architecture for Deep Reinforcement Learning .................................................................................................. 93 Abstract ................................................................................................... 93 Introduction ............................................................................................. 94 Related Work ........................................................................................... 95 Ensemble Methods for Deep Reinforcement Learning .............................. 97 Experiments ........................................................................................... 100 Conclusion ............................................................................................ 102 References ............................................................................................. 104

x

Section 2: Deep Learning Techniques Applied in Biology Chapter 6

Fish Detection Using Deep Learning ..................................................... 109 Abstract ................................................................................................. 109 Introduction ........................................................................................... 110 Literature Review ................................................................................... 111 Materials and Methods .......................................................................... 113 Data Augmentation ................................................................................ 118 Results and Discussion .......................................................................... 126 Conclusion ............................................................................................ 129 Acknowledgments ................................................................................. 130 References ............................................................................................. 131

Chapter 7

Can Deep Learning Identify Tomato Leaf Disease? ............................... 135 Abstract ................................................................................................. 135 Introduction ........................................................................................... 136 Related Work ......................................................................................... 137 Materials and Methods .......................................................................... 138 Experiments and Results ........................................................................ 143 Conclusion ............................................................................................ 149 Acknowledgments ................................................................................. 150 References ............................................................................................. 151

Chapter 8

Deep Learning For Plant Identification In Natural Environment ........... 157 Abstract ................................................................................................. 157 Introduction ........................................................................................... 158 Proposed Bjfu100 Dataset and Deep Learning Model ............................ 159 Experiments and Results ........................................................................ 162 Resnet26 on Flavia Dataset .................................................................... 165 Conclusion ............................................................................................ 166 Acknowledgments ................................................................................. 167 References ............................................................................................. 168

Chapter 9

Applying Deep Learning Models to Mouse Behavior Recognition ......... 171 Abstract ................................................................................................. 171 Introduction ........................................................................................... 172

xi

The Mouse Behavior Dataset ................................................................. 174 Experiments and Results ........................................................................ 175 Conclusions ........................................................................................... 186 Acknowledgements ............................................................................... 186 References ............................................................................................. 187 Section 3: Deep learning Applications in Medicine Chapter 10 Application of Deep Learning in Neuroradiology: Brain Hemorrhage Classification Using Transfer Learning ............................. 191 Abstract ................................................................................................. 191 Introduction ........................................................................................... 192 Related Work ......................................................................................... 194 Convolutional Neural Network .............................................................. 195 Transfer Learning ................................................................................... 196 Materials and Methods .......................................................................... 197 Results and Discussion .......................................................................... 204 Limitations ............................................................................................. 210 Conclusion ............................................................................................ 211 References ............................................................................................. 212 Chapter 11 A Review of the Application of Deep Learning in Brachytherapy.......... 217 Abstract ................................................................................................. 217 Introduction ........................................................................................... 218 Organ Delineation and Segmentation .................................................... 219 Segmentation and Reconstruction of the Applicator (Interstitial Needles) ..................................................................... 220 Dose Calculation ................................................................................... 222 Application of Treatment Planning System ............................................. 222 Others ................................................................................................... 223 Conclusions ........................................................................................... 224 References ............................................................................................. 225

xii

Chapter 12 Exploring Deep Learning and Transfer Learning for Colonic Polyp Classification ............................................................................... 229 Abstract ................................................................................................. 229 Introduction ........................................................................................... 230 Materials and Methods .......................................................................... 232 Results and Discussion .......................................................................... 242 Conclusion ............................................................................................ 250 Acknowledgments ................................................................................. 251 References ............................................................................................. 252 Chapter 13 Deep Learning Algorithm For Brain-Computer Interface ...................... 259 Abstract ................................................................................................. 259 Introduction ........................................................................................... 260 Critical Review of the Related Literature ................................................ 273 Comparison of Classification Algorithms ................................................ 276 Discussion ............................................................................................. 277 Methodology ......................................................................................... 280 Conclusion ............................................................................................ 281 References ............................................................................................. 282 Section 4: Deep Learning in Pattern Recognition Tasks Chapter 14 The Application of Deep Learning In Airport Visibility Forecast ........... 287 Abstract ................................................................................................. 287 Introduction ........................................................................................... 288 Deep Learning ....................................................................................... 288 The Establishment of Prediction Model .................................................. 289 Predictive Effect Test............................................................................... 291 Conclusions ........................................................................................... 295 References ............................................................................................. 297 Chapter 15 Hierarchical Representations Feature Deep Learning For Face Recognition............................................................................. 299 Abstract ................................................................................................. 299 Introduction ........................................................................................... 300 Images Preprocessing............................................................................. 302 Feature Extraction .................................................................................. 304 xiii

Designing the Classifiers of Supervised Learning .................................... 307 Designing the Classifier Combining Unsupervised and Supervised Learning .............................................................. 315 Experiments ........................................................................................... 322 Conclusion ............................................................................................ 332 Acknowledgements ............................................................................... 332 References ............................................................................................. 334 Chapter 16 Review of Research on Text Sentiment Analysis Based on Deep Learning .................................................................................. 341 Abstract ................................................................................................. 341 Introduction ........................................................................................... 342 Brief Review on the Research Progress of Text Sentiment Analysis .......... 343 Introduction to Text Sentiment Analysis Based on Deep Learning ........... 344 Summary and Prospect .......................................................................... 348 References ............................................................................................. 350 Chapter 17 Classifying Hand Written Digits With Deep Learning ........................... 353 Abstract ................................................................................................. 353 Introduction ........................................................................................... 354 Digit Classification with Deep Networks................................................ 354 Experiment ............................................................................................ 360 Conclusions ........................................................................................... 361 References ............................................................................................. 364 Chapter 18 Bitcoin Price Prediction Based on Deep Learning Methods ................................................................................................ 367 Abstract ................................................................................................. 367 Introduction ........................................................................................... 368 Dataset Exploration................................................................................ 368 Pre-Processing ....................................................................................... 369 Models .................................................................................................. 369 Results ................................................................................................... 371 Conclusion and Discussion.................................................................... 375 References ............................................................................................. 376 Index ..................................................................................................... 377

xiv

LIST OF CONTRIBUTORS Md Nazmus Saadat University of Kuala Lumpur, Malaysia Muhammad Shuaib University of Kuala Lumpur, Malaysia Junta Wu Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518071, China Huiyun Li Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518071, China Soohyun Park Korea University, Seoul, Republic of Korea Dohyun Kim Naver Webtoon Corporation, Seongnam, Republic of Korea Joongheon Kim Korea University, Seoul, Republic of Korea Qi Yue Xi’an Institute of Optics and Precision Mechanics, CAS, Xi’an 710119, China University of Chinese Academy of Sciences, Beijing 100039, China Xi’an University of Posts and Telecommunications, Xi’an 710121, China Caiwen Ma Xi’an Institute of Optics and Precision Mechanics, CAS, Xi’an 710119, China Xi-liang Chen Institute of Command Information System, PLA University of Science and Technology, No. 1, Hai Fu Road, Guang Hua Road, Qin Huai District, Nanjing City, Jiangsu Province 210007, China

Lei Cao Institute of Command Information System, PLA University of Science and Technology, No. 1, Hai Fu Road, Guang Hua Road, Qin Huai District, Nanjing City, Jiangsu Province 210007, China Chen-xi Li Institute of Command Information System, PLA University of Science and Technology, No. 1, Hai Fu Road, Guang Hua Road, Qin Huai District, Nanjing City, Jiangsu Province 210007, China Zhi-xiong Xu Institute of Command Information System, PLA University of Science and Technology, No. 1, Hai Fu Road, Guang Hua Road, Qin Huai District, Nanjing City, Jiangsu Province 210007, China Jun Lai Institute of Command Information System, PLA University of Science and Technology, No. 1, Hai Fu Road, Guang Hua Road, Qin Huai District, Nanjing City, Jiangsu Province 210007, China Suxia Cui Department of Electrical and Computer Engineering, Prairie View A&M University, Prairie View, TX 77446, USA Yu Zhou Department of Electrical and Computer Engineering, Prairie View A&M University, Prairie View, TX 77446, USA Yonghui Wang Department of Computer Science, Prairie View A&M University, Prairie View, TX 77446, USA Lujun Zhai Department of Electrical and Computer Engineering, Prairie View A&M University, Prairie View, TX 77446, USA Keke Zhang College of Engineering, Northeast Agricultural University, Harbin 150030, China Qiufeng Wu College of Science, Northeast Agricultural University, Harbin 150030, China Anwang Liu College of Engineering, Northeast Agricultural University, Harbin 150030, China xvi

Xiangyan Meng College of Science, Northeast Agricultural University, Harbin 150030, China Yu Sun School of Information Science and Technology, Beijing Forestry University, Beijing 100083, China Yuan Liu School of Information Science and Technology, Beijing Forestry University, Beijing 100083, China Guan Wang School of Information Science and Technology, Beijing Forestry University, Beijing 100083, China Haiyan Zhang School of Information Science and Technology, Beijing Forestry University, Beijing 100083, China Ngoc Giang Nguyen Graduate School of Natural Science and Technology, Kanazawa University, Kanazawa, Japan; Dau Phan Graduate School of Natural Science and Technology, Kanazawa University, Kanazawa, Japan; Favorisen Rosyking Lumbanraja Graduate School of Natural Science and Technology, Kanazawa University, Kanazawa, Japan; Mohammad Reza Faisal Graduate School of Natural Science and Technology, Kanazawa University, Kanazawa, Japan; Bahriddin Abapihi Graduate School of Natural Science and Technology, Kanazawa University, Kanazawa, Japan; Bedy Purnama Graduate School of Natural Science and Technology, Kanazawa University, Kanazawa, Japan;

xvii

Mera Kartika Delimayanti Graduate School of Natural Science and Technology, Kanazawa University, Kanazawa, Japan; Kunti Robiatul Mahmudah Graduate School of Natural Science and Technology, Kanazawa University, Kanazawa, Japan; Mamoru Kubo Institute of Science and Engineering, Kanazawa University, Kanazawa, Japan Kenji Satou Institute of Science and Engineering, Kanazawa University, Kanazawa, Japan Awwal Muhammad Dawud Department of Computer Engineering, Cyprus International University, Nicosia, Cyprus Kamil Yurtkan Department of Computer Engineering, Cyprus International University, Nicosia, Cyprus Huseyin Oztoprak Department of Computer Engineering, Cyprus International University, Nicosia, Cyprus Hai Hu Applied Nuclear Technology in Geosciences Key Laboratory of Sichuan Province, Chengdu University of Technology, Chengdu, China Yang Shao Applied Nuclear Technology in Geosciences Key Laboratory of Sichuan Province, Chengdu University of Technology, Chengdu, China Shijie Hu Applied Nuclear Technology in Geosciences Key Laboratory of Sichuan Province, Chengdu University of Technology, Chengdu, China Eduardo Ribeiro Department of Computer Sciences, University of Salzburg, Salzburg, Austria Department of Computer Sciences, Federal University of Tocantins, Palmas, TO, Brazil Andreas Uhl Department of Computer Sciences, University of Salzburg, Salzburg, Austria Georg Wimmer Department of Computer Sciences, University of Salzburg, Salzburg, Austria xviii

Michael Häfner St. Elisabeth Hospital, Vienna, Austria Asif Mansoor National University of Sciences and Technology, Islamabad, Pakistan Muhammad Waleed Usman National University of Computer and Emerging Sciences, Islamabad, Pakistan Noreen Jamil National University of Computer and Emerging Sciences, Islamabad, Pakistan M. Asif Naeem National University of Computer and Emerging Sciences, Islamabad, Pakistan Lei Zhu Training Center of Xinjiang Air Traffic Management Bureau, Urumqi, China Guodong Zhu College of Atmospheric Science, Nanjing University, Nanjing, China Meteorological Center of Xinjiang Air Traffic Management Bureau, Urumqi, China Lei Han Meteorological Center of Xinjiang Air Traffic Management Bureau, Urumqi, China Nan Wang Meteorological Center of Xinjiang Air Traffic Management Bureau, Urumqi, China Haijun Zhang Guangdong Provincial Key Laboratory of Conservation and Precision Utilization of Characteristic Agricultural Resources in Mountainous Areas, Meizhou, China School of Computing, Jiaying University, Meizhou, China Yinghui Chen Guangdong Provincial Key Laboratory of Conservation and Precision Utilization of Characteristic Agricultural Resources in Mountainous Areas, Meizhou, China School of Mathematics, Jiaying University, Meizhou, China Wenling Li College of Science, Yanbian University, Yanji, China Bo Jin College of Science, Yanbian University, Yanji, China

xix

Yu Quan Department of Economics and Management of Yanbian University, Yanji, China Ruzhang Yang Shanghai Foreign Language School, Shanghai, China Xiangxi Jiang Barstow School of Ningbo, Ningbo, China

xx

LIST OF ABBREVIATIONS  

    

AUV

Autonomous underwater vehicle

BA

Boltzmann addition

BM

Boltzmann multiplication

CPU

Central processing unit

CAD

Computer-aided diagnosis

CT

Computer tomography

CV

Computer vision

CNN

Convolution neural network

DBN

Deep belief network

DCNN

Deep convolution neural network

DNNs

Deep neural network

DRL

Deep reinforcement learning

DRBM

Deep restricted Boltzmann machine

DPG

Deterministic policy gradient

GRU

Gated recurrent units

GPU

Graphical processing unit

HBPNNs

Hybrid BP neural networks

ICH

Intracranial haemorrhage

JELSR

Joint embedding learning and sparse regression

LR

Logistic regression

LSTM

Long short-term memory network

MV

Majority voting

MDP

Markov decision processes

MRSF

Minimum redundancy spectral feature selection

MVEP

Motion-onset visual evoked potential

MLP

Multi-layer perceptron

NIRS

Near-infrared spectroscopy

PMI

Pointwise mutual information

PFCL

Prior fully connected layers

QDA

Quadratic discriminant analysis

RBF

Radial Basis Function

RH

Relative humidity

RBM

Restricted Boltzmann machine

RMSE

Root mean square error

SVD

Singular value decomposition

SCP

Slow cortical potentials

TORCS

The open racing car simulator

xxii

PREFACE

The Deep learning is a branch of machine learning based on data presentation via complex representations with high degree of abstraction - that are obtained by applying learned nonlinear transformations. Deep learning methods find their application in important areas of artificial intelligence, such as: computer vision, natural language processing, speech and sound comprehension, as well as in bioinformatics. Deep learning is a class of machine learning algorithms that: 

uses multilayer nonlinear processor units to extract and transform features. Each subsequent layer takes as input the output elements of the previous layer.  learns in a supervised and / or unsupervised manner.  learns a number of levels of representation - corresponding to different degrees of abstraction.  uses some form of descending gradient algorithm to train through error backpropagation. The layers used in deep programming include the hidden layers of the artificial neural network and a multitude of statement formulas. This book covers the most important discriminant and generative deep models with special emphasis on practical implementations. We cover the key elements of classical neural networks and provides an overview of the building blocks, regularization techniques, and learning methods that are specific to deep models. Also we consider the deep convolutional models and illustrates their application in image classification and natural language processing. The generative deep models are often used in computer vision applications and natural language processing. Sequence modeling by deep feedback neural networks can be applied in the field of natural language processing. Practical implementations of deep learning are made in modern dynamic languages (Python, Lua or Julia), and also with application frameworks for deep learning (e.g. Theano, TensorFlow, Torch). This edition covers different topics from deep learning algorithms, including: methods and approaches for deep learning, deep learning applications in biology, deep learning applications in medicine, and deep learning applications in pattern recognition systems. Section 1 focuses on methods and approaches for deep learning, describing advancements in deep learning theory and applications - perspective in 2020 and beyond; deep ensemble reinforcement learning with multiple deep deterministic policy gradient algorithm; dynamic decision-making for stabilized deep learning software

platforms; deep learning for hyperspectral data classification through exponential momentum deep convolution neural networks; and ensemble network architecture for deep reinforcement learning. Section 2 focuses on deep learning applications in biology, describing fish detection using deep learning; deep learning identification of tomato leaf disease; deep learning for plant identification in natural environment; and applying deep learning models to mouse behavior recognition. Section 3 focuses on deep learning applications in medicine, describing application of deep learning in neuroradiology: brain hemorrhage classification using transfer learning; a review of the application of deep learning in brachytherapy; exploring deep learning and transfer learning for colonic polyp classification; and deep learning algorithm for brain-computer interface. Section 4 focuses on deep learning applications in pattern recognition systems, describing application of deep learning in airport visibility forecast; hierarchical representations feature deep learning for face recognition; review of research on text sentiment analysis based on deep learning; classifying hand written digits with deep learning; and bitcoin price prediction based on deep learning methods.

SECTION 1:

Methods and Approaches for Deep Learning

CHAPTER 1

ADVANCEMENTS IN DEEP LEARNING THEORY AND APPLICATIONS: PERSPECTIVE IN 2020 AND BEYOND Md Nazmus Saadat and Muhammad Shuaib University of Kuala Lumpur, Malaysia

ABSTRACT The aim of this chapter is to introduce newcomers to deep learning, deep learning platforms, algorithms, applications, and open-source datasets. This chapter will give you a broad overview of the term deep learning, in context to deep learning machine learning, and Artificial Intelligence (AI) is also introduced. In Introduction, there is a brief overview of the research achievements of deep learning. After Introduction, a brief history of deep learning has been also discussed. The history started from a famous scientist called Allen Turing (1951) to 2020. In the start of a chapter after

Citation: Md Nazmus Saadat and Muhammad Shuaib (December 9th 2020). Advancements in Deep Learning Theory and Applications: Perspective in 2020 and beyond, Advances and Applications in Deep Learning, Marco Antonio Aceves-Fernandez, IntechOpen, DOI: 10.5772/intechopen.92271. Copyright: © 2020 by authors and IntechOpen. This paper is an open access article distributed under a Creative Commons Attribution 3.0 License .

4

Deep Learning Algorithms

Introduction, there are some commonly used terminologies, which are used in deep learning. The main focus is on the most recent applications, the most commonly used algorithms, modern platforms, and relevant opensource databases or datasets available online. While discussing the most recent applications and platforms of deep learning, their scope in future is also discussed. Future research directions are discussed in applications and platforms. The natural language processing and auto-pilot vehicles were considered the state-of-the-art application, and these applications still need a good portion of further research. Any reader from undergraduate and postgraduate students, data scientist, and researchers would be benefitted from this. Keywords:- Deep learning, machine learning      , neural networks

INTRODUCTION Deep learning is focusing comprehensively on video, image, text and audio recognition, autonomous driving, robotics, healthcare, etc. [1]. Deep learning is a result orientated field of study that why getting very much attention from researcher and academicians. The Rina Dechter introduced the word of deep learning in 1986, the main motivation behind the advent of field deep learning was making an intelligent machine that mimic the human brain. In humans, the brain is the most important and decision-making organ; brain takes decision based on sight, smell, touch, and sounds. The brain also can store memory and solve complex problems based on their experience. For the last few decades, the researchers dreamed of making a machine that is as intelligent as, like our brains, they started studying the biological structure and working of the human brain. Making a robot that performs certain duties and self-driving cars is to reduce roadside incidents. Because according to the World Health Organization (WHO), 1.35 million people die every year in road incidents [2] and approximately 90% of the incidents are due to human errors [3]. To develop state-of-the-art devices for the applications listed above, ones need to think in a different way of           of the most innovative paradigms that make it possible up to some extent. In deep learning, the word deep indicates the number of layers through which               ! "     intelligence machine learning or deep learning because all these overlap

Advancements In Deep Learning Theory And Applications: ...

5

each other some way or the other. Machine learning is any sort of computer program that can learn by their own without having specially programmed by the programmer. There are two types of machine learning: supervised learning and unsupervised learning. In supervised learning, you teach or train the machine with a fully labeled data, the machine learns from the labeled data and then anticipate the unforeseen data. In supervised learning, the machine can only give you correct output when the input is already experienced in training phase; it is based on experience; the more is the training dataset or experience of your machine the higher is the chances of getting the actual output. It is a time-consuming process and also required a lot of expertise in data science. On the other hand, in unsupervised learning, supervision of a model is not needed, rather the model work on its own catches new data and discovers the information inside the data. It usually deals with label-less data; compared to supervised learning, unsupervised          patterns. Deep learning models are agile and result oriented in terms of complicated abstractions. Deep learning models are mostly based on ANN, categorically CNNs, although there are deep belief networks, generative models, propositional formulas and Boltzmann machine also play their part (Figure 1).

Figure 1. Deep learning a subset of machine learning and AI.

Deep learning has been evaluated as a game-changer in AI and computer vision. Today, state-of-the-art object detection is possible only due to deep learning [4]; traditional methods of object detection are not enough to cater with detection so smartly. To understand the whole image of object detection,               #    

6

Deep Learning Algorithms

calculate the concept and locations of the objects in every image, that is, object detection which is based on face detection, pedestrian detection, and skeleton detection [5]. Deep learning has cutting-edge technology               healthcare. It has a very deep impact on the life of the people or societies because its application is always the need of the day. The deep learning     $   # data analytics. Big data analytics is the number of complicated processes '       *  methods used to identify the hidden patterns, unknown correlations market trends, and customer preference from huge dataset. Big data analytics can  # #           #            Deep learning is an emerging area of research and modern application. The deep learning       + covers industry, business, and healthcare; it combines all the hot research